Betty, Penn’s new off-campus supercomputer, joins the AI arms race

The new, glass-walled supercomputer called Betty sits on the top floor of a high-fenced brick data center on a hill 30 miles northwest — by road and two separate twin-fiber data lines — from its operators at the University of Pennsylvania.

Betty is a stack of central processing units, a “SuperPOD” of graphics processing units, plus data storage, made by Nvidia, the world’s most valuable company, thanks to soaring demand for high-speed and artificial-intelligence queries.

Betty is designed to run AI models that search, analyze, and report findings from videos, images, texts, and databanks. The models are designed to improve with each query, per researchers’ ever-more-specialized instructions.

At its home in Montgomery County, Betty hums louder than a commercial jet rising from an airport. It burns enough electricity to power a village of 1,000 homes.

The site, named for pioneering Penn computer scientist Frances “Betty” Holberton (1917-2001), is one of the tenants at Flexential Corp.’s data center in Collegeville. It’s run by the Penn Advanced Research Computing Center, its humming, flashing champion in the high-stakes academic struggle to meet fast-growing demand.

Penn expects Betty to land on the Top500.org list of the world’s most powerful supercomputers.

“Top universities’ faculty are competing for grants and for students. If they don’t have the resources to compete, they won’t be in good shape,” said Jaime Combariza, a computer scientist and systems manager Penn recruited from Johns Hopkins to run PARCC.

“So all universities are providing these centralized research groups. The idea is to keep people happy, so they use the facility and will succeed,” Combariza said.

During two years of meetings among deans and researchers from the university’s 12 colleges, the plan for Betty was approved quickly and built in less than a year — record time for Penn decision-making, according to Michael Borda, Penn’s associate vice provost for research.

“No question, there’s an arms race in computing,” Borda said. “We were not offering enough capacity.”

Betty quadruples the university’s computing capacity, with plenty of room to grow, Combariza said. Today’s denser, faster processor racks not only handle bigger, more complex queries but are much more efficient than machines built just a few years ago.

On July 11, info tech leaders from most of Penn’s 12 colleges walked through prison-style “man trap” doors and into the glass enclosure to visit Betty, for the first time.

The visitors inserted earplugs and filed into the space between Betty’s twin man-high server banks, scouting for the machines that handle their research queries. They shared discoveries like first-year students visiting their new dorm.

Penn scientists and their grad students “are doing a lot of research with large-language [AI] models and processing large images with large file sizes, so this is another computational capability we provide,” said Meredith Fetters, associate chief financial officer of research information systems at Penn’s Perelman School of Medicine.

“We’re going from looking at single images to looking at entire videos — hundreds of images [for each search] per minute. You’re scaling up the computations,” said Kenneth Charney, associate director for AI and Technology for PARCC. AI is “used for language, video, and audio. The more, the better.”

The three-story building housing Betty “colocates” corporate tenants, including GlaxoSmithKline, which constructed the building to support its neighboring labs but outsourced the site and its rapid hardware, software, and connectivity updates to Flexential in 2017.

So why is Betty so far from Penn’s campus?

“The hard thing in Philly is power,” as in electrical capacity, Charney said. It’s hard to get an extra megawatt from Peco, the city’s dominant power supplier. To have a data center in the city, Penn would have to build its own power plant, and suitable sites are scarce.

“This will grow. We’ll need that next megawatt” and more, Charney said.

The third floor of the building, where Betty sits, has a view of the cooling towers of the Limerick nuclear power plant, whose capacity owner Constellation Energy wants to increase by 300 megawatts, or about 15% — enough to power 200,000 homes, or 200 Betty-size centers.

Pennsylvania will need more electricity to support larger AI-serving data centers planned in places including the Susquehanna Valley and at the former U.S. Steel Fairless Works in Bucks County. A proposal next to PBF’s oil refinery in Delaware City, Del., would require more power than is currently generated to serve the state’s nearly 1 million residents.

Flexential’s Collegeville center is fed by two separate Peco substations at high voltage. But even short interruptions can crash AI applications and disrupt users for days or longer, so the first floor holds eight electric power generators. Fuel stored on site can keep the center running for several days in the case of an extended public power outage, said Patrick Doherty, Flexential’s chief revenue officer.

Flexential runs 40 data centers around the United States. National competitors include Digital Realty and NTT.

Access to electric power is important, along with strong fiber communications networks, land for expansion, and population, “where the users are. Philadelphia checks all the boxes,” Doherty said.

Data centers “are at historically low vacancy rates nationwide,” he said. “There is simply a lack of capacity and a need for endless computing power. AI is driving this.”

Research universities, with hundreds of professors, acting like small businesses with their own peculiar needs, are accelerating as “fast adapters of AI technologies,” Doherty said. Outsourcing “allows them to free up real estate, deploy faster, and apply for more grants,” and he sees Penn’s Betty as a model.

To demonstrate PARCC’s capacity, Charney runs a video of a quadruped robot being taught to walk a yoga ball in simulation, and then demonstrate what it has learned by crossing the intersection on a real-world yoga ball, at the busy intersection of 34th and Walnut Streets on Penn’s campus, in a project set up by Penn computer scientists Dinesh Jayaraman and Osbert Bastani.

“They are asking ChatGPT to tell if the robot is doing a good job or not,” Charney said. “Before we would have had to go to [an outside] data center or to Nvidia to schedule time on their systems,” which are often “fully booked. But with Betty, we can do this research completely at Penn.”

A Betty project set up by Marylyn D. Ritchie, vice dean for AI and data science, and her team at the medical school’s Institute for Biomedical Informatics is a genome study sequencing and rechecking the genes of 60,000 Americans — requiring more than 100 gigabytes per person at some stages. The project resembles the United Kingdom’s Hundred Thousand Genomes Project, which has provided unprecedented data for communities in Britain.

It’s not just professors and their graduate students who will rely on Betty, Charney said. Today’s undergraduates ask AI programs to write software code for them. “They can make a full-stack Web deployment, back end, front end, user interface. It used to be you couldn’t do something like that until several years into your job,” Chaney said.

“We always want to be looking forward,” he said. “Everyone is now employing these GPUs. Soon we want to be able to deploy quantum computing. Neumorphic processes. Maybe in five years.”