How Supercomputing Will Evolve, According to Jack Dongarra

High-performance supercomputing—once the exclusive domain of scientific research—is now a strategic resource for training increasingly complex artificial intelligence models. This convergence of AI and HPC is redefining not only these technologies, but also the ways in which knowledge is produced, and takes a strategic position in the global landscape.

To discuss how HPC is evolving, in July WIRED caught up with Jack Dongarra, a US computer scientist who has been a key contributor to the development of HPC software over the past four decades—so much so that in 2021 he earned the prestigious Turing Award. The meeting took place at the 74th Nobel Laureate Meeting in Lindau, Germany, which brought together dozens of Nobel laureates as well as more than 600 emerging scientists from around the world.

This interview has been edited for length and clarity.

WIRED: What will be the role of artificial intelligence and quantum computing in scientific and technological development in the coming years?

Jack Dongarra: I would say AI is already playing an important role in how science is done: We’re using AI in many ways to help with scientific discovery. It’s being used in terms of computing and helping us to approximate how things behave. So I think of AI as a way to get an approximation, and then maybe refine the approximation with the traditional techniques.

Today we have traditional techniques for modeling and simulation, and those are run on computers. If you have a very demanding problem, then you would turn to a supercomputer to understand how to compute the solution. AI is going to make that faster, better, more efficient.

AI is also going to have an impact beyond science—it’s going to be more important than the internet was when it arrived. It’s going to be so pervasive in what we do. It’s going to be used in so many ways that we haven’t really discovered today. It’s going to serve more of a purpose than the internet has played in the past 15, 20 years.

Quantum computing is interesting. It’s really a wonderful area for research, but my feeling is we have a long way to go. Today we have examples of quantum computers—hardware always arrives before software—but those examples are very primitive. With a digital computer, we think of doing a computation and getting an answer. The quantum computer is instead going to give us a probability distribution of where the answer is, and you’re going to make a number of, we’ll call it runs on the quantum computer, and it’ll give you a number of potential solutions to the problem, but it’s not going to give you the answer. So it’s going to be different.

With quantum computing, are we caught in a moment of hype?

I think unfortunately it’s been oversold—there’s too much hype associated with quantum. The result of that typically is that people will get all excited about it, and then it doesn’t live up to any of the promises that were made, and then the excitement will collapse.

We’ve seen this before: AI has gone through that cycle and has recovered. And now today AI is a real thing. People use it, it’s productive, and it’s going to serve a purpose for all of us in a very substantial way. I think quantum has to go through that winter, where people will be discouraged by it, they’ll ignore it, and then there’ll be some bright people who figure out how to use it and how to make it so that it is more competitive with traditional things.

There are many issues that have to be worked out. Quantum computers are very easy to disturb. They’re going to have a lot of “faults”—they will break down because of the nature of how fragile the computation is. Until we can make things more resistant to those failures, it’s not going to do quite the job that we hope that it can do. I don’t think we’ll ever have a laptop that’s a quantum laptop. I may be wrong, but certainly I don’t think it’ll happen in my lifetime.

Quantum computers also need quantum algorithms, and today we have very few algorithms that can effectively be run on a quantum computer. So quantum computing is at its infancy, and along with that the infrastructure that will use the quantum computer. So quantum algorithms, quantum software, the techniques that we have, all of those are very primitive.

When can we expect—if ever—the transition from traditional to quantum systems?

So today we have many supercomputing centers around the world, and they have very powerful computers. Those are digital computers. Sometimes the digital computer gets augmented with something to enhance performance—an accelerator. Today those accelerators are GPUs, graphics processing units. The GPU does something very well, and it just does that thing well, it’s been architected to do that. In the old days, that was important for graphics; today we’re refactoring that so that we can use a GPU to satisfy some of the computational needs that we have.

In the future, I think that we will augment the CPU and the GPU with other devices. Perhaps quantum would be another device that we would add to that. Maybe it would be neuromorphic—computing that sort of imitates how our brain works. And then we have optical computers. So think of shining light and having that light interfere, and the interference basically is the computation you want it to do. Think of an optical computer that takes two beams of light, and in the light is encoded numbers, and when they interact in this computing device, it produces an output, which is the multiplication of those numbers. And that happens at the speed of light. So that’s incredibly fast. So that’s a device that perhaps could fit into this CPU, GPU, quantum, neuromorphic computer device. Those are all things that perhaps could combine.

How is the current geopolitical competition—between China, the United States, and beyond—affecting the development and sharing of technology?

The US is restricting computing at a certain level from going to China. Certain parts from Nvidia are no longer allowed to be sold there, for example. But they’re sold to areas around China, and when I go visit Chinese colleagues and look at what they have in their their computers, they have a lot of Nvidia stuff. So there’s an unofficial pathway.

At the same time, China has pivoted from buying Western technology to investing in its own technology, putting more funding into the research necessary to advance it. Perhaps this restriction that’s been imposed has backfired by causing China to accelerate the development of parts that they can control very much more than they could otherwise.

The Chinese have also decided that information about their supercomputers should not be advertised. We do know about them—what they look like, and what their potential is, and what they’ve done—but there’s no metric that allows us to benchmark and compare in a very controlled way how those computers compare against the machines that we have. They have very powerful machines that are probably equal to the power of the most significant machines that we have in the US.

They’re built on technology that was invented or designed in China. They’ve designed their own chips. They compete with the chips that we have in the computers that are in the West. And the question that people ask is: Where were the chips fabricated? Most chips used in the West are fabricated by the Taiwan Semiconductor Manufacturing Company. China has technology, which is a generation or two behind the technology that TSMC has, but they’re going to catch up.

My guess is that some of the Chinese chips are also fabricated in Taiwan. When I ask my Chinese friends “Where were your chips manufactured?” they say China. And if I push them and say “Well, were they manufactured in Taiwan?” the answer to that comes back eventually is Taiwan is part of China.

How will the role of programmers and developers change as AI evolves? Will we get to write software using only natural language?

AI has a very important role I think in helping to take away some of the time-consuming parts of developing programs. It’s gotten all the information about everybody else’s programs that’s available and then it synthesizes that and then can push that forward. I’ve been very impressed when I have asked some of these systems to write a piece of software to do a certain task; the AI does a pretty good job. And then I can refine that with another prompt, saying “Optimize this for this kind of computer,” and it does a pretty good job of that. In the future, I think more and more we will be using language to describe a story to AI, and then have it write a program to carry out that function.

Now of course, there are limits—and we have to be careful about hallucinations or something giving us the wrong results. But maybe we can build in some checks to verify the solutions that AI produces and we can use that as a way of measuring the potential accuracy of that solution. We should be aware of the potential problems, but I think we have to move ahead in this front.

This story originally appeared on WIRED Italia and has been translated from Italian.