High-Performance Computing (HPC)

Definition

High-Performance Computing (HPC) refers to the use of aggregated computing power — typically supercomputers or clusters of tightly interconnected servers — to solve large, complex computational problems far faster than a standard computer could.

Overview

HPC systems are built to deliver massive computational throughput by combining thousands of processor cores, often paired with GPUs or other accelerators, connected via extremely low-latency, high-bandwidth networks. Unlike loosely coupled Grid Computing, an HPC cluster is usually housed in a single facility and engineered so nodes can exchange data with minimal delay, which is essential for problems where different parts of the computation depend heavily on one another. HPC workloads rely heavily on Parallel Computing techniques, splitting a problem into pieces that run simultaneously across many nodes and cores, coordinated through specialized libraries such as MPI (Message Passing Interface) for inter-node communication and OpenMP for shared-memory parallelism within a node. Performance is commonly benchmarked in FLOPS (floating-point operations per second), and the world's fastest systems — tracked on the TOP500 list — now operate at the exascale (over one quintillion operations per second). Historically associated with government research labs and universities running weather models, nuclear simulations, and physics research, HPC has become central to modern AI as well: training large language models and other deep learning systems demands the same class of tightly interconnected, GPU-dense infrastructure that traditional HPC pioneered, blurring the line between classic supercomputing and AI infrastructure. It is often mentioned alongside GPU Computing in this space.

Key Concepts

• Aggregates thousands of CPU and GPU cores in a tightly interconnected cluster
• Ultra-low-latency, high-bandwidth networking between nodes (e.g. InfiniBand)
• Performance measured in FLOPS, benchmarked publicly via lists like TOP500
• Specialized parallel programming libraries such as MPI and OpenMP
• Shared, high-throughput parallel file systems for large dataset access
• Job scheduling systems that queue and allocate workloads across the cluster
• Purpose-built cooling and power infrastructure for dense compute racks
• Increasingly used to train and run large-scale AI models

Use Cases

Frequently Asked Questions

Related in Computer Science