MPI Basics in Fortran

Analogy categoryAnalogies

Introduction to MPI in Fortran

The Message Passing Interface (MPI) is a library standard for distributed-memory parallel programming, where a program runs as a set of independent processes — each with its own private memory — that communicate exclusively by sending and receiving explicit messages. In modern Fortran, MPI is accessed through the use mpi_f08 module (the type-safe, recommended binding since MPI-3), which provides derived types such as MPI_Comm and MPI_Status instead of the older raw-integer use mpi interface, catching many argument-mismatch bugs at compile time that used to only surface at run time.

🏏

Cricket analogy: Like international teams that never share a dressing room and can only exchange information by physically sending a courier with a written note between grounds, unlike the single shared dressing room of OpenMP's threads.

Initialization, Rank, and Size

Every MPI program begins with a call to mpi_init and ends with mpi_finalize, bracketing the section of code that runs in parallel. Immediately after initialization, each process typically calls mpi_comm_rank(MPI_COMM_WORLD, rank, ierr) to discover its own unique identifier within the default communicator (numbered 0 through size-1) and mpi_comm_size(MPI_COMM_WORLD, nprocs, ierr) to discover how many processes total are participating; these two values, rank and nprocs, are almost always used to decide which slice of data or work a given process is responsible for.

🏏

Cricket analogy: Like every fielder being assigned a fixed jersey number (rank) at the start of a match and told the total squad size, then using that number to know which fielding position they're responsible for covering.

fortran

program mpi_hello
  use mpi_f08
  implicit none
  integer :: rank, nprocs, ierr

  call mpi_init(ierr)
  call mpi_comm_rank(MPI_COMM_WORLD, rank, ierr)
  call mpi_comm_size(MPI_COMM_WORLD, nprocs, ierr)

  print '(A,I0,A,I0)', 'Hello from rank ', rank, ' of ', nprocs

  call mpi_finalize(ierr)
end program mpi_hello

Point-to-Point Communication: Send and Recv

The most fundamental MPI operations are mpi_send and mpi_recv, which transfer a typed buffer of data from one specific rank to another specific rank. A call like call mpi_send(data, count, MPI_DOUBLE_PRECISION, dest, tag, MPI_COMM_WORLD, ierr) must be matched on the receiving rank by a corresponding mpi_recv naming the same source, tag, and communicator, and a common beginner mistake is a mismatched send/recv pair — wrong count, wrong datatype, or a rank that calls mpi_recv but whose matching sender never actually calls mpi_send — which causes the program to hang indefinitely rather than crash.

🏏

Cricket analogy: Like a fielder throwing the ball directly to a named teammate at a specific base; if that teammate isn't watching for it (never calls the matching catch), the throw effectively goes nowhere and the play stalls.

A mismatched or missing mpi_recv for a given mpi_send is one of the most common causes of an MPI program hanging forever with no error message. Because mpi_send can block until a matching receive is posted (depending on message size and MPI implementation buffering), always double-check that every send has a correctly matched receive with the same source/dest, tag, and datatype before assuming a slow-running job is just doing heavy computation.

Collective Operations: Broadcast and Reduce

Beyond point-to-point messages, MPI provides collective operations that involve every process in a communicator at once. mpi_bcast(data, count, datatype, root, comm, ierr) sends a copy of data from the root rank to every other rank in a single call, which is far simpler and typically faster than a manual loop of individual sends. mpi_reduce(sendbuf, recvbuf, count, datatype, MPI_SUM, root, comm, ierr) combines a value contributed by every rank (using an operation like MPI_SUM, MPI_MAX, or MPI_MIN) into a single result delivered to the root rank, while mpi_allreduce delivers that combined result to every rank instead of just one.

🏏

Cricket analogy: Like a team manager announcing the day's fielding plan to the whole squad at once over the intercom (broadcast), versus each fielder walking over individually to be told separately, and a coach adding up every fielder's private catch tally into one team total (reduce).

MPI collectives like mpi_bcast, mpi_reduce, and mpi_allreduce are implemented by the MPI library using optimized tree or ring algorithms internally, so they nearly always outperform an equivalent hand-written loop of mpi_send/mpi_recv calls, especially as the number of processes grows into the hundreds or thousands.

MPI processes have separate, private memory and communicate only by explicitly sending and receiving messages.
Every MPI program brackets its parallel section with mpi_init and mpi_finalize.
mpi_comm_rank and mpi_comm_size give each process its unique ID and the total process count, driving work division.
mpi_send and mpi_recv must be matched by source/dest, tag, and datatype, or the program can hang indefinitely.
mpi_bcast distributes data from one root process to all others in a single call.
mpi_reduce combines a per-process value into one result at a root rank; mpi_allreduce delivers that result to every rank.
The mpi_f08 module provides type-safe derived types (MPI_Comm, MPI_Status) and is the recommended modern Fortran binding.