Prerequisites and Download
Flink requires a Java runtime, specifically Java 11 or Java 17 for recent Flink versions (Flink 1.18+), since the JVM hosts both the JobManager and TaskManager processes. You download a binary distribution from the Apache Flink website or via a package manager, choosing the Scala-free distribution unless you rely on the legacy Scala DataStream API. After extracting the tarball, the resulting directory contains bin/ for startup scripts, conf/ for configuration files like flink-conf.yaml, and lib/ where you drop connector JARs such as flink-connector-kafka.
Cricket analogy: Downloading the right Flink distribution is like a franchise picking the correct kit and equipment set before a season, such as making sure the bowling machine and pitch curator tools match the ground's specifications.
Starting a Standalone Cluster
Running ./bin/start-cluster.sh launches a local standalone cluster: one JobManager and, by default, one TaskManager, both as background JVM processes on your machine. The Web UI becomes available at http://localhost:8081, showing running and completed jobs, TaskManager resource usage, and checkpoint history. You can add more TaskManagers by running ./bin/taskmanager.sh start again, which is useful for testing parallelism locally before deploying to a real cluster on Kubernetes or YARN.
Cricket analogy: Starting a standalone cluster is like setting up nets for a solo practice session, one bowler and one batsman, before scaling up to a full squad training day.
Submitting a Job
Once a cluster is running, you submit a compiled job JAR using ./bin/flink run -c com.example.MyJob path/to/job.jar, optionally passing --parallelism to override the default and program arguments after the JAR path. The CLI immediately reports the JobID, and you can track status with ./bin/flink list, request a savepoint with ./bin/flink savepoint <jobId> <targetDir>, or cancel the job gracefully with ./bin/flink cancel <jobId>. The Web UI mirrors all of this visually, letting you drag-and-drop a JAR, inspect the operator DAG, and view backpressure metrics per task.
Cricket analogy: Submitting a job via the CLI is like a captain handing the team sheet to the umpire before the toss, formally registering exactly who plays and in what order.
# Start a local standalone cluster
./bin/start-cluster.sh
# Submit a job JAR with a custom parallelism
./bin/flink run -c com.example.WordCountJob --parallelism 4 ./examples/wordcount.jar --input /data/input.txt
# List running jobs
./bin/flink list
# Take a savepoint before making changes, then cancel the job
./bin/flink savepoint <jobId> file:///tmp/flink-savepoints
./bin/flink cancel <jobId>
# Stop the local cluster
./bin/stop-cluster.shThe Flink Web UI at http://localhost:8081 lets you inspect the JobGraph visually, drill into individual operators' subtask metrics, and check backpressure indicators, which is often the fastest way to diagnose a slow pipeline during local development.
Always take a savepoint (./bin/flink savepoint) before cancelling a job you intend to restart with upgraded code, otherwise you lose the ability to resume exactly where the job left off with consistent state.
- Flink requires Java 11 or 17 depending on version, and downloads as a binary tarball with bin/, conf/, and lib/ directories.
- ./bin/start-cluster.sh starts a local standalone cluster with one JobManager and one TaskManager by default.
- The Web UI at localhost:8081 shows jobs, TaskManager resources, and checkpoint history.
- Jobs are submitted via ./bin/flink run with the main class and JAR path.
- ./bin/flink list, savepoint, and cancel manage running jobs from the CLI.
- Savepoints should be taken before cancelling a job you plan to upgrade and restart.
- Additional TaskManagers can be started manually to test parallelism locally.
Practice what you learned
1. What Java versions are required for recent Flink releases (1.18+)?
2. Which script starts a local standalone Flink cluster?
3. What port does the Flink Web UI listen on by default?
4. What should you do before cancelling a running job that you plan to upgrade and restart?
5. Which CLI command submits a compiled job JAR to a running cluster?
Was this page helpful?
You May Also Like
What Is Apache Flink?
An introduction to Apache Flink, the open-source distributed engine for stateful computations over unbounded and bounded data streams.
Flink Architecture
A tour of Flink's distributed architecture: JobManager, TaskManagers, slots, and how a job is deployed and coordinated across a cluster.
Your First Flink Job
A hands-on walkthrough of writing, running, and understanding a simple Flink DataStream job that reads events, transforms them, and writes results.
Related Reading
Related Study Notes in Programming
Browse all study notesApache Spark Study Notes
Programming · 30 topics
ProgrammingHadoop Study Notes
Programming · 30 topics
ProgrammingSnowflake Study Notes
Programming · 30 topics
ProgrammingApache Airflow Study Notes
Programming · 30 topics
Programmingdbt (Data Build Tool) Study Notes
Programming · 30 topics
ProgrammingPowerShell Study Notes
Programming · 30 topics