Big Data & Distributed Computing
Apache Spark, Hadoop ecosystem, distributed query engines, streaming with Kafka and large-scale data processing patterns
100% Free · All lessons unlocked
Course Content
Foundations
Core concepts and groundwork
CAP Theorem, Consistency and Availability
Reading
Batch vs Stream Processing Trade-offs
Reading
MapReduce Programming Model
Reading
HDFS Architecture and Data Locality
Reading
YARN Resource Management
Reading
Practice — MapReduce Word Count on HDFS
Exercise
Core Skills
Essential techniques and patterns
Spark Architecture — Driver, Executor, RDD
Reading
DataFrame API and Spark SQL
Reading
Transformations vs Actions and Lazy Evaluation
Reading
Reading and Writing Parquet, ORC and Delta
Reading
Partitioning, Shuffling and Skew Handling
Reading
Practice — Spark Batch Job on Clickstream Data
Exercise
Applied Practice
Hands-on, real-world scenarios
PySpark DataFrames — Joins, Windows and Aggregations
Reading
User-Defined Functions (UDFs) and Pandas UDFs
Reading
Spark MLlib for Pipeline Preprocessing
Reading
Spark UI and Performance Tuning
Reading
Delta Lake — ACID Transactions on Data Lake
Reading
Practice — PySpark Feature Engineering Pipeline
Exercise
Advanced Topics
Deeper, more complex material
Kafka Architecture — Brokers, Topics, Partitions
Reading
Producers and Consumers in Python
Reading
Consumer Groups and Offset Management
Reading
Schema Registry and Avro Serialisation
Reading
Kafka Connect for Source and Sink Connectors
Reading
Practice — Build a Kafka Producer-Consumer Pipeline
Exercise
Production & Scale
Building for the real world
Spark Structured Streaming Fundamentals
Reading
Watermarks and Late-Arriving Data
Reading
Apache Flink Overview and Stateful Processing
Reading
Windowing — Tumbling, Sliding and Session
Reading
Exactly-Once Delivery in Streaming Pipelines
Reading
Practice — Real-Time Anomaly Detection Pipeline
Exercise
Mastery & Capstone
Projects and final review
100% Free · All lessons unlocked
Topic Overview
What you'll learn
- Core concepts and fundamentals of Big Data & Distributed Computing
- Industry best practices and design patterns
- Hands-on exercises with real-world scenarios
- Performance optimization and advanced techniques
- Building production-ready applications