Collection of Java labs for distributed data processing with Apache Spark, Dockerized runtimes and analytical data workloads. The repository contains independent tasks covering batch analytics, text processing, sports analytics and Apache Iceberg experiments.
| Area | Tools |
|---|---|
| Language | Java |
| Data processing | Apache Spark, Spark SQL |
| Storage/table format | Apache Iceberg |
| Build | Maven |
| Runtime | Docker, Docker Compose |
PDD/
├── lab1/ # Spark job packaged as a Dockerized Java app
├── lab2/ # Retail analytics with Spark SQL
├── lab3/ # Word statistics and text processing
├── lab4/ # Player efficiency analytics
├── lab5/ # NBA salary analytics with Apache Iceberg
├── DSCA/ # Additional Spark SQL task
└── DSCA_4/ # Additional distributed systems coursework files
- Building Java applications for Apache Spark.
- Packaging Spark jobs with Maven Shade Plugin.
- Running isolated data processing tasks in Docker.
- Working with structured analytics datasets.
- Using Apache Iceberg for analytical table experiments.
Each lab is self-contained and usually includes its own pom.xml, Dockerfile and docker-compose.yml.
cd lab5
mvn -DskipTests clean package
docker compose up --buildUse the corresponding lab directory as the working directory when running commands.
This repository is useful for demonstrating practical experience with Spark-based data processing, Java build tooling and containerized analytical workloads.