GPU Systems Lab

A collection of focused projects exploring the foundations of GPU-oriented systems programming.

The repository begins with native interoperability, continues through explicit Vulkan compute, and gradually evolves toward reusable GPU systems and numerical compute abstractions.

The goal is not to build a graphics engine or a machine-learning framework.

Instead, each project introduces one important concept required for understanding modern GPU software stacks:

• native interoperability
• explicit Vulkan resource ownership
• GPU memory management
• compute pipelines and synchronization
• compute graphs and dependency scheduling
• reusable execution APIs
• parallel numerical algorithms
• CPU/GPU backend abstraction
• heterogeneous compute infrastructure
• GPU-oriented systems design

---

🎯 Repository Goals

The lab is designed to answer the following questions:

• How does Rust interact with native C and C++ libraries?
• How is a Vulkan compute device initialized?
• How are buffers and GPU memory managed explicitly?
• How are compute shaders dispatched and synchronized?
• How can Vulkan resources be modeled safely with Rust ownership?
• How can low-level GPU components be composed into reusable APIs?
• How can the same program run locally and on ARM64 GPU hardware?
• How can GPU workloads be submitted through an HPC scheduler?
• What abstractions are built on top of low-level GPU APIs?
• How can compute operations be represented as a dependency graph?
• How can GPU buffers and memory allocations be reused efficiently?
• How do parallel reductions scale beyond a single workgroup?
• How can numerical operations share CPU and GPU execution interfaces?
• How can these pieces be composed into a small compute framework?

---

🧭 Learning Path

The repository is organized into phases.

Each phase builds on the previous one.

Native interoperability
        |
        V
Vulkan foundations
        |
        V
Reusable GPU execution
        |
        V
GPU systems
        |
        V
Numerical and HPC-oriented compute

---

Phase I --- GPU Foundations

The first phase develops the low-level foundation required to understand and control GPU workloads directly.

Project	Topic	Status
01	Rust <-> C FFI	✅
02	Rust <-> C++ FFI	✅
03	Vulkan Instance & Device	✅
04	Vulkan Buffer & First Compute Shader	✅
05	Compute Resources & Synchronization	✅
06	GPU Compute Pipeline	✅

---

Project 01 --- Rust <-> C FFI

Introduces:

• C ABI
• extern "C"
• native linking
• ownership across language boundaries
• unsafe Rust interfaces

---

Project 02 --- Rust <-> C++ FFI

Introduces:

• C-compatible wrappers around C++
• opaque native objects
• constructor and destructor boundaries
• exception-safe FFI design
• Rust ownership of C++ resources

---

Project 03 --- Vulkan Instance & Device

Introduces:

• Vulkan instance creation
• physical device enumeration
• queue family discovery
• logical device creation
• compute-capable hardware inspection

---

Project 04 --- Vulkan Buffer & First Compute Shader

Introduces:

• Vulkan buffers
• device memory
• descriptor sets
• compute pipelines
• command buffers
• shader dispatch
• first GPU result readback

The compute shader performs:

value = value * 2

---

Project 05 --- Compute Resources & Synchronization

Transforms the procedural Vulkan example into reusable Rust components.

Introduces:

• RAII
• resource ownership
• Drop
• reusable buffer abstractions
• reusable pipeline abstractions
• command execution encapsulation
• fence-based synchronization

---

Project 06 --- GPU Compute Pipeline

Combines the reusable components into a high-level API:

let pipeline = GpuComputePipeline::new()?;
let output = pipeline.execute(&input)?;

Introduces:

• high-level GPU execution interfaces
• component composition
• separation of public API and implementation details
• long-lived and per-execution resources
• complete workload lifecycle management
• validation through Slurm on Jetson Orin Nano

Project 06 completes the GPU foundations phase.

---

Phase II --- GPU Systems

The second phase moves beyond learning Vulkan API mechanics.

The goal is to explore the kinds of systems and abstractions that are built on top of explicit GPU execution.

Project	Topic	Status
07	Compute Graph Execution	✅
08	GPU Memory Management	✅
09	Numerical GPU Algorithms	✅
10	Mini Compute Framework	✅

---

Project 07 --- Compute Graph Execution

Introduces a DAG-based execution model for composing multiple compute operations.

Introduces:

• compute graph nodes and edges
• data dependencies
• Kahn-style topological ordering
• graph validation
• branching graphs
• runtime-selected CPU/GPU execution
• JSON graph definitions

---

Project 08 --- GPU Memory Management

Builds reusable memory management on top of the Vulkan execution layer.

Introduces:

• reusable buffer pools
• capacity-based buffer reuse
• pool eviction
• Vulkan memory suballocation
• alignment-aware allocation
• reusable freed regions
• free-region merging
• device-local storage
• upload and readback staging buffers
• memory statistics and lifetime tracking

---

Project 09 --- Numerical GPU Algorithms

Implements general numerical operations using Vulkan compute.

Algorithms include:

• vector addition
• scalar multiplication
• SAXPY
• sum reduction
• max reduction
• dot product
• L2 normalization
• Jacobi iteration

Introduces:

• parallel decomposition
• workgroup-local synchronization
• hierarchical multi-workgroup reductions
• floating-point numerical validation
• CPU/GPU comparison
• basic performance measurement

---

Project 10 --- Mini Compute Framework

Combines the previous projects into a small backend-independent compute framework.

The framework supports:

• compute graphs
• graph validation
• topological scheduling
• runtime Vector, Scalar, and Matrix values
• a shared OperationExecutor abstraction
• CPU and Vulkan GPU backends
• reusable numerical kernels
• GPU memory pooling and suballocation
• JSON graph definitions
• CLI backend selection
• structured JSON results
• execution metadata
• local and Slurm execution

---

🖥 Infrastructure Track

The repository includes infrastructure for validating selected projects on real ARM64 GPU hardware.

Cluster topology:

                    Laptop
            Rust development
            ARM64 cross compilation
                      |
                      V
              Raspberry Pi 4
              Slurm controller
                      |
                      V
             Jetson Orin Nano
             Slurm GPU worker
                      |
                      V
              Vulkan execution

Infrastructure components:

• Ansible
• Slurm
• Munge
• NFS
• ARM64 cross compilation
• Vulkan
• NVIDIA GPU access
• Linux identity synchronization

The infrastructure is documented under:

infra/
|-- README.md
|-- ansible/
|-- slurm/
|-- docs/
    |-- architecture.md
    |-- cluster_bootstrap.md
    |-- troubleshooting.md

---

🔧 Cross Compilation

Development happens on an x86_64 laptop.

Selected projects are compiled for:

aarch64-unknown-linux-gnu

Install the required toolchain:

sudo apt update
sudo apt install gcc-aarch64-linux-gnu
rustup target add aarch64-unknown-linux-gnu

Example:

cargo build   --release   --target aarch64-unknown-linux-gnu   -p gpu_compute_pipeline

The ARM64 executable can then be copied to the Jetson or deployed to shared NFS storage.

---

🚀 Hardware and Slurm Validation

Selected GPU projects are validated through the complete heterogeneous workflow:

Rust source
    |
    V
x86_64 laptop
    |
    | ARM64 cross compilation
    V
shared NFS storage
    |
    V
Raspberry Pi Slurm controller
    |
    | GPU job submission
    V
Jetson Orin Nano worker
    |
    | Vulkan compute dispatch
    V
real GPU output

This validates more than the shader itself.

It also validates:

• cross compilation
• binary portability
• shared storage
• scheduling
• resource allocation
• process identity
• GPU device permissions
• service recovery
• reproducible infrastructure

---

🧠 Design Principles

The lab follows several principles:

• build incrementally
• keep abstractions understandable
• expose low-level behavior before hiding it
• give every native resource one clear owner
• use RAII for cleanup
• separate long-lived and temporary resources
• verify assumptions on real hardware
• treat debugging as part of learning
• document infrastructure problems and their root causes
• prefer small, complete projects over one oversized codebase

---

📚 What This Repository Demonstrates

The repository demonstrates:

• Rust interoperability with C and C++
• explicit Vulkan initialization and device selection
• Vulkan buffers and memory allocation
• descriptor sets and compute pipelines
• command recording, submission, and synchronization
• RAII ownership of Vulkan resources
• reusable GPU execution abstractions
• compute graph construction and validation
• dependency-aware topological execution
• CPU and GPU execution backends
• reusable GPU buffer pools
• Vulkan memory suballocation
• alignment-aware memory management
• device-local GPU memory with staging transfers
• workgroup synchronization
• single- and multi-workgroup reductions
• numerical GPU algorithms
• floating-point CPU/GPU validation
• JSON-defined compute graphs
• a CLI-driven mini compute framework
• x86_64 to ARM64 cross compilation
• Slurm GPU job execution on NVIDIA Jetson
• NFS-based artifact sharing
• Ansible-managed heterogeneous infrastructure
• Linux GPU permission debugging

---

🗺 Completed Learning Path

Native interoperability
        |
        V
Vulkan foundations
        |
        V
Reusable GPU execution
        |
        V
Compute graphs
        |
        V
GPU memory management
        |
        V
Numerical GPU algorithms
        |
        V
Mini compute framework
        |
        V
Jetson + Slurm validation

---

# 📌 Summary

GPU Systems Lab is a practical learning journey from native systems programming to a small heterogeneous compute framework.

The completed progression is:

```text
Rust/C/C++ FFI
    ->
Vulkan device and memory fundamentals
    ->
compute shaders and synchronization
    ->
reusable GPU execution
    ->
compute graphs
    ->
GPU memory pooling and suballocation
    ->
parallel numerical algorithms
    ->
CPU/GPU compute framework
    ->
Jetson execution through Slurm