scripts updated as per LUMI container release and Roihu infrastructure

AWQ/README.md

@@ -6,33 +6,51 @@ In order to target weight and activation scaling locations within the model, the
 
 ## Installations
 
+## LUMI
+
+To run AWQ quantization scripts on LUMI, you can use the `gptqmodel` library that already ships inside the LUMI AI Factory container — no extra packages or virtual environment are required for AWQ. `torch` , `transformers`, and `datasets` are also already provided in the container.
+
+The script loads the `Singularity` container environment and sets the container image path:
+
+```bash
+module purge
+module use /appl/local/laifs/modules
+module load lumi-aif-singularity-bindings
+
+export SIF=/appl/local/laifs/containers/lumi-multitorch-u24r70f21m50t210-20260731_122833/lumi-multitorch-full-u24r70f21m50t210-20260731_122833.sif
+
+```
+
+---
+
+## Roihu
+
 The CSC preinstalled PyTorch module covers most of the libraries needed to run these examples
-(torch, transformers, datasets, accelerate). The rest can be installed on top of the module in a virtual environment.
+(torch, transformers, datasets, accelerate). Llmcompressor can be installed on top of the module in a virtual environment.
 
 ### Load the module
 ```bash
 module purge
-module use /appl/local/csc/modulefiles
-module load pytorch/2.7
+module load python-pytorch/2.10
 ```
 ### Create and activate a virtual environment using system packages
 ```bash
 python3 -m venv --system-site-packages venv
 source venv/bin/activate
 ```
 ### Install packages
+
 ```bash
-pip install optimum==1.27.0 llmcompressor==0.7.1 --cache-dir ./.pip-cache
-```
-The flag --cache-dir points the pip cache to the current (scratch) folder instead of the default (home directory), to avoid filling up home directory quota. 
+(venv)> pip install llmcompressor==0.12.0 --cache-dir ./.pip-cache
+``` 
+---
 
 ## Usage
 
 The launch scripts are: 
 
-- `run-awq-modifier-lumi.sh` - quantizes model on LUMI with 1 GPU 
-- `run-awq-modifier-mahti.sh` - quantizes model on Mahti with 1 GPU
-- `run-awq-modifier-puhti.sh` - quantizes model on Puhti with 1 GPU
+- `run-awq-modifier-lumi.sh` - quantizes model on LUMI with 1 GPU
+- `run-awq-modifier-roihu.sh` - quantizes model on Roihu with 1 GPU
 
 **Note:** the scripts are made to be run on `gputest` or `dev-g` partition with a 30 minutes time-limit. You have to select the proper partition for longer jobs for your real runs. Additionally, change the `--account` parameter to your own project code. 
 
@@ -55,6 +73,6 @@ Meaning the script quantizes the model’s linear layers using a mixed-precision
 - Model size (MB) before and after quantization.
 
 ## Notes
-- The current scripts use **Falcon-RW-1B** for fast experimentation. You can replace `model_name` with a larger model. In this case, you might want to disable saving the full model.
+- The current scripts use **TinyLlama-1.1B-Chat-v1.0** for fast experimentation. You can replace `model_name` with a larger model. In this case, you might want to disable saving the full model.
 - For large models, `device_map="auto"` allows the model modules to be moved between the CPU and GPU for quantization.
 - Feel free to experiment with different values for `num_calibration_samples` and `max_seq_lenght` and to modify the quantization recipe.

AWQ/awq-modifier.py

@@ -11,7 +11,7 @@
 from llmcompressor.modifiers.awq import AWQModifier
 from llmcompressor.utils import dispatch_for_generation
 
-model_name = "tiiuae/falcon-rw-1b"
+model_name = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
 dataset_name = "HuggingFaceH4/ultrachat_200k"
 dataset_split = "train_sft"
 num_calibration_samples = 256

AWQ/run-awq-modifier-lumi.sh

@@ -11,15 +11,16 @@
 
 # Load the module
 module purge
-module use /appl/local/csc/modulefiles
-module load pytorch/2.7
+module use /appl/local/laifs/modules
+module load lumi-aif-singularity-bindings
 
-# Activate the virtual environment from your current directory or change to the appropriate path
-source venv/bin/activate
+# export path to used container image
+export SIF=/appl/local/laifs/containers/lumi-multitorch-u24r70f21m50t210-20260731_122833/lumi-multitorch-full-u24r70f21m50>
 
+# Activate the virtual environment from your current directory or change to th
 # This will store all the Hugging Face cache such as downloaded models
 # and datasets in the project's scratch folder
 export HF_HOME=/scratch/${SLURM_JOB_ACCOUNT}/${USER}/hf-cache
 mkdir -p $HF_HOME
 
-srun python3 awq-modifier.py
+srun singularity exec "$SIF" python3 awq-modifier.py

AWQ/run-awq-modifier-roihu.sh

@@ -0,0 +1,22 @@
+#!/bin/bash
+#SBATCH --account=project_xxxxxxx
+#SBATCH --partition=gputest
+#SBATCH --time=00:15:00
+#SBATCH --nodes=1
+#SBATCH --tasks-per-node=1
+#SBATCH --cpus-per-task=72
+#SBATCH --gres=gpu:gh200:1
+#SBATCH --mem=32G
+
+module purge
+module load python-pytorch/2.10
+
+# Activate the virtual environment from your current directory or change to the appropriate path
+source venv/bin/activate
+
+# Set hf cache to the project's scratch
+export HF_HOME=/scratch/$SLURM_JOB_ACCOUNT/$USER/hf-cache/
+mkdir -p $HF_HOME
+
+srun python3 awq-modifier.py
+

BitsAndBytes/README.md

@@ -4,15 +4,14 @@ This example demonstrates quantizing the **OPT-125M** model using the [bitsandby
 
 ## Running the script
 
-All of the libraries needed to run this example (transformers, bitsandbytes, accelerate) are covered by the CSC preinstalled PyTorch module.
+All of the libraries needed to run this example (transformers, bitsandbytes, accelerate) are covered by the AI Factory provided Container on LUMI or the CSC preinstalled PyTorch module on Roihu.
 
 The script `bnb-quantization.py` will quantize the OPT-125M model to nf4 or NormalFloat 4-bit, introduced to use with QLoRA technique, a parameter efficient fine-tuning technique. It can be used with QLoRA for fine-tuning, or without just for reducing model size. 
 
 The launch scripts are: 
 
-- `run-bnb-quantization-lumi.sh` - quantizes model on LUMI with 1 GPU 
-- `run-bnb-quantization-mahti.sh` - quantizes model on Mahti with 1 GPU
-- `run-bnb-quantization-puhti.sh` - quantizes model on Puhti with 1 GPU
+- `run-bnb-quantization-lumi.sh` - quantizes model on LUMI with 1 GPU
+- `run-bnb-quantization-roihu.sh` - quantizes model on Roihu with 1 GPU
 
 **Note:** the scripts are made to be run on `gputest` or `dev-g` partition with a 30 minutes time-limit. You have to select the proper partition for longer jobs for your real runs. Additionally, change the `--account` parameter to your own project code. 
 

BitsAndBytes/bnb-quantization.py

@@ -27,6 +27,7 @@ def benchmark(model, tokenizer, prompt):
             **inputs,
             max_new_tokens=50,
             do_sample=True,
+            use_cache=False,
             temperature=0.7,
 )
 

BitsAndBytes/run-bnb-quantization-lumi.sh

@@ -11,12 +11,16 @@
 
 # Load the module
 module purge
-module use /appl/local/csc/modulefiles
-module load pytorch/2.7
+module use /appl/local/laifs/modules
+module load lumi-aif-singularity-bindings
+
+# export path to used container image
+export SIF=/appl/local/laifs/containers/lumi-multitorch-u24r70f21m50t210-20260731_122833/lumi-multitorch-full-u24r70f21m50t210-20260731_122833.sif
 
 # This will store all the Hugging Face cache such as downloaded models
 # and datasets in the project's scratch folder
-export HF_HOME=/scratch/${SLURM_JOB_ACCOUNT}/${USER}/hf-cache
+export HF_HOME=/scratch/$SLURM_JOB_ACCOUNT/$USER/llm-quantization-scripts/BitsAndBytes/hf-cache
 mkdir -p $HF_HOME
+export SINGULARITYENV_HF_HOME=$HF_HOME
 
-srun python3 bnb-quantization.py
+srun singularity exec "$SIF" bash -c 'python3 bnb-quantization.py'

BitsAndBytes/run-bnb-quantization-roihu.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+#SBATCH --account=project_xxxxxxx
+#SBATCH --partition=gputest
+#SBATCH --time=00:15:00
+#SBATCH --nodes=1
+#SBATCH --tasks-per-node=1
+#SBATCH --cpus-per-task=72
+#SBATCH --gres=gpu:gh200:1
+#SBATCH --mem=32G
+
+module purge
+module load python-pytorch/2.10
+
+# Activate the virtual environment from your current directory or change to the appropriate path
+source venv/bin/activate
+
+# Set hf cache to the project's scratch
+export HF_HOME=/scratch/$SLURM_JOB_ACCOUNT/$USER/hf-cache/
+mkdir -p $HF_HOME
+
+srun python3 bnb-quantization.py