SyWeDaG: Synthetic Weather Data Generator

A desktop application for generating and visualizing synthetic meteorological scenarios using historical weather data from multiple sources (AEMET for Spain, extensible to other countries).

Project Structure

├── assets/                          # Images and resources
├── config/ 
│   └── config.json                  # Application and data-source configuration
├── data/                            # Local SQLite database files
├── examples/                        # Offline, network-free tutorial (see Testing below)
├── sample_pred_excels/              # Sample prediction Excel files
├── tests/                           # pytest suite, mirrors the src/ layout
├── src/ 
│   ├── _version.py                  # Single source of truth for the app version
│   ├── main.py                      # Streamlit entry point
│   ├── application/                 # Application/business logic (UI-independent)
│   │   ├── map_services.py          # Geocoding + GeoJSON coverage logic
│   │   └── config_services.py       # Validation + fetch/generate orchestration
│   ├── ui/                          # Presentation layer (Streamlit/Folium)
│   │   ├── styles/                  # UI styles per page/component
│   │   ├── map_component.py         # Interactive map page/component
│   │   ├── config_page.py           # Data/generation configuration page
│   │   └── results_page.py          # Results and visualization page
│   ├── data_sources/                # Weather source adapters
│   │   ├── base_source.py           # Common source interface/models
│   │   ├── aemet_source.py          # AEMET implementation
│   │   └── source_selector.py       # Source factory/selector
│   ├── generators/                  # Synthetic data generation logic
│   │   ├── synthetic_generator.py   # Main orchestration for daily/hourly generation
│   │   ├── daily_correctors/        # Secondary-variable correction models
│   │   │   ├── k_neighbors.py
│   │   │   ├── xgboost_model.py
│   │   │   └── mbc_correction.py
│   │   ├── monthly_adjustments/     # Monthly prediction adjustment logic
│   │   │   ├── temperature_adjuster.py
│   │   │   └── precipitation_adjuster.py
│   │   └── hourly_generation/       # Daily-to-hourly interpolation helpers
│   │       └── hourly_interpolator.py
│   ├── database/
│   │   └── sqliteDB.py              # DB schema and persistence helpers
│   ├── documentation                # Detailed documentation of the software structure and supported models
│   ├── modelValidation              # Instructions and utilities for validating the supported models
│   └── utils/                       # Shared utility helpers
│       ├── data_parsing.py
│       ├── geospatial.py
│       ├── historical_data_treatment.py
│       └── system_utils.py
├── .github/workflows/ci.yml         # CI: test matrix + app startup smoke test
├── build_desktop.bat                # Desktop build script
├── SyWeDaG.spec              # PyInstaller spec (generated/used in builds)
├── requirements.txt                 # Python dependencies
├── requirements-dev.txt             # Additional dependencies for running tests
├── CONTRIBUTING.md                  # Development setup, conventions, versioning policy
├── CHANGELOG.md                     # Notable changes, per Keep a Changelog
└── README.md

Features

• Interactive Map: Select geographical points in Spain using OpenStreetMap
• Search Functionality: Search for locations by name
• Zoom Controls: Navigate the map with custom zoom buttons
• Data Source Highlighting: Visual indication of areas with available data
• Modular Design: Easy to add new data sources for other countries

Platform Support

The software has been tested on the following platforms:

Operating System	Web	Desktop	Support Status
Ubuntu 22.04	✅	⚠️	Partially supported
Windows	✅	✅	Supported
macOS	—	—	Not tested

On Ubuntu 22.04, the project can be successfully compiled and executed in a web-based environment. However, compatibility issues currently prevent the desktop version from running correctly.

Linux desktop support is currently under development. We are actively working to resolve these issues and provide full Linux desktop support.

Installation

Install Python dependencies:

pip install -r requirements.txt

Running the Application

From the src directory:

streamlit run main.py

For desktop mode (from root directory):

build_desktop.bat

This will create a standalone executable in the dist folder.

Try it without an API key

examples/run_offline_demo.py runs the full generation pipeline on a bundled sample dataset, no AEMET API key or network access required. See examples/README.md.

Configuration

Edit config/config.json to:

• Add new data sources
• Modify default map settings
• Configure data source geographical boundaries

Technologies

• Streamlit: Web framework for the UI
• Folium: Interactive maps
• SQLite: Local data storage
• Pandas/NumPy: Data manipulation
• Plotly: Data visualization

Testing

pip install -r requirements-dev.txt
pytest --cov=src --cov-report=term-missing

The suite focuses on the generation pipeline's scientific properties rather than just execution: monthly adjustment invariants (e.g. Tmin <= Tmean <= Tmax after adjustment, monthly means matching predictions within tolerance), hourly interpolation consistency against daily aggregates, and the SQLite persistence and ZIP export/import round trips. Network calls to AEMET and Open-Meteo are mocked, so no test requires internet access or an API key.

The Streamlit UI layer (src/ui/) is not unit-tested; it is instead covered by a CI job that launches the packaged app and confirms it responds. The MBCn corrector (generators/daily_correctors/mbc_correction.py) is implemented but not wired into the generation pipeline, and is untested accordingly.

CI (.github/workflows/ci.yml) runs the full suite on Linux, Windows, and macOS across Python 3.11-3.12 on every push and pull request (numpy 2.3.5, pinned in requirements.txt, requires Python >= 3.11).

Versioning

SyWeDaG follows Semantic Versioning. The current version is defined in src/_version.py; see CHANGELOG.md for the history of notable changes.

Contributing

See CONTRIBUTING.md for development setup, test instructions, code conventions, and how to report issues.