Skip to content

qa/qc APIs and utilities - #10

Open
joefutrelle wants to merge 3 commits into
mainfrom
qaqc
Open

qa/qc APIs and utilities#10
joefutrelle wants to merge 3 commits into
mainfrom
qaqc

Conversation

@joefutrelle

Copy link
Copy Markdown
Collaborator

This pull request introduces a major new quality control (QC) subsystem to the ifcbkit library, providing comprehensive integrity checks for IFCB data and products. It also adds a command-line interface (ifcbkit-qc), improves ADC/header parsing, and exposes new APIs for working with ADC files and corrections. The changes include new documentation, new exports in the main package, and a minor version bump to 0.3.0.

The most important changes are:

Quality Control Subsystem:

  • Adds the ifcbkit.qc module, which provides functions like check_bin, check_collection, check_fileset, and check_products to report whether data is intact (present, parseable, internally consistent) and not whether it is scientifically "good". The QC system distinguishes between errors, warnings, and info-level findings, and supports opt-in checks, cost levels, and detailed reporting. A new CLI tool, ifcbkit-qc, is included for running QC from the command line. [1] [2] [3]

  • The QC system reports on ADC corrections, product coverage, and missing/optional files, and is fully documented in the expanded README.md. [1] [2]

ADC and Header Parsing Improvements:

  • Adds diagnostic reporting to ADC parsing via an optional diagnostics channel in iter_adc_targets, allowing consumers to see why lines were skipped (e.g., blank, short, unparseable, zero geometry). Introduces new constants for skip reasons and improves error handling in header parsing (parse failures are now reported, not raised). [1] [2] [3]

  • Exposes new utility functions: columns_for_bin_id, columns_from_declaration, and parse_adc_file_format for working with ADC file formats and column layouts. [1] [2]

ADC Correction File Handling:

  • Makes adcmod_path a public API and ensures that ADC correction files are resolved consistently in both sync and async code paths. QC now reports on corrections (e.g., deltas, orphans, invalid corrections) instead of silently preferring them. [1] [2] [3]

Package and CLI Updates:

  • Bumps the version to 0.3.0 and registers the ifcbkit-qc CLI in pyproject.toml. Adds h5py as an optional dependency for class-score QC. [1] [2] [3]

Documentation:

  • Expands the README.md with detailed documentation and usage examples for the new QC system, CLI, and product directory conventions. [1] [2]

These changes lay the foundation for robust, transparent data integrity checking in IFCB workflows and make the library more extensible and user-friendly for both programmatic and command-line use.

@joefutrelle
joefutrelle requested a review from johnwaalsh August 18, 2026 12:56
@joefutrelle

Copy link
Copy Markdown
Collaborator Author

Note that we'll need to resolve the conflict over version number by having this be a 0.4.0 release (since it contains a new feature)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant