diff --git a/ARCHITECTURE.adoc b/ARCHITECTURE.adoc new file mode 100644 index 0000000..1c0a7a6 --- /dev/null +++ b/ARCHITECTURE.adoc @@ -0,0 +1,48 @@ +== Architecture + +=== Overview + +This repository follows a modular, maintainable architecture designed +for clarity, scalability, and long-term sustainability. + +=== Directory Structure + +.... +. +├── src/ # Source code +├── tests/ # Test suites +├── docs/ # Documentation +├── scripts/ # Utility scripts +├── config/ # Configuration files +├── LICENSE # License file +├── LICENSES/ # Full license texts +└── README.adoc # Project documentation +.... + +=== Design Principles + +* *Separation of Concerns*: Each module has a single responsibility +* *Testability*: Code is written to be easily testable +* *Documentation*: All public APIs are documented +* *Configuration*: Environment-specific settings are externalized + +=== Dependencies + +* External dependencies are minimized and clearly declared +* Version pinning is used for reproducibility + +=== Security Considerations + +* Sensitive data is never committed to the repository +* Secrets are managed through environment variables or secure vaults +* Regular dependency audits are performed + +=== Maintainability + +* Code follows consistent style guidelines +* Pull requests require review and CI checks +* Issues and discussions are tracked transparently + +''''' + +_Last updated: 2026-07-18_ diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md deleted file mode 100644 index 607e3d8..0000000 --- a/ARCHITECTURE.md +++ /dev/null @@ -1,47 +0,0 @@ -# Architecture - -## Overview - -This repository follows a modular, maintainable architecture designed for clarity, scalability, and long-term sustainability. - -## Directory Structure - -``` -. -├── src/ # Source code -├── tests/ # Test suites -├── docs/ # Documentation -├── scripts/ # Utility scripts -├── config/ # Configuration files -├── LICENSE # License file -├── LICENSES/ # Full license texts -└── README.adoc # Project documentation -``` - -## Design Principles - -- **Separation of Concerns**: Each module has a single responsibility -- **Testability**: Code is written to be easily testable -- **Documentation**: All public APIs are documented -- **Configuration**: Environment-specific settings are externalized - -## Dependencies - -- External dependencies are minimized and clearly declared -- Version pinning is used for reproducibility - -## Security Considerations - -- Sensitive data is never committed to the repository -- Secrets are managed through environment variables or secure vaults -- Regular dependency audits are performed - -## Maintainability - -- Code follows consistent style guidelines -- Pull requests require review and CI checks -- Issues and discussions are tracked transparently - ---- - -*Last updated: 2026-07-18* diff --git a/CHANGELOG.adoc b/CHANGELOG.adoc new file mode 100644 index 0000000..55fdee2 --- /dev/null +++ b/CHANGELOG.adoc @@ -0,0 +1,86 @@ +== Changelog + +All notable changes to `+http-capability-gateway+` will be documented in +this file. + +This file is generated from conventional commits by the +https://github.com/hyperpolymath/standards/blob/main/.github/workflows/changelog-reusable.yml[`+changelog-reusable.yml+`] +workflow (`+hyperpolymath/standards#206+`). Adopt the workflow in this +repo’s CI to keep this file in sync automatically — see +https://github.com/hyperpolymath/standards/blob/main/templates/cliff.toml[`+templates/cliff.toml+`] +for the canonical config. + +The format follows https://keepachangelog.com/en/1.1.0/[Keep a +Changelog]; this project aims to follow +https://semver.org/spec/v2.0.0.html[Semantic Versioning]. + +=== [Unreleased] + +==== Added + +* feat(perf): Phase D benchmark harness scaffold (standards#99 / #91 +channel) (#12) +* feat(policy): catalog mode — auto-policy from BoJ cartridge.json +* feat(p2): bug fixes, unit tests, P2 productization docs +* feat(p1): gateway hardening — bug fixes, concurrency tests, +benchmarks, docs +* feat(crg): add crg-grade and crg-badge justfile recipes +* feat: add VeriSimDB async audit log client (capgw:audit) +* feat: wire conflow config validation pipeline +* feat: add k9iser.toml and generate K9 contracts +* feat: add stapeln.toml layer-based container definitionfrom existing +Containerfile to stapeln format.Chainguard base, security hardening, +SBOM generation.-Authored-By: Claude Opus 4.6 (1M context) +noreply@anthropic.com +* feat: deploy UX Manifesto infrastructure + +==== Fixed + +* fix(ci): switch CodeQL language matrix to `+actions+` (#13) +* fix(ci): bump a2ml/k9-validate-action pins to canonical (standards#85) +(#7) +* fix(ci): sync hypatia-scan.yml to canonical (kill cd-scanner build +drift) (#6) +* fix(ci): adopt canonical hypatia-scan.yml (env.HOME/scanner-layout + +Comment-step gate) (#5) +* fix(p0): resolve all 5 P0 release blockers +* fix: set correct Groove capability type (was: custom) +* fix: replace String.to_atom with String.to_existing_atom +* fix(scorecard): enforce granular permissions and add fuzzing +placeholder +* fix(ci): Resolve workflow-linter self-matching and metadata issues +* fix: global AGPL-3.0-or-later → PMPL-1.0-or-later replacement + +==== Changed + +* perf(d-2): wire in-process loopback backend so proxy-200 scenario +measures real dial-and-read (#14) +* refactor: migrate 6SCM → 6A2 (.scm → .a2ml format) + +==== Documentation + +* docs: record tech-debt audit findings (2026-05-26) (#16) +* docs: substantive CRG C annotation (EXPLAINME.adoc) +* docs: add TEST-NEEDS.md and/or PROOF-NEEDS.md from audit +* docs: add comprehensive v2.0 roadmap for irresistible gateway +* docs: add CONTRIBUTING.md +* docs: add checkpoint files for state tracking + +==== CI + +* ci: redistribute concurrency-cancel guard to read-only check workflows +(#9) +* ci: bump actions/upload-artifact SHA to current v4 (#4) +* ci: SHA-pin hyperpolymath validate-actions in dogfood-gate +* ci: wire hypatia-scan.yml to query own Dependabot alerts +* ci: deploy dogfood-gate, add Groove manifest and CRG tests + +=== Pre-history + +Prior commits to this file’s introduction are recorded in git history +but not formally classified into Keep-a-Changelog sections. To backfill, +run `+git cliff -o CHANGELOG.md+` locally using the canonical +https://github.com/hyperpolymath/standards/blob/main/templates/cliff.toml[`+cliff.toml+`] +— this is one-shot mechanical work. + +''''' diff --git a/CHANGELOG.md b/CHANGELOG.md deleted file mode 100644 index 81bb0f8..0000000 --- a/CHANGELOG.md +++ /dev/null @@ -1,76 +0,0 @@ - -# Changelog - -All notable changes to `http-capability-gateway` will be documented in this file. - -This file is generated from conventional commits by the -[`changelog-reusable.yml`](https://github.com/hyperpolymath/standards/blob/main/.github/workflows/changelog-reusable.yml) -workflow (`hyperpolymath/standards#206`). Adopt the workflow in this repo's CI to keep this file in sync automatically — see -[`templates/cliff.toml`](https://github.com/hyperpolymath/standards/blob/main/templates/cliff.toml) -for the canonical config. - -The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); -this project aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - -## [Unreleased] - -### Added - -- feat(perf): Phase D benchmark harness scaffold (standards#99 / #91 channel) (#12) -- feat(policy): catalog mode — auto-policy from BoJ cartridge.json -- feat(p2): bug fixes, unit tests, P2 productization docs -- feat(p1): gateway hardening — bug fixes, concurrency tests, benchmarks, docs -- feat(crg): add crg-grade and crg-badge justfile recipes -- feat: add VeriSimDB async audit log client (capgw:audit) -- feat: wire conflow config validation pipeline -- feat: add k9iser.toml and generate K9 contracts -- feat: add stapeln.toml layer-based container definition\n\nConverted from existing Containerfile to stapeln format.\nIncludes Chainguard base, security hardening, SBOM generation.\n\nCo-Authored-By: Claude Opus 4.6 (1M context) -- feat: deploy UX Manifesto infrastructure - -### Fixed - -- fix(ci): switch CodeQL language matrix to `actions` (#13) -- fix(ci): bump a2ml/k9-validate-action pins to canonical (standards#85) (#7) -- fix(ci): sync hypatia-scan.yml to canonical (kill cd-scanner build drift) (#6) -- fix(ci): adopt canonical hypatia-scan.yml (env.HOME/scanner-layout + Comment-step gate) (#5) -- fix(p0): resolve all 5 P0 release blockers -- fix: set correct Groove capability type (was: custom) -- fix: replace String.to_atom with String.to_existing_atom -- fix(scorecard): enforce granular permissions and add fuzzing placeholder -- fix(ci): Resolve workflow-linter self-matching and metadata issues -- fix: global AGPL-3.0-or-later → PMPL-1.0-or-later replacement - -### Changed - -- perf(d-2): wire in-process loopback backend so proxy-200 scenario measures real dial-and-read (#14) -- refactor: migrate 6SCM → 6A2 (.scm → .a2ml format) - -### Documentation - -- docs: record tech-debt audit findings (2026-05-26) (#16) -- docs: substantive CRG C annotation (EXPLAINME.adoc) -- docs: add TEST-NEEDS.md and/or PROOF-NEEDS.md from audit -- docs: add comprehensive v2.0 roadmap for irresistible gateway -- docs: add CONTRIBUTING.md -- docs: add checkpoint files for state tracking - -### CI - -- ci: redistribute concurrency-cancel guard to read-only check workflows (#9) -- ci: bump actions/upload-artifact SHA to current v4 (#4) -- ci: SHA-pin hyperpolymath validate-actions in dogfood-gate -- ci: wire hypatia-scan.yml to query own Dependabot alerts -- ci: deploy dogfood-gate, add Groove manifest and CRG tests - -## Pre-history - -Prior commits to this file's introduction are recorded in git history but not formally classified into Keep-a-Changelog sections. To backfill, run `git cliff -o CHANGELOG.md` locally using the canonical [`cliff.toml`](https://github.com/hyperpolymath/standards/blob/main/templates/cliff.toml) — this is one-shot mechanical work. - ---- - - diff --git a/CODE_OF_CONDUCT.adoc b/CODE_OF_CONDUCT.adoc new file mode 100644 index 0000000..c643d11 --- /dev/null +++ b/CODE_OF_CONDUCT.adoc @@ -0,0 +1,340 @@ +== Code of Conduct + +=== Our Pledge + +We as members, contributors, and leaders pledge to make participation in +Http Capability Gateway a harassment-free experience for everyone, +regardless of age, body size, visible or invisible disability, +ethnicity, sex characteristics, gender identity and expression, level of +experience, education, socio-economic status, nationality, personal +appearance, race, caste, colour, religion, or sexual identity and +orientation. + +We pledge to act and interact in ways that contribute to an open, +welcoming, diverse, inclusive, and healthy community. + +We recognise that a thriving open source community requires +*psychological safety* — an environment where people can contribute, ask +questions, make mistakes, and learn without fear of ridicule or +retaliation. + +''''' + +=== Our Standards + +==== Expected Behaviour + +The following behaviours contribute to a positive environment: + +*Communication* - Using welcoming and inclusive language - Being +respectful of differing viewpoints and experiences - Giving and +gracefully accepting constructive feedback - Assuming good intent while +addressing impact - Communicating clearly and patiently, especially with +newcomers + +*Collaboration* - Focusing on what is best for the community - Showing +empathy and kindness toward other community members - Being +collaborative rather than competitive - Mentoring and supporting less +experienced contributors - Celebrating others’ contributions and +successes + +*Professionalism* - Accepting responsibility and apologising to those +affected by our mistakes - Learning from the experience and avoiding +repetition - Respecting others’ time and attention - Staying on topic in +project spaces - Following project guidelines and conventions + +*Accessibility* - Using plain language and avoiding unnecessary jargon - +Providing alt text for images and transcripts for audio/video - Being +patient with those using assistive technologies - Accommodating +different communication styles and needs - Recognising that not everyone +communicates the same way + +==== Unacceptable Behaviour + +The following behaviours are considered harassment and are unacceptable: + +*Harassment* - The use of sexualised language or imagery, and sexual +attention or advances of any kind - Trolling, insulting or derogatory +comments, and personal or political attacks - Public or private +harassment - Deliberate intimidation, stalking, or following (online or +in-person) - Unwelcome physical contact or simulated physical contact +(e.g., emoji) - Sustained disruption of talks, events, or online +discussions + +*Discrimination* - Discriminatory jokes and language - Posting or +threatening to post others’ personally identifying information +("`doxing`") - Advocating for, or encouraging, any of the above +behaviour - Microaggressions — subtle, often unintentional, +discriminatory comments or actions + +*Professional Misconduct* - Publishing others’ private information +without explicit permission - Misrepresenting affiliation or +contributions - Plagiarism or claiming credit for others’ work - +Retaliating against anyone who reports a Code of Conduct violation - +Other conduct which could reasonably be considered inappropriate in a +professional setting + +==== Grey Areas + +Some situations require judgement. When uncertain: + +* *Intent vs Impact*: Good intentions do not excuse harmful impact. +Focus on making things right. +* *Power Dynamics*: Those with more power (maintainers, employers, +experienced contributors) must be especially mindful of their impact. +* *Cultural Differences*: What’s acceptable varies by culture. When in +doubt, err on the side of caution and ask. +* *Humour*: Jokes at others’ expense are rarely funny to everyone. Punch +up, not down. + +''''' + +=== Scope + +This Code of Conduct applies within all community spaces, including: + +*Online Spaces* - Repository discussions, issues, and pull/merge +requests - Project chat channels (Matrix, Discord, Slack, IRC) - Mailing +lists and forums - Social media when representing the project - Video +calls and virtual meetings + +*In-Person Spaces* - Conferences, meetups, and events - Workshops and +training sessions - Any gathering where you represent the project + +*Representation* This Code of Conduct also applies when an individual is +officially representing the community in public spaces. Examples +include: + +* Using an official project email address +* Posting via an official social media account +* Acting as an appointed representative at an event +* Speaking on behalf of the project + +''''' + +=== Enforcement + +==== Reporting + +If you experience or witness unacceptable behaviour, or have any other +concerns, please report it as soon as possible. + +*How to Report* + +[width="99%",cols="30%,33%,37%",options="header",] +|=== +|Method |Details |Best For +|*Email* |\{\{CONDUCT_EMAIL}} |Detailed reports, sensitive matters + +|*Private Message* |Contact any maintainer directly |Quick questions, +minor issues + +|*Anonymous Form* |[Link to form if available] |When you need anonymity +|=== + +*What to Include* + +* Your contact information (unless anonymous) +* Names/usernames of those involved +* Description of what happened +* When and where it occurred +* Any witnesses +* Any supporting evidence (screenshots, links) +* How you would like us to respond (if you have a preference) + +*What Happens Next* + +[arabic] +. You will receive acknowledgment within *\{\{RESPONSE_TIME}}* +. The \{\{CONDUCT_TEAM}} will review the report +. We may ask for additional information +. We will determine appropriate action +. We will inform you of the outcome (respecting others’ privacy) + +==== Confidentiality + +All reports will be handled with discretion: + +* Reporter identity is protected by default +* Details are shared only with those who need to know +* We will ask before naming you in any communication +* Anonymous reports are accepted and investigated + +==== Conflicts of Interest + +If a \{\{CONDUCT_TEAM}} member is involved in an incident: + +* They will recuse themselves from the process +* Another maintainer or external party will handle the report +* We will disclose any potential conflicts + +''''' + +=== Enforcement Guidelines + +The \{\{CONDUCT_TEAM}} will follow these guidelines in determining +consequences: + +==== 1. Correction + +*Community Impact*: Use of inappropriate language or other behaviour +deemed unprofessional or unwelcome. + +*Consequence*: A private, written warning providing clarity around the +nature of the violation and an explanation of why the behaviour was +inappropriate. A public apology may be requested. + +*Duration*: Immediate + +==== 2. Warning + +*Community Impact*: A violation through a single incident or series of +actions. + +*Consequence*: A warning with consequences for continued behaviour. No +interaction with the people involved, including unsolicited interaction +with those enforcing the Code of Conduct, for a specified period. This +includes avoiding interactions in community spaces as well as external +channels like social media. Violating these terms may lead to a +temporary or permanent ban. + +*Duration*: 1-4 weeks + +==== 3. Temporary Ban + +*Community Impact*: A serious violation of community standards, +including sustained inappropriate behaviour. + +*Consequence*: A temporary ban from any sort of interaction or public +communication with the community for a specified period. No public or +private interaction with the people involved, including unsolicited +interaction with those enforcing the Code of Conduct, is allowed during +this period. Violating these terms may lead to a permanent ban. + +*Duration*: 1-6 months + +==== 4. Permanent Ban + +*Community Impact*: Demonstrating a pattern of violation of community +standards, including sustained inappropriate behaviour, harassment of an +individual, or aggression toward or disparagement of classes of +individuals. + +*Consequence*: A permanent ban from any sort of public interaction +within the community. + +*Duration*: Permanent (with appeal rights after 12 months) + +==== Enforcement Across Perimeters + +For contributors with elevated access (Perimeter 2 or 1): + +[cols=",",options="header",] +|=== +|Level |Additional Consequence +|Correction |Noted in contributor record +|Warning |Access privileges may be temporarily reduced +|Temporary Ban |Access reduced to Perimeter 3 for ban duration +|Permanent Ban |All access revoked +|=== + +''''' + +=== Appeals + +If you believe an enforcement decision was made in error: + +[arabic] +. *Wait 7 days* after the decision (cooling-off period) +. *Email* \{\{CONDUCT_EMAIL}} with subject line "`Appeal: [Original +Report ID]`" +. *Explain* why you believe the decision should be reconsidered +. *Provide* any new information not previously available + +*Appeals Process* + +* Appeals are reviewed by a different \{\{CONDUCT_TEAM}} member than the +original +* You will receive a response within 14 days +* The appeals decision is final +* You may only appeal once per incident + +*Grounds for Appeal* + +* Procedural errors in the original investigation +* New evidence not previously available +* Disproportionate response to the violation +* Misunderstanding of facts + +''''' + +=== Supporting Those Who Report + +We are committed to supporting those who report violations: + +*We Will* - Believe and take all reports seriously - Respect your +privacy and confidentiality preferences - Keep you informed of progress +(if you wish) - Take steps to protect you from retaliation - Provide +resources if you need support + +*We Will Not* - Require you to confront the person directly - Dismiss +reports without investigation - Reveal your identity without consent - +Tolerate retaliation against reporters - Rush you to make decisions + +''''' + +=== Prevention + +Beyond enforcement, we actively work to prevent issues: + +*Onboarding* - All contributors are expected to read this Code of +Conduct - Perimeter 2 applicants must confirm they’ve read and +understood it - Maintainers receive additional training on enforcement + +*Culture* - We model the behaviour we expect - We intervene early when +we see potential issues - We thank people for positive contributions - +We create opportunities for diverse voices + +*Review* - This Code of Conduct is reviewed annually - Community +feedback is welcomed - Changes are communicated clearly + +''''' + +=== Acknowledgments + +This Code of Conduct is adapted from: + +* https://www.contributor-covenant.org/[Contributor Covenant], version +2.1 +* https://www.djangoproject.com/conduct/[Django Code of Conduct] +* https://www.rust-lang.org/policies/code-of-conduct[Rust Code of +Conduct] +* https://www.python.org/psf/conduct/[Python Community Code of Conduct] + +We thank these communities for their leadership in creating welcoming +spaces. + +''''' + +=== Questions? + +If you have questions about this Code of Conduct: + +* Open a +https://github.com/hyperpolymath/http-capability-gateway/discussions[Discussion] +(for general questions) +* Email \{\{CONDUCT_EMAIL}} (for private questions) +* Contact any maintainer directly + +''''' + +=== Summary + +*Be kind. Be respectful. Be collaborative.* + +We’re all here because we care about this project. Let’s make it a place +where everyone can do their best work. + +''''' + +Last updated: 2026 · Based on Contributor Covenant 2.1 diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md deleted file mode 100644 index f26886f..0000000 --- a/CODE_OF_CONDUCT.md +++ /dev/null @@ -1,331 +0,0 @@ - -# Code of Conduct - - - -## Our Pledge - -We as members, contributors, and leaders pledge to make participation in Http Capability Gateway a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, colour, religion, or sexual identity and orientation. - -We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community. - -We recognise that a thriving open source community requires **psychological safety** — an environment where people can contribute, ask questions, make mistakes, and learn without fear of ridicule or retaliation. - ---- - -## Our Standards - -### Expected Behaviour - -The following behaviours contribute to a positive environment: - -**Communication** -- Using welcoming and inclusive language -- Being respectful of differing viewpoints and experiences -- Giving and gracefully accepting constructive feedback -- Assuming good intent while addressing impact -- Communicating clearly and patiently, especially with newcomers - -**Collaboration** -- Focusing on what is best for the community -- Showing empathy and kindness toward other community members -- Being collaborative rather than competitive -- Mentoring and supporting less experienced contributors -- Celebrating others' contributions and successes - -**Professionalism** -- Accepting responsibility and apologising to those affected by our mistakes -- Learning from the experience and avoiding repetition -- Respecting others' time and attention -- Staying on topic in project spaces -- Following project guidelines and conventions - -**Accessibility** -- Using plain language and avoiding unnecessary jargon -- Providing alt text for images and transcripts for audio/video -- Being patient with those using assistive technologies -- Accommodating different communication styles and needs -- Recognising that not everyone communicates the same way - -### Unacceptable Behaviour - -The following behaviours are considered harassment and are unacceptable: - -**Harassment** -- The use of sexualised language or imagery, and sexual attention or advances of any kind -- Trolling, insulting or derogatory comments, and personal or political attacks -- Public or private harassment -- Deliberate intimidation, stalking, or following (online or in-person) -- Unwelcome physical contact or simulated physical contact (e.g., emoji) -- Sustained disruption of talks, events, or online discussions - -**Discrimination** -- Discriminatory jokes and language -- Posting or threatening to post others' personally identifying information ("doxing") -- Advocating for, or encouraging, any of the above behaviour -- Microaggressions — subtle, often unintentional, discriminatory comments or actions - -**Professional Misconduct** -- Publishing others' private information without explicit permission -- Misrepresenting affiliation or contributions -- Plagiarism or claiming credit for others' work -- Retaliating against anyone who reports a Code of Conduct violation -- Other conduct which could reasonably be considered inappropriate in a professional setting - -### Grey Areas - -Some situations require judgement. When uncertain: - -- **Intent vs Impact**: Good intentions do not excuse harmful impact. Focus on making things right. -- **Power Dynamics**: Those with more power (maintainers, employers, experienced contributors) must be especially mindful of their impact. -- **Cultural Differences**: What's acceptable varies by culture. When in doubt, err on the side of caution and ask. -- **Humour**: Jokes at others' expense are rarely funny to everyone. Punch up, not down. - ---- - -## Scope - -This Code of Conduct applies within all community spaces, including: - -**Online Spaces** -- Repository discussions, issues, and pull/merge requests -- Project chat channels (Matrix, Discord, Slack, IRC) -- Mailing lists and forums -- Social media when representing the project -- Video calls and virtual meetings - -**In-Person Spaces** -- Conferences, meetups, and events -- Workshops and training sessions -- Any gathering where you represent the project - -**Representation** -This Code of Conduct also applies when an individual is officially representing the community in public spaces. Examples include: - -- Using an official project email address -- Posting via an official social media account -- Acting as an appointed representative at an event -- Speaking on behalf of the project - ---- - -## Enforcement - -### Reporting - -If you experience or witness unacceptable behaviour, or have any other concerns, please report it as soon as possible. - -**How to Report** - -| Method | Details | Best For | -|--------|---------|----------| -| **Email** | {{CONDUCT_EMAIL}} | Detailed reports, sensitive matters | -| **Private Message** | Contact any maintainer directly | Quick questions, minor issues | -| **Anonymous Form** | [Link to form if available] | When you need anonymity | - -**What to Include** - -- Your contact information (unless anonymous) -- Names/usernames of those involved -- Description of what happened -- When and where it occurred -- Any witnesses -- Any supporting evidence (screenshots, links) -- How you would like us to respond (if you have a preference) - -**What Happens Next** - -1. You will receive acknowledgment within **{{RESPONSE_TIME}}** -2. The {{CONDUCT_TEAM}} will review the report -3. We may ask for additional information -4. We will determine appropriate action -5. We will inform you of the outcome (respecting others' privacy) - -### Confidentiality - -All reports will be handled with discretion: - -- Reporter identity is protected by default -- Details are shared only with those who need to know -- We will ask before naming you in any communication -- Anonymous reports are accepted and investigated - -### Conflicts of Interest - -If a {{CONDUCT_TEAM}} member is involved in an incident: - -- They will recuse themselves from the process -- Another maintainer or external party will handle the report -- We will disclose any potential conflicts - ---- - -## Enforcement Guidelines - -The {{CONDUCT_TEAM}} will follow these guidelines in determining consequences: - -### 1. Correction - -**Community Impact**: Use of inappropriate language or other behaviour deemed unprofessional or unwelcome. - -**Consequence**: A private, written warning providing clarity around the nature of the violation and an explanation of why the behaviour was inappropriate. A public apology may be requested. - -**Duration**: Immediate - -### 2. Warning - -**Community Impact**: A violation through a single incident or series of actions. - -**Consequence**: A warning with consequences for continued behaviour. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban. - -**Duration**: 1-4 weeks - -### 3. Temporary Ban - -**Community Impact**: A serious violation of community standards, including sustained inappropriate behaviour. - -**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban. - -**Duration**: 1-6 months - -### 4. Permanent Ban - -**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behaviour, harassment of an individual, or aggression toward or disparagement of classes of individuals. - -**Consequence**: A permanent ban from any sort of public interaction within the community. - -**Duration**: Permanent (with appeal rights after 12 months) - -### Enforcement Across Perimeters - -For contributors with elevated access (Perimeter 2 or 1): - -| Level | Additional Consequence | -|-------|----------------------| -| Correction | Noted in contributor record | -| Warning | Access privileges may be temporarily reduced | -| Temporary Ban | Access reduced to Perimeter 3 for ban duration | -| Permanent Ban | All access revoked | - ---- - -## Appeals - -If you believe an enforcement decision was made in error: - -1. **Wait 7 days** after the decision (cooling-off period) -2. **Email** {{CONDUCT_EMAIL}} with subject line "Appeal: [Original Report ID]" -3. **Explain** why you believe the decision should be reconsidered -4. **Provide** any new information not previously available - -**Appeals Process** - -- Appeals are reviewed by a different {{CONDUCT_TEAM}} member than the original -- You will receive a response within 14 days -- The appeals decision is final -- You may only appeal once per incident - -**Grounds for Appeal** - -- Procedural errors in the original investigation -- New evidence not previously available -- Disproportionate response to the violation -- Misunderstanding of facts - ---- - -## Supporting Those Who Report - -We are committed to supporting those who report violations: - -**We Will** -- Believe and take all reports seriously -- Respect your privacy and confidentiality preferences -- Keep you informed of progress (if you wish) -- Take steps to protect you from retaliation -- Provide resources if you need support - -**We Will Not** -- Require you to confront the person directly -- Dismiss reports without investigation -- Reveal your identity without consent -- Tolerate retaliation against reporters -- Rush you to make decisions - ---- - -## Prevention - -Beyond enforcement, we actively work to prevent issues: - -**Onboarding** -- All contributors are expected to read this Code of Conduct -- Perimeter 2 applicants must confirm they've read and understood it -- Maintainers receive additional training on enforcement - -**Culture** -- We model the behaviour we expect -- We intervene early when we see potential issues -- We thank people for positive contributions -- We create opportunities for diverse voices - -**Review** -- This Code of Conduct is reviewed annually -- Community feedback is welcomed -- Changes are communicated clearly - ---- - -## Acknowledgments - -This Code of Conduct is adapted from: - -- [Contributor Covenant](https://www.contributor-covenant.org/), version 2.1 -- [Django Code of Conduct](https://www.djangoproject.com/conduct/) -- [Rust Code of Conduct](https://www.rust-lang.org/policies/code-of-conduct) -- [Python Community Code of Conduct](https://www.python.org/psf/conduct/) - -We thank these communities for their leadership in creating welcoming spaces. - ---- - -## Questions? - -If you have questions about this Code of Conduct: - -- Open a [Discussion](https://github.com/hyperpolymath/http-capability-gateway/discussions) (for general questions) -- Email {{CONDUCT_EMAIL}} (for private questions) -- Contact any maintainer directly - ---- - -## Summary - -**Be kind. Be respectful. Be collaborative.** - -We're all here because we care about this project. Let's make it a place where everyone can do their best work. - ---- - -Last updated: 2026 · Based on Contributor Covenant 2.1 diff --git a/CONTRIBUTING.adoc b/CONTRIBUTING.adoc new file mode 100644 index 0000000..f0a4f9e --- /dev/null +++ b/CONTRIBUTING.adoc @@ -0,0 +1,109 @@ +== Clone the repository + +git clone https://github.com/hyperpolymath/http-capability-gateway.git +cd http-capability-gateway + +== Using Nix (recommended for reproducibility) + +nix develop + +== Or using toolbox/distrobox + +toolbox create http-capability-gateway-dev toolbox enter +http-capability-gateway-dev # Install dependencies manually + +== Verify setup + +just check # or: cargo check / mix compile / etc. just test # Run test +suite + +.... + +### Repository Structure +.... + +http-capability-gateway/ ├── src/ # Source code (Perimeter 1-2) ├── lib/ +# Library code (Perimeter 1-2) ├── extensions/ # Extensions (Perimeter +2) ├── plugins/ # Plugins (Perimeter 2) ├── tools/ # Tooling (Perimeter +2) ├── docs/ # Documentation (Perimeter 3) │ ├── architecture/ # ADRs, +specs (Perimeter 2) │ └── proposals/ # RFCs (Perimeter 3) ├── examples/ +# Examples (Perimeter 3) ├── spec/ # Spec tests (Perimeter 3) ├── tests/ +# Test suite (Perimeter 2-3) ├── .well-known/ # Protocol files +(Perimeter 1-3) ├── .github/ # GitHub config (Perimeter 1) │ ├── +ISSUE_TEMPLATE/ │ └── workflows/ ├── CHANGELOG.md ├── CODE_OF_CONDUCT.md +├── CONTRIBUTING.md # This file ├── GOVERNANCE.md ├── LICENSE ├── +MAINTAINERS.md ├── README.adoc ├── SECURITY.md ├── flake.nix # Nix flake +(Perimeter 1) └── Justfile # Task runner (Perimeter 1) + +.... + +--- + +## How to Contribute + +### Reporting Bugs + +**Before reporting**: +1. Search existing issues +2. Check if it's already fixed in `main` +3. Determine which perimeter the bug affects + +**When reporting**: + +Use the [bug report template](.github/ISSUE_TEMPLATE/bug_report.md) and include: + +- Clear, descriptive title +- Environment details (OS, versions, toolchain) +- Steps to reproduce +- Expected vs actual behaviour +- Logs, screenshots, or minimal reproduction + +### Suggesting Features + +**Before suggesting**: +1. Check the [roadmap](ROADMAP.md) if available +2. Search existing issues and discussions +3. Consider which perimeter the feature belongs to + +**When suggesting**: + +Use the [feature request template](.github/ISSUE_TEMPLATE/feature_request.md) and include: + +- Problem statement (what pain point does this solve?) +- Proposed solution +- Alternatives considered +- Which perimeter this affects + +### Your First Contribution + +Look for issues labelled: + +- [`good first issue`](https://github.com/hyperpolymath/http-capability-gateway/labels/good%20first%20issue) — Simple Perimeter 3 tasks +- [`help wanted`](https://github.com/hyperpolymath/http-capability-gateway/labels/help%20wanted) — Community help needed +- [`documentation`](https://github.com/hyperpolymath/http-capability-gateway/labels/documentation) — Docs improvements +- [`perimeter-3`](https://github.com/hyperpolymath/http-capability-gateway/labels/perimeter-3) — Community sandbox scope + +--- + +## Development Workflow + +### Branch Naming +.... + +docs/short-description # Documentation (P3) test/what-added # Test +additions (P3) feat/short-description # New features (P2) +fix/issue-number-description # Bug fixes (P2) refactor/what-changed # +Code improvements (P2) security/what-fixed # Security fixes (P1-2) + +.... + +### Commit Messages + +We follow [Conventional Commits](https://www.conventionalcommits.org/): +.... + +(): + +{empty}[optional body] + +{empty}[optional footer] diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md deleted file mode 100644 index f074004..0000000 --- a/CONTRIBUTING.md +++ /dev/null @@ -1,120 +0,0 @@ - -# Clone the repository -git clone https://github.com/hyperpolymath/http-capability-gateway.git -cd http-capability-gateway - -# Using Nix (recommended for reproducibility) -nix develop - -# Or using toolbox/distrobox -toolbox create http-capability-gateway-dev -toolbox enter http-capability-gateway-dev -# Install dependencies manually - -# Verify setup -just check # or: cargo check / mix compile / etc. -just test # Run test suite -``` - -### Repository Structure -``` -http-capability-gateway/ -├── src/ # Source code (Perimeter 1-2) -├── lib/ # Library code (Perimeter 1-2) -├── extensions/ # Extensions (Perimeter 2) -├── plugins/ # Plugins (Perimeter 2) -├── tools/ # Tooling (Perimeter 2) -├── docs/ # Documentation (Perimeter 3) -│ ├── architecture/ # ADRs, specs (Perimeter 2) -│ └── proposals/ # RFCs (Perimeter 3) -├── examples/ # Examples (Perimeter 3) -├── spec/ # Spec tests (Perimeter 3) -├── tests/ # Test suite (Perimeter 2-3) -├── .well-known/ # Protocol files (Perimeter 1-3) -├── .github/ # GitHub config (Perimeter 1) -│ ├── ISSUE_TEMPLATE/ -│ └── workflows/ -├── CHANGELOG.md -├── CODE_OF_CONDUCT.md -├── CONTRIBUTING.md # This file -├── GOVERNANCE.md -├── LICENSE -├── MAINTAINERS.md -├── README.adoc -├── SECURITY.md -├── flake.nix # Nix flake (Perimeter 1) -└── Justfile # Task runner (Perimeter 1) -``` - ---- - -## How to Contribute - -### Reporting Bugs - -**Before reporting**: -1. Search existing issues -2. Check if it's already fixed in `main` -3. Determine which perimeter the bug affects - -**When reporting**: - -Use the [bug report template](.github/ISSUE_TEMPLATE/bug_report.md) and include: - -- Clear, descriptive title -- Environment details (OS, versions, toolchain) -- Steps to reproduce -- Expected vs actual behaviour -- Logs, screenshots, or minimal reproduction - -### Suggesting Features - -**Before suggesting**: -1. Check the [roadmap](ROADMAP.md) if available -2. Search existing issues and discussions -3. Consider which perimeter the feature belongs to - -**When suggesting**: - -Use the [feature request template](.github/ISSUE_TEMPLATE/feature_request.md) and include: - -- Problem statement (what pain point does this solve?) -- Proposed solution -- Alternatives considered -- Which perimeter this affects - -### Your First Contribution - -Look for issues labelled: - -- [`good first issue`](https://github.com/hyperpolymath/http-capability-gateway/labels/good%20first%20issue) — Simple Perimeter 3 tasks -- [`help wanted`](https://github.com/hyperpolymath/http-capability-gateway/labels/help%20wanted) — Community help needed -- [`documentation`](https://github.com/hyperpolymath/http-capability-gateway/labels/documentation) — Docs improvements -- [`perimeter-3`](https://github.com/hyperpolymath/http-capability-gateway/labels/perimeter-3) — Community sandbox scope - ---- - -## Development Workflow - -### Branch Naming -``` -docs/short-description # Documentation (P3) -test/what-added # Test additions (P3) -feat/short-description # New features (P2) -fix/issue-number-description # Bug fixes (P2) -refactor/what-changed # Code improvements (P2) -security/what-fixed # Security fixes (P1-2) -``` - -### Commit Messages - -We follow [Conventional Commits](https://www.conventionalcommits.org/): -``` -(): - -[optional body] - -[optional footer] diff --git a/GOVERNANCE.adoc b/GOVERNANCE.adoc new file mode 100644 index 0000000..9b836fb --- /dev/null +++ b/GOVERNANCE.adoc @@ -0,0 +1,60 @@ +== Governance + +=== Overview + +This project is governed by the following principles and structures to +ensure transparent, inclusive, and effective decision-making. + +=== Roles and Responsibilities + +==== Maintainers + +Maintainers are responsible for: - Reviewing and merging pull requests - +Managing releases and versioning - Ensuring code quality and standards - +Triaging issues and bug reports - Community engagement and support + +==== Contributors + +Contributors are expected to: - Follow the code of conduct - Submit +well-documented pull requests - Write tests for new functionality - +Maintain existing tests - Update documentation as needed + +=== Decision Making + +==== Minor Changes + +* Can be made by any maintainer +* Include bug fixes, documentation updates, dependency updates + +==== Major Changes + +* Require discussion in issues or pull requests +* Include new features, architectural changes, API changes +* Need approval from at least 2 maintainers + +==== Breaking Changes + +* Require RFC (Request for Comments) process +* Need approval from majority of maintainers +* Must include migration guide + +=== Code of Conduct + +All participants are expected to follow our Code of Conduct. Violations +can be reported to the maintainers. + +=== Communication + +* *Issues*: For bug reports and feature requests +* *Discussions*: For questions and general discussion +* *Pull Requests*: For code contributions + +=== Licensing + +All contributions are made under the terms of the repository’s LICENSE +file. By submitting a pull request, you agree to license your +contributions accordingly. + +''''' + +_Last updated: 2026-07-18_ diff --git a/GOVERNANCE.md b/GOVERNANCE.md deleted file mode 100644 index e27364c..0000000 --- a/GOVERNANCE.md +++ /dev/null @@ -1,60 +0,0 @@ -# Governance - -## Overview - -This project is governed by the following principles and structures to ensure transparent, inclusive, and effective decision-making. - -## Roles and Responsibilities - -### Maintainers - -Maintainers are responsible for: -- Reviewing and merging pull requests -- Managing releases and versioning -- Ensuring code quality and standards -- Triaging issues and bug reports -- Community engagement and support - -### Contributors - -Contributors are expected to: -- Follow the code of conduct -- Submit well-documented pull requests -- Write tests for new functionality -- Maintain existing tests -- Update documentation as needed - -## Decision Making - -### Minor Changes -- Can be made by any maintainer -- Include bug fixes, documentation updates, dependency updates - -### Major Changes -- Require discussion in issues or pull requests -- Include new features, architectural changes, API changes -- Need approval from at least 2 maintainers - -### Breaking Changes -- Require RFC (Request for Comments) process -- Need approval from majority of maintainers -- Must include migration guide - -## Code of Conduct - -All participants are expected to follow our Code of Conduct. Violations can be reported to the maintainers. - -## Communication - -- **Issues**: For bug reports and feature requests -- **Discussions**: For questions and general discussion -- **Pull Requests**: For code contributions - -## Licensing - -All contributions are made under the terms of the repository's LICENSE file. -By submitting a pull request, you agree to license your contributions accordingly. - ---- - -*Last updated: 2026-07-18* diff --git a/IMPLEMENTATION-ROADMAP.adoc b/IMPLEMENTATION-ROADMAP.adoc new file mode 100644 index 0000000..a8c2840 --- /dev/null +++ b/IMPLEMENTATION-ROADMAP.adoc @@ -0,0 +1,652 @@ +== http-capability-gateway - Implementation Roadmap + +____ +*HISTORICAL DOCUMENT (2026-04-16):* This document was written before any +code existed and is no longer accurate. The repository now has 19 Elixir +modules, 7 test files, 2 Zig FFI parsers, and 2 Idris2 ABI modules. The +"`What’s Missing`" list below is mostly *completed*. See +`+ROADMAP.adoc+` and `+STATE.adoc+` for the current state. +____ + +*Created:* 2026-01-22 *Status at time of writing:* 30% (Design Phase) — +*now ~55% with code implemented, verification lagging* *Target:* MVP +v0.1.0 *Estimated Effort:* 40-60 hours + +''''' + +=== Current State + +==== What Exists + +* ✅ README with clear MVP scope +* ✅ Architecture design (Policy DSL → Elixir Gateway) +* ✅ Example policy.yaml structure +* ✅ Verb Governance Spec (DSL v1) defined +* ✅ Directory structure (contractiles/, docs/, config/) +* ✅ STATE.scm, ECOSYSTEM.scm files + +==== What’s Missing + +* ❌ No Elixir application (`+mix.exs+`) +* ❌ No source code (`+lib/+` directory empty/missing) +* ❌ No policy loader implementation +* ❌ No validator (schema validation) +* ❌ No compiler (DSL → enforcement rules) +* ❌ No HTTP gateway (Plug/Cowboy) +* ❌ No enforcement engine +* ❌ No tests + +''''' + +=== Phase 1: Foundation (MVP) - 40-50 hours + +==== 1.1 Elixir Application Scaffold (2-3h) + +*Priority:* CRITICAL *Files to Create:* + +.... +http-capability-gateway/ +├── mix.exs # Mix project config +├── lib/ +│ └── http_capability_gateway/ +│ ├── application.ex # OTP application +│ └── gateway.ex # Main supervisor +└── config/ + ├── config.exs # Base config + ├── dev.exs # Dev environment + ├── prod.exs # Production environment + └── policy.yaml # ✓ Already exists +.... + +*Dependencies to Add:* - `+plug_cowboy+` - HTTP server - `+jason+` - +JSON encoding/decoding - `+yaml_elixir+` - YAML parser - `+telemetry+` - +Metrics/logging - `+ex_json_schema+` - JSON Schema validation + +*Tasks:* - [ ] Create `+mix.exs+` with dependencies - [ ] Generate OTP +application structure - [ ] Configure logger (structured JSON output) - +[ ] Set up Telemetry for observability + +''''' + +==== 1.2 Policy Loader (4-6h) + +*Priority:* CRITICAL *Module:* `+HttpCapabilityGateway.PolicyLoader+` + +*Implementation:* + +[source,elixir] +---- +defmodule HttpCapabilityGateway.PolicyLoader do + @moduledoc """ + Loads Verb Governance Spec (YAML) from disk. + Parses YAML into Elixir map structure. + """ + + @spec load_policy(path :: String.t()) :: {:ok, map()} | {:error, term()} + def load_policy(path) do + # Read YAML file + # Parse with YamlElixir + # Return structured policy map + end +end +---- + +*Schema to Parse:* + +[source,yaml] +---- +service: + name: string + version: integer + environment: string + +verbs: + GET: {exposure: public | authenticated | internal} + POST: {exposure: public | authenticated | internal} + PUT: {exposure: public | authenticated | internal} + DELETE: {exposure: public | authenticated | internal} + PATCH: {exposure: public | authenticated | internal} + HEAD: {exposure: public | authenticated | internal} + OPTIONS: {exposure: public | authenticated | internal} + +routes: + - path: string (regex-capable) + verbs: {verb-specific overrides with narrative} + +stealth: + profiles: + limited: + unauthenticated: 401 | 403 | 404 | 405 + untrusted: 401 | 403 | 404 | 405 + +narrative: + purpose: string +---- + +*Tasks:* - [ ] Implement YAML file reading - [ ] Parse service metadata +- [ ] Parse verb exposure levels - [ ] Parse route-specific overrides - +[ ] Parse stealth profiles - [ ] Handle file errors gracefully + +''''' + +==== 1.3 Policy Validator (6-8h) + +*Priority:* CRITICAL *Module:* `+HttpCapabilityGateway.PolicyValidator+` + +*Implementation:* + +[source,elixir] +---- +defmodule HttpCapabilityGateway.PolicyValidator do + @moduledoc """ + Validates loaded policy against JSON Schema. + Ensures all required fields present. + Checks exposure levels are valid. + Validates route paths are valid regexes. + """ + + @spec validate(policy :: map()) :: :ok | {:error, [validation_error()]} + def validate(policy) do + # Validate service metadata + # Validate verb exposure levels + # Validate route paths (regex compilation) + # Validate stealth profile codes + # Return aggregated errors + end +end +---- + +*Validation Rules:* - `+service.name+` - required, non-empty string - +`+service.version+` - required, positive integer - +`+service.environment+` - required, one of: dev, staging, prod - +`+verbs..exposure+` - required, one of: public, authenticated, +internal - `+routes[].path+` - valid regex pattern - +`+stealth.profiles..+` - valid HTTP status code +(401-599) + +*Tasks:* - [ ] Define JSON Schema for DSL v1 - [ ] Implement schema +validation with ex_json_schema - [ ] Validate regex patterns compile - [ +] Validate exposure levels are recognized - [ ] Validate stealth codes +are valid HTTP codes - [ ] Generate human-readable error messages + +''''' + +==== 1.4 Policy Compiler (8-12h) + +*Priority:* CRITICAL *Module:* `+HttpCapabilityGateway.PolicyCompiler+` + +*Implementation:* + +[source,elixir] +---- +defmodule HttpCapabilityGateway.PolicyCompiler do + @moduledoc """ + Compiles DSL policy into fast enforcement rules. + Builds ETS table with compiled patterns for O(1) lookups. + """ + + defmodule CompiledRule do + @moduledoc "Single enforcement rule" + defstruct [ + :path_regex, # Compiled Regex for route matching + :verb, # HTTP method atom (:get, :post, etc.) + :exposure, # :public | :authenticated | :internal + :stealth_profile, # :none | :limited (from policy) + :narrative # Optional explanation string + ] + end + + @spec compile(policy :: map()) :: {:ok, :ets.tid()} | {:error, term()} + def compile(policy) do + # Compile global verb rules + # Compile route-specific overrides (take precedence) + # Store in ETS table: {path_pattern, verb} => CompiledRule + # Return ETS table reference + end +end +---- + +*Compilation Strategy:* 1. Parse global verb rules (apply to all routes) +2. Parse route-specific overrides (highest precedence) 3. Compile regex +patterns for efficient matching 4. Store in ETS table for fast +concurrent lookups 5. Return ETS table reference for enforcement + +*ETS Schema:* + +.... +Key: {path_regex, verb_atom} +Value: %CompiledRule{...} +.... + +*Tasks:* - [ ] Implement global verb rule compilation - [ ] Implement +route-specific override compilation - [ ] Compile regex patterns with +error handling - [ ] Create ETS table with appropriate options (public, +read_concurrency) - [ ] Handle pattern conflicts (routes should override +global) - [ ] Benchmark lookup performance + +''''' + +==== 1.5 HTTP Gateway (10-14h) + +*Priority:* CRITICAL *Module:* `+HttpCapabilityGateway.Gateway+` + +*Implementation:* + +[source,elixir] +---- +defmodule HttpCapabilityGateway.Gateway do + @moduledoc """ + Plug-based HTTP gateway. + Enforces compiled rules on incoming requests. + Proxies allowed requests to backend. + Logs all decisions. + """ + + use Plug.Router + + plug :match + plug :dispatch + + # Match all methods and paths + match _ do + # Extract request metadata + # Lookup enforcement rule + # Make decision (allow/deny/stealth) + # Log decision with provenance + # Forward or reject + end +end +---- + +*Request Flow:* + +.... +Incoming Request + | + v +Extract Metadata (method, path, auth, trust_level) + | + v +Lookup Enforcement Rule (ETS) + | + v +Apply Exposure Policy + | + v +Check Stealth Profile + | + v +Make Decision (allow | deny | stealth_response) + | + v +Log Decision (JSON) + | + v +Forward to Backend (if allowed) | Return Error +.... + +*Decision Logic:* + +[source,elixir] +---- +def decide(request, rule) do + case {request.trust_level, rule.exposure} do + {_, :public} -> :allow + {:authenticated, :authenticated} -> :allow + {:internal, :internal} -> :allow + {:internal, :authenticated} -> :allow + {:authenticated, :public} -> :allow + _ -> apply_stealth(request, rule) + end +end +---- + +*Tasks:* - [ ] Implement Plug.Router for HTTP gateway - [ ] Extract +request metadata (method, path, headers) - [ ] Determine trust level +(public/authenticated/internal) - [ ] Lookup compiled rule from ETS - [ +] Implement decision logic with exposure levels - [ ] Implement stealth +profile responses - [ ] Forward allowed requests to backend (proxy) - [ +] Structure JSON logs (see section 1.6) - [ ] Handle backend errors +gracefully + +''''' + +==== 1.6 Structured Logging (4-6h) + +*Priority:* HIGH *Module:* `+HttpCapabilityGateway.DecisionLogger+` + +*Log Format (JSON):* + +[source,json] +---- +{ + "timestamp": "2026-01-22T20:30:00.123Z", + "service": "ledger-api", + "environment": "dev", + "request": { + "method": "DELETE", + "path": "/accounts/42", + "source_ip": "192.168.1.100", + "trust_level": "public" + }, + "rule": { + "exposure": "internal", + "narrative": "Account deletion requires internal trust." + }, + "decision": "deny", + "response": { + "status": 404, + "stealth_profile": "limited" + }, + "duration_ms": 1.23 +} +---- + +*Tasks:* - [ ] Define log schema (above) - [ ] Implement JSON encoder +for decisions - [ ] Integrate with Elixir Logger - [ ] Add Telemetry +metrics (decision counts, latency) - [ ] Support log levels (info for +allow, warn for deny) - [ ] Ensure no sensitive data in logs + +''''' + +==== 1.7 Configuration & Environment (2-3h) + +*Priority:* MEDIUM *Files:* `+config/config.exs+`, `+config/dev.exs+`, +`+config/prod.exs+` + +*Configuration Schema:* + +[source,elixir] +---- +config :http_capability_gateway, + # Policy + policy_path: "config/policy.yaml", + reload_on_change: false, # Hot reload (Phase 2) + + # HTTP Gateway + port: 8080, + backend_url: "http://localhost:8081", + timeout_ms: 5000, + + # Trust Detection + trust_header: "X-Trust-Level", # authenticated | internal + auth_header: "Authorization", + + # Logging + log_format: :json, + log_level: :info + +# Dev overrides +import_config "#{Mix.env()}.exs" +---- + +*Tasks:* - [ ] Define config schema - [ ] Implement env-specific +overrides (dev/prod) - [ ] Add config validation on startup - [ ] +Document all config options + +''''' + +==== 1.8 Backend Proxy (4-6h) + +*Priority:* HIGH *Module:* `+HttpCapabilityGateway.Proxy+` + +*Implementation:* + +[source,elixir] +---- +defmodule HttpCapabilityGateway.Proxy do + @moduledoc """ + Forwards allowed requests to backend service. + Preserves headers, body, and method. + Returns backend response to client. + """ + + @spec forward(conn :: Plug.Conn.t(), backend_url :: String.t()) :: Plug.Conn.t() + def forward(conn, backend_url) do + # Build backend request (method, path, headers, body) + # Send HTTP request to backend + # Stream response back to client + # Handle errors (timeout, connection refused) + end +end +---- + +*HTTP Client:* Use `+Req+` or `+Finch+` for backend requests + +*Tasks:* - [ ] Implement HTTP client for backend forwarding - [ ] +Preserve all headers from original request - [ ] Forward request body +(streaming for large payloads) - [ ] Stream response back to client - [ +] Handle backend timeouts - [ ] Handle backend connection errors - [ ] +Add retry logic (optional) + +''''' + +=== Phase 2: Testing & Quality (12-16h) + +==== 2.1 Unit Tests (6-8h) + +*Test Coverage:* - PolicyLoader - YAML parsing, error handling - +PolicyValidator - All validation rules, error messages - PolicyCompiler +- Rule compilation, ETS lookups, regex matching - Gateway - Decision +logic for all exposure/trust combinations - Proxy - Forwarding, error +handling + +*Tools:* ExUnit, Mox (for HTTP backend mocking) + +*Tasks:* - [ ] Test policy loader with valid/invalid YAML - [ ] Test +validator with valid/invalid policies - [ ] Test compiler regex matching +- [ ] Test gateway decision logic (all paths) - [ ] Test stealth profile +responses - [ ] Test proxy forwarding and error handling - [ ] Achieve +80%+ code coverage + +''''' + +==== 2.2 Integration Tests (4-6h) + +*Test Scenarios:* 1. Full flow: Load policy → Validate → Compile → +Enforce 2. Allowed request forwarded to backend 3. Denied request +returns correct stealth code 4. Route-specific override takes precedence +5. Backend timeout handling 6. Invalid policy rejection on startup + +*Tasks:* - [ ] Set up test backend (simple Plug app) - [ ] Test +end-to-end request flows - [ ] Test policy loading from file - [ ] Test +enforcement with real HTTP requests - [ ] Test log output (structured +JSON) + +''''' + +==== 2.3 Property Testing (Optional, 2-4h) + +*Library:* StreamData + +*Properties to Test:* - Any valid policy compiles without error - +Compiled rules match expected exposure - Decision logic is deterministic +- Stealth responses never leak internal info + +''''' + +=== Phase 3: Documentation (4-6h) + +==== 3.1 API Documentation (2-3h) + +*Tasks:* - [ ] Add @moduledoc to all modules - [ ] Add @doc to all +public functions - [ ] Add @spec for all function signatures - [ ] +Generate ExDoc documentation - [ ] Deploy docs to GitHub Pages + +''''' + +==== 3.2 User Guide (2-3h) + +*Sections:* - Quick Start (5 minutes) - Policy DSL Reference - +Configuration Options - Deployment Guide (systemd, Docker, K8s) - +Troubleshooting + +*Tasks:* - [ ] Write quick start guide - [ ] Document DSL syntax with +examples - [ ] Document all config options - [ ] Add deployment examples +- [ ] Add troubleshooting section + +''''' + +=== Phase 4: Production Readiness (8-12h) + +==== 4.1 Performance Optimization (4-6h) + +*Targets:* - < 1ms decision latency (p50) - < 5ms decision latency (p99) +- > 10,000 req/s throughput (single node) + +*Tasks:* - [ ] Benchmark ETS lookup performance - [ ] Benchmark regex +matching - [ ] Optimize hot paths - [ ] Add telemetry for latency +tracking - [ ] Load test with wrk/k6 + +''''' + +==== 4.2 Observability (2-3h) + +*Tasks:* - [ ] Add Prometheus metrics exporter - [ ] Add health check +endpoint (`+/health+`) - [ ] Add readiness check endpoint (`+/ready+`) - +[ ] Add metrics endpoint (`+/metrics+`) - [ ] Document metrics schema + +*Metrics to Export:* - +`+http_capability_gateway_decisions_total{decision, verb, exposure}+` +(counter) - `+http_capability_gateway_decision_duration_seconds+` +(histogram) - `+http_capability_gateway_backend_requests_total{status}+` +(counter) + +''''' + +==== 4.3 Containerization (2-3h) + +*Tasks:* - [ ] Create Dockerfile (multi-stage build) - [ ] Create +docker-compose.yml for local dev - [ ] Create Kubernetes manifests +(Deployment, Service, ConfigMap) - [ ] Document container deployment + +''''' + +=== Definition of Done (MVP v0.1.0) + +A feature is considered complete when: - ✅ Code implemented and +compiles - ✅ Unit tests written and passing - ✅ Integration tests +passing - ✅ Documentation updated (API docs + user guide) - ✅ +Benchmarked (meets performance targets) - ✅ Reviewed (at least 1 +reviewer) + +*MVP Completion Criteria:* - ✅ Policy loaded from YAML - ✅ Policy +validated against schema - ✅ Policy compiled to ETS rules - ✅ HTTP +gateway enforces rules - ✅ Decisions logged as JSON - ✅ Allowed +requests forwarded to backend - ✅ Denied requests return stealth codes +- ✅ Tests passing (80%+ coverage) - ✅ Documentation complete - ✅ +Performance targets met + +''''' + +=== Effort Estimate Summary + +[width="100%",cols="27%,48%,25%",options="header",] +|=== +|Phase |Description |Hours +|1. Foundation (MVP) |Elixir app, loader, validator, compiler, gateway, +proxy |40-50 + +|2. Testing & Quality |Unit tests, integration tests |12-16 + +|3. Documentation |API docs, user guide |4-6 + +|4. Production Readiness |Performance, observability, containers |8-12 + +|*TOTAL* |*MVP v0.1.0 (80-90% complete)* |*64-84 hours* +|=== + +*Revised Estimate (Focused MVP):* 40-60 hours *Priority:* Complete Phase +1 (Foundation) first → 40-50 hours + +''''' + +=== Dependencies + +*External:* - Elixir 1.15+ with OTP 26+ - Backend service to proxy to +(e.g., WordPress, API) + +*Internal (lcb-website stack):* - indieweb2-bastion - Consent portal +(provides trust levels via header) - svalinn - Edge gateway (sits in +front of capability gateway) - WordPress - Backend service + +*Integration Points:* - Receives requests from Svalinn - Reads +`+X-Trust-Level+` header from indieweb2-bastion - Forwards allowed +requests to WordPress/backend + +''''' + +=== Risk Assessment + +[width="100%",cols="17%,33%,20%,30%",options="header",] +|=== +|Risk |Probability |Impact |Mitigation +|Policy DSL too complex |Medium |High |Keep v1 minimal, defer complex +features + +|ETS performance issues |Low |High |Benchmark early, optimize compiler + +|Backend proxy errors |Medium |Medium |Comprehensive error handling, +retries + +|Regex DoS attacks |Medium |High |Validate patterns at compile time, set +limits + +|Integration with indieweb2-bastion |High |Medium |Define header +contract early, test integration +|=== + +''''' + +=== Next Steps (Immediate) + +[arabic] +. *Create Elixir scaffold* (2-3h) +* `+mix new http_capability_gateway --sup+` +* Add dependencies to `+mix.exs+` +* Set up directory structure +. *Implement PolicyLoader* (4-6h) +* Read and parse `+config/policy.yaml+` +* Return structured policy map +. *Implement PolicyValidator* (6-8h) +* Define JSON Schema for DSL v1 +* Validate policy against schema +* Test with valid/invalid policies +. *Implement PolicyCompiler* (8-12h) +* Compile policy to ETS rules +* Benchmark lookup performance +. *Implement Gateway + Proxy* (14-20h) +* HTTP gateway with Plug +* Decision logic +* Backend forwarding +* Structured logging + +*Target:* MVP v0.1.0 in 40-50 hours of focused work + +''''' + +=== Questions for Stakeholder + +[arabic] +. *Backend URL:* What’s the WordPress backend URL? (default: +`+http://localhost:8081+`) +. *Trust Header:* What header does indieweb2-bastion set? (assumed: +`+X-Trust-Level+`) +. *Port:* What port should the gateway listen on? (default: 8080) +. *Stealth Profiles:* Are the example stealth codes (404, 405) correct? +. *Narrative:* Should narrative be logged or just for documentation? + +''''' + +=== Conclusion + +*http-capability-gateway* is currently at *30% completion* (design +phase). Implementing the MVP will require *40-60 hours* of focused +Elixir development. The architecture is well-defined, and the scope is +minimal and achievable. The primary blocker is the *lack of +implementation* - all the design work is complete, but no code exists +yet. + +*Recommendation:* Prioritize this after completing Cerro Torre (95% → +100%) and before WordPress integration, as it’s a *critical frontend +component* for the hardened stack. diff --git a/IMPLEMENTATION-ROADMAP.md b/IMPLEMENTATION-ROADMAP.md deleted file mode 100644 index 2d74f87..0000000 --- a/IMPLEMENTATION-ROADMAP.md +++ /dev/null @@ -1,661 +0,0 @@ - -# http-capability-gateway - Implementation Roadmap - -> **HISTORICAL DOCUMENT (2026-04-16):** This document was written before any code existed and is no longer accurate. The repository now has 19 Elixir modules, 7 test files, 2 Zig FFI parsers, and 2 Idris2 ABI modules. The "What's Missing" list below is mostly **completed**. See `ROADMAP.adoc` and `STATE.adoc` for the current state. - -**Created:** 2026-01-22 -**Status at time of writing:** 30% (Design Phase) — **now ~55% with code implemented, verification lagging** -**Target:** MVP v0.1.0 -**Estimated Effort:** 40-60 hours - ---- - -## Current State - -### What Exists -- ✅ README with clear MVP scope -- ✅ Architecture design (Policy DSL → Elixir Gateway) -- ✅ Example policy.yaml structure -- ✅ Verb Governance Spec (DSL v1) defined -- ✅ Directory structure (contractiles/, docs/, config/) -- ✅ STATE.scm, ECOSYSTEM.scm files - -### What's Missing -- ❌ No Elixir application (`mix.exs`) -- ❌ No source code (`lib/` directory empty/missing) -- ❌ No policy loader implementation -- ❌ No validator (schema validation) -- ❌ No compiler (DSL → enforcement rules) -- ❌ No HTTP gateway (Plug/Cowboy) -- ❌ No enforcement engine -- ❌ No tests - ---- - -## Phase 1: Foundation (MVP) - 40-50 hours - -### 1.1 Elixir Application Scaffold (2-3h) -**Priority:** CRITICAL -**Files to Create:** -``` -http-capability-gateway/ -├── mix.exs # Mix project config -├── lib/ -│ └── http_capability_gateway/ -│ ├── application.ex # OTP application -│ └── gateway.ex # Main supervisor -└── config/ - ├── config.exs # Base config - ├── dev.exs # Dev environment - ├── prod.exs # Production environment - └── policy.yaml # ✓ Already exists -``` - -**Dependencies to Add:** -- `plug_cowboy` - HTTP server -- `jason` - JSON encoding/decoding -- `yaml_elixir` - YAML parser -- `telemetry` - Metrics/logging -- `ex_json_schema` - JSON Schema validation - -**Tasks:** -- [ ] Create `mix.exs` with dependencies -- [ ] Generate OTP application structure -- [ ] Configure logger (structured JSON output) -- [ ] Set up Telemetry for observability - ---- - -### 1.2 Policy Loader (4-6h) -**Priority:** CRITICAL -**Module:** `HttpCapabilityGateway.PolicyLoader` - -**Implementation:** -```elixir -defmodule HttpCapabilityGateway.PolicyLoader do - @moduledoc """ - Loads Verb Governance Spec (YAML) from disk. - Parses YAML into Elixir map structure. - """ - - @spec load_policy(path :: String.t()) :: {:ok, map()} | {:error, term()} - def load_policy(path) do - # Read YAML file - # Parse with YamlElixir - # Return structured policy map - end -end -``` - -**Schema to Parse:** -```yaml -service: - name: string - version: integer - environment: string - -verbs: - GET: {exposure: public | authenticated | internal} - POST: {exposure: public | authenticated | internal} - PUT: {exposure: public | authenticated | internal} - DELETE: {exposure: public | authenticated | internal} - PATCH: {exposure: public | authenticated | internal} - HEAD: {exposure: public | authenticated | internal} - OPTIONS: {exposure: public | authenticated | internal} - -routes: - - path: string (regex-capable) - verbs: {verb-specific overrides with narrative} - -stealth: - profiles: - limited: - unauthenticated: 401 | 403 | 404 | 405 - untrusted: 401 | 403 | 404 | 405 - -narrative: - purpose: string -``` - -**Tasks:** -- [ ] Implement YAML file reading -- [ ] Parse service metadata -- [ ] Parse verb exposure levels -- [ ] Parse route-specific overrides -- [ ] Parse stealth profiles -- [ ] Handle file errors gracefully - ---- - -### 1.3 Policy Validator (6-8h) -**Priority:** CRITICAL -**Module:** `HttpCapabilityGateway.PolicyValidator` - -**Implementation:** -```elixir -defmodule HttpCapabilityGateway.PolicyValidator do - @moduledoc """ - Validates loaded policy against JSON Schema. - Ensures all required fields present. - Checks exposure levels are valid. - Validates route paths are valid regexes. - """ - - @spec validate(policy :: map()) :: :ok | {:error, [validation_error()]} - def validate(policy) do - # Validate service metadata - # Validate verb exposure levels - # Validate route paths (regex compilation) - # Validate stealth profile codes - # Return aggregated errors - end -end -``` - -**Validation Rules:** -- `service.name` - required, non-empty string -- `service.version` - required, positive integer -- `service.environment` - required, one of: dev, staging, prod -- `verbs..exposure` - required, one of: public, authenticated, internal -- `routes[].path` - valid regex pattern -- `stealth.profiles..` - valid HTTP status code (401-599) - -**Tasks:** -- [ ] Define JSON Schema for DSL v1 -- [ ] Implement schema validation with ex_json_schema -- [ ] Validate regex patterns compile -- [ ] Validate exposure levels are recognized -- [ ] Validate stealth codes are valid HTTP codes -- [ ] Generate human-readable error messages - ---- - -### 1.4 Policy Compiler (8-12h) -**Priority:** CRITICAL -**Module:** `HttpCapabilityGateway.PolicyCompiler` - -**Implementation:** -```elixir -defmodule HttpCapabilityGateway.PolicyCompiler do - @moduledoc """ - Compiles DSL policy into fast enforcement rules. - Builds ETS table with compiled patterns for O(1) lookups. - """ - - defmodule CompiledRule do - @moduledoc "Single enforcement rule" - defstruct [ - :path_regex, # Compiled Regex for route matching - :verb, # HTTP method atom (:get, :post, etc.) - :exposure, # :public | :authenticated | :internal - :stealth_profile, # :none | :limited (from policy) - :narrative # Optional explanation string - ] - end - - @spec compile(policy :: map()) :: {:ok, :ets.tid()} | {:error, term()} - def compile(policy) do - # Compile global verb rules - # Compile route-specific overrides (take precedence) - # Store in ETS table: {path_pattern, verb} => CompiledRule - # Return ETS table reference - end -end -``` - -**Compilation Strategy:** -1. Parse global verb rules (apply to all routes) -2. Parse route-specific overrides (highest precedence) -3. Compile regex patterns for efficient matching -4. Store in ETS table for fast concurrent lookups -5. Return ETS table reference for enforcement - -**ETS Schema:** -``` -Key: {path_regex, verb_atom} -Value: %CompiledRule{...} -``` - -**Tasks:** -- [ ] Implement global verb rule compilation -- [ ] Implement route-specific override compilation -- [ ] Compile regex patterns with error handling -- [ ] Create ETS table with appropriate options (public, read_concurrency) -- [ ] Handle pattern conflicts (routes should override global) -- [ ] Benchmark lookup performance - ---- - -### 1.5 HTTP Gateway (10-14h) -**Priority:** CRITICAL -**Module:** `HttpCapabilityGateway.Gateway` - -**Implementation:** -```elixir -defmodule HttpCapabilityGateway.Gateway do - @moduledoc """ - Plug-based HTTP gateway. - Enforces compiled rules on incoming requests. - Proxies allowed requests to backend. - Logs all decisions. - """ - - use Plug.Router - - plug :match - plug :dispatch - - # Match all methods and paths - match _ do - # Extract request metadata - # Lookup enforcement rule - # Make decision (allow/deny/stealth) - # Log decision with provenance - # Forward or reject - end -end -``` - -**Request Flow:** -``` -Incoming Request - | - v -Extract Metadata (method, path, auth, trust_level) - | - v -Lookup Enforcement Rule (ETS) - | - v -Apply Exposure Policy - | - v -Check Stealth Profile - | - v -Make Decision (allow | deny | stealth_response) - | - v -Log Decision (JSON) - | - v -Forward to Backend (if allowed) | Return Error -``` - -**Decision Logic:** -```elixir -def decide(request, rule) do - case {request.trust_level, rule.exposure} do - {_, :public} -> :allow - {:authenticated, :authenticated} -> :allow - {:internal, :internal} -> :allow - {:internal, :authenticated} -> :allow - {:authenticated, :public} -> :allow - _ -> apply_stealth(request, rule) - end -end -``` - -**Tasks:** -- [ ] Implement Plug.Router for HTTP gateway -- [ ] Extract request metadata (method, path, headers) -- [ ] Determine trust level (public/authenticated/internal) -- [ ] Lookup compiled rule from ETS -- [ ] Implement decision logic with exposure levels -- [ ] Implement stealth profile responses -- [ ] Forward allowed requests to backend (proxy) -- [ ] Structure JSON logs (see section 1.6) -- [ ] Handle backend errors gracefully - ---- - -### 1.6 Structured Logging (4-6h) -**Priority:** HIGH -**Module:** `HttpCapabilityGateway.DecisionLogger` - -**Log Format (JSON):** -```json -{ - "timestamp": "2026-01-22T20:30:00.123Z", - "service": "ledger-api", - "environment": "dev", - "request": { - "method": "DELETE", - "path": "/accounts/42", - "source_ip": "192.168.1.100", - "trust_level": "public" - }, - "rule": { - "exposure": "internal", - "narrative": "Account deletion requires internal trust." - }, - "decision": "deny", - "response": { - "status": 404, - "stealth_profile": "limited" - }, - "duration_ms": 1.23 -} -``` - -**Tasks:** -- [ ] Define log schema (above) -- [ ] Implement JSON encoder for decisions -- [ ] Integrate with Elixir Logger -- [ ] Add Telemetry metrics (decision counts, latency) -- [ ] Support log levels (info for allow, warn for deny) -- [ ] Ensure no sensitive data in logs - ---- - -### 1.7 Configuration & Environment (2-3h) -**Priority:** MEDIUM -**Files:** `config/config.exs`, `config/dev.exs`, `config/prod.exs` - -**Configuration Schema:** -```elixir -config :http_capability_gateway, - # Policy - policy_path: "config/policy.yaml", - reload_on_change: false, # Hot reload (Phase 2) - - # HTTP Gateway - port: 8080, - backend_url: "http://localhost:8081", - timeout_ms: 5000, - - # Trust Detection - trust_header: "X-Trust-Level", # authenticated | internal - auth_header: "Authorization", - - # Logging - log_format: :json, - log_level: :info - -# Dev overrides -import_config "#{Mix.env()}.exs" -``` - -**Tasks:** -- [ ] Define config schema -- [ ] Implement env-specific overrides (dev/prod) -- [ ] Add config validation on startup -- [ ] Document all config options - ---- - -### 1.8 Backend Proxy (4-6h) -**Priority:** HIGH -**Module:** `HttpCapabilityGateway.Proxy` - -**Implementation:** -```elixir -defmodule HttpCapabilityGateway.Proxy do - @moduledoc """ - Forwards allowed requests to backend service. - Preserves headers, body, and method. - Returns backend response to client. - """ - - @spec forward(conn :: Plug.Conn.t(), backend_url :: String.t()) :: Plug.Conn.t() - def forward(conn, backend_url) do - # Build backend request (method, path, headers, body) - # Send HTTP request to backend - # Stream response back to client - # Handle errors (timeout, connection refused) - end -end -``` - -**HTTP Client:** Use `Req` or `Finch` for backend requests - -**Tasks:** -- [ ] Implement HTTP client for backend forwarding -- [ ] Preserve all headers from original request -- [ ] Forward request body (streaming for large payloads) -- [ ] Stream response back to client -- [ ] Handle backend timeouts -- [ ] Handle backend connection errors -- [ ] Add retry logic (optional) - ---- - -## Phase 2: Testing & Quality (12-16h) - -### 2.1 Unit Tests (6-8h) -**Test Coverage:** -- PolicyLoader - YAML parsing, error handling -- PolicyValidator - All validation rules, error messages -- PolicyCompiler - Rule compilation, ETS lookups, regex matching -- Gateway - Decision logic for all exposure/trust combinations -- Proxy - Forwarding, error handling - -**Tools:** ExUnit, Mox (for HTTP backend mocking) - -**Tasks:** -- [ ] Test policy loader with valid/invalid YAML -- [ ] Test validator with valid/invalid policies -- [ ] Test compiler regex matching -- [ ] Test gateway decision logic (all paths) -- [ ] Test stealth profile responses -- [ ] Test proxy forwarding and error handling -- [ ] Achieve 80%+ code coverage - ---- - -### 2.2 Integration Tests (4-6h) -**Test Scenarios:** -1. Full flow: Load policy → Validate → Compile → Enforce -2. Allowed request forwarded to backend -3. Denied request returns correct stealth code -4. Route-specific override takes precedence -5. Backend timeout handling -6. Invalid policy rejection on startup - -**Tasks:** -- [ ] Set up test backend (simple Plug app) -- [ ] Test end-to-end request flows -- [ ] Test policy loading from file -- [ ] Test enforcement with real HTTP requests -- [ ] Test log output (structured JSON) - ---- - -### 2.3 Property Testing (Optional, 2-4h) -**Library:** StreamData - -**Properties to Test:** -- Any valid policy compiles without error -- Compiled rules match expected exposure -- Decision logic is deterministic -- Stealth responses never leak internal info - ---- - -## Phase 3: Documentation (4-6h) - -### 3.1 API Documentation (2-3h) -**Tasks:** -- [ ] Add @moduledoc to all modules -- [ ] Add @doc to all public functions -- [ ] Add @spec for all function signatures -- [ ] Generate ExDoc documentation -- [ ] Deploy docs to GitHub Pages - ---- - -### 3.2 User Guide (2-3h) -**Sections:** -- Quick Start (5 minutes) -- Policy DSL Reference -- Configuration Options -- Deployment Guide (systemd, Docker, K8s) -- Troubleshooting - -**Tasks:** -- [ ] Write quick start guide -- [ ] Document DSL syntax with examples -- [ ] Document all config options -- [ ] Add deployment examples -- [ ] Add troubleshooting section - ---- - -## Phase 4: Production Readiness (8-12h) - -### 4.1 Performance Optimization (4-6h) -**Targets:** -- < 1ms decision latency (p50) -- < 5ms decision latency (p99) -- > 10,000 req/s throughput (single node) - -**Tasks:** -- [ ] Benchmark ETS lookup performance -- [ ] Benchmark regex matching -- [ ] Optimize hot paths -- [ ] Add telemetry for latency tracking -- [ ] Load test with wrk/k6 - ---- - -### 4.2 Observability (2-3h) -**Tasks:** -- [ ] Add Prometheus metrics exporter -- [ ] Add health check endpoint (`/health`) -- [ ] Add readiness check endpoint (`/ready`) -- [ ] Add metrics endpoint (`/metrics`) -- [ ] Document metrics schema - -**Metrics to Export:** -- `http_capability_gateway_decisions_total{decision, verb, exposure}` (counter) -- `http_capability_gateway_decision_duration_seconds` (histogram) -- `http_capability_gateway_backend_requests_total{status}` (counter) - ---- - -### 4.3 Containerization (2-3h) -**Tasks:** -- [ ] Create Dockerfile (multi-stage build) -- [ ] Create docker-compose.yml for local dev -- [ ] Create Kubernetes manifests (Deployment, Service, ConfigMap) -- [ ] Document container deployment - ---- - -## Definition of Done (MVP v0.1.0) - -A feature is considered complete when: -- ✅ Code implemented and compiles -- ✅ Unit tests written and passing -- ✅ Integration tests passing -- ✅ Documentation updated (API docs + user guide) -- ✅ Benchmarked (meets performance targets) -- ✅ Reviewed (at least 1 reviewer) - -**MVP Completion Criteria:** -- ✅ Policy loaded from YAML -- ✅ Policy validated against schema -- ✅ Policy compiled to ETS rules -- ✅ HTTP gateway enforces rules -- ✅ Decisions logged as JSON -- ✅ Allowed requests forwarded to backend -- ✅ Denied requests return stealth codes -- ✅ Tests passing (80%+ coverage) -- ✅ Documentation complete -- ✅ Performance targets met - ---- - -## Effort Estimate Summary - -| Phase | Description | Hours | -|-------|-------------|-------| -| 1. Foundation (MVP) | Elixir app, loader, validator, compiler, gateway, proxy | 40-50 | -| 2. Testing & Quality | Unit tests, integration tests | 12-16 | -| 3. Documentation | API docs, user guide | 4-6 | -| 4. Production Readiness | Performance, observability, containers | 8-12 | -| **TOTAL** | **MVP v0.1.0 (80-90% complete)** | **64-84 hours** | - -**Revised Estimate (Focused MVP):** 40-60 hours -**Priority:** Complete Phase 1 (Foundation) first → 40-50 hours - ---- - -## Dependencies - -**External:** -- Elixir 1.15+ with OTP 26+ -- Backend service to proxy to (e.g., WordPress, API) - -**Internal (lcb-website stack):** -- indieweb2-bastion - Consent portal (provides trust levels via header) -- svalinn - Edge gateway (sits in front of capability gateway) -- WordPress - Backend service - -**Integration Points:** -- Receives requests from Svalinn -- Reads `X-Trust-Level` header from indieweb2-bastion -- Forwards allowed requests to WordPress/backend - ---- - -## Risk Assessment - -| Risk | Probability | Impact | Mitigation | -|------|-------------|--------|------------| -| Policy DSL too complex | Medium | High | Keep v1 minimal, defer complex features | -| ETS performance issues | Low | High | Benchmark early, optimize compiler | -| Backend proxy errors | Medium | Medium | Comprehensive error handling, retries | -| Regex DoS attacks | Medium | High | Validate patterns at compile time, set limits | -| Integration with indieweb2-bastion | High | Medium | Define header contract early, test integration | - ---- - -## Next Steps (Immediate) - -1. **Create Elixir scaffold** (2-3h) - - `mix new http_capability_gateway --sup` - - Add dependencies to `mix.exs` - - Set up directory structure - -2. **Implement PolicyLoader** (4-6h) - - Read and parse `config/policy.yaml` - - Return structured policy map - -3. **Implement PolicyValidator** (6-8h) - - Define JSON Schema for DSL v1 - - Validate policy against schema - - Test with valid/invalid policies - -4. **Implement PolicyCompiler** (8-12h) - - Compile policy to ETS rules - - Benchmark lookup performance - -5. **Implement Gateway + Proxy** (14-20h) - - HTTP gateway with Plug - - Decision logic - - Backend forwarding - - Structured logging - -**Target:** MVP v0.1.0 in 40-50 hours of focused work - ---- - -## Questions for Stakeholder - -1. **Backend URL:** What's the WordPress backend URL? (default: `http://localhost:8081`) -2. **Trust Header:** What header does indieweb2-bastion set? (assumed: `X-Trust-Level`) -3. **Port:** What port should the gateway listen on? (default: 8080) -4. **Stealth Profiles:** Are the example stealth codes (404, 405) correct? -5. **Narrative:** Should narrative be logged or just for documentation? - ---- - -## Conclusion - -**http-capability-gateway** is currently at **30% completion** (design phase). Implementing the MVP will require **40-60 hours** of focused Elixir development. The architecture is well-defined, and the scope is minimal and achievable. The primary blocker is the **lack of implementation** - all the design work is complete, but no code exists yet. - -**Recommendation:** Prioritize this after completing Cerro Torre (95% → 100%) and before WordPress integration, as it's a **critical frontend component** for the hardened stack. diff --git a/MULTI-PROTOCOL.md b/MULTI-PROTOCOL.adoc similarity index 71% rename from MULTI-PROTOCOL.md rename to MULTI-PROTOCOL.adoc index c5a9928..81811fb 100644 --- a/MULTI-PROTOCOL.md +++ b/MULTI-PROTOCOL.adoc @@ -1,25 +1,25 @@ - -# Multi-Protocol API Gateway - -> **⚠️ SCOPE NOTICE (2026-04-16):** This document describes the vision for -> multi-protocol support. Today, only **HTTP/REST** is production-ready. gRPC -> and GraphQL handlers exist but are **stubs**: `GraphQLHandler.check_operation_policy/2` -> always returns true, and `GRPCHandler.forward_grpc_request/5` returns a -> hardcoded response with no actual forwarding. Do not route gRPC or GraphQL -> traffic through this gateway in production. -> -> See `docs/SUPPORTED-FEATURES.md` for the authoritative status of each -> protocol, and `ROADMAP.adoc` for the MVP scope. - -The HTTP Capability Gateway aims to support multiple protocols: HTTP/REST, gRPC, and GraphQL. -Only HTTP/REST is currently supported end-to-end. - -## Architecture - -``` +== Multi-Protocol API Gateway + +____ +*⚠️ SCOPE NOTICE (2026-04-16):* This document describes the vision for +multi-protocol support. Today, only *HTTP/REST* is production-ready. +gRPC and GraphQL handlers exist but are *stubs*: +`+GraphQLHandler.check_operation_policy/2+` always returns true, and +`+GRPCHandler.forward_grpc_request/5+` returns a hardcoded response with +no actual forwarding. Do not route gRPC or GraphQL traffic through this +gateway in production. + +See `+docs/SUPPORTED-FEATURES.md+` for the authoritative status of each +protocol, and `+ROADMAP.adoc+` for the MVP scope. +____ + +The HTTP Capability Gateway aims to support multiple protocols: +HTTP/REST, gRPC, and GraphQL. Only HTTP/REST is currently supported +end-to-end. + +=== Architecture + +.... ┌─────────────────────────────────────────────────────────────┐ │ Idris2 ABI Layer │ │ (src/abi/Protocol.idr, Types.idr) │ @@ -50,21 +50,23 @@ Only HTTP/REST is currently supported end-to-end. │ │(HTTP) │ │Handler│ │Handler │ │ │ │ │ └─────────┘ └──────┘ └────────┘ └──────┘ │ └────────────────────────────────────────────────────────────┘ -``` +.... -## Protocol Detection +=== Protocol Detection Automatic protocol detection based on request characteristics: -1. **GraphQL**: POST to `/graphql` endpoint -2. **gRPC**: HTTP/2 with `content-type: application/grpc*` -3. **HTTP/REST**: Everything else (default) +[arabic] +. *GraphQL*: POST to `+/graphql+` endpoint +. *gRPC*: HTTP/2 with `+content-type: application/grpc*+` +. *HTTP/REST*: Everything else (default) -## Configuration +=== Configuration -Add to `config/config.exs`: +Add to `+config/config.exs+`: -```elixir +[source,elixir] +---- config :http_capability_gateway, # Enable multi-protocol support enable_grpc: true, @@ -77,13 +79,14 @@ config :http_capability_gateway, # Policy enforcement grpc_methods_policy: "config/grpc-policy.yaml", graphql_operations_policy: "config/graphql-policy.yaml" -``` +---- -## Policy Examples +=== Policy Examples -### gRPC Policy +==== gRPC Policy -```yaml +[source,yaml] +---- # config/grpc-policy.yaml dsl_version: "1" @@ -102,11 +105,12 @@ grpc_methods: - service: "myapp.AdminService" method: "*" # All methods exposure: "internal" -``` +---- -### GraphQL Policy +==== GraphQL Policy -```yaml +[source,yaml] +---- # config/graphql-policy.yaml dsl_version: "1" @@ -123,13 +127,15 @@ graphql: blocked_operations: - "AdminMutation" - "DangerousQuery" -``` +---- -## Building FFI Layer +=== Building FFI Layer -The Zig FFI layer must be compiled before the gateway can handle gRPC/GraphQL: +The Zig FFI layer must be compiled before the gateway can handle +gRPC/GraphQL: -```bash +[source,bash] +---- cd ffi/zig zig build @@ -138,11 +144,12 @@ zig build test # Install library sudo cp zig-out/lib/libgateway.so /usr/local/lib/ -``` +---- -## Testing +=== Testing -```bash +[source,bash] +---- # Test gRPC endpoint grpcurl -plaintext -d '{"service":"health"}' localhost:4000 grpc.health.v1.Health/Check @@ -153,29 +160,28 @@ curl -X POST http://localhost:4000/graphql \ # Test HTTP/REST (existing) curl http://localhost:4000/api/users -``` +---- -## Performance +=== Performance -Protocol parsing performance (Zig FFI): -- **gRPC**: ~5μs per request -- **GraphQL**: ~10μs per query -- **HTTP**: ~2μs per request +Protocol parsing performance (Zig FFI): - *gRPC*: ~5μs per request - +*GraphQL*: ~10μs per query - *HTTP*: ~2μs per request All parsers are memory-safe with zero-copy parsing where possible. -## Security +=== Security -- **Type Safety**: Idris2 dependent types prove correctness -- **Memory Safety**: Zig compile-time checks prevent buffer overflows -- **Policy Enforcement**: All protocols subject to verb governance -- **Stealth Mode**: Supported for all protocols (custom error codes) +* *Type Safety*: Idris2 dependent types prove correctness +* *Memory Safety*: Zig compile-time checks prevent buffer overflows +* *Policy Enforcement*: All protocols subject to verb governance +* *Stealth Mode*: Supported for all protocols (custom error codes) -## Limitations +=== Limitations -Current implementation is in **beta**: -- gRPC: HTTP/2 frame parsing stub (TODO: full HPACK decoder) -- GraphQL: Basic operation detection (TODO: full AST parsing) -- Policy integration: Stubs in place (TODO: wire to PolicyCompiler) +Current implementation is in *beta*: - gRPC: HTTP/2 frame parsing stub +(TODO: full HPACK decoder) - GraphQL: Basic operation detection (TODO: +full AST parsing) - Policy integration: Stubs in place (TODO: wire to +PolicyCompiler) -See `ffi/zig/README.md` and `src/abi/README.md` for implementation details. +See `+ffi/zig/README.md+` and `+src/abi/README.md+` for implementation +details. diff --git a/PROOF-NEEDS.adoc b/PROOF-NEEDS.adoc new file mode 100644 index 0000000..c1ace01 --- /dev/null +++ b/PROOF-NEEDS.adoc @@ -0,0 +1,42 @@ +== SPDX-License-Identifier: CC-BY-SA-4.0 + +== Copyright (c) Jonathan D.A. Jewell j.d.a.jewell@open.ac.uk + +== Proof Needs (echo-types audit) + +This file records cross-repo proof obligations relevant to this +repository, including an explicit echo-types audit per estate convention +(`+feedback_proofs_must_check_and_cross_doc_echo_types.md+`). + +=== Echo-types + +*Status: record-as-not-relevant (2026-06-02).* + +`+hyperpolymath/echo-types+` was audited for relevant existing proofs +that this repository should reuse or extend. Findings: + +* The trust hierarchy already has a mechanised proof in +`+proven/SafeTrust.idr+` (referenced from +`+lib/http_capability_gateway/safe_trust.ex+`). +* No L3 (echo) obligation is in scope for the gateway: the gateway does +not participate in any echo protocol; it terminates inbound HTTP and +forwards to a single backend. +* The new `+EgressPolicy+` module’s decision function is a pure +first-order predicate (host + verb membership in an allowlist). It does +not interact with echo types. + +*Action:* none. Re-check this entry when egress acquires a +chimichanga-attenuation seam (#84-3 follow-up); attenuation may bring an +L3 obligation into scope at that point. + +=== SafeTrust monotonicity + +Already mechanised in `+proven/SafeTrust.idr+` (Idris2). The Elixir +implementation in `+lib/http_capability_gateway/safe_trust.ex+` mirrors +the specification one-to-one. No new proof debt added by this PR. + +=== EgressPolicy + +The egress policy decision function `+EgressPolicy.decide/3+` is small +enough to mechanise (membership in a list of host+verb pairs). Filing as +proof debt for a future PR; not blocking egress scaffold landing. diff --git a/PROOF-NEEDS.md b/PROOF-NEEDS.md deleted file mode 100644 index 1b86a55..0000000 --- a/PROOF-NEEDS.md +++ /dev/null @@ -1,40 +0,0 @@ -# SPDX-License-Identifier: CC-BY-SA-4.0 -# Copyright (c) Jonathan D.A. Jewell - -# Proof Needs (echo-types audit) - -This file records cross-repo proof obligations relevant to this repository, -including an explicit echo-types audit per estate convention -(`feedback_proofs_must_check_and_cross_doc_echo_types.md`). - -## Echo-types - -**Status: record-as-not-relevant (2026-06-02).** - -`hyperpolymath/echo-types` was audited for relevant existing proofs that -this repository should reuse or extend. Findings: - -- The trust hierarchy already has a mechanised proof in - `proven/SafeTrust.idr` (referenced from `lib/http_capability_gateway/safe_trust.ex`). -- No L3 (echo) obligation is in scope for the gateway: the gateway does not - participate in any echo protocol; it terminates inbound HTTP and forwards - to a single backend. -- The new `EgressPolicy` module's decision function is a pure first-order - predicate (host + verb membership in an allowlist). It does not interact - with echo types. - -**Action:** none. Re-check this entry when egress acquires a -chimichanga-attenuation seam (#84-3 follow-up); attenuation may bring an -L3 obligation into scope at that point. - -## SafeTrust monotonicity - -Already mechanised in `proven/SafeTrust.idr` (Idris2). The Elixir -implementation in `lib/http_capability_gateway/safe_trust.ex` mirrors the -specification one-to-one. No new proof debt added by this PR. - -## EgressPolicy - -The egress policy decision function `EgressPolicy.decide/3` is small enough -to mechanise (membership in a list of host+verb pairs). Filing as proof -debt for a future PR; not blocking egress scaffold landing. diff --git a/PROOFS_DONE.adoc b/PROOFS_DONE.adoc new file mode 100644 index 0000000..8b13789 --- /dev/null +++ b/PROOFS_DONE.adoc @@ -0,0 +1 @@ + diff --git a/PROOFS_DONE.md b/PROOFS_DONE.md deleted file mode 100644 index 2cbfe79..0000000 --- a/PROOFS_DONE.md +++ /dev/null @@ -1,4 +0,0 @@ - diff --git a/PROOFS_NEEDED.adoc b/PROOFS_NEEDED.adoc new file mode 100644 index 0000000..7ac1f88 --- /dev/null +++ b/PROOFS_NEEDED.adoc @@ -0,0 +1,81 @@ +== PROOF-NEEDS.md — http-capability-gateway + +=== Current State + +* **src/abi/*.idr**: YES — `+Protocol.idr+`, `+Types.idr+`, +`+MTLSPolicy.idr+` (obligation stub) +* *Dangerous patterns*: 0 (`+believe_me+` reference in Protocol.idr is +documentation only) +* *LOC*: ~9,500 +* *ABI layer*: Idris2 definitions present + +=== What Needs Proving + +[width="100%",cols="51%,27%,22%",options="header",] +|=== +|Component |What |Why +|Capability token validation |Token issuance/revocation correctness +|Security-critical: malformed tokens bypass access control + +|Protocol state machine |Session state transitions are total |Prevent +stuck/invalid protocol states + +|Permission composition |Capability intersection/union laws |Ensure +composed permissions don’t escalate + +|ABI type safety |FFI boundary type marshalling correctness |Prevent +memory corruption at language boundary + +|mTLS trust policy |An unverified client cert is never mapped to a +privileged trust class (`+classify CertUnverified _ = Untrusted+`) +|Security core of Phase B mTLS-as-primary-path: forged/unverified certs +must be indistinguishable from anonymous +|=== + +==== mTLS trust policy (Phase B / standards#97) + +* *Claim stated:* `+src/abi/MTLSPolicy.idr+` — +`+unverifiedNeverPrivileged+`. +* *Status:* PENDING (scheduled for Phase C/D — standards#98/#99). The +obligation is recorded as a `+0+`-multiplicity hole; not yet discharged +and intentionally excluded from `+gateway.ipkg modules+` so it does not +gate the build before discharge. +* *Mirrors:* the runtime decision in +`+Gateway.determine_trust_level_from_cert/2+` and the listener +fail-closed contract in `+HttpCapabilityGateway.Application+`. + +=== Recommended Prover + +*Idris2* — ABI layer already in Idris2; extend existing `+Protocol.idr+` +and `+Types.idr+` with dependent type proofs for capability correctness. + +=== Priority + +*MEDIUM* — Has working ABI layer with clean Idris2 code. Main gap is +proving capability composition and protocol state transitions. + +=== Open contract surfaces (2026-06-02 self-audit, #31) + +The audit issue #31 (priority 3) calls out two integration gaps that are +not yet code but want to be tracked as proof obligations: + +* *chimichanga capability attenuation*: when the `+capability+` field +(PR #33) acquires a partial-order lattice via +`+hyperpolymath/chimichanga+`, the attenuation predicate +`+client_capability ≤ rule.capability+` becomes a new proof obligation +alongside the existing trust-total-order proof in +`+proven/SafeTrust.idr+`. See `+docs/CAPABILITY-INTEGRATION.md+` §2. +* *groove-protocol service discovery*: resolving a backend URL via +service discovery does NOT itself need a proof, but the _fallback +contract_ ("`on resolution failure, return 503 — NEVER fall back to a +static URL`") wants a mechanised statement so future refactors do not +introduce a silent trust-downgrade. See +`+docs/CAPABILITY-INTEGRATION.md+` §3. + +==== Echo-types audit (2026-06-02) + +Per estate convention +(`+feedback_proofs_must_check_and_cross_doc_echo_types.md+`), echo-types +was audited. *Status: record-as-not-relevant* for the current scaffold. +Re-check this entry when chimichanga attenuation lands: the lattice may +introduce an L3 obligation at that point. diff --git a/PROOFS_NEEDED.md b/PROOFS_NEEDED.md deleted file mode 100644 index a625c1e..0000000 --- a/PROOFS_NEEDED.md +++ /dev/null @@ -1,65 +0,0 @@ - -# PROOF-NEEDS.md — http-capability-gateway - -## Current State - -- **src/abi/*.idr**: YES — `Protocol.idr`, `Types.idr`, `MTLSPolicy.idr` (obligation stub) -- **Dangerous patterns**: 0 (`believe_me` reference in Protocol.idr is documentation only) -- **LOC**: ~9,500 -- **ABI layer**: Idris2 definitions present - -## What Needs Proving - -| Component | What | Why | -|-----------|------|-----| -| Capability token validation | Token issuance/revocation correctness | Security-critical: malformed tokens bypass access control | -| Protocol state machine | Session state transitions are total | Prevent stuck/invalid protocol states | -| Permission composition | Capability intersection/union laws | Ensure composed permissions don't escalate | -| ABI type safety | FFI boundary type marshalling correctness | Prevent memory corruption at language boundary | -| mTLS trust policy | An unverified client cert is never mapped to a privileged trust class (`classify CertUnverified _ = Untrusted`) | Security core of Phase B mTLS-as-primary-path: forged/unverified certs must be indistinguishable from anonymous | - -### mTLS trust policy (Phase B / standards#97) - -- **Claim stated:** `src/abi/MTLSPolicy.idr` — `unverifiedNeverPrivileged`. -- **Status:** PENDING (scheduled for Phase C/D — standards#98/#99). The - obligation is recorded as a `0`-multiplicity hole; not yet discharged and - intentionally excluded from `gateway.ipkg modules` so it does not gate the - build before discharge. -- **Mirrors:** the runtime decision in - `Gateway.determine_trust_level_from_cert/2` and the listener fail-closed - contract in `HttpCapabilityGateway.Application`. - -## Recommended Prover - -**Idris2** — ABI layer already in Idris2; extend existing `Protocol.idr` and `Types.idr` with dependent type proofs for capability correctness. - -## Priority - -**MEDIUM** — Has working ABI layer with clean Idris2 code. Main gap is proving capability composition and protocol state transitions. - -## Open contract surfaces (2026-06-02 self-audit, #31) - -The audit issue #31 (priority 3) calls out two integration gaps that are -not yet code but want to be tracked as proof obligations: - -- **chimichanga capability attenuation**: when the `capability` field - (PR #33) acquires a partial-order lattice via `hyperpolymath/chimichanga`, - the attenuation predicate `client_capability ≤ rule.capability` becomes - a new proof obligation alongside the existing trust-total-order proof in - `proven/SafeTrust.idr`. See `docs/CAPABILITY-INTEGRATION.md` §2. -- **groove-protocol service discovery**: resolving a backend URL via - service discovery does NOT itself need a proof, but the *fallback - contract* ("on resolution failure, return 503 — NEVER fall back to a - static URL") wants a mechanised statement so future refactors do not - introduce a silent trust-downgrade. See - `docs/CAPABILITY-INTEGRATION.md` §3. - -### Echo-types audit (2026-06-02) - -Per estate convention (`feedback_proofs_must_check_and_cross_doc_echo_types.md`), -echo-types was audited. **Status: record-as-not-relevant** for the current -scaffold. Re-check this entry when chimichanga attenuation lands: the -lattice may introduce an L3 obligation at that point. diff --git a/README.adoc b/README.adoc new file mode 100644 index 0000000..8010064 --- /dev/null +++ b/README.adoc @@ -0,0 +1,498 @@ +Jonathan D.A. Jewell :toc: preamble :icons: +font :sectnums: + +https://opensource.org/licenses/MPL-2.0[image:https://img.shields.io/badge/License-MPL_2.0-blue.svg[License: +MPL-2.0]] +image:https://img.shields.io/badge/Elixir-1.19+-purple.svg[Elixir 1.19+] +image:https://img.shields.io/badge/OTP-27+-red.svg[OTP 27+] + +== Overview + +http-capability-gateway is a lightweight, policy-driven HTTP governance +layer that enforces a declarative, auditable model of HTTP verb exposure +in front of existing services. + +This project introduces a minimal viable implementation of a new +category: a capability gateway for HTTP. It does not replace nginx or +Apache. Instead, it governs what they are allowed to do. + +The gateway loads a Verb Governance Spec (DSL v1), validates it, +compiles it into fast enforcement rules, and applies those rules to real +HTTP traffic. Every decision is logged in structured form for audit and +introspection. + +=== Current Status + +This repository contains a real Elixir gateway implementation, but it +should currently be treated as a narrow, in-progress API governance +layer rather than a fully proven front door for an entire site. + +* The core policy pipeline exists: loader, validator, compiler, gateway, +proxy, telemetry. +* The main remaining gaps are security depth, end-to-end verification, +and benchmark evidence. +* Read `+ROADMAP.adoc+`, `+TEST-NEEDS.md+`, and `+PROOFS_NEEDED.md+` +together when judging readiness. + +It provides: + +* *Declarative Verb Governance*: Define allowed HTTP verbs globally and +per-route +* *Stealth Mode*: Return configurable status codes (404, 403, etc.) for +unauthorized requests +* *Fast Policy Enforcement Architecture*: ETS-backed lookups and +compiled policy rules; benchmark evidence still needs to be formalized +* *Trust Level Integration*: Current implementation is header-based, +with mTLS-oriented direction documented but not the primary proved path +yet +* *Comprehensive Logging*: Structured JSON logs with telemetry metrics +* *Backend Proxy*: Transparent proxying to backend services with header +preservation + +Wondering how this works? See EXPLAINME.adoc. + +== Why This Exists + +Modern systems expose HTTP methods inconsistently and often +accidentally. DELETE, PUT, PATCH, OPTIONS, and even HEAD can leak +capabilities or create attack surface when left unmanaged. + +Traditional reverse proxies do not provide: + +* per-verb governance +* narrative or provenance +* reversible policy artefacts +* trust-aware verb exposure +* structured constraints +* intentional stealthing or deception + +http-capability-gateway introduces a principled, schema-driven approach +to HTTP method governance without disrupting existing infrastructure. + +== MVP Scope + +The MVP focuses on the smallest coherent loop: + +[arabic] +. Load a Verb Governance Spec from disk +. Validate it against a top-level schema +. Compile it into fast, matchable rules +. Enforce those rules on real HTTP traffic +. Emit structured logs for every decision + +No trust engine, no dynamic scoring, no control plane, no VeriSimDB +integration. Those will grow around this core in later phases. + +== Quick Start + +=== Installation + +[source,bash] +---- +# Clone the repository +git clone https://github.com/hyperpolymath/http-capability-gateway.git +cd http-capability-gateway + +# Install dependencies +mix deps.get + +# Compile +mix compile +---- + +=== Basic Usage + +[arabic] +. *Create a policy file* (`+config/policy.yaml+`): ++ +[source,yaml] +---- +dsl_version: "1" +governance: + global_verbs: + - GET + - POST + routes: + - path: "/api/admin" + verbs: [GET] + - path: "/api/users/[0-9]+" + verbs: [GET, PUT, DELETE] +stealth: + enabled: true + status_code: 404 +---- + +[arabic] +. *Configure backend* (`+config/dev.exs+`): ++ +[source,elixir] +---- +config :http_capability_gateway, + policy_path: "examples/policy-dev.yaml", + backend_url: "http://localhost:4000", + port: 4000 +---- + +[arabic] +. *Start the gateway*: ++ +[source,bash] +---- +mix run --no-halt + +# Or with interactive shell +iex -S mix +---- + +[arabic] +. *Test requests*: ++ +[source,bash] +---- +# Allowed: GET on global route +curl http://localhost:4000/api/public +# Returns: proxied response from backend + +# Denied: DELETE not in global verbs +curl -X DELETE http://localhost:4000/api/public +# Returns: 404 (stealth mode) + +# Allowed: PUT on specific route +curl -X PUT http://localhost:4000/api/users/123 +# Returns: proxied response from backend +---- + +== Verb Governance Spec (DSL v1) + +=== Structure + +[source,yaml] +---- +dsl_version: "1" # Required: policy format version + +governance: + # Global verbs: allowed on all routes unless overridden + global_verbs: + - GET + - POST + + # Route-specific rules (optional) + routes: + - path: "/api/admin" + verbs: [GET] # Only GET allowed, overrides global + - path: "/api/users/[0-9]+" # Regex patterns supported + verbs: [GET, PUT, DELETE] + +stealth: # Optional stealth mode configuration + enabled: true + status_code: 404 # Status code for denied requests +---- + +Earlier-format DSL (v0) uses a richer +`+service+`/`+verbs+`/`+narrative+` structure: + +[source,yaml] +---- +service: + name: ledger-api + version: 1 + environment: dev + +verbs: + GET: { exposure: public } + POST: { exposure: authenticated } + DELETE: { exposure: internal } + +routes: + - path: /accounts + verbs: + DELETE: + exposure: internal + narrative: "Account deletion requires internal trust." + +stealth: + profiles: + limited: + unauthenticated: 405 + untrusted: 404 + +narrative: + purpose: "Define safe verb exposure for ledger operations." +---- + +=== Supported HTTP Verbs + +`+GET+`, `+POST+`, `+PUT+`, `+DELETE+`, `+PATCH+`, `+HEAD+`, `+OPTIONS+` + +=== Path Matching + +* *Literal paths*: `+/api/users+` +* *Regex patterns*: `+/api/users/[0-9]++` (numeric user IDs) +* *Wildcard patterns*: `+/api/posts/.++` (any post path) + +=== Stealth Mode + +When a request is denied: + +* *Stealth enabled*: Returns configured status code (e.g., 404) with +empty body +* *Stealth disabled*: Returns 403 Forbidden + +Valid stealth status codes: `+200+`, `+301+`, `+302+`, `+403+`, `+404+`, +`+410+`, `+500+`, `+503+` + +== Configuration + +=== Environment Variables + +[source,bash] +---- +# Policy file path (default: config/policy.yaml) +export POLICY_PATH=/path/to/policy.yaml + +# Backend URL (required) +export BACKEND_URL=http://backend:4000 + +# Gateway port (default: 4000) +export PORT=4000 + +# Trust level header name (default: x-trust-level) +export TRUST_LEVEL_HEADER=x-trust-level +---- + +=== Elixir Config + +`+config/config.exs+` (shared config): + +[source,elixir] +---- +import Config + +config :http_capability_gateway, + backend_url: System.get_env("BACKEND_URL", "http://localhost:4000"), + port: String.to_integer(System.get_env("PORT", "4000")), + trust_level_header: System.get_env("TRUST_LEVEL_HEADER", "x-trust-level") +---- + +`+config/dev.exs+` (development): + +[source,elixir] +---- +import Config + +config :http_capability_gateway, + policy_path: "examples/policy-dev.yaml", + log_level: :debug +---- + +`+config/prod.exs+` (production): + +[source,elixir] +---- +import Config + +config :http_capability_gateway, + policy_path: System.get_env("POLICY_PATH"), + log_level: :info +---- + +== Trust Levels + +Extract trust levels from mTLS certificates or HTTP headers: + +[source,bash] +---- +# From header (current implementation) +curl -H "X-Trust-Level: high" http://localhost:4000/api/admin +---- + +Trust levels can be used for: + +* Audit logging +* Fine-grained access control (future feature) +* Rate limiting (future feature) + +== Logging + +Structured JSON logs with telemetry: + +[source,json] +---- +{ + "timestamp": "2026-01-22T23:00:00.000Z", + "level": "info", + "message": "request_handled", + "request_id": "req-abc123", + "method": "GET", + "path": "/api/users/123", + "trust_level": "high", + "verb_allowed": true, + "stealth_triggered": false, + "response_status": 200, + "duration_ms": 45 +} +---- + +== Testing + +[source,bash] +---- +# Run all tests +mix test + +# Run specific test file +mix test test/policy_loader_test.exs + +# Run property-based tests only +mix test --only property + +# Run performance tests +mix test --only performance + +# Run with coverage +mix test --cover +---- + +Current tests cover the policy pipeline and some gateway behavior, but +the repo still needs materially stronger evidence in the following +areas: + +* security tests for token validation, request sanitization, and SSRF +resistance +* end-to-end request lifecycle tests +* concurrency and reload testing +* formal benchmark runs + +Do not treat the current suite as sufficient proof for whole-site +gateway deployment. + +== Architecture (MVP) + +[source,ascii] +---- +Policy File (DSL) + | + v +Policy Loader → Validator → Compiler + | + v +Gateway (Elixir) + | + v +HTTP Traffic → Enforcement → JSON Logs +---- + +See TOPOLOGY for a full visual architecture map and completion +dashboard. + +== Performance Optimisations (v1.0.0) + +=== Tiered ETS Lookup + +Policy lookups use a three-tier strategy that eliminates full-table +scans: + +* *Tier 1 — Exact Path (O(1))*: Literal route patterns (no regex +metacharacters) are stored with `+{:exact,+` `+path,+` `+verb}+` ETS +keys. A direct hash lookup resolves 90%+ of requests instantly. +* *Tier 2 — Regex Routes (O(r))*: Patterns containing regex +metacharacters are tested only against other regex routes, not the +entire table. +* *Tier 3 — Global Rules (O(1))*: If no route matches, a final +`+{:global,+` `+verb}+` lookup catches global verb defaults. + +For a 1000-route policy, this reduces from ~1000 regex evaluations per +request to 1 hash lookup (typical case) or ~50 regex evaluations (edge +case). + +Performance-oriented design is present, but the benchmark story is not +yet strong enough to advertise hard numbers as release evidence. A +performance test file exists; benchmarking and concurrency validation +are still tracked as open work in `+ROADMAP.adoc+` and +`+TEST-NEEDS.md+`. + +=== Security Headers + +All responses (including health/metrics endpoints) include +OWASP-recommended security headers: + +* `+X-Content-Type-Options:+` `+nosniff+` — prevent MIME-type sniffing +* `+X-Frame-Options:+` `+DENY+` — prevent clickjacking +* `+Referrer-Policy:+` `+strict-origin-when-cross-origin+` — limit +referrer leakage +* `+Cache-Control:+` `+no-store,+` `+no-cache,+` `+must-revalidate+` — +prevent caching of policy decisions +* `+Connection:+` `+close+` — prevent connection reuse across trust +boundaries + +== Project Structure + +.... +http-capability-gateway/ +├── lib/ +│ └── http_capability_gateway/ +│ ├── application.ex # OTP application +│ ├── gateway.ex # HTTP gateway (Plug.Router) +│ ├── proxy.ex # Backend proxy (Req) +│ ├── policy_loader.ex # YAML policy loading +│ ├── policy_validator.ex # DSL v1 validation +│ ├── policy_compiler.ex # ETS compilation +│ ├── logging.ex # Structured logging +│ └── log_formatter.ex # JSON log formatter +├── test/ # Current automated tests +├── config/ # Elixir config files and default policy +├── examples/ # Example policies +└── docs/ # API documentation +.... + +== Roadmap + +Use `+ROADMAP.adoc+` as the current roadmap. The short version: + +* core policy pipeline exists +* tests exist but are not yet strong enough for broad production claims +* the recommended near-term role is route-scoped API prefiltering, not +full-site gateway responsibility +* benchmark, E2E, and security-hardening work remain open + +=== Phase 2 + +* Rate limits +* Expanded stealth profiles +* Hot policy reloads + +=== Phase 3 + +* Control plane +* VeriSimDB integration for provenance +* Distributed gateways + +=== Phase 4 + +* Agent introspection +* Constraint engine +* Formal proofs + +== Contributing + +Contributions welcome! Please: + +[arabic] +. Fork the repository +. Create a feature branch +. Add tests for new features +. Ensure `+mix+` `+test+` passes +. Submit a pull request + +== Philosophy + +This project treats governance as a first-class engineering concern. +Policies are artefacts. Artefacts are reversible. Decisions have +provenance. HTTP verbs become capabilities, not accidents. + +== License + +This project is licensed under the Mozilla Public License, v. 2.0. See +the `+LICENSE+` file for details. + +SPDX-License-Identifier: CC-BY-SA-4.0 diff --git a/README.md b/README.md deleted file mode 100644 index af26817..0000000 --- a/README.md +++ /dev/null @@ -1,527 +0,0 @@ - - -Jonathan D.A. Jewell \<[j.d.a.jewell@open.ac](j.d.a.jewell@open.ac).uk\> -:toc: preamble :icons: font :sectnums: - -[![License: MPL-2.0](https://img.shields.io/badge/License-MPL_2.0-blue.svg)](https://opensource.org/licenses/MPL-2.0) image:[Elixir -1.19+](https://img.shields.io/badge/Elixir-1.19+-purple.svg) image:[OTP -27+](https://img.shields.io/badge/OTP-27+-red.svg) - -# Overview - -http-capability-gateway is a lightweight, policy-driven HTTP governance -layer that enforces a declarative, auditable model of HTTP verb exposure -in front of existing services. - -This project introduces a minimal viable implementation of a new -category: a capability gateway for HTTP. It does not replace nginx or -Apache. Instead, it governs what they are allowed to do. - -The gateway loads a Verb Governance Spec (DSL v1), validates it, -compiles it into fast enforcement rules, and applies those rules to real -HTTP traffic. Every decision is logged in structured form for audit and -introspection. - -## Current Status - -This repository contains a real Elixir gateway implementation, but it -should currently be treated as a narrow, in-progress API governance -layer rather than a fully proven front door for an entire site. - -- The core policy pipeline exists: loader, validator, compiler, gateway, - proxy, telemetry. - -- The main remaining gaps are security depth, end-to-end verification, - and benchmark evidence. - -- Read `ROADMAP.adoc`, `TEST-NEEDS.md`, and `PROOFS_NEEDED.md` together - when judging readiness. - -It provides: - -- **Declarative Verb Governance**: Define allowed HTTP verbs globally - and per-route - -- **Stealth Mode**: Return configurable status codes (404, 403, etc.) - for unauthorized requests - -- **Fast Policy Enforcement Architecture**: ETS-backed lookups and - compiled policy rules; benchmark evidence still needs to be formalized - -- **Trust Level Integration**: Current implementation is header-based, - with mTLS-oriented direction documented but not the primary proved - path yet - -- **Comprehensive Logging**: Structured JSON logs with telemetry metrics - -- **Backend Proxy**: Transparent proxying to backend services with - header preservation - -Wondering how this works? See [EXPLAINME.adoc](EXPLAINME.adoc). - -# Why This Exists - -Modern systems expose HTTP methods inconsistently and often -accidentally. DELETE, PUT, PATCH, OPTIONS, and even HEAD can leak -capabilities or create attack surface when left unmanaged. - -Traditional reverse proxies do not provide: - -- per-verb governance - -- narrative or provenance - -- reversible policy artefacts - -- trust-aware verb exposure - -- structured constraints - -- intentional stealthing or deception - -http-capability-gateway introduces a principled, schema-driven approach -to HTTP method governance without disrupting existing infrastructure. - -# MVP Scope - -The MVP focuses on the smallest coherent loop: - -1. Load a Verb Governance Spec from disk - -2. Validate it against a top-level schema - -3. Compile it into fast, matchable rules - -4. Enforce those rules on real HTTP traffic - -5. Emit structured logs for every decision - -No trust engine, no dynamic scoring, no control plane, no VeriSimDB -integration. Those will grow around this core in later phases. - -# Quick Start - -## Installation - -```bash -# Clone the repository -git clone https://github.com/hyperpolymath/http-capability-gateway.git -cd http-capability-gateway - -# Install dependencies -mix deps.get - -# Compile -mix compile -``` - -## Basic Usage - -1. **Create a policy file** (`config/policy.yaml`): - - ``` yaml - dsl_version: "1" - governance: - global_verbs: - - GET - - POST - routes: - - path: "/api/admin" - verbs: [GET] - - path: "/api/users/[0-9]+" - verbs: [GET, PUT, DELETE] - stealth: - enabled: true - status_code: 404 - ``` - - - -1. **Configure backend** (`config/dev.exs`): - - ``` elixir - config :http_capability_gateway, - policy_path: "examples/policy-dev.yaml", - backend_url: "http://localhost:4000", - port: 4000 - ``` - - - -1. **Start the gateway**: - - ``` bash - mix run --no-halt - - # Or with interactive shell - iex -S mix - ``` - - - -1. **Test requests**: - - ``` bash - # Allowed: GET on global route - curl http://localhost:4000/api/public - # Returns: proxied response from backend - - # Denied: DELETE not in global verbs - curl -X DELETE http://localhost:4000/api/public - # Returns: 404 (stealth mode) - - # Allowed: PUT on specific route - curl -X PUT http://localhost:4000/api/users/123 - # Returns: proxied response from backend - ``` - -# Verb Governance Spec (DSL v1) - -## Structure - -```yaml -dsl_version: "1" # Required: policy format version - -governance: - # Global verbs: allowed on all routes unless overridden - global_verbs: - - GET - - POST - - # Route-specific rules (optional) - routes: - - path: "/api/admin" - verbs: [GET] # Only GET allowed, overrides global - - path: "/api/users/[0-9]+" # Regex patterns supported - verbs: [GET, PUT, DELETE] - -stealth: # Optional stealth mode configuration - enabled: true - status_code: 404 # Status code for denied requests -``` - -Earlier-format DSL (v0) uses a richer `service`/`verbs`/`narrative` -structure: - -```yaml -service: - name: ledger-api - version: 1 - environment: dev - -verbs: - GET: { exposure: public } - POST: { exposure: authenticated } - DELETE: { exposure: internal } - -routes: - - path: /accounts - verbs: - DELETE: - exposure: internal - narrative: "Account deletion requires internal trust." - -stealth: - profiles: - limited: - unauthenticated: 405 - untrusted: 404 - -narrative: - purpose: "Define safe verb exposure for ledger operations." -``` - -## Supported HTTP Verbs - -`GET`, `POST`, `PUT`, `DELETE`, `PATCH`, `HEAD`, `OPTIONS` - -## Path Matching - -- **Literal paths**: `/api/users` - -- **Regex patterns**: `/api/users/[0-9]+` (numeric user IDs) - -- **Wildcard patterns**: `/api/posts/.+` (any post path) - -## Stealth Mode - -When a request is denied: - -- **Stealth enabled**: Returns configured status code (e.g., 404) with - empty body - -- **Stealth disabled**: Returns 403 Forbidden - -Valid stealth status codes: `200`, `301`, `302`, `403`, `404`, `410`, -`500`, `503` - -# Configuration - -## Environment Variables - -```bash -# Policy file path (default: config/policy.yaml) -export POLICY_PATH=/path/to/policy.yaml - -# Backend URL (required) -export BACKEND_URL=http://backend:4000 - -# Gateway port (default: 4000) -export PORT=4000 - -# Trust level header name (default: x-trust-level) -export TRUST_LEVEL_HEADER=x-trust-level -``` - -## Elixir Config - -`config/config.exs` (shared config): - -```elixir -import Config - -config :http_capability_gateway, - backend_url: System.get_env("BACKEND_URL", "http://localhost:4000"), - port: String.to_integer(System.get_env("PORT", "4000")), - trust_level_header: System.get_env("TRUST_LEVEL_HEADER", "x-trust-level") -``` - -`config/dev.exs` (development): - -```elixir -import Config - -config :http_capability_gateway, - policy_path: "examples/policy-dev.yaml", - log_level: :debug -``` - -`config/prod.exs` (production): - -```elixir -import Config - -config :http_capability_gateway, - policy_path: System.get_env("POLICY_PATH"), - log_level: :info -``` - -# Trust Levels - -Extract trust levels from mTLS certificates or HTTP headers: - -```bash -# From header (current implementation) -curl -H "X-Trust-Level: high" http://localhost:4000/api/admin -``` - -Trust levels can be used for: - -- Audit logging - -- Fine-grained access control (future feature) - -- Rate limiting (future feature) - -# Logging - -Structured JSON logs with telemetry: - -```json -{ - "timestamp": "2026-01-22T23:00:00.000Z", - "level": "info", - "message": "request_handled", - "request_id": "req-abc123", - "method": "GET", - "path": "/api/users/123", - "trust_level": "high", - "verb_allowed": true, - "stealth_triggered": false, - "response_status": 200, - "duration_ms": 45 -} -``` - -# Testing - -```bash -# Run all tests -mix test - -# Run specific test file -mix test test/policy_loader_test.exs - -# Run property-based tests only -mix test --only property - -# Run performance tests -mix test --only performance - -# Run with coverage -mix test --cover -``` - -Current tests cover the policy pipeline and some gateway behavior, but -the repo still needs materially stronger evidence in the following -areas: - -- security tests for token validation, request sanitization, and SSRF - resistance - -- end-to-end request lifecycle tests - -- concurrency and reload testing - -- formal benchmark runs - -Do not treat the current suite as sufficient proof for whole-site -gateway deployment. - -# Architecture (MVP) - -```ascii -Policy File (DSL) - | - v -Policy Loader → Validator → Compiler - | - v -Gateway (Elixir) - | - v -HTTP Traffic → Enforcement → JSON Logs -``` - -See TOPOLOGY for a full visual -architecture map and completion dashboard. - -# Performance Optimisations (v1.0.0) - -## Tiered ETS Lookup - -Policy lookups use a three-tier strategy that eliminates full-table -scans: - -- **Tier 1 — Exact Path (O(1))**: Literal route patterns (no regex - metacharacters) are stored with `{:exact,` `path,` `verb}` ETS keys. A - direct hash lookup resolves 90%+ of requests instantly. - -- **Tier 2 — Regex Routes (O(r))**: Patterns containing regex - metacharacters are tested only against other regex routes, not the - entire table. - -- **Tier 3 — Global Rules (O(1))**: If no route matches, a final - `{:global,` `verb}` lookup catches global verb defaults. - -For a 1000-route policy, this reduces from ~1000 regex evaluations per -request to 1 hash lookup (typical case) or ~50 regex evaluations (edge -case). - -Performance-oriented design is present, but the benchmark story is not -yet strong enough to advertise hard numbers as release evidence. A -performance test file exists; benchmarking and concurrency validation -are still tracked as open work in `ROADMAP.adoc` and `TEST-NEEDS.md`. - -## Security Headers - -All responses (including health/metrics endpoints) include -OWASP-recommended security headers: - -- `X-Content-Type-Options:` `nosniff` — prevent MIME-type sniffing - -- `X-Frame-Options:` `DENY` — prevent clickjacking - -- `Referrer-Policy:` `strict-origin-when-cross-origin` — limit referrer - leakage - -- `Cache-Control:` `no-store,` `no-cache,` `must-revalidate` — prevent - caching of policy decisions - -- `Connection:` `close` — prevent connection reuse across trust - boundaries - -# Project Structure - - http-capability-gateway/ - ├── lib/ - │ └── http_capability_gateway/ - │ ├── application.ex # OTP application - │ ├── gateway.ex # HTTP gateway (Plug.Router) - │ ├── proxy.ex # Backend proxy (Req) - │ ├── policy_loader.ex # YAML policy loading - │ ├── policy_validator.ex # DSL v1 validation - │ ├── policy_compiler.ex # ETS compilation - │ ├── logging.ex # Structured logging - │ └── log_formatter.ex # JSON log formatter - ├── test/ # Current automated tests - ├── config/ # Elixir config files and default policy - ├── examples/ # Example policies - └── docs/ # API documentation - -# Roadmap - -Use `ROADMAP.adoc` as the current roadmap. The short version: - -- core policy pipeline exists - -- tests exist but are not yet strong enough for broad production claims - -- the recommended near-term role is route-scoped API prefiltering, not - full-site gateway responsibility - -- benchmark, E2E, and security-hardening work remain open - -## Phase 2 - -- Rate limits - -- Expanded stealth profiles - -- Hot policy reloads - -## Phase 3 - -- Control plane - -- VeriSimDB integration for provenance - -- Distributed gateways - -## Phase 4 - -- Agent introspection - -- Constraint engine - -- Formal proofs - -# Contributing - -Contributions welcome! Please: - -1. Fork the repository - -2. Create a feature branch - -3. Add tests for new features - -4. Ensure `mix` `test` passes - -5. Submit a pull request - -# Philosophy - -This project treats governance as a first-class engineering concern. -Policies are artefacts. Artefacts are reversible. Decisions have -provenance. HTTP verbs become capabilities, not accidents. - -# License - -This project is licensed under the Mozilla Public License, v. 2.0. See -the `LICENSE` file for details. - -SPDX-License-Identifier: CC-BY-SA-4.0 diff --git a/ROADMAP-v2.adoc b/ROADMAP-v2.adoc new file mode 100644 index 0000000..5042907 --- /dev/null +++ b/ROADMAP-v2.adoc @@ -0,0 +1,488 @@ +== http-capability-gateway v2.0 Roadmap — HISTORICAL / ASPIRATIONAL + +____ +*⚠️ HISTORICAL DOCUMENT (2026-04-16):* This is an aspirational v2.0 +vision written when the project was still in design phase. It is NOT on +the current release path for v0.1.0 — the features listed here (web UI, +plugins, multi-protocol full support, AI policy suggestions, etc.) are +explicitly *out of MVP scope*. + +See `+ROADMAP.adoc+` and `+docs/SUPPORTED-FEATURES.md+` for the current +scope and what is actually supported today. + +Do not treat any item in this document as a commitment or an imminent +feature. It is preserved for reference and long-term direction only. +____ + +== Original Vision: The Gateway Everyone Wants + +Transform http-capability-gateway from "`good production gateway`" to +*"`the obvious choice for API governance`"* by making it: + +[arabic] +. *Bulletproof* - Never fails, always recovers +. *Effortless* - 5-minute setup, beautiful UI, zero config defaults +. *Powerful* - Handles any protocol, any scale, any use case +. *Extensible* - Plugin ecosystem, community contributions +. *Observable* - See everything, understand everything + +''''' + +=== 🛡️ DEPENDABILITY (Make it Bulletproof) + +==== Priority 1: Resilience + +* *Circuit Breakers* (#13) +** Automatic backend failure detection +** Exponential backoff with jitter +** Self-healing (auto-recovery) +** Per-backend circuit state +** Metrics: circuit_breaker_state, failure_rate +* *Graceful Degradation* +** Stale policy cache (continue with last good policy) +** Backup policy file support +** Read-only mode when policy invalid +** Queue requests during brief outages +* *Policy Rollback* +** Git-based policy versioning +** Automatic rollback on error spike +** Policy canary deployments (test on 1% traffic) +** Diff visualization before applying + +==== Priority 2: High Availability + +* *Zero-Downtime Updates* (#15) +** Hot-reload policies without restart +** Atomic ETS table swap +** Watch policy file for changes +** POST /admin/reload endpoint +* *Clustering Support* +** Distributed policy cache (libcluster) +** Consistent hashing for sticky sessions +** Leader election for policy updates +** Gossip protocol for policy sync + +''''' + +=== 🔒 SECURITY (Make it Fort Knox) + +==== Priority 1: Protection + +* *Rate Limiting* (#14) +** Per trust level (untrusted: 10/min, authenticated: 100/min, internal: +unlimited) +** Token bucket algorithm +** Distributed rate limiting (Redis) +** Return 429 with Retry-After +** Headers: X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset +* *Input Validation* +** Request size limits (body, headers) +** Path traversal protection +** SQL injection detection +** XSS filtering +** CSRF protection +* *DDoS Protection* +** Connection rate limiting +** Slowloris protection +** IP-based blocking +** Geofencing support + +==== Priority 2: Compliance + +* *Audit Logging* +** Who changed what when +** Policy change history +** Failed access attempts +** Compliance reports (SOC2, HIPAA) +* *Secrets Management* +** HashiCorp Vault integration +** AWS Secrets Manager +** Environment variable encryption +** Rotate credentials automatically + +''''' + +=== ⚡ PERFORMANCE (Make it Blazing Fast) + +==== Priority 1: Speed + +* *Caching Layer* (#17) +** HTTP caching (Cache-Control, ETag, Last-Modified) +** Cachex for in-memory cache +** Redis for distributed cache +** TTL per route in policy +** Cache hit/miss metrics +** Admin API for cache invalidation +* *Connection Pooling* +** Reuse backend connections +** Configurable pool size +** Connection health checks +** Automatic pool scaling +* *HTTP/2 Support* +** Server push for related resources +** Multiplexing +** Header compression + +==== Priority 2: Optimization + +* *Async Everything* +** Async logging (no blocking) +** Async metrics export +** Background policy compilation +* *Response Streaming* +** Stream large responses +** Chunked transfer encoding +** Server-sent events (SSE) + +''''' + +=== 🚀 FUNCTIONALITY (Make it Powerful) + +==== Priority 1: Protocol Support (#22) + +* *GraphQL* +** Query/mutation name matching +** Field-level permissions +** Introspection control +* *WebSocket* +** Long-lived connection policy checks +** Message-level governance +** Heartbeat support +* *gRPC* +** Protobuf method matching +** Service-level policies +** Streaming support + +==== Priority 2: Advanced Features + +* *Request/Response Transformation* +** Modify headers (add, remove, replace) +** Body transformation (JSON → JSON) +** URL rewriting +** Query parameter manipulation +* *Load Balancing* +** Multiple backend support +** Round-robin, least-connections, random +** Health checks +** Weighted routing +* *Multi-Tenancy* +** Tenant isolation +** Per-tenant policies +** Tenant-specific backends +** Usage tracking per tenant + +''''' + +=== 🎨 EASY TO USE (Make it Effortless) + +==== Priority 1: Beautiful UI (#16) + +* *Phoenix LiveView Dashboard* +** Visual policy editor (drag-drop rules) +** Live policy validation +** Metrics charts (requests/sec, latency, errors) +** Request logs viewer (real-time) +** Policy diff/compare +** Dark mode +** Mobile responsive +* *Interactive Tutorial* +** Onboarding wizard +** Example policies gallery +** Interactive playground +** Video walkthroughs + +==== Priority 2: Developer Experience + +* *Policy Testing Framework* (#19) +** `+assert_allows(policy, :GET, "/api/users", trust: "authenticated")+` +** `+assert_denies(policy, :DELETE, "/api/admin")+` +** Property-based testing +** Coverage reports +** `+mix policy.test+` command +* *CLI Tool* +** `+hcg init+` - scaffolding +** `+hcg validate policy.yaml+` - validation +** `+hcg test policy.yaml+` - testing +** `+hcg deploy+` - deployment +** `+hcg analyze+` - policy optimization suggestions +* *Policy Templates Library* +** REST API gateway +** Microservices mesh +** Internal service gateway +** Public API gateway +** Multi-tenant SaaS +** B2B API platform + +''''' + +=== 🔌 EXTENSIBILITY (Make it Hackable) + +==== Priority 1: Plugin System (#21) + +* *Behavior-Based Plugins* ++ +[source,elixir] +---- +defmodule MyAuthPlugin do + @behaviour HttpCapabilityGateway.Plugin.Auth + + def authenticate(conn, config) do + # Custom auth logic + end +end +---- +* *Plugin Types* +** `+Auth+` - Custom authentication +** `+RequestFilter+` - Modify requests +** `+ResponseFilter+` - Modify responses +** `+PolicyLoader+` - Load from custom sources +** `+MetricsExporter+` - Export to custom systems +* *Plugin Discovery* +** Hex.pm package ecosystem +** Auto-discovery via Mix config +** Plugin marketplace on website + +==== Priority 2: Integrations + +* *API Gateway Integration* +** Kong plugin +** Traefik middleware +** Envoy filter +** AWS API Gateway authorizer +* *Observability Integration* +** Datadog APM +** New Relic +** Honeycomb +** Elastic APM + +''''' + +=== 📊 OBSERVABILITY (Make it Transparent) + +==== Priority 1: Visibility (#18) + +* *Distributed Tracing* +** OpenTelemetry integration +** Jaeger/Zipkin export +** Trace ID in logs +** Trace context propagation +** Span tags: verb, path, trust_level, decision +* *Real-Time Metrics Dashboard* +** Requests per second (live graph) +** Latency percentiles (p50, p95, p99) +** Error rates by endpoint +** Top endpoints by traffic +** Top denied requests +** Trust level distribution + +==== Priority 2: Insights + +* *Policy Analytics* +** Most-hit rules +** Unused rules detection +** Dead code in policies +** Optimization suggestions +* *Cost Optimization* +** Backend cost per endpoint +** Expensive routes identification +** Caching ROI calculation +** Rate limit effectiveness + +''''' + +=== 🎯 DEPLOYMENT (Make it Cloud-Native) + +==== Priority 1: Infrastructure as Code (#20) + +* *Kubernetes* +** Helm chart with sensible defaults +** HPA (Horizontal Pod Autoscaler) +** Ingress integration +** ConfigMap for policies +** Secret for TLS certs +** ServiceMonitor for Prometheus +* *Terraform Modules* +** AWS: ECS/Fargate deployment +** GCP: Cloud Run / GKE +** Azure: Container Instances / AKS +** Complete networking setup +** Load balancer configuration + +==== Priority 2: Developer Workflows + +* *GitOps Integration* +** ArgoCD application +** FluxCD kustomization +** Automatic policy sync from Git +** PR-based policy reviews +* *CI/CD Templates* +** GitHub Actions workflows +** GitLab CI templates +** Jenkins pipelines +** Policy validation in CI + +''''' + +=== 🌟 KILLER FEATURES (Make it Unique) + +==== The "`Wow`" Factors + +[arabic] +. *AI-Powered Policy Suggestions* +* Analyze traffic patterns +* Suggest optimal policies +* Detect anomalies +* Auto-generate policies from OpenAPI specs +. *Time-Travel Debugging* +* Replay requests from logs +* Step through policy evaluation +* "`Why was this request denied?`" +* Visual policy execution trace +. *Policy Simulation* +* "`What if`" analysis +* Test policy changes on historical traffic +* Impact prediction before deployment +* A/B test policies +. *Community Policy Library* +* Share policies on hub.hyperpolymath.org +* Star/fork popular policies +* Policy composition (import shared rules) +* Security policy marketplace +. *Zero-Config Mode* +* Start with `+hcg start+` +* Auto-generate policy from traffic +* Learn mode (observe only) +* Graduate to enforcement mode +. *Visual Policy Builder* +* Flowchart-style policy design +* Drag-drop nodes for rules +* Real-time validation +* Export to YAML + +''''' + +=== 📈 ADOPTION STRATEGY + +==== Make It Irresistible + +[arabic] +. *Quick Start Promise: "`5 Minutes to Production`"* ++ +[source,bash] +---- +curl -sSL https://get.hcg.dev | bash +hcg init +hcg start # That's it! +---- +. *Free Tier with All Features* +* No artificial limitations +* Open source forever +* Optional paid cloud hosting +* Enterprise support available +. *Viral Features* +* Beautiful public dashboards (share your metrics) +* Policy templates sharing +* Success stories showcase +* Badge: "`Protected by HCG`" +. *Integration Everywhere* +* Docker Hub official image +* AWS/Azure/GCP marketplaces +* Terraform registry +* Helm artifact hub +* VS Code extension +. *Community Building* +* Discord server +* Monthly webinars +* Blog with tutorials +* YouTube channel +* Twitter bot (gateway tips) + +''''' + +=== 🎯 IMPLEMENTATION PRIORITY + +==== Phase 1: Dependability (v1.1.0) - 2 weeks + +* Circuit breakers (#13) +* Hot-reload (#15) +* Graceful degradation + +==== Phase 2: Easy to Use (v1.2.0) - 3 weeks + +* Web UI dashboard (#16) +* Policy testing framework (#19) +* CLI tool + +==== Phase 3: Performance (v1.3.0) - 2 weeks + +* Caching layer (#17) +* Rate limiting (#14) +* Connection pooling + +==== Phase 4: Observability (v1.4.0) - 2 weeks + +* Distributed tracing (#18) +* Real-time metrics dashboard +* Policy analytics + +==== Phase 5: Extensibility (v2.0.0) - 3 weeks + +* Plugin system (#21) +* Protocol support (#22) +* Helm chart (#20) + +''''' + +=== 💡 COMPETITIVE ADVANTAGES + +==== vs Kong + +* ✅ Simpler policy language (DSL v1 vs Lua) +* ✅ Better stealth mode +* ✅ Native Elixir performance +* ✅ Built-in policy testing + +==== vs Traefik + +* ✅ More fine-grained verb control +* ✅ Trust-level based governance +* ✅ Better observability out-of-box + +==== vs Envoy + +* ✅ Much easier to configure +* ✅ Beautiful UI (Envoy has none) +* ✅ Policy-first design +* ✅ Smaller learning curve + +==== vs AWS API Gateway + +* ✅ Self-hosted (no vendor lock-in) +* ✅ Open source +* ✅ No per-request pricing +* ✅ More flexible policies + +''''' + +=== 🚀 Success Metrics + +*Adoption:* - 1,000 GitHub stars in 6 months - 100 production +deployments - 10 community plugins + +*Performance:* - <5ms latency overhead - 10,000+ req/sec on single +instance - 99.99% uptime in production + +*Community:* - 50 policy templates shared - 100 Discord members - 10 +blog posts/tutorials + +''''' + +=== 🎊 The Ultimate Goal + +*"`The Gateway Everyone Wants`"* + +When developers think "`I need an API gateway,`" their first thought +should be: > "`http-capability-gateway - duh! It’s so easy, so powerful, +and that UI is gorgeous!`" + +*Make it so good they can’t resist.* diff --git a/ROADMAP-v2.md b/ROADMAP-v2.md deleted file mode 100644 index 364b855..0000000 --- a/ROADMAP-v2.md +++ /dev/null @@ -1,492 +0,0 @@ - -# http-capability-gateway v2.0 Roadmap — HISTORICAL / ASPIRATIONAL - -> **⚠️ HISTORICAL DOCUMENT (2026-04-16):** This is an aspirational v2.0 vision -> written when the project was still in design phase. It is NOT on the current -> release path for v0.1.0 — the features listed here (web UI, plugins, multi-protocol -> full support, AI policy suggestions, etc.) are explicitly **out of MVP scope**. -> -> See `ROADMAP.adoc` and `docs/SUPPORTED-FEATURES.md` for the current scope and -> what is actually supported today. -> -> Do not treat any item in this document as a commitment or an imminent feature. -> It is preserved for reference and long-term direction only. - -# Original Vision: The Gateway Everyone Wants - -Transform http-capability-gateway from "good production gateway" to **"the obvious choice for API governance"** by making it: - -1. **Bulletproof** - Never fails, always recovers -2. **Effortless** - 5-minute setup, beautiful UI, zero config defaults -3. **Powerful** - Handles any protocol, any scale, any use case -4. **Extensible** - Plugin ecosystem, community contributions -5. **Observable** - See everything, understand everything - ---- - -## 🛡️ DEPENDABILITY (Make it Bulletproof) - -### Priority 1: Resilience -- **Circuit Breakers** (#13) - - Automatic backend failure detection - - Exponential backoff with jitter - - Self-healing (auto-recovery) - - Per-backend circuit state - - Metrics: circuit_breaker_state, failure_rate - -- **Graceful Degradation** - - Stale policy cache (continue with last good policy) - - Backup policy file support - - Read-only mode when policy invalid - - Queue requests during brief outages - -- **Policy Rollback** - - Git-based policy versioning - - Automatic rollback on error spike - - Policy canary deployments (test on 1% traffic) - - Diff visualization before applying - -### Priority 2: High Availability -- **Zero-Downtime Updates** (#15) - - Hot-reload policies without restart - - Atomic ETS table swap - - Watch policy file for changes - - POST /admin/reload endpoint - -- **Clustering Support** - - Distributed policy cache (libcluster) - - Consistent hashing for sticky sessions - - Leader election for policy updates - - Gossip protocol for policy sync - ---- - -## 🔒 SECURITY (Make it Fort Knox) - -### Priority 1: Protection -- **Rate Limiting** (#14) - - Per trust level (untrusted: 10/min, authenticated: 100/min, internal: unlimited) - - Token bucket algorithm - - Distributed rate limiting (Redis) - - Return 429 with Retry-After - - Headers: X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset - -- **Input Validation** - - Request size limits (body, headers) - - Path traversal protection - - SQL injection detection - - XSS filtering - - CSRF protection - -- **DDoS Protection** - - Connection rate limiting - - Slowloris protection - - IP-based blocking - - Geofencing support - -### Priority 2: Compliance -- **Audit Logging** - - Who changed what when - - Policy change history - - Failed access attempts - - Compliance reports (SOC2, HIPAA) - -- **Secrets Management** - - HashiCorp Vault integration - - AWS Secrets Manager - - Environment variable encryption - - Rotate credentials automatically - ---- - -## ⚡ PERFORMANCE (Make it Blazing Fast) - -### Priority 1: Speed -- **Caching Layer** (#17) - - HTTP caching (Cache-Control, ETag, Last-Modified) - - Cachex for in-memory cache - - Redis for distributed cache - - TTL per route in policy - - Cache hit/miss metrics - - Admin API for cache invalidation - -- **Connection Pooling** - - Reuse backend connections - - Configurable pool size - - Connection health checks - - Automatic pool scaling - -- **HTTP/2 Support** - - Server push for related resources - - Multiplexing - - Header compression - -### Priority 2: Optimization -- **Async Everything** - - Async logging (no blocking) - - Async metrics export - - Background policy compilation - -- **Response Streaming** - - Stream large responses - - Chunked transfer encoding - - Server-sent events (SSE) - ---- - -## 🚀 FUNCTIONALITY (Make it Powerful) - -### Priority 1: Protocol Support (#22) -- **GraphQL** - - Query/mutation name matching - - Field-level permissions - - Introspection control - -- **WebSocket** - - Long-lived connection policy checks - - Message-level governance - - Heartbeat support - -- **gRPC** - - Protobuf method matching - - Service-level policies - - Streaming support - -### Priority 2: Advanced Features -- **Request/Response Transformation** - - Modify headers (add, remove, replace) - - Body transformation (JSON → JSON) - - URL rewriting - - Query parameter manipulation - -- **Load Balancing** - - Multiple backend support - - Round-robin, least-connections, random - - Health checks - - Weighted routing - -- **Multi-Tenancy** - - Tenant isolation - - Per-tenant policies - - Tenant-specific backends - - Usage tracking per tenant - ---- - -## 🎨 EASY TO USE (Make it Effortless) - -### Priority 1: Beautiful UI (#16) -- **Phoenix LiveView Dashboard** - - Visual policy editor (drag-drop rules) - - Live policy validation - - Metrics charts (requests/sec, latency, errors) - - Request logs viewer (real-time) - - Policy diff/compare - - Dark mode - - Mobile responsive - -- **Interactive Tutorial** - - Onboarding wizard - - Example policies gallery - - Interactive playground - - Video walkthroughs - -### Priority 2: Developer Experience -- **Policy Testing Framework** (#19) - - `assert_allows(policy, :GET, "/api/users", trust: "authenticated")` - - `assert_denies(policy, :DELETE, "/api/admin")` - - Property-based testing - - Coverage reports - - `mix policy.test` command - -- **CLI Tool** - - `hcg init` - scaffolding - - `hcg validate policy.yaml` - validation - - `hcg test policy.yaml` - testing - - `hcg deploy` - deployment - - `hcg analyze` - policy optimization suggestions - -- **Policy Templates Library** - - REST API gateway - - Microservices mesh - - Internal service gateway - - Public API gateway - - Multi-tenant SaaS - - B2B API platform - ---- - -## 🔌 EXTENSIBILITY (Make it Hackable) - -### Priority 1: Plugin System (#21) -- **Behavior-Based Plugins** - ```elixir - defmodule MyAuthPlugin do - @behaviour HttpCapabilityGateway.Plugin.Auth - - def authenticate(conn, config) do - # Custom auth logic - end - end - ``` - -- **Plugin Types** - - `Auth` - Custom authentication - - `RequestFilter` - Modify requests - - `ResponseFilter` - Modify responses - - `PolicyLoader` - Load from custom sources - - `MetricsExporter` - Export to custom systems - -- **Plugin Discovery** - - Hex.pm package ecosystem - - Auto-discovery via Mix config - - Plugin marketplace on website - -### Priority 2: Integrations -- **API Gateway Integration** - - Kong plugin - - Traefik middleware - - Envoy filter - - AWS API Gateway authorizer - -- **Observability Integration** - - Datadog APM - - New Relic - - Honeycomb - - Elastic APM - ---- - -## 📊 OBSERVABILITY (Make it Transparent) - -### Priority 1: Visibility (#18) -- **Distributed Tracing** - - OpenTelemetry integration - - Jaeger/Zipkin export - - Trace ID in logs - - Trace context propagation - - Span tags: verb, path, trust_level, decision - -- **Real-Time Metrics Dashboard** - - Requests per second (live graph) - - Latency percentiles (p50, p95, p99) - - Error rates by endpoint - - Top endpoints by traffic - - Top denied requests - - Trust level distribution - -### Priority 2: Insights -- **Policy Analytics** - - Most-hit rules - - Unused rules detection - - Dead code in policies - - Optimization suggestions - -- **Cost Optimization** - - Backend cost per endpoint - - Expensive routes identification - - Caching ROI calculation - - Rate limit effectiveness - ---- - -## 🎯 DEPLOYMENT (Make it Cloud-Native) - -### Priority 1: Infrastructure as Code (#20) -- **Kubernetes** - - Helm chart with sensible defaults - - HPA (Horizontal Pod Autoscaler) - - Ingress integration - - ConfigMap for policies - - Secret for TLS certs - - ServiceMonitor for Prometheus - -- **Terraform Modules** - - AWS: ECS/Fargate deployment - - GCP: Cloud Run / GKE - - Azure: Container Instances / AKS - - Complete networking setup - - Load balancer configuration - -### Priority 2: Developer Workflows -- **GitOps Integration** - - ArgoCD application - - FluxCD kustomization - - Automatic policy sync from Git - - PR-based policy reviews - -- **CI/CD Templates** - - GitHub Actions workflows - - GitLab CI templates - - Jenkins pipelines - - Policy validation in CI - ---- - -## 🌟 KILLER FEATURES (Make it Unique) - -### The "Wow" Factors - -1. **AI-Powered Policy Suggestions** - - Analyze traffic patterns - - Suggest optimal policies - - Detect anomalies - - Auto-generate policies from OpenAPI specs - -2. **Time-Travel Debugging** - - Replay requests from logs - - Step through policy evaluation - - "Why was this request denied?" - - Visual policy execution trace - -3. **Policy Simulation** - - "What if" analysis - - Test policy changes on historical traffic - - Impact prediction before deployment - - A/B test policies - -4. **Community Policy Library** - - Share policies on hub.hyperpolymath.org - - Star/fork popular policies - - Policy composition (import shared rules) - - Security policy marketplace - -5. **Zero-Config Mode** - - Start with `hcg start` - - Auto-generate policy from traffic - - Learn mode (observe only) - - Graduate to enforcement mode - -6. **Visual Policy Builder** - - Flowchart-style policy design - - Drag-drop nodes for rules - - Real-time validation - - Export to YAML - ---- - -## 📈 ADOPTION STRATEGY - -### Make It Irresistible - -1. **Quick Start Promise: "5 Minutes to Production"** - ```bash - curl -sSL https://get.hcg.dev | bash - hcg init - hcg start # That's it! - ``` - -2. **Free Tier with All Features** - - No artificial limitations - - Open source forever - - Optional paid cloud hosting - - Enterprise support available - -3. **Viral Features** - - Beautiful public dashboards (share your metrics) - - Policy templates sharing - - Success stories showcase - - Badge: "Protected by HCG" - -4. **Integration Everywhere** - - Docker Hub official image - - AWS/Azure/GCP marketplaces - - Terraform registry - - Helm artifact hub - - VS Code extension - -5. **Community Building** - - Discord server - - Monthly webinars - - Blog with tutorials - - YouTube channel - - Twitter bot (gateway tips) - ---- - -## 🎯 IMPLEMENTATION PRIORITY - -### Phase 1: Dependability (v1.1.0) - 2 weeks -- Circuit breakers (#13) -- Hot-reload (#15) -- Graceful degradation - -### Phase 2: Easy to Use (v1.2.0) - 3 weeks -- Web UI dashboard (#16) -- Policy testing framework (#19) -- CLI tool - -### Phase 3: Performance (v1.3.0) - 2 weeks -- Caching layer (#17) -- Rate limiting (#14) -- Connection pooling - -### Phase 4: Observability (v1.4.0) - 2 weeks -- Distributed tracing (#18) -- Real-time metrics dashboard -- Policy analytics - -### Phase 5: Extensibility (v2.0.0) - 3 weeks -- Plugin system (#21) -- Protocol support (#22) -- Helm chart (#20) - ---- - -## 💡 COMPETITIVE ADVANTAGES - -### vs Kong -- ✅ Simpler policy language (DSL v1 vs Lua) -- ✅ Better stealth mode -- ✅ Native Elixir performance -- ✅ Built-in policy testing - -### vs Traefik -- ✅ More fine-grained verb control -- ✅ Trust-level based governance -- ✅ Better observability out-of-box - -### vs Envoy -- ✅ Much easier to configure -- ✅ Beautiful UI (Envoy has none) -- ✅ Policy-first design -- ✅ Smaller learning curve - -### vs AWS API Gateway -- ✅ Self-hosted (no vendor lock-in) -- ✅ Open source -- ✅ No per-request pricing -- ✅ More flexible policies - ---- - -## 🚀 Success Metrics - -**Adoption:** -- 1,000 GitHub stars in 6 months -- 100 production deployments -- 10 community plugins - -**Performance:** -- <5ms latency overhead -- 10,000+ req/sec on single instance -- 99.99% uptime in production - -**Community:** -- 50 policy templates shared -- 100 Discord members -- 10 blog posts/tutorials - ---- - -## 🎊 The Ultimate Goal - -**"The Gateway Everyone Wants"** - -When developers think "I need an API gateway," their first thought should be: -> "http-capability-gateway - duh! It's so easy, so powerful, and that UI is gorgeous!" - -**Make it so good they can't resist.** diff --git a/SECURITY.adoc b/SECURITY.adoc new file mode 100644 index 0000000..1deb34a --- /dev/null +++ b/SECURITY.adoc @@ -0,0 +1,440 @@ +== Security Policy + +We take security seriously. We appreciate your efforts to responsibly +disclose vulnerabilities and will make every effort to acknowledge your +contributions. + +=== Table of Contents + +* link:#reporting-a-vulnerability[Reporting a Vulnerability] +* link:#what-to-include[What to Include] +* link:#response-timeline[Response Timeline] +* link:#disclosure-policy[Disclosure Policy] +* link:#scope[Scope] +* link:#safe-harbour[Safe Harbour] +* link:#recognition[Recognition] +* link:#security-updates[Security Updates] +* link:#security-best-practices[Security Best Practices] + +''''' + +=== Reporting a Vulnerability + +==== Preferred Method: GitHub Security Advisories + +The preferred method for reporting security vulnerabilities is through +GitHub’s Security Advisory feature: + +[arabic] +. Navigate to +https://github.com/hyperpolymath/http-capability-gateway/security/advisories/new[Report +a Vulnerability] +. Click *"`Report a vulnerability`"* +. Complete the form with as much detail as possible +. Submit — we’ll receive a private notification + +This method ensures: + +* End-to-end encryption of your report +* Private discussion space for collaboration +* Coordinated disclosure tooling +* Automatic credit when the advisory is published + +==== Alternative: Encrypted Email + +If you cannot use GitHub Security Advisories, you may email us directly: + +[cols=",",] +|=== +|*Email* |6759885+hyperpolymath@users.noreply.github.com +|*Fingerprint* |`+[PGP fingerprint not set]+` +|=== + +____ +*⚠️ Important:* Do not report security vulnerabilities through public +GitHub issues, pull requests, discussions, or social media. +____ + +''''' + +=== What to Include + +A good vulnerability report helps us understand and reproduce the issue +quickly. + +==== Required Information + +* *Description*: Clear explanation of the vulnerability +* *Impact*: What an attacker could achieve (confidentiality, integrity, +availability) +* *Affected versions*: Which versions/commits are affected +* *Reproduction steps*: Detailed steps to reproduce the issue + +==== Helpful Additional Information + +* *Proof of concept*: Code, scripts, or screenshots demonstrating the +vulnerability +* *Attack scenario*: Realistic attack scenario showing exploitability +* *CVSS score*: Your assessment of severity (use +https://www.first.org/cvss/calculator/3.1[CVSS 3.1 Calculator]) +* *CWE ID*: Common Weakness Enumeration identifier if known +* *Suggested fix*: If you have ideas for remediation +* *References*: Links to related vulnerabilities, research, or +advisories + +==== Example Report Structure + +[source,markdown] +---- +## Summary +[One-sentence description of the vulnerability] + +## Vulnerability Type +[e.g., SQL Injection, XSS, SSRF, Path Traversal, etc.] + +## Affected Component +[File path, function name, API endpoint, etc.] + +## Affected Versions +[Version range or specific commits] + +## Severity Assessment +- CVSS 3.1 Score: [X.X] +- CVSS Vector: [CVSS:3.1/AV:X/AC:X/PR:X/UI:X/S:X/C:X/I:X/A:X] + +## Description +[Detailed technical description] + +## Steps to Reproduce +1. [First step] +2. [Second step] +3. [...] + +## Proof of Concept +[Code, curl commands, screenshots, etc.] + +## Impact +[What can an attacker achieve?] + +## Suggested Remediation +[Optional: your ideas for fixing] + +## References +[Links to related issues, CVEs, research] +---- + +''''' + +=== Response Timeline + +We commit to the following response times: + +[width="100%",cols="24%,35%,41%",options="header",] +|=== +|Stage |Timeframe |Description +|*Initial Response* |48 hours |We acknowledge receipt and confirm we’re +investigating + +|*Triage* |7 days |We assess severity, confirm the vulnerability, and +estimate timeline + +|*Status Update* |Every 7 days |Regular updates on remediation progress + +|*Resolution* |90 days |Target for fix development and release (complex +issues may take longer) + +|*Disclosure* |90 days |Public disclosure after fix is available +(coordinated with you) +|=== + +____ +*Note:* These are targets, not guarantees. Complex vulnerabilities may +require more time. We’ll communicate openly about any delays. +____ + +''''' + +=== Disclosure Policy + +We follow *coordinated disclosure* (also known as responsible +disclosure): + +[arabic] +. *You report* the vulnerability privately +. *We acknowledge* and begin investigation +. *We develop* a fix and prepare a release +. *We coordinate* disclosure timing with you +. *We publish* security advisory and fix simultaneously +. *You may publish* your research after disclosure + +==== Our Commitments + +* We will not take legal action against researchers who follow this +policy +* We will work with you to understand and resolve the issue +* We will credit you in the security advisory (unless you prefer +anonymity) +* We will notify you before public disclosure +* We will publish advisories with sufficient detail for users to assess +risk + +==== Your Commitments + +* Report vulnerabilities promptly after discovery +* Give us reasonable time to address the issue before disclosure +* Do not access, modify, or delete data beyond what’s necessary to +demonstrate the vulnerability +* Do not degrade service availability (no DoS testing on production) +* Do not share vulnerability details with others until coordinated +disclosure + +==== Disclosure Timeline + +.... +Day 0 You report vulnerability +Day 1-2 We acknowledge receipt +Day 7 We confirm vulnerability and share initial assessment +Day 7-90 We develop and test fix +Day 90 Coordinated public disclosure + (earlier if fix is ready; later by mutual agreement) +.... + +If we cannot reach agreement on disclosure timing, we default to 90 days +from your initial report. + +''''' + +=== Scope + +==== In Scope ✅ + +The following are within scope for security research: + +* This repository (`+hyperpolymath/http-capability-gateway+`) and all +its code +* Official releases and packages published from this repository +* Documentation that could lead to security issues +* Build and deployment configurations in this repository +* Dependencies (report here, we’ll coordinate with upstream) + +==== Out of Scope ❌ + +The following are *not* in scope: + +* Third-party services we integrate with (report directly to them) +* Social engineering attacks against maintainers +* Physical security +* Denial of service attacks against production infrastructure +* Spam, phishing, or other non-technical attacks +* Issues already reported or publicly known +* Theoretical vulnerabilities without proof of concept + +==== Qualifying Vulnerabilities + +We’re particularly interested in: + +* Remote code execution +* SQL injection, command injection, code injection +* Authentication/authorisation bypass +* Cross-site scripting (XSS) and cross-site request forgery (CSRF) +* Server-side request forgery (SSRF) +* Path traversal / local file inclusion +* Information disclosure (credentials, PII, secrets) +* Cryptographic weaknesses +* Deserialisation vulnerabilities +* Memory safety issues (buffer overflows, use-after-free, etc.) +* Supply chain vulnerabilities (dependency confusion, etc.) +* Significant logic flaws + +==== Non-Qualifying Issues + +The following generally do not qualify as security vulnerabilities: + +* Missing security headers on non-sensitive pages +* Clickjacking on pages without sensitive actions +* Self-XSS (requires victim to paste code) +* Missing rate limiting (unless it enables a specific attack) +* Username/email enumeration (unless high-risk context) +* Missing cookie flags on non-sensitive cookies +* Software version disclosure +* Verbose error messages (unless exposing secrets) +* Best practice deviations without demonstrable impact + +''''' + +=== Safe Harbour + +We support security research conducted in good faith. + +==== Our Promise + +If you conduct security research in accordance with this policy: + +* ✅ We will not initiate legal action against you +* ✅ We will not report your activity to law enforcement +* ✅ We will work with you in good faith to resolve issues +* ✅ We consider your research authorised under the Computer Fraud and +Abuse Act (CFAA), UK Computer Misuse Act, and similar laws +* ✅ We waive any potential claim against you for circumvention of +security controls + +==== Good Faith Requirements + +To qualify for safe harbour, you must: + +* Comply with this security policy +* Report vulnerabilities promptly +* Avoid privacy violations (do not access others’ data) +* Avoid service degradation (no destructive testing) +* Not exploit vulnerabilities beyond proof-of-concept +* Not use vulnerabilities for profit (beyond bug bounties where offered) + +____ +*⚠️ Important:* This safe harbour does not extend to third-party +systems. Always check their policies before testing. +____ + +''''' + +=== Recognition + +We believe in recognising security researchers who help us improve. + +==== Hall of Fame + +Researchers who report valid vulnerabilities will be acknowledged in our +link:SECURITY-ACKNOWLEDGMENTS.md[Security Acknowledgments] (unless they +prefer anonymity). + +Recognition includes: + +* Your name (or chosen alias) +* Link to your website/profile (optional) +* Brief description of the vulnerability class +* Date of report + +==== What We Offer + +* ✅ Public credit in security advisories +* ✅ Acknowledgment in release notes +* ✅ Entry in our Hall of Fame +* ✅ Reference/recommendation letter upon request (for significant +findings) + +==== What We Don’t Currently Offer + +* ❌ Monetary bug bounties +* ❌ Hardware or swag +* ❌ Paid security research contracts + +____ +*Note:* We’re a community project with limited resources. Your +contributions help everyone who uses this software. +____ + +''''' + +=== Security Updates + +==== Receiving Updates + +To stay informed about security updates: + +* *Watch this repository*: Click "`Watch`" → "`Custom`" → Select +"`Security alerts`" +* *GitHub Security Advisories*: Published at +https://github.com/hyperpolymath/http-capability-gateway/security/advisories[Security +Advisories] +* *Release notes*: Security fixes noted in link:CHANGELOG.md[CHANGELOG] + +==== Update Policy + +[cols=",",options="header",] +|=== +|Severity |Response +|*Critical/High* |Patch release as soon as fix is ready +|*Medium* |Included in next scheduled release (or earlier) +|*Low* |Included in next scheduled release +|=== + +==== Supported Versions + +[cols=",,",options="header",] +|=== +|Version |Supported |Notes +|`+main+` branch |✅ Yes |Latest development +|Latest release |✅ Yes |Current stable +|Previous minor release |✅ Yes |Security fixes backported +|Older versions |❌ No |Please upgrade +|=== + +''''' + +=== Security Best Practices + +When using Http Capability Gateway, we recommend: + +==== General + +* Keep dependencies up to date +* Use the latest stable release +* Subscribe to security notifications +* Review configuration against security documentation +* Follow principle of least privilege + +==== For Contributors + +* Never commit secrets, credentials, or API keys +* Use signed commits (`+git config commit.gpgsign true+`) +* Review dependencies before adding them +* Run security linters locally before pushing +* Report any concerns about existing code + +''''' + +=== Additional Resources + +* https://github.com/hyperpolymath/http-capability-gateway/security/advisories[Security +Advisories] +* link:CHANGELOG.md[Changelog] +* link:CONTRIBUTING.md[Contributing Guidelines] +* https://cve.mitre.org/[CVE Database] +* https://www.first.org/cvss/calculator/3.1[CVSS Calculator] + +''''' + +=== Contact + +[width="100%",cols="50%,50%",options="header",] +|=== +|Purpose |Contact +|*Security issues* +|https://github.com/hyperpolymath/http-capability-gateway/security/advisories/new[Report +via GitHub] or 6759885+hyperpolymath@users.noreply.github.com + +|*General questions* +|https://github.com/hyperpolymath/http-capability-gateway/discussions[GitHub +Discussions] + +|*Other enquiries* |See link:README.md[README] for contact information +|=== + +''''' + +=== Policy Changes + +This security policy may be updated from time to time. Significant +changes will be: + +* Committed to this repository with a clear commit message +* Noted in the changelog +* Announced via GitHub Discussions (for major changes) + +''''' + +_Thank you for helping keep Http Capability Gateway and its users safe._ +🛡️ + +''''' + +Last updated: 2026 · Policy version: 1.0.0 diff --git a/SECURITY.md b/SECURITY.md deleted file mode 100644 index 9b721c8..0000000 --- a/SECURITY.md +++ /dev/null @@ -1,380 +0,0 @@ - -# Security Policy - -We take security seriously. We appreciate your efforts to responsibly disclose vulnerabilities and will make every effort to acknowledge your contributions. - -## Table of Contents - -- [Reporting a Vulnerability](#reporting-a-vulnerability) -- [What to Include](#what-to-include) -- [Response Timeline](#response-timeline) -- [Disclosure Policy](#disclosure-policy) -- [Scope](#scope) -- [Safe Harbour](#safe-harbour) -- [Recognition](#recognition) -- [Security Updates](#security-updates) -- [Security Best Practices](#security-best-practices) - ---- - -## Reporting a Vulnerability - -### Preferred Method: GitHub Security Advisories - -The preferred method for reporting security vulnerabilities is through GitHub's Security Advisory feature: - -1. Navigate to [Report a Vulnerability](https://github.com/hyperpolymath/http-capability-gateway/security/advisories/new) -2. Click **"Report a vulnerability"** -3. Complete the form with as much detail as possible -4. Submit — we'll receive a private notification - -This method ensures: - -- End-to-end encryption of your report -- Private discussion space for collaboration -- Coordinated disclosure tooling -- Automatic credit when the advisory is published - -### Alternative: Encrypted Email - -If you cannot use GitHub Security Advisories, you may email us directly: - -| | | -|---|---| -| **Email** | 6759885+hyperpolymath@users.noreply.github.com | -| **Fingerprint** | `[PGP fingerprint not set]` | - - -> **⚠️ Important:** Do not report security vulnerabilities through public GitHub issues, pull requests, discussions, or social media. - ---- - -## What to Include - -A good vulnerability report helps us understand and reproduce the issue quickly. - -### Required Information - -- **Description**: Clear explanation of the vulnerability -- **Impact**: What an attacker could achieve (confidentiality, integrity, availability) -- **Affected versions**: Which versions/commits are affected -- **Reproduction steps**: Detailed steps to reproduce the issue - -### Helpful Additional Information - -- **Proof of concept**: Code, scripts, or screenshots demonstrating the vulnerability -- **Attack scenario**: Realistic attack scenario showing exploitability -- **CVSS score**: Your assessment of severity (use [CVSS 3.1 Calculator](https://www.first.org/cvss/calculator/3.1)) -- **CWE ID**: Common Weakness Enumeration identifier if known -- **Suggested fix**: If you have ideas for remediation -- **References**: Links to related vulnerabilities, research, or advisories - -### Example Report Structure - -```markdown -## Summary -[One-sentence description of the vulnerability] - -## Vulnerability Type -[e.g., SQL Injection, XSS, SSRF, Path Traversal, etc.] - -## Affected Component -[File path, function name, API endpoint, etc.] - -## Affected Versions -[Version range or specific commits] - -## Severity Assessment -- CVSS 3.1 Score: [X.X] -- CVSS Vector: [CVSS:3.1/AV:X/AC:X/PR:X/UI:X/S:X/C:X/I:X/A:X] - -## Description -[Detailed technical description] - -## Steps to Reproduce -1. [First step] -2. [Second step] -3. [...] - -## Proof of Concept -[Code, curl commands, screenshots, etc.] - -## Impact -[What can an attacker achieve?] - -## Suggested Remediation -[Optional: your ideas for fixing] - -## References -[Links to related issues, CVEs, research] -``` - ---- - -## Response Timeline - -We commit to the following response times: - -| Stage | Timeframe | Description | -|-------|-----------|-------------| -| **Initial Response** | 48 hours | We acknowledge receipt and confirm we're investigating | -| **Triage** | 7 days | We assess severity, confirm the vulnerability, and estimate timeline | -| **Status Update** | Every 7 days | Regular updates on remediation progress | -| **Resolution** | 90 days | Target for fix development and release (complex issues may take longer) | -| **Disclosure** | 90 days | Public disclosure after fix is available (coordinated with you) | - -> **Note:** These are targets, not guarantees. Complex vulnerabilities may require more time. We'll communicate openly about any delays. - ---- - -## Disclosure Policy - -We follow **coordinated disclosure** (also known as responsible disclosure): - -1. **You report** the vulnerability privately -2. **We acknowledge** and begin investigation -3. **We develop** a fix and prepare a release -4. **We coordinate** disclosure timing with you -5. **We publish** security advisory and fix simultaneously -6. **You may publish** your research after disclosure - -### Our Commitments - -- We will not take legal action against researchers who follow this policy -- We will work with you to understand and resolve the issue -- We will credit you in the security advisory (unless you prefer anonymity) -- We will notify you before public disclosure -- We will publish advisories with sufficient detail for users to assess risk - -### Your Commitments - -- Report vulnerabilities promptly after discovery -- Give us reasonable time to address the issue before disclosure -- Do not access, modify, or delete data beyond what's necessary to demonstrate the vulnerability -- Do not degrade service availability (no DoS testing on production) -- Do not share vulnerability details with others until coordinated disclosure - -### Disclosure Timeline - -``` -Day 0 You report vulnerability -Day 1-2 We acknowledge receipt -Day 7 We confirm vulnerability and share initial assessment -Day 7-90 We develop and test fix -Day 90 Coordinated public disclosure - (earlier if fix is ready; later by mutual agreement) -``` - -If we cannot reach agreement on disclosure timing, we default to 90 days from your initial report. - ---- - -## Scope - -### In Scope ✅ - -The following are within scope for security research: - -- This repository (`hyperpolymath/http-capability-gateway`) and all its code -- Official releases and packages published from this repository -- Documentation that could lead to security issues -- Build and deployment configurations in this repository -- Dependencies (report here, we'll coordinate with upstream) - -### Out of Scope ❌ - -The following are **not** in scope: - -- Third-party services we integrate with (report directly to them) -- Social engineering attacks against maintainers -- Physical security -- Denial of service attacks against production infrastructure -- Spam, phishing, or other non-technical attacks -- Issues already reported or publicly known -- Theoretical vulnerabilities without proof of concept - -### Qualifying Vulnerabilities - -We're particularly interested in: - -- Remote code execution -- SQL injection, command injection, code injection -- Authentication/authorisation bypass -- Cross-site scripting (XSS) and cross-site request forgery (CSRF) -- Server-side request forgery (SSRF) -- Path traversal / local file inclusion -- Information disclosure (credentials, PII, secrets) -- Cryptographic weaknesses -- Deserialisation vulnerabilities -- Memory safety issues (buffer overflows, use-after-free, etc.) -- Supply chain vulnerabilities (dependency confusion, etc.) -- Significant logic flaws - -### Non-Qualifying Issues - -The following generally do not qualify as security vulnerabilities: - -- Missing security headers on non-sensitive pages -- Clickjacking on pages without sensitive actions -- Self-XSS (requires victim to paste code) -- Missing rate limiting (unless it enables a specific attack) -- Username/email enumeration (unless high-risk context) -- Missing cookie flags on non-sensitive cookies -- Software version disclosure -- Verbose error messages (unless exposing secrets) -- Best practice deviations without demonstrable impact - ---- - -## Safe Harbour - -We support security research conducted in good faith. - -### Our Promise - -If you conduct security research in accordance with this policy: - -- ✅ We will not initiate legal action against you -- ✅ We will not report your activity to law enforcement -- ✅ We will work with you in good faith to resolve issues -- ✅ We consider your research authorised under the Computer Fraud and Abuse Act (CFAA), UK Computer Misuse Act, and similar laws -- ✅ We waive any potential claim against you for circumvention of security controls - -### Good Faith Requirements - -To qualify for safe harbour, you must: - -- Comply with this security policy -- Report vulnerabilities promptly -- Avoid privacy violations (do not access others' data) -- Avoid service degradation (no destructive testing) -- Not exploit vulnerabilities beyond proof-of-concept -- Not use vulnerabilities for profit (beyond bug bounties where offered) - -> **⚠️ Important:** This safe harbour does not extend to third-party systems. Always check their policies before testing. - ---- - -## Recognition - -We believe in recognising security researchers who help us improve. - -### Hall of Fame - -Researchers who report valid vulnerabilities will be acknowledged in our [Security Acknowledgments](SECURITY-ACKNOWLEDGMENTS.md) (unless they prefer anonymity). - -Recognition includes: - -- Your name (or chosen alias) -- Link to your website/profile (optional) -- Brief description of the vulnerability class -- Date of report - -### What We Offer - -- ✅ Public credit in security advisories -- ✅ Acknowledgment in release notes -- ✅ Entry in our Hall of Fame -- ✅ Reference/recommendation letter upon request (for significant findings) - -### What We Don't Currently Offer - -- ❌ Monetary bug bounties -- ❌ Hardware or swag -- ❌ Paid security research contracts - -> **Note:** We're a community project with limited resources. Your contributions help everyone who uses this software. - ---- - -## Security Updates - -### Receiving Updates - -To stay informed about security updates: - -- **Watch this repository**: Click "Watch" → "Custom" → Select "Security alerts" -- **GitHub Security Advisories**: Published at [Security Advisories](https://github.com/hyperpolymath/http-capability-gateway/security/advisories) -- **Release notes**: Security fixes noted in [CHANGELOG](CHANGELOG.md) - -### Update Policy - -| Severity | Response | -|----------|----------| -| **Critical/High** | Patch release as soon as fix is ready | -| **Medium** | Included in next scheduled release (or earlier) | -| **Low** | Included in next scheduled release | - -### Supported Versions - - - -| Version | Supported | Notes | -|---------|-----------|-------| -| `main` branch | ✅ Yes | Latest development | -| Latest release | ✅ Yes | Current stable | -| Previous minor release | ✅ Yes | Security fixes backported | -| Older versions | ❌ No | Please upgrade | - ---- - -## Security Best Practices - -When using Http Capability Gateway, we recommend: - -### General - -- Keep dependencies up to date -- Use the latest stable release -- Subscribe to security notifications -- Review configuration against security documentation -- Follow principle of least privilege - -### For Contributors - -- Never commit secrets, credentials, or API keys -- Use signed commits (`git config commit.gpgsign true`) -- Review dependencies before adding them -- Run security linters locally before pushing -- Report any concerns about existing code - ---- - -## Additional Resources - -- [Security Advisories](https://github.com/hyperpolymath/http-capability-gateway/security/advisories) -- [Changelog](CHANGELOG.md) -- [Contributing Guidelines](CONTRIBUTING.md) -- [CVE Database](https://cve.mitre.org/) -- [CVSS Calculator](https://www.first.org/cvss/calculator/3.1) - ---- - -## Contact - -| Purpose | Contact | -|---------|---------| -| **Security issues** | [Report via GitHub](https://github.com/hyperpolymath/http-capability-gateway/security/advisories/new) or 6759885+hyperpolymath@users.noreply.github.com | -| **General questions** | [GitHub Discussions](https://github.com/hyperpolymath/http-capability-gateway/discussions) | -| **Other enquiries** | See [README](README.md) for contact information | - ---- - -## Policy Changes - -This security policy may be updated from time to time. Significant changes will be: - -- Committed to this repository with a clear commit message -- Noted in the changelog -- Announced via GitHub Discussions (for major changes) - ---- - -*Thank you for helping keep Http Capability Gateway and its users safe.* 🛡️ - ---- - -Last updated: 2026 · Policy version: 1.0.0 diff --git a/TEST-NEEDS.adoc b/TEST-NEEDS.adoc new file mode 100644 index 0000000..bc9a7f5 --- /dev/null +++ b/TEST-NEEDS.adoc @@ -0,0 +1,100 @@ +== TEST-NEEDS.md — http-capability-gateway + +=== CRG Grade: C — ACHIEVED 2026-04-04 + +____ +Generated 2026-03-29 by punishing audit. Superseded 2026-04-16 by the +P0/P1/P2 test work documented below. +____ + +=== Current State (updated 2026-04-16) + +[width="100%",cols="50%,25%,25%",options="header",] +|=== +|Category |Count |Notes +|Unit tests |9 |gateway, policy_compiler, policy_loader, +policy_validator, policy_property, performance, http_capability_gateway, +*circuit_breaker*, *k9_contract* + +|Security |1 |security_test.exs: sanitization, headers, SSRF, capability +tokens (30+ tests) + +|E2E |1 |e2e_test.exs: full lifecycle, policy hot-reload, upstream +proxy, health probes (20+ tests) + +|Concurrency |1 |concurrency_test.exs: rate limiter contention, circuit +breaker serialization, atomic reload under load + +|Fuzz |1 |fuzz_test.exs: property-based fuzzing with StreamData (6 +properties) + +|Benchmarks |2 |performance_test.exs (existing) + benchmark_test.exs +(rate limiter / circuit breaker / route lookup) +|=== + +*Source modules:* ~19 Elixir modules + 2 Idris2 ABI + 2 Zig FFI parsers. + +=== Coverage Summary + +==== ✅ Covered + +* *P2P (Property-Based) Tests* +** Policy compilation: arbitrary YAML through compiler +(`+test/fuzz_test.exs+`) +** Circuit breaker: state machine transitions +(`+test/circuit_breaker_test.exs+`) +** Rate limiter: token bucket under contention +(`+test/concurrency_test.exs+`) +* *E2E Tests* +** Full request lifecycle (`+test/e2e_test.exs+`) +** Policy hot-reload under load (`+test/concurrency_test.exs+`) +** Health check / readiness probe validation (`+test/e2e_test.exs+`) +* *Aspect Tests* +** *Security:* Request sanitization, header injection, SSRF prevention, +capability token validation (`+test/security_test.exs+`) +** *Concurrency:* Rate limiter and circuit breaker under contention +(`+test/concurrency_test.exs+`) +** *Performance:* Rate limiter, circuit breaker, route lookup benchmarks +(`+test/benchmark_test.exs+`) +* *Benchmarks* +** Rate limiter throughput (`+test/benchmark_test.exs+`) +** Circuit breaker state transition cost (`+test/benchmark_test.exs+`) +** Exact vs regex vs global-fallback route lookup +(`+test/benchmark_test.exs+`) +** Policy evaluation overhead (`+test/performance_test.exs+`) +** Full plug pipeline throughput (`+test/benchmark_test.exs+`) + +==== ⚠️ Still Missing + +* *Multi-protocol routing tests* — GraphQL/gRPC handlers are stubs per +`+docs/SUPPORTED-FEATURES.md+`, so this is out of MVP scope rather than +"`missing`". +* *Zig FFI integration test execution* — requires zig toolchain; covered +by separate FFI build step. +* *Container build smoke test* — performed in CI, not in `+mix test+`. +* *Error handling: upstream timeout* — Req receive_timeout covered +implicitly; no dedicated test. +* [line-through]#*Real-CA mTLS integration test* — code uses +`+Record.extract+` accessors but no live cert in test fixtures.# *CLOSED +(Phase B / standards#97):* `+test/mtls_test.exs+` drives the cert→trust +pipeline with a real test CA (`+test/fixtures/mtls/+`) and proves the CA +trust invariant via `+:public_key.pkix_path_validation/3+`. Live-socket +handshake test across the gateway↔BoJ seam is Phase C scope. +* *Self-tests for config validation on startup* — Application.start +refuses without policy, but no dedicated assertion. + +=== Priority + +Originally *CRITICAL* when only 7 unit tests covered 19 modules. Now: +the release gate in `+docs/RELEASE-CRITERIA.md+` maps every MVP claim to +a concrete test file. Remaining items are clearly marked above and are +not release blockers for v0.1.0. + +=== FUZZ STATUS + +* `+tests/fuzz/placeholder.txt+` has been removed (was a scorecard +placeholder, not real fuzzing). +* Real property-based fuzz tests added in `+test/fuzz_test.exs+` using +StreamData. +* Covers: arbitrary HTTP methods, trust strings, paths, policies, and +combined input fuzzing. diff --git a/TEST-NEEDS.md b/TEST-NEEDS.md deleted file mode 100644 index fefcbdc..0000000 --- a/TEST-NEEDS.md +++ /dev/null @@ -1,71 +0,0 @@ - -# TEST-NEEDS.md — http-capability-gateway - -## CRG Grade: C — ACHIEVED 2026-04-04 - -> Generated 2026-03-29 by punishing audit. Superseded 2026-04-16 by the -> P0/P1/P2 test work documented below. - -## Current State (updated 2026-04-16) - -| Category | Count | Notes | -|-------------|-------|-------| -| Unit tests | 9 | gateway, policy_compiler, policy_loader, policy_validator, policy_property, performance, http_capability_gateway, **circuit_breaker**, **k9_contract** | -| Security | 1 | security_test.exs: sanitization, headers, SSRF, capability tokens (30+ tests) | -| E2E | 1 | e2e_test.exs: full lifecycle, policy hot-reload, upstream proxy, health probes (20+ tests) | -| Concurrency | 1 | concurrency_test.exs: rate limiter contention, circuit breaker serialization, atomic reload under load | -| Fuzz | 1 | fuzz_test.exs: property-based fuzzing with StreamData (6 properties) | -| Benchmarks | 2 | performance_test.exs (existing) + benchmark_test.exs (rate limiter / circuit breaker / route lookup) | - -**Source modules:** ~19 Elixir modules + 2 Idris2 ABI + 2 Zig FFI parsers. - -## Coverage Summary - -### ✅ Covered - -- **P2P (Property-Based) Tests** - - Policy compilation: arbitrary YAML through compiler (`test/fuzz_test.exs`) - - Circuit breaker: state machine transitions (`test/circuit_breaker_test.exs`) - - Rate limiter: token bucket under contention (`test/concurrency_test.exs`) - -- **E2E Tests** - - Full request lifecycle (`test/e2e_test.exs`) - - Policy hot-reload under load (`test/concurrency_test.exs`) - - Health check / readiness probe validation (`test/e2e_test.exs`) - -- **Aspect Tests** - - **Security:** Request sanitization, header injection, SSRF prevention, capability token validation (`test/security_test.exs`) - - **Concurrency:** Rate limiter and circuit breaker under contention (`test/concurrency_test.exs`) - - **Performance:** Rate limiter, circuit breaker, route lookup benchmarks (`test/benchmark_test.exs`) - -- **Benchmarks** - - Rate limiter throughput (`test/benchmark_test.exs`) - - Circuit breaker state transition cost (`test/benchmark_test.exs`) - - Exact vs regex vs global-fallback route lookup (`test/benchmark_test.exs`) - - Policy evaluation overhead (`test/performance_test.exs`) - - Full plug pipeline throughput (`test/benchmark_test.exs`) - -### ⚠️ Still Missing - -- **Multi-protocol routing tests** — GraphQL/gRPC handlers are stubs per `docs/SUPPORTED-FEATURES.md`, so this is out of MVP scope rather than "missing". -- **Zig FFI integration test execution** — requires zig toolchain; covered by separate FFI build step. -- **Container build smoke test** — performed in CI, not in `mix test`. -- **Error handling: upstream timeout** — Req receive_timeout covered implicitly; no dedicated test. -- ~~**Real-CA mTLS integration test** — code uses `Record.extract` accessors but no live cert in test fixtures.~~ **CLOSED (Phase B / standards#97):** `test/mtls_test.exs` drives the cert→trust pipeline with a real test CA (`test/fixtures/mtls/`) and proves the CA trust invariant via `:public_key.pkix_path_validation/3`. Live-socket handshake test across the gateway↔BoJ seam is Phase C scope. -- **Self-tests for config validation on startup** — Application.start refuses without policy, but no dedicated assertion. - -## Priority - -Originally **CRITICAL** when only 7 unit tests covered 19 modules. -Now: the release gate in `docs/RELEASE-CRITERIA.md` maps every MVP claim -to a concrete test file. Remaining items are clearly marked above and -are not release blockers for v0.1.0. - -## FUZZ STATUS - -- `tests/fuzz/placeholder.txt` has been removed (was a scorecard placeholder, not real fuzzing). -- Real property-based fuzz tests added in `test/fuzz_test.exs` using StreamData. -- Covers: arbitrary HTTP methods, trust strings, paths, policies, and combined input fuzzing. diff --git a/TOPOLOGY.md b/TOPOLOGY.adoc similarity index 53% rename from TOPOLOGY.md rename to TOPOLOGY.adoc index 733fdb0..a541f54 100644 --- a/TOPOLOGY.md +++ b/TOPOLOGY.adoc @@ -1,22 +1,18 @@ - - - +____ +*Note (2026-04-16):* The "`completion percentage`" model used in earlier +versions of this document was misleading — components claimed as +"`100%`" (e.g., mTLS) were verified-broken in review. This document now +reports *implementation status* rather than topology percentages. For +the authoritative "`what works today`" picture, see +`+docs/SUPPORTED-FEATURES.md+`. For release gating, see +`+docs/RELEASE-CRITERIA.md+`. +____ -> **Note (2026-04-16):** The "completion percentage" model used in earlier -> versions of this document was misleading — components claimed as "100%" -> (e.g., mTLS) were verified-broken in review. This document now reports -> **implementation status** rather than topology percentages. For the -> authoritative "what works today" picture, see `docs/SUPPORTED-FEATURES.md`. -> For release gating, see `docs/RELEASE-CRITERIA.md`. +== http-capability-gateway — Project Topology -# http-capability-gateway — Project Topology +=== System Architecture -## System Architecture - -``` +.... ┌─────────────────────────────────────────┐ │ HTTP TRAFFIC │ │ (GET, POST, DELETE, etc.) │ @@ -56,66 +52,110 @@ Copyright (c) Jonathan D.A. Jewell │ Justfile / mix.exs .machine_readable/ │ │ Mustfile / Docker 0-AI-MANIFEST.a2ml │ └─────────────────────────────────────────┘ -``` - -## Component Status - -Statuses below are backed by executed tests. See `docs/SUPPORTED-FEATURES.md` -for detailed caveats. - -| Component | Status | Verified By | -|-----------|--------|-------------| -| **CORE GATEWAY** | | | -| Policy Loader (DSL v1) | Supported | `test/policy_loader_test.exs` | -| Validator | Supported | `test/policy_validator_test.exs` | -| Compiler (Tiered Lookup) | Supported | `test/policy_compiler_test.exs`, `test/benchmark_test.exs` | -| Enforcement Engine | Supported | `test/gateway_test.exs`, `test/e2e_test.exs` | -| Security Headers | Supported | `test/security_test.exs` | -| Atomic Policy Reload | Supported | `test/e2e_test.exs`, `test/concurrency_test.exs` | -| **INTERFACES & LOGS** | | | -| HTTP Proxy Layer | Supported | `test/e2e_test.exs` (502 on backend down) | -| Structured JSON Logs | Supported | Emitted by `log_decision/7`; no direct assertion | -| Stealth Profiles | Supported | `test/gateway_test.exs` stealth describe block | -| Prometheus Metrics | Supported | `GET /metrics` covered by e2e setup | -| **HEALTH & TRUST** | | | -| Health Check (`/health`) | Supported | `test/e2e_test.exs` | -| Readiness Check (`/ready`) | Supported | `test/e2e_test.exs` | -| Trust Header Extraction | Supported | `test/security_test.exs` | -| Trust Header Spoofing Protection | Supported | `test/security_test.exs` | -| mTLS Trust Extraction | Supported with caveats | Code uses `Record.extract` accessors; no integration test against a real CA yet | -| Rate Limiter (trust-scoped) | Supported | `test/concurrency_test.exs`, `test/benchmark_test.exs` | -| Circuit Breaker | Supported | `test/circuit_breaker_test.exs`, `test/concurrency_test.exs` | -| K9 Service Contracts | Supported | `test/k9_contract_test.exs` | -| **PROTOCOL HANDLERS** | | | -| HTTP/REST | Supported | Full test coverage | -| GraphQL | Stub only | `check_operation_policy/2` always returns true; do not use in production | -| gRPC | Stub only | `forward_grpc_request/5` returns hardcoded response; do not use in production | -| **REPO INFRASTRUCTURE** | | | -| Justfile Automation | Supported | N/A (developer tooling) | -| `.machine_readable/` | Supported | `STATE.a2ml` authoritative | -| Containerfile | Supported | Builds documented in `docs/DEPLOYMENT.md` | - -## Key Dependencies - -``` +.... + +=== Component Status + +Statuses below are backed by executed tests. See +`+docs/SUPPORTED-FEATURES.md+` for detailed caveats. + +[width="100%",cols="35%,25%,40%",options="header",] +|=== +|Component |Status |Verified By +|*CORE GATEWAY* | | + +|Policy Loader (DSL v1) |Supported |`+test/policy_loader_test.exs+` + +|Validator |Supported |`+test/policy_validator_test.exs+` + +|Compiler (Tiered Lookup) |Supported |`+test/policy_compiler_test.exs+`, +`+test/benchmark_test.exs+` + +|Enforcement Engine |Supported |`+test/gateway_test.exs+`, +`+test/e2e_test.exs+` + +|Security Headers |Supported |`+test/security_test.exs+` + +|Atomic Policy Reload |Supported |`+test/e2e_test.exs+`, +`+test/concurrency_test.exs+` + +|*INTERFACES & LOGS* | | + +|HTTP Proxy Layer |Supported |`+test/e2e_test.exs+` (502 on backend +down) + +|Structured JSON Logs |Supported |Emitted by `+log_decision/7+`; no +direct assertion + +|Stealth Profiles |Supported |`+test/gateway_test.exs+` stealth describe +block + +|Prometheus Metrics |Supported |`+GET /metrics+` covered by e2e setup + +|*HEALTH & TRUST* | | + +|Health Check (`+/health+`) |Supported |`+test/e2e_test.exs+` + +|Readiness Check (`+/ready+`) |Supported |`+test/e2e_test.exs+` + +|Trust Header Extraction |Supported |`+test/security_test.exs+` + +|Trust Header Spoofing Protection |Supported |`+test/security_test.exs+` + +|mTLS Trust Extraction |Supported with caveats |Code uses +`+Record.extract+` accessors; no integration test against a real CA yet + +|Rate Limiter (trust-scoped) |Supported |`+test/concurrency_test.exs+`, +`+test/benchmark_test.exs+` + +|Circuit Breaker |Supported |`+test/circuit_breaker_test.exs+`, +`+test/concurrency_test.exs+` + +|K9 Service Contracts |Supported |`+test/k9_contract_test.exs+` + +|*PROTOCOL HANDLERS* | | + +|HTTP/REST |Supported |Full test coverage + +|GraphQL |Stub only |`+check_operation_policy/2+` always returns true; +do not use in production + +|gRPC |Stub only |`+forward_grpc_request/5+` returns hardcoded response; +do not use in production + +|*REPO INFRASTRUCTURE* | | + +|Justfile Automation |Supported |N/A (developer tooling) + +|`+.machine_readable/+` |Supported |`+STATE.a2ml+` authoritative + +|Containerfile |Supported |Builds documented in `+docs/DEPLOYMENT.md+` +|=== + +=== Key Dependencies + +.... Policy Spec (DSL) ───► Validator ───► Compiler ───► Rule Table │ ▼ HTTP Traffic ───────► Enforcement ───────────────► Forward / Block -``` +.... -## Update Protocol +=== Update Protocol This file is maintained by both humans and AI agents. When updating: -1. **Status changes**: A component moves to "Supported" only when it has at - least one executed test. Do not claim completion based on code presence. -2. **Adding a component**: Add a new row with the test file that verifies it. - If no test exists, mark as "Stub only" or "Not implemented". -3. **Architectural changes**: Update the ASCII diagram in the System Architecture section. -4. **Date**: Update the `Last updated` comment at the top of this file. -5. **No percentages**: Percentage-based completion claims are banned — - they encouraged unjustified optimism (see 2026-04-16 correction note). - -Progress bars use: `█` (filled) and `░` (empty), 10 characters wide. -Percentages: 0%, 10%, 20%, ... 100% (in 10% increments). +[arabic] +. *Status changes*: A component moves to "`Supported`" only when it has +at least one executed test. Do not claim completion based on code +presence. +. *Adding a component*: Add a new row with the test file that verifies +it. If no test exists, mark as "`Stub only`" or "`Not implemented`". +. *Architectural changes*: Update the ASCII diagram in the System +Architecture section. +. *Date*: Update the `+Last updated+` comment at the top of this file. +. *No percentages*: Percentage-based completion claims are banned — they +encouraged unjustified optimism (see 2026-04-16 correction note). + +Progress bars use: `+█+` (filled) and `+░+` (empty), 10 characters wide. +Percentages: 0%, 10%, 20%, … 100% (in 10% increments). diff --git a/docs/API.adoc b/docs/API.adoc new file mode 100644 index 0000000..7fe4017 --- /dev/null +++ b/docs/API.adoc @@ -0,0 +1,665 @@ +== API Documentation + +Complete API reference for HTTP Capability Gateway modules. + +=== Table of Contents + +[arabic] +. link:#policyloader[PolicyLoader] +. link:#policyvalidator[PolicyValidator] +. link:#policycompiler[PolicyCompiler] +. link:#gateway[Gateway] +. link:#proxy[Proxy] +. link:#logging[Logging] +. link:#logformatter[LogFormatter] +. link:#application[Application] + +''''' + +=== PolicyLoader + +*Module*: `+HttpCapabilityGateway.PolicyLoader+` + +Loads and parses YAML policy files. + +==== Functions + +===== `+load_policy/1+` + +Loads a policy from a YAML string. + +*Signature*: + +[source,elixir] +---- +@spec load_policy(binary()) :: {:ok, map()} | {:error, binary()} +---- + +*Parameters*: - `+yaml_content+` (binary): YAML policy content as string + +*Returns*: - `+{:ok, policy}+` - Successfully parsed policy map - +`+{:error, reason}+` - Error message string + +*Examples*: + +[source,elixir] +---- +# Success case +yaml = """ +dsl_version: "1" +governance: + global_verbs: + - GET + - POST +""" + +{:ok, policy} = PolicyLoader.load_policy(yaml) +# => {:ok, %{"dsl_version" => "1", "governance" => %{"global_verbs" => ["GET", "POST"]}}} + +# Error case +{:error, reason} = PolicyLoader.load_policy("invalid: yaml: [") +# => {:error, "YAML parsing error: ..."} +---- + +''''' + +===== `+load_from_file/1+` + +Loads a policy from a YAML file path. + +*Signature*: + +[source,elixir] +---- +@spec load_from_file(binary()) :: {:ok, map()} | {:error, binary()} +---- + +*Parameters*: - `+file_path+` (binary): Path to YAML policy file + +*Returns*: - `+{:ok, policy}+` - Successfully loaded and parsed policy - +`+{:error, reason}+` - Error message (file not found, parse error) + +*Examples*: + +[source,elixir] +---- +# Success +{:ok, policy} = PolicyLoader.load_from_file("priv/config/policy.dev.yaml") + +# File not found +{:error, "File not found: /path/to/missing.yaml"} = + PolicyLoader.load_from_file("/path/to/missing.yaml") +---- + +''''' + +=== PolicyValidator + +*Module*: `+HttpCapabilityGateway.PolicyValidator+` + +Validates policy structure against DSL v1 schema. + +==== Functions + +===== `+validate/1+` + +Validates a policy map against the DSL v1 schema. + +*Signature*: + +[source,elixir] +---- +@spec validate(map()) :: :ok | {:error, binary()} +---- + +*Parameters*: - `+policy+` (map): Policy map from PolicyLoader + +*Returns*: - `+:ok+` - Policy is valid - `+{:error, reason}+` - +Validation error message + +*Validation Checks*: 1. `+dsl_version+` is present and equals `+"1"+` 2. +`+governance+` is present and is a map 3. `+governance.global_verbs+` is +present and non-empty array 4. All verbs are valid HTTP verbs 5. Routes +(if present) have `+path+` and `+verbs+` fields 6. Stealth config (if +present) has `+enabled+` and `+status_code+` + +*Examples*: + +[source,elixir] +---- +# Valid policy +policy = %{ + "dsl_version" => "1", + "governance" => %{ + "global_verbs" => ["GET", "POST"] + } +} +:ok = PolicyValidator.validate(policy) + +# Invalid: missing dsl_version +policy = %{"governance" => %{"global_verbs" => ["GET"]}} +{:error, "dsl_version: must be present"} = PolicyValidator.validate(policy) + +# Invalid: empty global_verbs +policy = %{ + "dsl_version" => "1", + "governance" => %{"global_verbs" => []} +} +{:error, "global_verbs: must not be empty"} = PolicyValidator.validate(policy) + +# Invalid: bad HTTP verb +policy = %{ + "dsl_version" => "1", + "governance" => %{"global_verbs" => ["GET", "INVALID_VERB"]} +} +{:error, "Invalid HTTP verb: INVALID_VERB"} = PolicyValidator.validate(policy) +---- + +''''' + +=== PolicyCompiler + +*Module*: `+HttpCapabilityGateway.PolicyCompiler+` + +Compiles validated policies to ETS tables for fast lookups. + +==== Functions + +===== `+compile/1+` + +Compiles a policy to ETS tables. + +*Signature*: + +[source,elixir] +---- +@spec compile(map()) :: :ok | {:error, binary()} +---- + +*Parameters*: - `+policy+` (map): Validated policy from PolicyValidator + +*Returns*: - `+:ok+` - Policy compiled successfully - +`+{:error, reason}+` - Compilation error + +*Side Effects*: - Creates or updates `+:gateway_rules+` ETS table - +Creates or updates `+:stealth_config+` ETS table + +*Examples*: + +[source,elixir] +---- +policy = %{ + "dsl_version" => "1", + "governance" => %{ + "global_verbs" => ["GET", "POST"], + "routes" => [ + %{"path" => "/api/admin", "verbs" => ["GET"]} + ] + }, + "stealth" => %{ + "enabled" => true, + "status_code" => 404 + } +} + +:ok = PolicyCompiler.compile(policy) +---- + +''''' + +===== `+is_verb_allowed?/2+` + +Checks if an HTTP verb is allowed for a given path. + +*Signature*: + +[source,elixir] +---- +@spec is_verb_allowed?(binary(), binary()) :: boolean() +---- + +*Parameters*: - `+path+` (binary): Request path (e.g., +`+"/api/users/123"+`) - `+verb+` (binary): HTTP verb (e.g., `+"GET"+`) + +*Returns*: - `+true+` - Verb is allowed for this path - `+false+` - Verb +is not allowed + +*Logic*: 1. Check if path matches any route pattern 2. If matched, check +route-specific verbs 3. If not matched, check global verbs + +*Examples*: + +[source,elixir] +---- +# Assuming policy compiled with: +# global_verbs: ["GET", "POST"] +# routes: [%{"path" => "/api/admin", "verbs" => ["GET"]}] + +# Global verb on unspecified route +true = PolicyCompiler.is_verb_allowed?("/api/public", "GET") + +# Non-global verb on unspecified route +false = PolicyCompiler.is_verb_allowed?("/api/public", "DELETE") + +# Route-specific verb +true = PolicyCompiler.is_verb_allowed?("/api/admin", "GET") +false = PolicyCompiler.is_verb_allowed?("/api/admin", "POST") + +# Regex route matching +true = PolicyCompiler.is_verb_allowed?("/api/users/123", "GET") +false = PolicyCompiler.is_verb_allowed?("/api/users/abc", "DELETE") +---- + +''''' + +===== `+get_stealth_config/0+` + +Retrieves the compiled stealth configuration. + +*Signature*: + +[source,elixir] +---- +@spec get_stealth_config() :: %{enabled: boolean(), status_code: integer()} | nil +---- + +*Returns*: - `+%{enabled: true, status_code: 404}+` - Stealth config - +`+%{enabled: false, status_code: 403}+` - Stealth disabled - `+nil+` - +No stealth config (default to 403) + +*Examples*: + +[source,elixir] +---- +# With stealth enabled +%{enabled: true, status_code: 404} = PolicyCompiler.get_stealth_config() + +# With stealth disabled +%{enabled: false, status_code: 403} = PolicyCompiler.get_stealth_config() +---- + +''''' + +=== Gateway + +*Module*: `+HttpCapabilityGateway.Gateway+` + +HTTP gateway with verb enforcement using Plug. + +==== Functions + +===== `+call/2+` + +Plug callback for handling HTTP requests. + +*Signature*: + +[source,elixir] +---- +@spec call(Plug.Conn.t(), any()) :: Plug.Conn.t() +---- + +*Parameters*: - `+conn+` (Plug.Conn.t): Incoming connection - `+_opts+` +(any): Options (unused) + +*Returns*: - Modified connection with response or proxy + +*Request Flow*: 1. Extract request ID (or generate) 2. Extract trust +level from header 3. Check if verb is allowed for path 4. If allowed: +proxy to backend 5. If denied: return stealth response or 403 + +*Examples*: + +[source,elixir] +---- +# This is typically called by Plug.Cowboy, not manually +conn = %Plug.Conn{method: "GET", request_path: "/api/users"} +conn = Gateway.call(conn, []) +# => Proxied response or error response +---- + +*Assigns*: - `+conn.assigns.request_id+` - UUID for request tracking - +`+conn.assigns.trust_level+` - Extracted trust level (e.g., "`high`") + +''''' + +=== Proxy + +*Module*: `+HttpCapabilityGateway.Proxy+` + +HTTP proxy for forwarding requests to backend services. + +==== Functions + +===== `+forward/1+` + +Forwards a request to the configured backend URL. + +*Signature*: + +[source,elixir] +---- +@spec forward(Plug.Conn.t()) :: Plug.Conn.t() +---- + +*Parameters*: - `+conn+` (Plug.Conn.t): Connection with allowed verb + +*Returns*: - Connection with backend response + +*Behavior*: 1. Constructs backend URL (backend_url + request_path) 2. +Forwards HTTP method, headers, and body 3. Streams response back to +client 4. Preserves status code and headers + +*Examples*: + +[source,elixir] +---- +# Proxy a GET request +conn = %Plug.Conn{ + method: "GET", + request_path: "/api/users/123", + req_headers: [{"x-request-id", "req-abc"}] +} + +conn = Proxy.forward(conn) +# => conn.status = 200, conn.resp_body = "..." (from backend) +---- + +*Forwarded Headers*: - `+X-Request-ID+` - Request correlation ID - +`+X-Trust-Level+` - Extracted trust level - `+X-Forwarded-For+` - Client +IP address - All original request headers + +''''' + +=== Logging + +*Module*: `+HttpCapabilityGateway.Logging+` + +Structured logging with telemetry integration. + +==== Functions + +===== `+log_request/3+` + +Logs a handled request with metadata. + +*Signature*: + +[source,elixir] +---- +@spec log_request(Plug.Conn.t(), boolean(), integer()) :: :ok +---- + +*Parameters*: - `+conn+` (Plug.Conn.t): Request connection - +`+verb_allowed+` (boolean): Whether verb was allowed - `+duration_ms+` +(integer): Request duration in milliseconds + +*Returns*: - `+:ok+` + +*Side Effects*: - Emits telemetry event: +`+[:http_capability_gateway, :request, :handled]+` - Logs structured +JSON log entry + +*Log Fields*: - `+timestamp+` - ISO 8601 timestamp - `+level+` - Log +level (info, warn, error) - `+message+` - "`request_handled`" - +`+request_id+` - Request correlation ID - `+method+` - HTTP method - +`+path+` - Request path - `+trust_level+` - Extracted trust level - +`+verb_allowed+` - Boolean - `+stealth_triggered+` - Boolean - +`+response_status+` - HTTP status code - `+duration_ms+` - Request +duration + +*Examples*: + +[source,elixir] +---- +conn = %Plug.Conn{ + assigns: %{request_id: "req-123", trust_level: "high"}, + method: "GET", + request_path: "/api/users", + status: 200 +} + +Logging.log_request(conn, true, 45) +# Logs: {"timestamp": "...", "message": "request_handled", ...} +---- + +''''' + +=== LogFormatter + +*Module*: `+HttpCapabilityGateway.LogFormatter+` + +JSON log formatter for structured logging. + +==== Functions + +===== `+format/4+` + +Formats log entries as JSON. + +*Signature*: + +[source,elixir] +---- +@spec format(atom(), term(), Logger.Formatter.time(), keyword()) :: IO.chardata() +---- + +*Parameters*: - `+level+` (atom): Log level (:debug, :info, :warn, +:error) - `+message+` (term): Log message - `+timestamp+` +(Logger.Formatter.time): Log timestamp - `+metadata+` (keyword): Log +metadata + +*Returns*: - JSON-formatted log entry as IO.chardata + +*Examples*: + +[source,elixir] +---- +# This is used internally by Logger +Logger.configure_backend(:console, format: {LogFormatter, :format}) + +# Logs will be JSON: +# {"timestamp": "2026-01-22T23:00:00.000Z", "level": "info", "message": "request_handled", ...} +---- + +''''' + +=== Application + +*Module*: `+HttpCapabilityGateway.Application+` + +OTP application for gateway lifecycle management. + +==== Callbacks + +===== `+start/2+` + +Starts the application supervision tree. + +*Signature*: + +[source,elixir] +---- +@spec start(any(), any()) :: {:ok, pid()} | {:error, term()} +---- + +*Behavior*: 1. Loads policy from configured file 2. Validates policy 3. +Compiles policy to ETS 4. Starts HTTP server (Plug.Cowboy) + +*Configuration*: + +[source,elixir] +---- +config :http_capability_gateway, + policy_file: "priv/config/policy.dev.yaml", + backend_url: "http://localhost:4000", + port: 8080 +---- + +*Examples*: + +[source,elixir] +---- +# Start application (usually via mix) +{:ok, pid} = Application.start(:http_capability_gateway) + +# Application will: +# 1. Load policy.yaml +# 2. Validate DSL v1 +# 3. Compile to ETS +# 4. Start HTTP server on port 8080 +---- + +''''' + +=== Type Specifications + +==== Policy Types + +[source,elixir] +---- +@type policy :: %{ + required(String.t()) => String.t() | map() | list(), + "dsl_version" => String.t(), + "governance" => governance(), + "stealth" => stealth() | nil +} + +@type governance :: %{ + required(String.t()) => list() | list(route()), + "global_verbs" => [http_verb()], + "routes" => [route()] | nil +} + +@type route :: %{ + required(String.t()) => String.t() | [http_verb()], + "path" => String.t(), + "verbs" => [http_verb()] +} + +@type stealth :: %{ + required(String.t()) => boolean() | integer(), + "enabled" => boolean(), + "status_code" => integer() +} + +@type http_verb :: String.t() +# Valid: "GET" | "POST" | "PUT" | "DELETE" | "PATCH" | "HEAD" | "OPTIONS" +---- + +''''' + +=== Error Handling + +==== Common Errors + +*PolicyLoader Errors*: - `+{:error, "File not found: ..."}+` - Policy +file doesn’t exist - `+{:error, "YAML parsing error: ..."}+` - Invalid +YAML syntax - `+{:error, "Empty policy"}+` - Policy file is empty + +*PolicyValidator Errors*: - `+{:error, "dsl_version: must be present"}+` +- Missing version field - `+{:error, "Invalid HTTP verb: ..."}+` - +Unknown verb in policy - `+{:error, "global_verbs: must not be empty"}+` +- Empty verb list + +*PolicyCompiler Errors*: - `+{:error, "Invalid regex pattern: ..."}+` - +Bad route pattern + +*Runtime Errors*: - Gateway returns 500 if backend connection fails - +Logs error with request_id for debugging + +''''' + +=== Performance + +==== ETS Lookups + +* *Global verbs*: O(1) lookup +* *Route matching*: O(n) where n = number of routes +* *Verb checking*: O(1) for route match + +==== Benchmarks + +See `+test/performance_test.exs+` for detailed benchmarks: + +[source,bash] +---- +mix test --only performance +---- + +Expected results: - Policy compilation (1000 routes): <100ms - Verb +check: <1ms - Request handling: <5ms (excluding backend) - Throughput: +>1000 req/s + +''''' + +=== Examples + +==== Complete Usage Example + +[source,elixir] +---- +# 1. Load policy +{:ok, policy} = PolicyLoader.load_from_file("policy.yaml") + +# 2. Validate +:ok = PolicyValidator.validate(policy) + +# 3. Compile +:ok = PolicyCompiler.compile(policy) + +# 4. Check verbs +true = PolicyCompiler.is_verb_allowed?("/api/users", "GET") +false = PolicyCompiler.is_verb_allowed?("/api/admin", "DELETE") + +# 5. Get stealth config +%{enabled: true, status_code: 404} = PolicyCompiler.get_stealth_config() + +# 6. Gateway handles requests automatically via Plug +---- + +''''' + +=== Testing + +==== Unit Tests + +[source,bash] +---- +# Test PolicyLoader +mix test test/policy_loader_test.exs + +# Test PolicyValidator +mix test test/policy_validator_test.exs + +# Test PolicyCompiler +mix test test/policy_compiler_test.exs + +# Test Gateway +mix test test/gateway_test.exs +---- + +==== Integration Tests + +[source,bash] +---- +# Full request flow tests +mix test test/gateway_test.exs +---- + +==== Property-Based Tests + +[source,bash] +---- +# Invariant testing with StreamData +mix test test/policy_property_test.exs +---- + +''''' + +=== Support + +For API questions: - *GitHub Discussions*: +https://github.com/hyperpolymath/http-capability-gateway/discussions - +*API Issues*: +https://github.com/hyperpolymath/http-capability-gateway/issues diff --git a/docs/API.md b/docs/API.md deleted file mode 100644 index e196b93..0000000 --- a/docs/API.md +++ /dev/null @@ -1,670 +0,0 @@ - -# API Documentation - -Complete API reference for HTTP Capability Gateway modules. - -## Table of Contents - -1. [PolicyLoader](#policyloader) -2. [PolicyValidator](#policyvalidator) -3. [PolicyCompiler](#policycompiler) -4. [Gateway](#gateway) -5. [Proxy](#proxy) -6. [Logging](#logging) -7. [LogFormatter](#logformatter) -8. [Application](#application) - ---- - -## PolicyLoader - -**Module**: `HttpCapabilityGateway.PolicyLoader` - -Loads and parses YAML policy files. - -### Functions - -#### `load_policy/1` - -Loads a policy from a YAML string. - -**Signature**: -```elixir -@spec load_policy(binary()) :: {:ok, map()} | {:error, binary()} -``` - -**Parameters**: -- `yaml_content` (binary): YAML policy content as string - -**Returns**: -- `{:ok, policy}` - Successfully parsed policy map -- `{:error, reason}` - Error message string - -**Examples**: - -```elixir -# Success case -yaml = """ -dsl_version: "1" -governance: - global_verbs: - - GET - - POST -""" - -{:ok, policy} = PolicyLoader.load_policy(yaml) -# => {:ok, %{"dsl_version" => "1", "governance" => %{"global_verbs" => ["GET", "POST"]}}} - -# Error case -{:error, reason} = PolicyLoader.load_policy("invalid: yaml: [") -# => {:error, "YAML parsing error: ..."} -``` - ---- - -#### `load_from_file/1` - -Loads a policy from a YAML file path. - -**Signature**: -```elixir -@spec load_from_file(binary()) :: {:ok, map()} | {:error, binary()} -``` - -**Parameters**: -- `file_path` (binary): Path to YAML policy file - -**Returns**: -- `{:ok, policy}` - Successfully loaded and parsed policy -- `{:error, reason}` - Error message (file not found, parse error) - -**Examples**: - -```elixir -# Success -{:ok, policy} = PolicyLoader.load_from_file("priv/config/policy.dev.yaml") - -# File not found -{:error, "File not found: /path/to/missing.yaml"} = - PolicyLoader.load_from_file("/path/to/missing.yaml") -``` - ---- - -## PolicyValidator - -**Module**: `HttpCapabilityGateway.PolicyValidator` - -Validates policy structure against DSL v1 schema. - -### Functions - -#### `validate/1` - -Validates a policy map against the DSL v1 schema. - -**Signature**: -```elixir -@spec validate(map()) :: :ok | {:error, binary()} -``` - -**Parameters**: -- `policy` (map): Policy map from PolicyLoader - -**Returns**: -- `:ok` - Policy is valid -- `{:error, reason}` - Validation error message - -**Validation Checks**: -1. `dsl_version` is present and equals `"1"` -2. `governance` is present and is a map -3. `governance.global_verbs` is present and non-empty array -4. All verbs are valid HTTP verbs -5. Routes (if present) have `path` and `verbs` fields -6. Stealth config (if present) has `enabled` and `status_code` - -**Examples**: - -```elixir -# Valid policy -policy = %{ - "dsl_version" => "1", - "governance" => %{ - "global_verbs" => ["GET", "POST"] - } -} -:ok = PolicyValidator.validate(policy) - -# Invalid: missing dsl_version -policy = %{"governance" => %{"global_verbs" => ["GET"]}} -{:error, "dsl_version: must be present"} = PolicyValidator.validate(policy) - -# Invalid: empty global_verbs -policy = %{ - "dsl_version" => "1", - "governance" => %{"global_verbs" => []} -} -{:error, "global_verbs: must not be empty"} = PolicyValidator.validate(policy) - -# Invalid: bad HTTP verb -policy = %{ - "dsl_version" => "1", - "governance" => %{"global_verbs" => ["GET", "INVALID_VERB"]} -} -{:error, "Invalid HTTP verb: INVALID_VERB"} = PolicyValidator.validate(policy) -``` - ---- - -## PolicyCompiler - -**Module**: `HttpCapabilityGateway.PolicyCompiler` - -Compiles validated policies to ETS tables for fast lookups. - -### Functions - -#### `compile/1` - -Compiles a policy to ETS tables. - -**Signature**: -```elixir -@spec compile(map()) :: :ok | {:error, binary()} -``` - -**Parameters**: -- `policy` (map): Validated policy from PolicyValidator - -**Returns**: -- `:ok` - Policy compiled successfully -- `{:error, reason}` - Compilation error - -**Side Effects**: -- Creates or updates `:gateway_rules` ETS table -- Creates or updates `:stealth_config` ETS table - -**Examples**: - -```elixir -policy = %{ - "dsl_version" => "1", - "governance" => %{ - "global_verbs" => ["GET", "POST"], - "routes" => [ - %{"path" => "/api/admin", "verbs" => ["GET"]} - ] - }, - "stealth" => %{ - "enabled" => true, - "status_code" => 404 - } -} - -:ok = PolicyCompiler.compile(policy) -``` - ---- - -#### `is_verb_allowed?/2` - -Checks if an HTTP verb is allowed for a given path. - -**Signature**: -```elixir -@spec is_verb_allowed?(binary(), binary()) :: boolean() -``` - -**Parameters**: -- `path` (binary): Request path (e.g., `"/api/users/123"`) -- `verb` (binary): HTTP verb (e.g., `"GET"`) - -**Returns**: -- `true` - Verb is allowed for this path -- `false` - Verb is not allowed - -**Logic**: -1. Check if path matches any route pattern -2. If matched, check route-specific verbs -3. If not matched, check global verbs - -**Examples**: - -```elixir -# Assuming policy compiled with: -# global_verbs: ["GET", "POST"] -# routes: [%{"path" => "/api/admin", "verbs" => ["GET"]}] - -# Global verb on unspecified route -true = PolicyCompiler.is_verb_allowed?("/api/public", "GET") - -# Non-global verb on unspecified route -false = PolicyCompiler.is_verb_allowed?("/api/public", "DELETE") - -# Route-specific verb -true = PolicyCompiler.is_verb_allowed?("/api/admin", "GET") -false = PolicyCompiler.is_verb_allowed?("/api/admin", "POST") - -# Regex route matching -true = PolicyCompiler.is_verb_allowed?("/api/users/123", "GET") -false = PolicyCompiler.is_verb_allowed?("/api/users/abc", "DELETE") -``` - ---- - -#### `get_stealth_config/0` - -Retrieves the compiled stealth configuration. - -**Signature**: -```elixir -@spec get_stealth_config() :: %{enabled: boolean(), status_code: integer()} | nil -``` - -**Returns**: -- `%{enabled: true, status_code: 404}` - Stealth config -- `%{enabled: false, status_code: 403}` - Stealth disabled -- `nil` - No stealth config (default to 403) - -**Examples**: - -```elixir -# With stealth enabled -%{enabled: true, status_code: 404} = PolicyCompiler.get_stealth_config() - -# With stealth disabled -%{enabled: false, status_code: 403} = PolicyCompiler.get_stealth_config() -``` - ---- - -## Gateway - -**Module**: `HttpCapabilityGateway.Gateway` - -HTTP gateway with verb enforcement using Plug. - -### Functions - -#### `call/2` - -Plug callback for handling HTTP requests. - -**Signature**: -```elixir -@spec call(Plug.Conn.t(), any()) :: Plug.Conn.t() -``` - -**Parameters**: -- `conn` (Plug.Conn.t): Incoming connection -- `_opts` (any): Options (unused) - -**Returns**: -- Modified connection with response or proxy - -**Request Flow**: -1. Extract request ID (or generate) -2. Extract trust level from header -3. Check if verb is allowed for path -4. If allowed: proxy to backend -5. If denied: return stealth response or 403 - -**Examples**: - -```elixir -# This is typically called by Plug.Cowboy, not manually -conn = %Plug.Conn{method: "GET", request_path: "/api/users"} -conn = Gateway.call(conn, []) -# => Proxied response or error response -``` - -**Assigns**: -- `conn.assigns.request_id` - UUID for request tracking -- `conn.assigns.trust_level` - Extracted trust level (e.g., "high") - ---- - -## Proxy - -**Module**: `HttpCapabilityGateway.Proxy` - -HTTP proxy for forwarding requests to backend services. - -### Functions - -#### `forward/1` - -Forwards a request to the configured backend URL. - -**Signature**: -```elixir -@spec forward(Plug.Conn.t()) :: Plug.Conn.t() -``` - -**Parameters**: -- `conn` (Plug.Conn.t): Connection with allowed verb - -**Returns**: -- Connection with backend response - -**Behavior**: -1. Constructs backend URL (backend_url + request_path) -2. Forwards HTTP method, headers, and body -3. Streams response back to client -4. Preserves status code and headers - -**Examples**: - -```elixir -# Proxy a GET request -conn = %Plug.Conn{ - method: "GET", - request_path: "/api/users/123", - req_headers: [{"x-request-id", "req-abc"}] -} - -conn = Proxy.forward(conn) -# => conn.status = 200, conn.resp_body = "..." (from backend) -``` - -**Forwarded Headers**: -- `X-Request-ID` - Request correlation ID -- `X-Trust-Level` - Extracted trust level -- `X-Forwarded-For` - Client IP address -- All original request headers - ---- - -## Logging - -**Module**: `HttpCapabilityGateway.Logging` - -Structured logging with telemetry integration. - -### Functions - -#### `log_request/3` - -Logs a handled request with metadata. - -**Signature**: -```elixir -@spec log_request(Plug.Conn.t(), boolean(), integer()) :: :ok -``` - -**Parameters**: -- `conn` (Plug.Conn.t): Request connection -- `verb_allowed` (boolean): Whether verb was allowed -- `duration_ms` (integer): Request duration in milliseconds - -**Returns**: -- `:ok` - -**Side Effects**: -- Emits telemetry event: `[:http_capability_gateway, :request, :handled]` -- Logs structured JSON log entry - -**Log Fields**: -- `timestamp` - ISO 8601 timestamp -- `level` - Log level (info, warn, error) -- `message` - "request_handled" -- `request_id` - Request correlation ID -- `method` - HTTP method -- `path` - Request path -- `trust_level` - Extracted trust level -- `verb_allowed` - Boolean -- `stealth_triggered` - Boolean -- `response_status` - HTTP status code -- `duration_ms` - Request duration - -**Examples**: - -```elixir -conn = %Plug.Conn{ - assigns: %{request_id: "req-123", trust_level: "high"}, - method: "GET", - request_path: "/api/users", - status: 200 -} - -Logging.log_request(conn, true, 45) -# Logs: {"timestamp": "...", "message": "request_handled", ...} -``` - ---- - -## LogFormatter - -**Module**: `HttpCapabilityGateway.LogFormatter` - -JSON log formatter for structured logging. - -### Functions - -#### `format/4` - -Formats log entries as JSON. - -**Signature**: -```elixir -@spec format(atom(), term(), Logger.Formatter.time(), keyword()) :: IO.chardata() -``` - -**Parameters**: -- `level` (atom): Log level (:debug, :info, :warn, :error) -- `message` (term): Log message -- `timestamp` (Logger.Formatter.time): Log timestamp -- `metadata` (keyword): Log metadata - -**Returns**: -- JSON-formatted log entry as IO.chardata - -**Examples**: - -```elixir -# This is used internally by Logger -Logger.configure_backend(:console, format: {LogFormatter, :format}) - -# Logs will be JSON: -# {"timestamp": "2026-01-22T23:00:00.000Z", "level": "info", "message": "request_handled", ...} -``` - ---- - -## Application - -**Module**: `HttpCapabilityGateway.Application` - -OTP application for gateway lifecycle management. - -### Callbacks - -#### `start/2` - -Starts the application supervision tree. - -**Signature**: -```elixir -@spec start(any(), any()) :: {:ok, pid()} | {:error, term()} -``` - -**Behavior**: -1. Loads policy from configured file -2. Validates policy -3. Compiles policy to ETS -4. Starts HTTP server (Plug.Cowboy) - -**Configuration**: -```elixir -config :http_capability_gateway, - policy_file: "priv/config/policy.dev.yaml", - backend_url: "http://localhost:4000", - port: 8080 -``` - -**Examples**: - -```elixir -# Start application (usually via mix) -{:ok, pid} = Application.start(:http_capability_gateway) - -# Application will: -# 1. Load policy.yaml -# 2. Validate DSL v1 -# 3. Compile to ETS -# 4. Start HTTP server on port 8080 -``` - ---- - -## Type Specifications - -### Policy Types - -```elixir -@type policy :: %{ - required(String.t()) => String.t() | map() | list(), - "dsl_version" => String.t(), - "governance" => governance(), - "stealth" => stealth() | nil -} - -@type governance :: %{ - required(String.t()) => list() | list(route()), - "global_verbs" => [http_verb()], - "routes" => [route()] | nil -} - -@type route :: %{ - required(String.t()) => String.t() | [http_verb()], - "path" => String.t(), - "verbs" => [http_verb()] -} - -@type stealth :: %{ - required(String.t()) => boolean() | integer(), - "enabled" => boolean(), - "status_code" => integer() -} - -@type http_verb :: String.t() -# Valid: "GET" | "POST" | "PUT" | "DELETE" | "PATCH" | "HEAD" | "OPTIONS" -``` - ---- - -## Error Handling - -### Common Errors - -**PolicyLoader Errors**: -- `{:error, "File not found: ..."}` - Policy file doesn't exist -- `{:error, "YAML parsing error: ..."}` - Invalid YAML syntax -- `{:error, "Empty policy"}` - Policy file is empty - -**PolicyValidator Errors**: -- `{:error, "dsl_version: must be present"}` - Missing version field -- `{:error, "Invalid HTTP verb: ..."}` - Unknown verb in policy -- `{:error, "global_verbs: must not be empty"}` - Empty verb list - -**PolicyCompiler Errors**: -- `{:error, "Invalid regex pattern: ..."}` - Bad route pattern - -**Runtime Errors**: -- Gateway returns 500 if backend connection fails -- Logs error with request_id for debugging - ---- - -## Performance - -### ETS Lookups - -- **Global verbs**: O(1) lookup -- **Route matching**: O(n) where n = number of routes -- **Verb checking**: O(1) for route match - -### Benchmarks - -See `test/performance_test.exs` for detailed benchmarks: - -```bash -mix test --only performance -``` - -Expected results: -- Policy compilation (1000 routes): <100ms -- Verb check: <1ms -- Request handling: <5ms (excluding backend) -- Throughput: >1000 req/s - ---- - -## Examples - -### Complete Usage Example - -```elixir -# 1. Load policy -{:ok, policy} = PolicyLoader.load_from_file("policy.yaml") - -# 2. Validate -:ok = PolicyValidator.validate(policy) - -# 3. Compile -:ok = PolicyCompiler.compile(policy) - -# 4. Check verbs -true = PolicyCompiler.is_verb_allowed?("/api/users", "GET") -false = PolicyCompiler.is_verb_allowed?("/api/admin", "DELETE") - -# 5. Get stealth config -%{enabled: true, status_code: 404} = PolicyCompiler.get_stealth_config() - -# 6. Gateway handles requests automatically via Plug -``` - ---- - -## Testing - -### Unit Tests - -```bash -# Test PolicyLoader -mix test test/policy_loader_test.exs - -# Test PolicyValidator -mix test test/policy_validator_test.exs - -# Test PolicyCompiler -mix test test/policy_compiler_test.exs - -# Test Gateway -mix test test/gateway_test.exs -``` - -### Integration Tests - -```bash -# Full request flow tests -mix test test/gateway_test.exs -``` - -### Property-Based Tests - -```bash -# Invariant testing with StreamData -mix test test/policy_property_test.exs -``` - ---- - -## Support - -For API questions: -- **GitHub Discussions**: https://github.com/hyperpolymath/http-capability-gateway/discussions -- **API Issues**: https://github.com/hyperpolymath/http-capability-gateway/issues diff --git a/docs/CAPABILITY-INTEGRATION.adoc b/docs/CAPABILITY-INTEGRATION.adoc new file mode 100644 index 0000000..669d057 --- /dev/null +++ b/docs/CAPABILITY-INTEGRATION.adoc @@ -0,0 +1,132 @@ +== Capability + Service-Discovery Integration + +This document describes the *contract surfaces* by which +`+http-capability-gateway+` connects (or, in v0.x, _will connect_) to: + +* the estate capability model (`+hyperpolymath/chimichanga+` capability +attenuation, `+hyperpolymath/boj-server+` cartridges), and +* service discovery (`+hyperpolymath/groove-protocol+`). + +It is intentionally written as a _contract_ rather than a feature list: +this PR adds documentation only. The implementation is filed as proof +debt in `+PROOFS_NEEDED.md+` and tracked in the linked audit issues. + +=== 1. Existing surface: BoJ cartridges + +`+PolicyLoader.load_from_boj_catalog/1+` already builds a Verb +Governance Spec from a directory of `+cartridge.json+` manifests. The +mapping is: + +[width="100%",cols="23%,26%,51%",options="header",] +|=== +|Cartridge field |DSL v1 field |Notes +|`+name+` |route `+name+` |One `+invoke+` route per cartridge + +|`+auth_method+` |route `+exposure+` |`+none+` → `+public+`, others → +`+authenticated+` + +|`+description+` |route `+narrative+` |Carried through for audit + +|(nothing yet) |route `+capability+` |*OPEN*: bind cartridge capability +label +|=== + +The third row is the gap this document calls out: cartridges declare +capability vocabularies but those vocabularies do not yet flow onto +compiled rules. PR #33 (`+audit/policy-schema-capability+`) adds the +schema field; the cartridge loader needs a follow-up to populate it. + +=== 2. Chimichanga capability attenuation (proposed surface) + +`+hyperpolymath/chimichanga+` defines a capability-attenuation lattice: +a holder of capability `+C+` can derive a strictly weaker capability +`+C'+` without consulting the issuer. The gateway is the right place to +enforce attenuation at the request boundary. + +==== Proposed contract + +Each compiled rule carries: + +* `+capability :: String.t() | nil+` (PR #33 lands this) +* `+attenuated_from :: [String.t()] | nil+` (NOT YET) — the set of +broader capabilities from which the rule’s capability is derivable. + +The gateway then enforces: + +.... +allow iff trust ≥ exposure + AND (rule.capability == nil + OR client_capability ≤ rule.capability) +.... + +where `+≤+` is the chimichanga lattice partial order, _not_ the trust +total order in `+SafeTrust+`. The two predicates are orthogonal: trust +is "`who is the caller`", capability is "`what may the caller do`". + +==== Open questions + +* *Where does `+client_capability+` come from?* Candidates: a signed JWT +claim, an mTLS certificate extension, a header populated by an upstream +attenuator. Out of scope for this doc. +* *Is the lattice mechanised in echo-types?* Filed for follow-up; see +`+PROOFS_NEEDED.md+`. Recorded as `+record-as-not-relevant+` for the +current scaffold PRs. + +=== 3. Service discovery via groove-protocol (proposed surface) + +`+backend_url+` is statically configured today +(`+config :http_capability_gateway, :backend_url, "http://localhost:8080"+`). +This breaks when estate apps move between hosts or scale horizontally. + +==== Proposed contract + +A new optional config key `+:backend_discovery+` that takes one of: + +* `+{:static, url}+` — current behaviour (the default) +* `+{:groove, service_name}+` — resolve the backend URL via +groove-protocol service discovery at request time, with a +circuit-breaker fallback to 503 on resolution failure (NOT to a static +URL — that would be a silent-trust-downgrade vector). + +`+Proxy.forward/2+` is the single integration point. The resolver is the +seam. + +==== Open questions + +* *What is the TTL of a discovery hit?* groove-protocol semantics +determine this; this gateway just consults it. +* *Does the resolved URL participate in policy?* Today the policy is +path-and-verb-keyed and backend-agnostic. If discovery returns multiple +candidate URLs, picking one is out of scope for v1. + +=== 4. Why this is documentation, not code + +A small documentation PR is the right step here because: + +[arabic] +. The cross-repo contracts (`+chimichanga+`, `+groove-protocol+`) are +still stabilising. +. The compiler-side schema change is in flight in #33 (which carries the +`+capability+` field) and a code-side discovery seam would prematurely +commit to a particular resolver shape. +. The egress mode in #32 is the more urgent consumer of the same +capability vocabulary; once it stabilises, the chimichanga binding is a +natural follow-up. + +=== 5. Echo-types audit + +`+hyperpolymath/echo-types+` was audited per estate convention. The +gateway does not currently participate in any echo protocol; trust +hierarchy proofs live in `+proven/SafeTrust.idr+`. *Status: +record-as-not-relevant.* + +The chimichanga attenuation surface _may_ introduce an L3 obligation +when it lands; this section will be revisited at that point. + +=== References + +* Audit issue: #31 +* Related PRs: #32 (egress scaffold), #33 (capability schema field) +* `+PROOFS_NEEDED.md+` (this repo) +* `+PolicyLoader.load_from_boj_catalog/1+` in +`+lib/http_capability_gateway/policy_loader.ex+` diff --git a/docs/CAPABILITY-INTEGRATION.md b/docs/CAPABILITY-INTEGRATION.md deleted file mode 100644 index cd3941d..0000000 --- a/docs/CAPABILITY-INTEGRATION.md +++ /dev/null @@ -1,125 +0,0 @@ - - - -# Capability + Service-Discovery Integration - -This document describes the **contract surfaces** by which -`http-capability-gateway` connects (or, in v0.x, *will connect*) to: - -- the estate capability model (`hyperpolymath/chimichanga` capability - attenuation, `hyperpolymath/boj-server` cartridges), and -- service discovery (`hyperpolymath/groove-protocol`). - -It is intentionally written as a *contract* rather than a feature list: -this PR adds documentation only. The implementation is filed as proof debt -in `PROOFS_NEEDED.md` and tracked in the linked audit issues. - -## 1. Existing surface: BoJ cartridges - -`PolicyLoader.load_from_boj_catalog/1` already builds a Verb Governance Spec -from a directory of `cartridge.json` manifests. The mapping is: - -| Cartridge field | DSL v1 field | Notes | -|----------------------|---------------------------|-----------------------------------------------------| -| `name` | route `name` | One `invoke` route per cartridge | -| `auth_method` | route `exposure` | `none` → `public`, others → `authenticated` | -| `description` | route `narrative` | Carried through for audit | -| (nothing yet) | route `capability` | **OPEN**: bind cartridge capability label | - -The third row is the gap this document calls out: cartridges declare -capability vocabularies but those vocabularies do not yet flow onto -compiled rules. PR #33 (`audit/policy-schema-capability`) adds the schema -field; the cartridge loader needs a follow-up to populate it. - -## 2. Chimichanga capability attenuation (proposed surface) - -`hyperpolymath/chimichanga` defines a capability-attenuation lattice: a -holder of capability `C` can derive a strictly weaker capability `C'` -without consulting the issuer. The gateway is the right place to enforce -attenuation at the request boundary. - -### Proposed contract - -Each compiled rule carries: - -- `capability :: String.t() | nil` (PR #33 lands this) -- `attenuated_from :: [String.t()] | nil` (NOT YET) — the set of broader - capabilities from which the rule's capability is derivable. - -The gateway then enforces: - -``` -allow iff trust ≥ exposure - AND (rule.capability == nil - OR client_capability ≤ rule.capability) -``` - -where `≤` is the chimichanga lattice partial order, *not* the trust total -order in `SafeTrust`. The two predicates are orthogonal: trust is "who is -the caller", capability is "what may the caller do". - -### Open questions - -- **Where does `client_capability` come from?** Candidates: a signed JWT - claim, an mTLS certificate extension, a header populated by an upstream - attenuator. Out of scope for this doc. -- **Is the lattice mechanised in echo-types?** Filed for follow-up; see - `PROOFS_NEEDED.md`. Recorded as `record-as-not-relevant` for the - current scaffold PRs. - -## 3. Service discovery via groove-protocol (proposed surface) - -`backend_url` is statically configured today -(`config :http_capability_gateway, :backend_url, "http://localhost:8080"`). -This breaks when estate apps move between hosts or scale horizontally. - -### Proposed contract - -A new optional config key `:backend_discovery` that takes one of: - -- `{:static, url}` — current behaviour (the default) -- `{:groove, service_name}` — resolve the backend URL via groove-protocol - service discovery at request time, with a circuit-breaker fallback to - 503 on resolution failure (NOT to a static URL — that would be a - silent-trust-downgrade vector). - -`Proxy.forward/2` is the single integration point. The resolver is the -seam. - -### Open questions - -- **What is the TTL of a discovery hit?** groove-protocol semantics - determine this; this gateway just consults it. -- **Does the resolved URL participate in policy?** Today the policy is - path-and-verb-keyed and backend-agnostic. If discovery returns multiple - candidate URLs, picking one is out of scope for v1. - -## 4. Why this is documentation, not code - -A small documentation PR is the right step here because: - -1. The cross-repo contracts (`chimichanga`, `groove-protocol`) are still - stabilising. -2. The compiler-side schema change is in flight in #33 (which carries the - `capability` field) and a code-side discovery seam would prematurely - commit to a particular resolver shape. -3. The egress mode in #32 is the more urgent consumer of the same - capability vocabulary; once it stabilises, the chimichanga binding is - a natural follow-up. - -## 5. Echo-types audit - -`hyperpolymath/echo-types` was audited per estate convention. The gateway -does not currently participate in any echo protocol; trust hierarchy -proofs live in `proven/SafeTrust.idr`. **Status: record-as-not-relevant.** - -The chimichanga attenuation surface *may* introduce an L3 obligation when -it lands; this section will be revisited at that point. - -## References - -- Audit issue: #31 -- Related PRs: #32 (egress scaffold), #33 (capability schema field) -- `PROOFS_NEEDED.md` (this repo) -- `PolicyLoader.load_from_boj_catalog/1` in - `lib/http_capability_gateway/policy_loader.ex` diff --git a/docs/CMS-COMPATIBILITY.md b/docs/CMS-COMPATIBILITY.adoc similarity index 67% rename from docs/CMS-COMPATIBILITY.md rename to docs/CMS-COMPATIBILITY.adoc index a9f76b6..817d70b 100644 --- a/docs/CMS-COMPATIBILITY.md +++ b/docs/CMS-COMPATIBILITY.adoc @@ -1,57 +1,62 @@ - -# CMS Compatibility Guide -# WordPress, Drupal, Moodle, and More +== CMS Compatibility Guide -## Overview +== WordPress, Drupal, Moodle, and More -http-capability-gateway is designed to be a **good citizen** in existing web infrastructure. It plays nicely with popular CMSs, doesn't break admin panels, and respects standard web patterns. +=== Overview ---- +http-capability-gateway is designed to be a *good citizen* in existing +web infrastructure. It plays nicely with popular CMSs, doesn’t break +admin panels, and respects standard web patterns. -## 🎯 Design Principles +''''' -1. **Preserve, Don't Replace** - Keep existing headers, cookies, sessions -2. **Bypass When Needed** - Admin paths shouldn't need verb governance -3. **Auto-Detect** - Recognize CMS patterns automatically -4. **Document Everything** - Clear examples for each CMS -5. **No Surprises** - Predictable behavior in proxy chains +=== 🎯 Design Principles ---- +[arabic] +. *Preserve, Don’t Replace* - Keep existing headers, cookies, sessions +. *Bypass When Needed* - Admin paths shouldn’t need verb governance +. *Auto-Detect* - Recognize CMS patterns automatically +. *Document Everything* - Clear examples for each CMS +. *No Surprises* - Predictable behavior in proxy chains -## 🔧 WordPress Compatibility +''''' -### Auto-Detection -HCG automatically detects WordPress by: -- `X-Powered-By: PHP` + presence of `/wp-admin/` -- WordPress cookies (`wordpress_*`, `wp-settings-*`) -- REST API at `/wp-json/` +=== 🔧 WordPress Compatibility -### Preserved Headers -```yaml +==== Auto-Detection + +HCG automatically detects WordPress by: - `+X-Powered-By: PHP+` + +presence of `+/wp-admin/+` - WordPress cookies (`+wordpress_*+`, +`+wp-settings-*+`) - REST API at `+/wp-json/+` + +==== Preserved Headers + +[source,yaml] +---- # Automatically preserved for WordPress headers: - X-WordPress-Nonce - X-WP-Total - X-WP-TotalPages - Link (REST API pagination) -``` +---- -### Preserved Cookies -```yaml +==== Preserved Cookies + +[source,yaml] +---- cookies: - wordpress_* # WordPress auth cookies - wp-settings-* # User settings - wordpress_test_cookie - wordpress_logged_in_* - wp_woocommerce_* # WooCommerce -``` +---- -### Recommended Policy +==== Recommended Policy -```yaml +[source,yaml] +---- # wordpress-policy.yaml dsl_version: "1" @@ -89,10 +94,12 @@ governance: stealth: enabled: true status_code: 404 -``` +---- + +==== WooCommerce Support -### WooCommerce Support -```yaml +[source,yaml] +---- routes: # Checkout flow - path: "/checkout/.*" @@ -105,29 +112,32 @@ routes: # Payment gateway callbacks - path: "/wc-api/.*" verbs: [GET, POST, PUT] -``` +---- -### Multisite Configuration -```yaml +==== Multisite Configuration + +[source,yaml] +---- routes: # Network admin - path: "/wp-admin/network/.*" verbs: [GET, POST, PUT, DELETE] bypass_if_cookie: "wordpress_logged_in_" -``` +---- + +''''' ---- +=== 🎨 Drupal Compatibility -## 🎨 Drupal Compatibility +==== Auto-Detection -### Auto-Detection -HCG detects Drupal by: -- `X-Generator: Drupal` -- Cache tags: `X-Drupal-Cache-Tags` -- Session cookies: `SESS*` +HCG detects Drupal by: - `+X-Generator: Drupal+` - Cache tags: +`+X-Drupal-Cache-Tags+` - Session cookies: `+SESS*+` -### Preserved Headers -```yaml +==== Preserved Headers + +[source,yaml] +---- headers: - X-Drupal-Cache - X-Drupal-Cache-Tags @@ -135,19 +145,22 @@ headers: - X-Drupal-Cache-Max-Age - X-Generator - X-Drupal-Dynamic-Cache -``` +---- + +==== Preserved Cookies -### Preserved Cookies -```yaml +[source,yaml] +---- cookies: - SESS* # Drupal session - SSESS* # Secure session - Drupal.visitor.* # Visitor tracking -``` +---- -### Recommended Policy +==== Recommended Policy -```yaml +[source,yaml] +---- # drupal-policy.yaml dsl_version: "1" @@ -186,10 +199,12 @@ governance: stealth: enabled: true status_code: 404 -``` +---- -### Drupal Cache Integration -```yaml +==== Drupal Cache Integration + +[source,yaml] +---- # Work with Drupal's cache system caching: respect_backend_headers: true @@ -197,35 +212,41 @@ caching: invalidate_on_tags: - "node:*" - "taxonomy_term:*" -``` +---- + +''''' + +=== 📚 Moodle Compatibility ---- +==== Auto-Detection -## 📚 Moodle Compatibility +* Session cookies: `+MoodleSession*+` +* Admin path: `+/admin/+` +* Login page: `+/login/index.php+` -### Auto-Detection -- Session cookies: `MoodleSession*` -- Admin path: `/admin/` -- Login page: `/login/index.php` +==== Preserved Headers -### Preserved Headers -```yaml +[source,yaml] +---- headers: - X-Moodle-Version - X-Frame-Options # Moodle sets this for embedding -``` +---- -### Preserved Cookies -```yaml +==== Preserved Cookies + +[source,yaml] +---- cookies: - MoodleSession* - MOODLEID_* - moodle_test_cookie -``` +---- -### Recommended Policy +==== Recommended Policy -```yaml +[source,yaml] +---- # moodle-policy.yaml dsl_version: "1" @@ -261,16 +282,18 @@ governance: stealth: enabled: true status_code: 404 -``` +---- + +''''' ---- +=== 🌐 Reverse Proxy Integration -## 🌐 Reverse Proxy Integration +==== Common Proxy Setups -### Common Proxy Setups +===== 1. CloudFlare → HCG → Backend -#### 1. CloudFlare → HCG → Backend -```yaml +[source,yaml] +---- # Preserve CloudFlare headers proxy: preserve_headers: @@ -285,10 +308,12 @@ proxy: - 173.245.48.0/20 - 103.21.244.0/22 # ... (CloudFlare IP ranges) -``` +---- -#### 2. nginx → HCG → Backend -```nginx +===== 2. nginx → HCG → Backend + +[source,nginx] +---- # /etc/nginx/sites-available/mysite upstream hcg { server 127.0.0.1:4000; @@ -319,10 +344,12 @@ server { proxy_read_timeout 60s; } } -``` +---- + +===== 3. Apache → HCG → Backend -#### 3. Apache → HCG → Backend -```apache +[source,apache] +---- # /etc/apache2/sites-available/mysite.conf ServerName example.com @@ -335,10 +362,12 @@ server { RequestHeader set X-Forwarded-Proto "http" RequestHeader set X-Forwarded-Port "80" -``` +---- -#### 4. Caddy → HCG → Backend -```caddyfile +===== 4. Caddy → HCG → Backend + +[source,caddyfile] +---- # Caddyfile example.com { reverse_proxy localhost:4000 { @@ -346,10 +375,12 @@ example.com { header_up X-Forwarded-Proto {scheme} } } -``` +---- + +===== 5. Varnish → HCG → Backend -#### 5. Varnish → HCG → Backend -```vcl +[source,vcl] +---- # /etc/varnish/default.vcl backend hcg { .host = "127.0.0.1"; @@ -371,17 +402,18 @@ sub vcl_recv { return (purge); } } -``` +---- ---- +''''' -## 🔒 .well-known/ Support +=== 🔒 .well-known/ Support -### Automatic Passthrough +==== Automatic Passthrough -HCG **automatically bypasses policy** for `.well-known/*` paths: +HCG *automatically bypasses policy* for `+.well-known/*+` paths: -```yaml +[source,yaml] +---- # Built-in, no configuration needed well_known: auto_passthrough: true @@ -393,12 +425,14 @@ well_known: - /.well-known/webfinger # Federation - /.well-known/nodeinfo # Fediverse - /.well-known/host-meta # Discovery -``` +---- -### ACME / Let's Encrypt Integration +==== ACME / Let’s Encrypt Integration -**Setup 1: Certbot with HCG** -```bash +*Setup 1: Certbot with HCG* + +[source,bash] +---- # HCG automatically serves ACME challenges certbot certonly --webroot \ -w /var/www/html \ @@ -410,41 +444,47 @@ certbot certonly --webroot \ certbot certonly --standalone \ --preferred-challenges http \ -d example.com -``` +---- + +*Setup 2: ACME challenges through HCG* -**Setup 2: ACME challenges through HCG** -```yaml +[source,yaml] +---- # config/prod.exs config :http_capability_gateway, acme_challenge_dir: "/var/lib/acme/challenges" -``` +---- -HCG will serve files from this directory at `/.well-known/acme-challenge/` +HCG will serve files from this directory at +`+/.well-known/acme-challenge/+` -### Security.txt +==== Security.txt -```yaml +[source,yaml] +---- # Expose security policy through HCG # Place file at: priv/static/.well-known/security.txt # HCG serves it automatically -``` +---- + +Example `+/var/www/html/.well-known/security.txt+`: -Example `/var/www/html/.well-known/security.txt`: -``` +.... Contact: mailto:security@example.com Expires: 2027-12-31T23:59:59z Encryption: https://example.com/pgp-key.txt Preferred-Languages: en Canonical: https://example.com/.well-known/security.txt -``` +.... ---- +''''' -## 🛡️ Security Headers +=== 🛡️ Security Headers -### Preserve Backend Headers +==== Preserve Backend Headers -```yaml +[source,yaml] +---- # HCG preserves these by default security_headers: preserve_from_backend: @@ -459,11 +499,12 @@ security_headers: - Cross-Origin-Embedder-Policy - Cross-Origin-Opener-Policy - Cross-Origin-Resource-Policy -``` +---- -### Add Missing Headers +==== Add Missing Headers -```yaml +[source,yaml] +---- # config/prod.exs config :http_capability_gateway, security_headers: @@ -473,32 +514,33 @@ config :http_capability_gateway, "X-Content-Type-Options": "nosniff" "Referrer-Policy": "strict-origin-when-cross-origin" "Permissions-Policy": "geolocation=(), microphone=(), camera=()" -``` +---- -### Header Merging Strategy +==== Header Merging Strategy -**Precedence** (highest to lowest): -1. Backend headers (always preserved) -2. Policy-defined headers -3. Gateway default headers +*Precedence* (highest to lowest): 1. Backend headers (always preserved) +2. Policy-defined headers 3. Gateway default headers -**Example:** -```yaml +*Example:* + +[source,yaml] +---- routes: - path: "/embed/.*" headers: set: X-Frame-Options: "ALLOW-FROM https://trusted.com" # This overrides default SAMEORIGIN -``` +---- ---- +''''' -## 🌍 CORS Support +=== 🌍 CORS Support -### Preserve CORS Headers +==== Preserve CORS Headers -```yaml +[source,yaml] +---- # Automatically preserved cors_headers: - Access-Control-Allow-Origin @@ -507,11 +549,12 @@ cors_headers: - Access-Control-Expose-Headers - Access-Control-Max-Age - Access-Control-Allow-Credentials -``` +---- -### OPTIONS Preflight +==== OPTIONS Preflight -```yaml +[source,yaml] +---- # Auto-handle OPTIONS for CORS governance: global_verbs: @@ -524,36 +567,40 @@ routes: enabled: true allow_origin: "https://app.example.com" allow_credentials: true -``` +---- ---- +''''' -## 📝 Resource Hints +=== 📝 Resource Hints -### Preserve Link Headers +==== Preserve Link Headers -```yaml +[source,yaml] +---- # Preserve resource hints from backend resource_hints: preserve: - Link # For preload, prefetch, dns-prefetch -``` +---- + +*Example backend response:* -**Example backend response:** -``` +.... Link: ; rel=preload; as=style Link: ; rel=preload; as=script Link: ; rel=dns-prefetch -``` +.... HCG passes these through unchanged. ---- +''''' -## 🧪 Testing CMS Compatibility +=== 🧪 Testing CMS Compatibility -### WordPress Health Check -```bash +==== WordPress Health Check + +[source,bash] +---- # Test admin access curl -I https://example.com/wp-admin/ # Should redirect to login or show 200 if logged in @@ -565,10 +612,12 @@ curl https://example.com/wp-json/wp/v2/posts # Test AJAX curl -X POST https://example.com/wp-admin/admin-ajax.php \ -d "action=heartbeat" -``` +---- + +==== Drupal Health Check -### Drupal Health Check -```bash +[source,bash] +---- # Test admin access curl -I https://example.com/admin # Should redirect to login or show 200 if logged in @@ -579,10 +628,12 @@ curl https://example.com/jsonapi/node/article # Test status page curl https://example.com/admin/reports/status -``` +---- -### Moodle Health Check -```bash +==== Moodle Health Check + +[source,bash] +---- # Test login page curl -I https://example.com/login/index.php # Should show 200 @@ -590,54 +641,67 @@ curl -I https://example.com/login/index.php # Test admin curl -I https://example.com/admin/ # Should redirect to login or show 200 if logged in -``` +---- + +''''' ---- +=== 🐛 Troubleshooting -## 🐛 Troubleshooting +==== Problem: Admin Panel Broken -### Problem: Admin Panel Broken -**Symptoms**: Can't access /wp-admin/, getting 404 -**Solution**: Add bypass rule for admin paths -```yaml +*Symptoms*: Can’t access /wp-admin/, getting 404 *Solution*: Add bypass +rule for admin paths + +[source,yaml] +---- routes: - path: "/wp-admin/.*" verbs: [GET, POST, PUT, DELETE, PATCH] bypass_if_authenticated: true -``` +---- + +==== Problem: AJAX Requests Failing + +*Symptoms*: Admin-ajax.php returns 404 *Solution*: Ensure POST is +allowed -### Problem: AJAX Requests Failing -**Symptoms**: Admin-ajax.php returns 404 -**Solution**: Ensure POST is allowed -```yaml +[source,yaml] +---- routes: - path: "/wp-admin/admin-ajax.php" verbs: [POST] -``` +---- + +==== Problem: Sessions Not Persisting + +*Symptoms*: Keep getting logged out *Solution*: Check cookie +preservation -### Problem: Sessions Not Persisting -**Symptoms**: Keep getting logged out -**Solution**: Check cookie preservation -```bash +[source,bash] +---- # Verify cookies are passed through curl -v -b "wordpress_logged_in_xxx=..." \ https://example.com/wp-admin/ -``` +---- -### Problem: Cache Headers Ignored -**Symptoms**: Pages not caching -**Solution**: Enable cache header respect -```yaml +==== Problem: Cache Headers Ignored + +*Symptoms*: Pages not caching *Solution*: Enable cache header respect + +[source,yaml] +---- caching: respect_backend_headers: true -``` +---- ---- +''''' -## 📚 Complete Examples +=== 📚 Complete Examples -### WordPress + nginx + Let's Encrypt -```nginx +==== WordPress + nginx + Let’s Encrypt + +[source,nginx] +---- server { listen 443 ssl http2; server_name example.com; @@ -659,10 +723,12 @@ server { proxy_set_header X-Forwarded-Proto $scheme; } } -``` +---- + +==== Drupal + Varnish + HCG -### Drupal + Varnish + HCG -```vcl +[source,vcl] +---- # Varnish caches, HCG enforces policy sub vcl_recv { # Bypass cache for admin @@ -680,36 +746,39 @@ sub vcl_backend_response { set beresp.http.X-VC-Cache-Tags = beresp.http.X-Drupal-Cache-Tags; } } -``` +---- ---- +''''' -## ✅ Compatibility Checklist +=== ✅ Compatibility Checklist Before deploying HCG with a CMS: -- [ ] Test admin panel access -- [ ] Test AJAX endpoints -- [ ] Verify session persistence -- [ ] Check file uploads work -- [ ] Test REST API endpoints -- [ ] Verify ACME challenges work -- [ ] Check security headers present -- [ ] Test CORS if needed -- [ ] Verify cache headers respected -- [ ] Test under load +* [ ] Test admin panel access +* [ ] Test AJAX endpoints +* [ ] Verify session persistence +* [ ] Check file uploads work +* [ ] Test REST API endpoints +* [ ] Verify ACME challenges work +* [ ] Check security headers present +* [ ] Test CORS if needed +* [ ] Verify cache headers respected +* [ ] Test under load ---- +''''' -## 🤝 Community Policies +=== 🤝 Community Policies -Share your CMS policy configurations: -- [WordPress policies on hub.hyperpolymath.org](https://hub.hyperpolymath.org/wordpress) -- [Drupal policies](https://hub.hyperpolymath.org/drupal) -- [Moodle policies](https://hub.hyperpolymath.org/moodle) +Share your CMS policy configurations: - +https://hub.hyperpolymath.org/wordpress[WordPress policies on +hub.hyperpolymath.org] - https://hub.hyperpolymath.org/drupal[Drupal +policies] - https://hub.hyperpolymath.org/moodle[Moodle policies] -**Submit yours**: `hcg policy publish wordpress-my-config.yaml` +*Submit yours*: `+hcg policy publish wordpress-my-config.yaml+` ---- +''''' -**tl;dr**: HCG plays nice with WordPress, Drupal, Moodle, and standard web infrastructure. It preserves what matters, bypasses admin paths intelligently, and respects web standards like .well-known/ and security headers. +*tl;dr*: HCG plays nice with WordPress, Drupal, Moodle, and standard web +infrastructure. It preserves what matters, bypasses admin paths +intelligently, and respects web standards like .well-known/ and security +headers. diff --git a/docs/DEPLOYMENT.adoc b/docs/DEPLOYMENT.adoc new file mode 100644 index 0000000..fb9a2c5 --- /dev/null +++ b/docs/DEPLOYMENT.adoc @@ -0,0 +1,847 @@ +== Deployment Guide — HTTP Capability Gateway v0.1.0-dev + +Practical guide for deploying the HTTP Capability Gateway to production +environments. Covers container-based deployment (Podman/Docker), +bare-metal OTP releases, policy file setup, health checks, monitoring, +security configuration, and troubleshooting. + +=== Table of Contents + +[arabic] +. link:#prerequisites[Prerequisites] +. link:#container-deployment[Container Deployment] +. link:#bare-metal-deployment[Bare-Metal Deployment] +. link:#policy-file-setup[Policy File Setup] +. link:#health-checks[Health Checks] +. link:#monitoring[Monitoring] +. link:#security[Security] +. link:#troubleshooting[Troubleshooting] + +''''' + +=== Prerequisites + +==== Runtime Requirements + +[cols=",,",options="header",] +|=== +|Requirement |Minimum |Recommended +|Elixir |1.19+ |1.19.4 +|Erlang/OTP |27+ |28.2 +|RAM |256 MB |1 GB+ +|CPU |1 core |2+ cores +|=== + +*Or* a container runtime: + +* *Podman* 4.0+ (preferred) +* *Docker* 24.0+ +* *nerdctl* 1.0+ + +The Containerfile uses OCI format and is compatible with all three +runtimes. + +==== Network Requirements + +* Inbound access on the configured port (default: `+4000+`) +* Outbound HTTPS/HTTP access to backend services (for proxy) +* DNS resolution for backend service hostnames + +''''' + +=== Container Deployment + +==== Building the Image + +The repository includes a multi-stage `+Containerfile+` that produces a +minimal Alpine-based runtime image (~30 MB) with no build tools or +source code. + +[source,bash] +---- +# Build with Podman (preferred) +podman build -t http-capability-gateway:0.1.0-dev -f Containerfile . + +# Build with Docker +docker build -t http-capability-gateway:0.1.0-dev -f Containerfile . +---- + +The builder stage uses +`+hexpm/elixir:1.19.4-erlang-28.2.2-alpine-3.22.1+`. The runtime stage +uses `+alpine:3.22.1+` with only `+libstdc+++`, `+ncurses-libs+`, and +`+openssl+` installed. The application runs as a non-root `+gateway+` +user. + +==== Running with Podman or Docker + +[source,bash] +---- +# Run the gateway container +podman run -d \ + --name http-capability-gateway \ + -p 8080:4000 \ + -e POLICY_PATH=/app/config/policy.yaml \ + -e BACKEND_URL=http://backend:4000 \ + -e PORT=4000 \ + -v ./my-policy.yaml:/app/config/policy.yaml:ro \ + http-capability-gateway:0.1.0-dev + +# Check logs +podman logs -f http-capability-gateway + +# Stop and remove +podman stop http-capability-gateway +podman rm http-capability-gateway +---- + +==== Running with podman-compose or docker-compose + +The repository includes a `+docker-compose.yml+` that starts the gateway +along with an example httpbin backend for testing. + +[source,bash] +---- +# Start all services (gateway + example backend) +podman-compose up --build -d + +# Or with Docker Compose +docker-compose up --build -d + +# View gateway logs +podman-compose logs -f gateway + +# Test the gateway +curl http://localhost:8080/health +curl http://localhost:8080/ready +curl http://localhost:8080/metrics + +# Stop all services +podman-compose down +---- + +The compose file maps host port `+8080+` to the gateway’s internal port +`+4000+` and mounts a policy file from `+./examples/policy-dev.yaml+`. + +==== Environment Variables + +[width="100%",cols="26%,23%,21%,30%",options="header",] +|=== +|Variable |Required |Default |Description +|`+POLICY_PATH+` |Yes (prod) |`+config/policy.yaml+` |Path to the DSL v1 +policy YAML file + +|`+BACKEND_URL+` |No |`+nil+` |URL of the backend service to proxy to + +|`+PORT+` |No |`+4000+` |HTTP listen port + +|`+TRUST_LEVEL_HEADER+` |No |`+x-trust-level+` |Header name for trust +level extraction + +|`+TRUST_LEVEL_SOURCE+` |No |`+header+` |Trust extraction source: +`+header+` or `+mtls+` + +|`+LOG_LEVEL+` |No |`+info+` |Logger level (`+debug+`, `+info+`, +`+warn+`, `+error+`) +|=== + +==== Volume Mounts + +[width="100%",cols="31%,44%,25%",options="header",] +|=== +|Host Path |Container Path |Purpose +|Your policy file |`+/app/config/policy.yaml+` |DSL v1 governance policy + +|TLS certificates (optional) |`+/app/certs/+` |mTLS client CA and server +certificates +|=== + +''''' + +=== Bare-Metal Deployment + +==== Mix Release Build + +[source,bash] +---- +# Clone and enter the repository +cd /opt/http-capability-gateway + +# Install dependencies (production only) +export MIX_ENV=prod +mix deps.get --only prod +mix compile + +# Build OTP release +mix release + +# The release is at: +# _build/prod/rel/http_capability_gateway/ +---- + +==== Running the Release + +[source,bash] +---- +# Start in foreground (useful for debugging) +_build/prod/rel/http_capability_gateway/bin/http_capability_gateway start + +# Start as background daemon +_build/prod/rel/http_capability_gateway/bin/http_capability_gateway daemon + +# Check if running +_build/prod/rel/http_capability_gateway/bin/http_capability_gateway pid + +# Stop the daemon +_build/prod/rel/http_capability_gateway/bin/http_capability_gateway stop + +# Remote console (attach to running node) +_build/prod/rel/http_capability_gateway/bin/http_capability_gateway remote +---- + +==== Systemd Service File + +Create `+/etc/systemd/system/http-capability-gateway.service+`: + +[source,ini] +---- +[Unit] +Description=HTTP Capability Gateway v0.1.0-dev +Documentation=https://github.com/hyperpolymath/http-capability-gateway +After=network.target + +[Service] +Type=exec +User=gateway +Group=gateway +WorkingDirectory=/opt/http-capability-gateway + +# Environment variables +Environment=POLICY_PATH=/etc/http-capability-gateway/policy.yaml +Environment=BACKEND_URL=http://backend.internal:4000 +Environment=PORT=4000 +Environment=TRUST_LEVEL_HEADER=x-trust-level +Environment=LOG_LEVEL=info + +# Start the OTP release +ExecStart=/opt/http-capability-gateway/_build/prod/rel/http_capability_gateway/bin/http_capability_gateway start +ExecStop=/opt/http-capability-gateway/_build/prod/rel/http_capability_gateway/bin/http_capability_gateway stop + +# Restart policy +Restart=on-failure +RestartSec=5s + +# Logging +StandardOutput=journal +StandardError=journal +SyslogIdentifier=http-capability-gateway + +# Security hardening +NoNewPrivileges=true +PrivateTmp=true +ProtectSystem=strict +ProtectHome=true +ReadWritePaths=/var/log/http-capability-gateway + +[Install] +WantedBy=multi-user.target +---- + +Enable and start the service: + +[source,bash] +---- +# Create the gateway user +sudo useradd --system --shell /usr/sbin/nologin gateway + +# Set ownership +sudo chown -R gateway:gateway /opt/http-capability-gateway + +# Install and start the service +sudo systemctl daemon-reload +sudo systemctl enable http-capability-gateway +sudo systemctl start http-capability-gateway + +# Check status +sudo systemctl status http-capability-gateway +journalctl -u http-capability-gateway -f +---- + +==== Configuration via Environment Variables + +For bare-metal deployments, environment variables can be set through: + +[arabic] +. *Systemd `+Environment=+` directives* (shown above) +. *Systemd `+EnvironmentFile=+`* pointing to a file: ++ +[source,ini] +---- +EnvironmentFile=/etc/http-capability-gateway/env +---- ++ +With `+/etc/http-capability-gateway/env+` containing: ++ +[source,bash] +---- +POLICY_PATH=/etc/http-capability-gateway/policy.yaml +BACKEND_URL=http://backend.internal:4000 +PORT=4000 +---- +. *Shell exports* when running interactively + +''''' + +=== Policy File Setup + +==== Where to Put Policy Files + +[cols=",",options="header",] +|=== +|Deployment |Recommended Location +|Container |`+/app/config/policy.yaml+` (volume mount) +|Bare-metal |`+/etc/http-capability-gateway/policy.yaml+` +|Development |`+config/policy.yaml+` (in-repo) +|=== + +==== DSL v1 Format Overview + +Policy files use YAML with the DSL v1 schema. A minimal policy: + +[source,yaml] +---- +dsl_version: "1" +governance: + global_verbs: + - GET + - POST + routes: + - path: "/api/v1/users/[0-9]+" + verbs: [GET, PUT, DELETE] + - path: "/health" + verbs: [GET] +stealth: + enabled: true + status_code: 404 +---- + +For the complete DSL v1 reference, including regex routes, stealth mode +options, and validation rules, see POLICY-DSL.md. + +==== Hot Reload via SIGHUP + +Policy files are loaded at startup. To reload the policy without +restarting the gateway, send a SIGHUP signal to the BEAM process: + +[source,bash] +---- +# Find the OS process ID (not the Erlang PID) +kill -HUP $(pidof beam.smp) +---- + +The gateway uses an *atomic dual-table swap* for zero-downtime reloads: + +[arabic] +. A new pair of ETS tables (main + regex) is compiled from the updated +policy +. If compilation succeeds, both table references are swapped atomically +. The old tables are deleted after the swap +. If compilation fails, the old tables remain active (no service +disruption) + +This guarantees that in-flight requests are never served from a +partially loaded policy. + +''''' + +=== Health Checks + +==== Liveness: `+GET /health+` + +Returns `+200 OK+` if the BEAM process is running. Does *not* check +policy loading or backend connectivity. Use this for container liveness +probes. + +[source,bash] +---- +curl -s http://localhost:4000/health | jq . +---- + +[source,json] +---- +{ + "status": "healthy", + "service": "http-capability-gateway", + "version": "0.1.0-dev", + "uptime_seconds": 3600 +} +---- + +==== Readiness: `+GET /ready+` + +Returns `+200 OK+` only if the policy is loaded and ETS tables are +operational. Returns `+503 Service Unavailable+` if the gateway is not +ready to serve traffic. Use this for container readiness probes and load +balancer health checks. + +[source,bash] +---- +curl -s http://localhost:4000/ready | jq . +---- + +[source,json] +---- +{ + "status": "ready", + "service": "http-capability-gateway", + "policy_rules": 12, + "main_table_rules": 10, + "regex_table_rules": 2, + "rate_limiter_buckets": 42 +} +---- + +The readiness response includes: + +* `+policy_rules+`: Total compiled rules (main + regex tables) +* `+main_table_rules+`: Exact-match (O(1)) and global rules +* `+regex_table_rules+`: Regex pattern routes (O(r) scan) +* `+rate_limiter_buckets+`: Active client rate limiter entries + +==== Container Health Check Configuration + +The Containerfile includes a built-in health check: + +.... +HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 + CMD wget --no-verbose --tries=1 --spider http://localhost:4000/health || exit 1 +.... + +For Kubernetes, configure liveness and readiness probes separately: + +[source,yaml] +---- +livenessProbe: + httpGet: + path: /health + port: 4000 + initialDelaySeconds: 10 + periodSeconds: 30 +readinessProbe: + httpGet: + path: /ready + port: 4000 + initialDelaySeconds: 5 + periodSeconds: 10 +---- + +''''' + +=== Monitoring + +==== Prometheus `+/metrics+` Endpoint + +The gateway exposes Prometheus-format metrics at `+GET /metrics+`: + +[source,bash] +---- +curl -s http://localhost:4000/metrics +---- + +===== Key Metrics to Watch + +[width="100%",cols="30%,22%,48%",options="header",] +|=== +|Metric |Type |Description +|`+http_capability_gateway_request_completed_count+` |Counter |Total +requests processed + +|`+http_capability_gateway_request_completed_duration+` |Distribution +|Request duration (microseconds) + +|`+http_capability_gateway_access_decision_count+` |Counter |Decisions +by type (`+allow+`/`+deny+`), verb, trust level + +|`+http_capability_gateway_backend_forward_count+` |Counter |Requests +forwarded to backend + +|`+http_capability_gateway_backend_response_duration+` |Distribution +|Backend response latency (microseconds) + +|`+http_capability_gateway_error_count+` |Counter |Errors by type + +|`+http_capability_gateway_minikaran_anomaly_count+` |Counter |Anomalies +detected by type +|=== + +===== Prometheus Scrape Configuration + +[source,yaml] +---- +scrape_configs: + - job_name: 'http-capability-gateway' + scrape_interval: 15s + static_configs: + - targets: ['gateway:4000'] + metrics_path: /metrics +---- + +==== Minikaran Anomaly Dashboard: `+GET /api/v1/minikaran+` + +The built-in Minikaran traffic anomaly detector provides a JSON +dashboard for monitoring unusual traffic patterns: + +[source,bash] +---- +curl -s http://localhost:4000/api/v1/minikaran | jq . +---- + +[source,json] +---- +{ + "status": { + "status": "active", + "windows_collected": 42, + "min_windows_required": 5, + "current_anomalies": 1, + "uptime_sec": 2520 + }, + "anomalies": [ + { + "type": "traffic_spike", + "path": "/api/v1/users", + "current": 150, + "baseline": 42.3 + } + ], + "baseline": { + "window_count": 41, + "avg_requests_per_minute": 85.2, + "trust_distribution": { + "untrusted": 0.6, + "authenticated": 0.35, + "internal": 0.05 + }, + "latency_p50_us": 1200, + "latency_p95_us": 8500, + "latency_p99_us": 25000, + "avg_error_rate": 0.02, + "known_paths": 45, + "avg_unique_clients": 120 + } +} +---- + +*Anomaly types detected:* + +[width="100%",cols="32%,68%",options="header",] +|=== +|Type |Description +|`+traffic_spike+` |Request volume z-score exceeds threshold for a path + +|`+trust_shift+` |Trust level distribution deviates from learned +baseline + +|`+latency_spike+` |p95 latency exceeds baseline by significant margin + +|`+path_novelty+` |Unusual number of never-before-seen request paths + +|`+error_spike+` |Error rate exceeds learned baseline +|=== + +Minikaran requires *5+ baseline windows* (1 minute each) before anomaly +detection activates. During the learning phase, `+status+` will be +`+"learning"+`. + +==== Log Monitoring + +The gateway emits structured JSON logs suitable for ingestion by ELK, +Splunk, DataDog, or CloudWatch: + +[source,json] +---- +{ + "timestamp": "2026-02-28T12:00:00.000Z", + "level": "info", + "message": "access_decision", + "request_id": "a1b2c3d4e5f6", + "path": "/api/v1/users/123", + "verb": "GET", + "trust_level": "authenticated", + "decision": "allow", + "duration_us": 45 +} +---- + +''''' + +=== Security + +==== mTLS Setup + +The gateway supports mTLS-based trust level extraction. To enable: + +[arabic] +. *Set `+TRUST_LEVEL_SOURCE=mtls+`* in your environment configuration. +. *Configure Cowboy for TLS* in `+config/prod.exs+`: ++ +[source,elixir] +---- +config :http_capability_gateway, + scheme: :https, + certfile: "/app/certs/server.pem", + keyfile: "/app/certs/server-key.pem", + cacertfile: "/app/certs/ca.pem", + verify: :verify_peer, + fail_if_no_peer_cert: false +---- +. *Trust level mapping from certificates:* +* `+internal+` – Verified certificate with OU = "`Internal Services`" +* `+authenticated+` – Any verified certificate from a trusted CA +* `+untrusted+` – No certificate or verification failed + +==== Trust Levels and How They Work + +The gateway uses a formally verified trust hierarchy (from +`+proven/SafeTrust.idr+`): + +.... +untrusted (rank 0) < authenticated (rank 1) < internal (rank 2) +.... + +Access decisions compare trust rank against exposure rank: + +[width="100%",cols="18%,24%,32%,26%",options="header",] +|=== +|Trust Level |Can Access Public |Can Access Authenticated |Can Access +Internal +|`+untrusted+` |Yes |No |No + +|`+authenticated+` |Yes |Yes |No + +|`+internal+` |Yes |Yes |Yes +|=== + +The monotonicity property guarantees that upgrading trust never revokes +previously granted access. + +==== Trust Header Spoofing Protection + +By default, the gateway strips the `+X-Trust-Level+` header from +requests that do not originate from a trusted proxy IP. This prevents +external clients from spoofing elevated trust levels. + +Configure trusted proxy IPs: + +[source,elixir] +---- +config :http_capability_gateway, + strip_trust_header: true, + trusted_proxies: ["127.0.0.1", "::1", "10.0.0.1"] +---- + +==== Rate Limiter Configuration + +The token bucket rate limiter enforces per-client, per-trust-level +limits: + +[cols=",,",options="header",] +|=== +|Trust Level |Default Rate |Default Burst +|`+untrusted+` |10 req/s |10 +|`+authenticated+` |100 req/s |100 +|`+internal+` |Unlimited |Unlimited +|=== + +Override defaults in configuration: + +[source,elixir] +---- +config :http_capability_gateway, :rate_limits, %{ + untrusted: {20, 20}, # {rate_per_sec, burst_capacity} + authenticated: {200, 200}, + internal: :unlimited +} +---- + +When a client exceeds their rate limit, the gateway returns: + +* *Status*: `+429 Too Many Requests+` +* *Header*: `+Retry-After: +` +* *Body*: JSON with retry timing + +Client identification uses `+X-Forwarded-For+` first entry (if present) +or the direct peer IP, combined with the trust level. + +==== OWASP Security Headers + +All responses include hardened security headers: + +* `+X-Content-Type-Options: nosniff+` +* `+X-Frame-Options: DENY+` +* `+Referrer-Policy: strict-origin-when-cross-origin+` +* `+Cache-Control: no-store, no-cache, must-revalidate+` +* `+Connection: close+` + +''''' + +=== Troubleshooting + +==== Gateway Fails to Start + +*Symptom*: Service exits immediately or loops in `+Restart+`. + +*Common causes:* + +[arabic] +. *Missing `+POLICY_PATH+`*: In production, `+POLICY_PATH+` is required. ++ +[source,bash] +---- +# Check the variable is set +echo $POLICY_PATH +ls -la $POLICY_PATH +---- +. *Invalid policy file*: The gateway validates the policy at startup and +refuses to start if validation fails. ++ +[source,bash] +---- +# Validate policy manually +mix run -e 'HttpCapabilityGateway.PolicyValidator.validate_file("policy.yaml")' +---- +. *Port already in use*: ++ +[source,bash] +---- +ss -tlnp | grep 4000 +---- +. *Check logs*: ++ +[source,bash] +---- +journalctl -u http-capability-gateway -n 50 --no-pager +# Or for containers: +podman logs http-capability-gateway +---- + +==== All Requests Return 404 (Stealth Mode) + +*Symptom*: Legitimate requests get `+404 Not Found+`. + +*Causes:* + +* The HTTP verb is not in `+global_verbs+` or the route’s `+verbs+` list +* The request path does not match any route regex pattern +* Stealth mode is enabled, masking the actual `+403+` as `+404+` + +*Debug steps:* + +[arabic] +. Check if the verb is allowed for the path in your policy file +. Test path matching: regex patterns must match the full path +. Temporarily disable stealth (`+stealth.enabled: false+`) to see real +403s +. Verify with the readiness endpoint that rules are loaded: ++ +[source,bash] +---- +curl -s http://localhost:4000/ready | jq .policy_rules +---- + +==== 429 Too Many Requests + +*Symptom*: Clients receiving rate limit errors. + +*Causes:* + +* Client exceeding per-trust-level rate limit +* Multiple clients behind the same proxy IP sharing a bucket + +*Solutions:* + +* Check the `+Retry-After+` header for when the client can retry +* Increase rate limits in configuration for the affected trust level +* Ensure `+X-Forwarded-For+` is set correctly by upstream proxies so +clients behind a shared IP get separate buckets + +==== 503 Circuit Breaker Open + +*Symptom*: Requests return `+503 Service Unavailable+` with "`Circuit +breaker open`". + +*Cause*: The backend has exceeded the breach threshold configured in +K9-SVC contracts, causing the circuit breaker to open. + +*Resolution:* + +* Check backend health: the circuit breaker will automatically probe +with half-open requests after the configured timeout +* Check Minikaran dashboard for latency spikes: +`+GET /api/v1/minikaran+` +* If the backend is healthy, the circuit breaker will recover +automatically + +==== 503 Service Configuration Unavailable + +*Symptom*: All requests return `+503+` with "`Service configuration +unavailable`". + +*Cause*: The policy table is nil, meaning policy compilation failed or +the gateway is still starting up. + +*Resolution:* + +* Check startup logs for policy validation errors +* Verify the policy file syntax and content +* Ensure `+POLICY_PATH+` points to a valid DSL v1 YAML file + +==== Backend Connection Errors + +*Symptom*: Allowed requests fail during proxying. + +*Debug steps:* + +[source,bash] +---- +# Test backend connectivity from the gateway host +curl -v $BACKEND_URL/health + +# Check DNS resolution +nslookup backend.internal + +# Check gateway logs for proxy errors +journalctl -u http-capability-gateway | grep -i proxy +---- + +==== High Memory Usage + +*Symptom*: Gateway process consuming excessive memory. + +*Possible causes:* + +[arabic] +. Very large policy file (10,000+ routes) expanding ETS tables +. Large number of unique client IPs filling rate limiter buckets +. Minikaran sliding window accumulating observations + +*Mitigation:* + +[source,bash] +---- +# Check ETS table sizes via remote console +_build/prod/rel/http_capability_gateway/bin/http_capability_gateway remote +> :ets.info(:rate_limiter_buckets, :size) +> :ets.info(:minikaran_observations, :size) + +# Reset rate limiter buckets if needed +> HttpCapabilityGateway.RateLimiter.reset() +---- + +''''' + +=== Support + +* *Issues*: +https://github.com/hyperpolymath/http-capability-gateway/issues +* *Discussions*: +https://github.com/hyperpolymath/http-capability-gateway/discussions diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md deleted file mode 100644 index cb3d8d6..0000000 --- a/docs/DEPLOYMENT.md +++ /dev/null @@ -1,744 +0,0 @@ - -# Deployment Guide — HTTP Capability Gateway v0.1.0-dev - -Practical guide for deploying the HTTP Capability Gateway to production environments. -Covers container-based deployment (Podman/Docker), bare-metal OTP releases, policy -file setup, health checks, monitoring, security configuration, and troubleshooting. - -## Table of Contents - -1. [Prerequisites](#prerequisites) -2. [Container Deployment](#container-deployment) -3. [Bare-Metal Deployment](#bare-metal-deployment) -4. [Policy File Setup](#policy-file-setup) -5. [Health Checks](#health-checks) -6. [Monitoring](#monitoring) -7. [Security](#security) -8. [Troubleshooting](#troubleshooting) - ---- - -## Prerequisites - -### Runtime Requirements - -| Requirement | Minimum | Recommended | -|-------------|---------|-------------| -| Elixir | 1.19+ | 1.19.4 | -| Erlang/OTP | 27+ | 28.2 | -| RAM | 256 MB | 1 GB+ | -| CPU | 1 core | 2+ cores | - -**Or** a container runtime: - -- **Podman** 4.0+ (preferred) -- **Docker** 24.0+ -- **nerdctl** 1.0+ - -The Containerfile uses OCI format and is compatible with all three runtimes. - -### Network Requirements - -- Inbound access on the configured port (default: `4000`) -- Outbound HTTPS/HTTP access to backend services (for proxy) -- DNS resolution for backend service hostnames - ---- - -## Container Deployment - -### Building the Image - -The repository includes a multi-stage `Containerfile` that produces a minimal -Alpine-based runtime image (~30 MB) with no build tools or source code. - -```bash -# Build with Podman (preferred) -podman build -t http-capability-gateway:0.1.0-dev -f Containerfile . - -# Build with Docker -docker build -t http-capability-gateway:0.1.0-dev -f Containerfile . -``` - -The builder stage uses `hexpm/elixir:1.19.4-erlang-28.2.2-alpine-3.22.1`. -The runtime stage uses `alpine:3.22.1` with only `libstdc++`, `ncurses-libs`, -and `openssl` installed. The application runs as a non-root `gateway` user. - -### Running with Podman or Docker - -```bash -# Run the gateway container -podman run -d \ - --name http-capability-gateway \ - -p 8080:4000 \ - -e POLICY_PATH=/app/config/policy.yaml \ - -e BACKEND_URL=http://backend:4000 \ - -e PORT=4000 \ - -v ./my-policy.yaml:/app/config/policy.yaml:ro \ - http-capability-gateway:0.1.0-dev - -# Check logs -podman logs -f http-capability-gateway - -# Stop and remove -podman stop http-capability-gateway -podman rm http-capability-gateway -``` - -### Running with podman-compose or docker-compose - -The repository includes a `docker-compose.yml` that starts the gateway along -with an example httpbin backend for testing. - -```bash -# Start all services (gateway + example backend) -podman-compose up --build -d - -# Or with Docker Compose -docker-compose up --build -d - -# View gateway logs -podman-compose logs -f gateway - -# Test the gateway -curl http://localhost:8080/health -curl http://localhost:8080/ready -curl http://localhost:8080/metrics - -# Stop all services -podman-compose down -``` - -The compose file maps host port `8080` to the gateway's internal port `4000` -and mounts a policy file from `./examples/policy-dev.yaml`. - -### Environment Variables - -| Variable | Required | Default | Description | -|----------|----------|---------|-------------| -| `POLICY_PATH` | Yes (prod) | `config/policy.yaml` | Path to the DSL v1 policy YAML file | -| `BACKEND_URL` | No | `nil` | URL of the backend service to proxy to | -| `PORT` | No | `4000` | HTTP listen port | -| `TRUST_LEVEL_HEADER` | No | `x-trust-level` | Header name for trust level extraction | -| `TRUST_LEVEL_SOURCE` | No | `header` | Trust extraction source: `header` or `mtls` | -| `LOG_LEVEL` | No | `info` | Logger level (`debug`, `info`, `warn`, `error`) | - -### Volume Mounts - -| Host Path | Container Path | Purpose | -|-----------|----------------|---------| -| Your policy file | `/app/config/policy.yaml` | DSL v1 governance policy | -| TLS certificates (optional) | `/app/certs/` | mTLS client CA and server certificates | - ---- - -## Bare-Metal Deployment - -### Mix Release Build - -```bash -# Clone and enter the repository -cd /opt/http-capability-gateway - -# Install dependencies (production only) -export MIX_ENV=prod -mix deps.get --only prod -mix compile - -# Build OTP release -mix release - -# The release is at: -# _build/prod/rel/http_capability_gateway/ -``` - -### Running the Release - -```bash -# Start in foreground (useful for debugging) -_build/prod/rel/http_capability_gateway/bin/http_capability_gateway start - -# Start as background daemon -_build/prod/rel/http_capability_gateway/bin/http_capability_gateway daemon - -# Check if running -_build/prod/rel/http_capability_gateway/bin/http_capability_gateway pid - -# Stop the daemon -_build/prod/rel/http_capability_gateway/bin/http_capability_gateway stop - -# Remote console (attach to running node) -_build/prod/rel/http_capability_gateway/bin/http_capability_gateway remote -``` - -### Systemd Service File - -Create `/etc/systemd/system/http-capability-gateway.service`: - -```ini -[Unit] -Description=HTTP Capability Gateway v0.1.0-dev -Documentation=https://github.com/hyperpolymath/http-capability-gateway -After=network.target - -[Service] -Type=exec -User=gateway -Group=gateway -WorkingDirectory=/opt/http-capability-gateway - -# Environment variables -Environment=POLICY_PATH=/etc/http-capability-gateway/policy.yaml -Environment=BACKEND_URL=http://backend.internal:4000 -Environment=PORT=4000 -Environment=TRUST_LEVEL_HEADER=x-trust-level -Environment=LOG_LEVEL=info - -# Start the OTP release -ExecStart=/opt/http-capability-gateway/_build/prod/rel/http_capability_gateway/bin/http_capability_gateway start -ExecStop=/opt/http-capability-gateway/_build/prod/rel/http_capability_gateway/bin/http_capability_gateway stop - -# Restart policy -Restart=on-failure -RestartSec=5s - -# Logging -StandardOutput=journal -StandardError=journal -SyslogIdentifier=http-capability-gateway - -# Security hardening -NoNewPrivileges=true -PrivateTmp=true -ProtectSystem=strict -ProtectHome=true -ReadWritePaths=/var/log/http-capability-gateway - -[Install] -WantedBy=multi-user.target -``` - -Enable and start the service: - -```bash -# Create the gateway user -sudo useradd --system --shell /usr/sbin/nologin gateway - -# Set ownership -sudo chown -R gateway:gateway /opt/http-capability-gateway - -# Install and start the service -sudo systemctl daemon-reload -sudo systemctl enable http-capability-gateway -sudo systemctl start http-capability-gateway - -# Check status -sudo systemctl status http-capability-gateway -journalctl -u http-capability-gateway -f -``` - -### Configuration via Environment Variables - -For bare-metal deployments, environment variables can be set through: - -1. **Systemd `Environment=` directives** (shown above) -2. **Systemd `EnvironmentFile=`** pointing to a file: - ```ini - EnvironmentFile=/etc/http-capability-gateway/env - ``` - With `/etc/http-capability-gateway/env` containing: - ```bash - POLICY_PATH=/etc/http-capability-gateway/policy.yaml - BACKEND_URL=http://backend.internal:4000 - PORT=4000 - ``` -3. **Shell exports** when running interactively - ---- - -## Policy File Setup - -### Where to Put Policy Files - -| Deployment | Recommended Location | -|------------|---------------------| -| Container | `/app/config/policy.yaml` (volume mount) | -| Bare-metal | `/etc/http-capability-gateway/policy.yaml` | -| Development | `config/policy.yaml` (in-repo) | - -### DSL v1 Format Overview - -Policy files use YAML with the DSL v1 schema. A minimal policy: - -```yaml -dsl_version: "1" -governance: - global_verbs: - - GET - - POST - routes: - - path: "/api/v1/users/[0-9]+" - verbs: [GET, PUT, DELETE] - - path: "/health" - verbs: [GET] -stealth: - enabled: true - status_code: 404 -``` - -For the complete DSL v1 reference, including regex routes, stealth mode options, -and validation rules, see [POLICY-DSL.md](POLICY-DSL.md). - -### Hot Reload via SIGHUP - -Policy files are loaded at startup. To reload the policy without restarting -the gateway, send a SIGHUP signal to the BEAM process: - -```bash -# Find the OS process ID (not the Erlang PID) -kill -HUP $(pidof beam.smp) -``` - -The gateway uses an **atomic dual-table swap** for zero-downtime reloads: - -1. A new pair of ETS tables (main + regex) is compiled from the updated policy -2. If compilation succeeds, both table references are swapped atomically -3. The old tables are deleted after the swap -4. If compilation fails, the old tables remain active (no service disruption) - -This guarantees that in-flight requests are never served from a partially -loaded policy. - ---- - -## Health Checks - -### Liveness: `GET /health` - -Returns `200 OK` if the BEAM process is running. Does **not** check policy -loading or backend connectivity. Use this for container liveness probes. - -```bash -curl -s http://localhost:4000/health | jq . -``` - -```json -{ - "status": "healthy", - "service": "http-capability-gateway", - "version": "0.1.0-dev", - "uptime_seconds": 3600 -} -``` - -### Readiness: `GET /ready` - -Returns `200 OK` only if the policy is loaded and ETS tables are operational. -Returns `503 Service Unavailable` if the gateway is not ready to serve traffic. -Use this for container readiness probes and load balancer health checks. - -```bash -curl -s http://localhost:4000/ready | jq . -``` - -```json -{ - "status": "ready", - "service": "http-capability-gateway", - "policy_rules": 12, - "main_table_rules": 10, - "regex_table_rules": 2, - "rate_limiter_buckets": 42 -} -``` - -The readiness response includes: - -- `policy_rules`: Total compiled rules (main + regex tables) -- `main_table_rules`: Exact-match (O(1)) and global rules -- `regex_table_rules`: Regex pattern routes (O(r) scan) -- `rate_limiter_buckets`: Active client rate limiter entries - -### Container Health Check Configuration - -The Containerfile includes a built-in health check: - -``` -HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 - CMD wget --no-verbose --tries=1 --spider http://localhost:4000/health || exit 1 -``` - -For Kubernetes, configure liveness and readiness probes separately: - -```yaml -livenessProbe: - httpGet: - path: /health - port: 4000 - initialDelaySeconds: 10 - periodSeconds: 30 -readinessProbe: - httpGet: - path: /ready - port: 4000 - initialDelaySeconds: 5 - periodSeconds: 10 -``` - ---- - -## Monitoring - -### Prometheus `/metrics` Endpoint - -The gateway exposes Prometheus-format metrics at `GET /metrics`: - -```bash -curl -s http://localhost:4000/metrics -``` - -#### Key Metrics to Watch - -| Metric | Type | Description | -|--------|------|-------------| -| `http_capability_gateway_request_completed_count` | Counter | Total requests processed | -| `http_capability_gateway_request_completed_duration` | Distribution | Request duration (microseconds) | -| `http_capability_gateway_access_decision_count` | Counter | Decisions by type (`allow`/`deny`), verb, trust level | -| `http_capability_gateway_backend_forward_count` | Counter | Requests forwarded to backend | -| `http_capability_gateway_backend_response_duration` | Distribution | Backend response latency (microseconds) | -| `http_capability_gateway_error_count` | Counter | Errors by type | -| `http_capability_gateway_minikaran_anomaly_count` | Counter | Anomalies detected by type | - -#### Prometheus Scrape Configuration - -```yaml -scrape_configs: - - job_name: 'http-capability-gateway' - scrape_interval: 15s - static_configs: - - targets: ['gateway:4000'] - metrics_path: /metrics -``` - -### Minikaran Anomaly Dashboard: `GET /api/v1/minikaran` - -The built-in Minikaran traffic anomaly detector provides a JSON dashboard -for monitoring unusual traffic patterns: - -```bash -curl -s http://localhost:4000/api/v1/minikaran | jq . -``` - -```json -{ - "status": { - "status": "active", - "windows_collected": 42, - "min_windows_required": 5, - "current_anomalies": 1, - "uptime_sec": 2520 - }, - "anomalies": [ - { - "type": "traffic_spike", - "path": "/api/v1/users", - "current": 150, - "baseline": 42.3 - } - ], - "baseline": { - "window_count": 41, - "avg_requests_per_minute": 85.2, - "trust_distribution": { - "untrusted": 0.6, - "authenticated": 0.35, - "internal": 0.05 - }, - "latency_p50_us": 1200, - "latency_p95_us": 8500, - "latency_p99_us": 25000, - "avg_error_rate": 0.02, - "known_paths": 45, - "avg_unique_clients": 120 - } -} -``` - -**Anomaly types detected:** - -| Type | Description | -|------|-------------| -| `traffic_spike` | Request volume z-score exceeds threshold for a path | -| `trust_shift` | Trust level distribution deviates from learned baseline | -| `latency_spike` | p95 latency exceeds baseline by significant margin | -| `path_novelty` | Unusual number of never-before-seen request paths | -| `error_spike` | Error rate exceeds learned baseline | - -Minikaran requires **5+ baseline windows** (1 minute each) before anomaly -detection activates. During the learning phase, `status` will be `"learning"`. - -### Log Monitoring - -The gateway emits structured JSON logs suitable for ingestion by ELK, -Splunk, DataDog, or CloudWatch: - -```json -{ - "timestamp": "2026-02-28T12:00:00.000Z", - "level": "info", - "message": "access_decision", - "request_id": "a1b2c3d4e5f6", - "path": "/api/v1/users/123", - "verb": "GET", - "trust_level": "authenticated", - "decision": "allow", - "duration_us": 45 -} -``` - ---- - -## Security - -### mTLS Setup - -The gateway supports mTLS-based trust level extraction. To enable: - -1. **Set `TRUST_LEVEL_SOURCE=mtls`** in your environment configuration. - -2. **Configure Cowboy for TLS** in `config/prod.exs`: - ```elixir - config :http_capability_gateway, - scheme: :https, - certfile: "/app/certs/server.pem", - keyfile: "/app/certs/server-key.pem", - cacertfile: "/app/certs/ca.pem", - verify: :verify_peer, - fail_if_no_peer_cert: false - ``` - -3. **Trust level mapping from certificates:** - - `internal` -- Verified certificate with OU = "Internal Services" - - `authenticated` -- Any verified certificate from a trusted CA - - `untrusted` -- No certificate or verification failed - -### Trust Levels and How They Work - -The gateway uses a formally verified trust hierarchy -(from `proven/SafeTrust.idr`): - -``` -untrusted (rank 0) < authenticated (rank 1) < internal (rank 2) -``` - -Access decisions compare trust rank against exposure rank: - -| Trust Level | Can Access Public | Can Access Authenticated | Can Access Internal | -|-------------|-------------------|-------------------------|---------------------| -| `untrusted` | Yes | No | No | -| `authenticated` | Yes | Yes | No | -| `internal` | Yes | Yes | Yes | - -The monotonicity property guarantees that upgrading trust never revokes -previously granted access. - -### Trust Header Spoofing Protection - -By default, the gateway strips the `X-Trust-Level` header from requests -that do not originate from a trusted proxy IP. This prevents external -clients from spoofing elevated trust levels. - -Configure trusted proxy IPs: - -```elixir -config :http_capability_gateway, - strip_trust_header: true, - trusted_proxies: ["127.0.0.1", "::1", "10.0.0.1"] -``` - -### Rate Limiter Configuration - -The token bucket rate limiter enforces per-client, per-trust-level limits: - -| Trust Level | Default Rate | Default Burst | -|-------------|-------------|---------------| -| `untrusted` | 10 req/s | 10 | -| `authenticated` | 100 req/s | 100 | -| `internal` | Unlimited | Unlimited | - -Override defaults in configuration: - -```elixir -config :http_capability_gateway, :rate_limits, %{ - untrusted: {20, 20}, # {rate_per_sec, burst_capacity} - authenticated: {200, 200}, - internal: :unlimited -} -``` - -When a client exceeds their rate limit, the gateway returns: - -- **Status**: `429 Too Many Requests` -- **Header**: `Retry-After: ` -- **Body**: JSON with retry timing - -Client identification uses `X-Forwarded-For` first entry (if present) or -the direct peer IP, combined with the trust level. - -### OWASP Security Headers - -All responses include hardened security headers: - -- `X-Content-Type-Options: nosniff` -- `X-Frame-Options: DENY` -- `Referrer-Policy: strict-origin-when-cross-origin` -- `Cache-Control: no-store, no-cache, must-revalidate` -- `Connection: close` - ---- - -## Troubleshooting - -### Gateway Fails to Start - -**Symptom**: Service exits immediately or loops in `Restart`. - -**Common causes:** - -1. **Missing `POLICY_PATH`**: In production, `POLICY_PATH` is required. - ```bash - # Check the variable is set - echo $POLICY_PATH - ls -la $POLICY_PATH - ``` - -2. **Invalid policy file**: The gateway validates the policy at startup and - refuses to start if validation fails. - ```bash - # Validate policy manually - mix run -e 'HttpCapabilityGateway.PolicyValidator.validate_file("policy.yaml")' - ``` - -3. **Port already in use**: - ```bash - ss -tlnp | grep 4000 - ``` - -4. **Check logs**: - ```bash - journalctl -u http-capability-gateway -n 50 --no-pager - # Or for containers: - podman logs http-capability-gateway - ``` - -### All Requests Return 404 (Stealth Mode) - -**Symptom**: Legitimate requests get `404 Not Found`. - -**Causes:** - -- The HTTP verb is not in `global_verbs` or the route's `verbs` list -- The request path does not match any route regex pattern -- Stealth mode is enabled, masking the actual `403` as `404` - -**Debug steps:** - -1. Check if the verb is allowed for the path in your policy file -2. Test path matching: regex patterns must match the full path -3. Temporarily disable stealth (`stealth.enabled: false`) to see real 403s -4. Verify with the readiness endpoint that rules are loaded: - ```bash - curl -s http://localhost:4000/ready | jq .policy_rules - ``` - -### 429 Too Many Requests - -**Symptom**: Clients receiving rate limit errors. - -**Causes:** - -- Client exceeding per-trust-level rate limit -- Multiple clients behind the same proxy IP sharing a bucket - -**Solutions:** - -- Check the `Retry-After` header for when the client can retry -- Increase rate limits in configuration for the affected trust level -- Ensure `X-Forwarded-For` is set correctly by upstream proxies so - clients behind a shared IP get separate buckets - -### 503 Circuit Breaker Open - -**Symptom**: Requests return `503 Service Unavailable` with "Circuit breaker open". - -**Cause**: The backend has exceeded the breach threshold configured in K9-SVC -contracts, causing the circuit breaker to open. - -**Resolution:** - -- Check backend health: the circuit breaker will automatically probe with - half-open requests after the configured timeout -- Check Minikaran dashboard for latency spikes: `GET /api/v1/minikaran` -- If the backend is healthy, the circuit breaker will recover automatically - -### 503 Service Configuration Unavailable - -**Symptom**: All requests return `503` with "Service configuration unavailable". - -**Cause**: The policy table is nil, meaning policy compilation failed or the -gateway is still starting up. - -**Resolution:** - -- Check startup logs for policy validation errors -- Verify the policy file syntax and content -- Ensure `POLICY_PATH` points to a valid DSL v1 YAML file - -### Backend Connection Errors - -**Symptom**: Allowed requests fail during proxying. - -**Debug steps:** - -```bash -# Test backend connectivity from the gateway host -curl -v $BACKEND_URL/health - -# Check DNS resolution -nslookup backend.internal - -# Check gateway logs for proxy errors -journalctl -u http-capability-gateway | grep -i proxy -``` - -### High Memory Usage - -**Symptom**: Gateway process consuming excessive memory. - -**Possible causes:** - -1. Very large policy file (10,000+ routes) expanding ETS tables -2. Large number of unique client IPs filling rate limiter buckets -3. Minikaran sliding window accumulating observations - -**Mitigation:** - -```bash -# Check ETS table sizes via remote console -_build/prod/rel/http_capability_gateway/bin/http_capability_gateway remote -> :ets.info(:rate_limiter_buckets, :size) -> :ets.info(:minikaran_observations, :size) - -# Reset rate limiter buckets if needed -> HttpCapabilityGateway.RateLimiter.reset() -``` - ---- - -## Support - -- **Issues**: -- **Discussions**: diff --git a/docs/POLICY-DSL.adoc b/docs/POLICY-DSL.adoc new file mode 100644 index 0000000..bc257d2 --- /dev/null +++ b/docs/POLICY-DSL.adoc @@ -0,0 +1,678 @@ +== Policy DSL v1 Reference — HTTP Capability Gateway + +Complete reference for the YAML-based Policy DSL v1 used by the HTTP +Capability Gateway to define verb governance rules. This document covers +the schema, field definitions, regex route patterns, stealth mode, +validation rules, and hot reload behaviour. + +=== Table of Contents + +[arabic] +. link:#overview[Overview] +. link:#dsl-v1-schema[DSL v1 Schema] +. link:#field-reference[Field Reference] +. link:#regex-routes-vs-literal-routes[Regex Routes vs Literal Routes] +. link:#global-rules[Global Rules] +. link:#stealth-mode-configuration[Stealth Mode Configuration] +. link:#example-policy-files[Example Policy Files] +. link:#validation-rules[Validation Rules] +. link:#hot-reload-behaviour[Hot Reload Behaviour] + +''''' + +=== Overview + +The Policy DSL v1 is a declarative YAML format for defining HTTP verb +governance. It controls which HTTP methods (GET, POST, PUT, DELETE, +PATCH, HEAD, OPTIONS) are allowed on which paths, and what happens when +a request is denied. + +*Key concepts:* + +* *Global verbs*: HTTP methods allowed on all routes by default +* *Route overrides*: Path-specific verb lists that take precedence over +globals +* *Stealth mode*: Configurable response for denied requests (hide the +API surface) +* *Tiered lookup*: Literal paths use O(1) ETS lookup; regex patterns use +O(r) scan + +The policy file is loaded at gateway startup and compiled into ETS +tables for sub-microsecond enforcement on every request. + +''''' + +=== DSL v1 Schema + +==== Root Structure + +[source,yaml] +---- +dsl_version: "1" # Required: must be "1" +governance: # Required: verb governance rules + global_verbs: [] # Required: array of HTTP verb strings + routes: [] # Optional: array of route override objects +stealth: # Optional: stealth mode configuration + enabled: boolean # Required if stealth present + status_code: integer # Required if stealth present +---- + +==== Route Object Structure + +[source,yaml] +---- +path: "/api/users" # Required: literal path or regex pattern +verbs: [GET, POST] # Required: array of HTTP verb strings +---- + +''''' + +=== Field Reference + +==== `+dsl_version+` + +[cols=",",options="header",] +|=== +|Property |Value +|Type |String +|Required |Yes +|Valid values |`+"1"+` +|=== + +Specifies the policy format version. Must be the string `+"1"+` for this +version of the gateway. Future versions will use `+"2"+`, `+"3"+`, etc. + +[source,yaml] +---- +dsl_version: "1" +---- + +*Validation:* - Must be present - Must be a string (not integer `+1+`) - +Must equal `+"1"+` exactly + +''''' + +==== `+governance+` + +[cols=",",options="header",] +|=== +|Property |Value +|Type |Object +|Required |Yes +|=== + +Container for all verb governance rules. Must contain `+global_verbs+` +and optionally `+routes+`. + +''''' + +==== `+governance.global_verbs+` + +[cols=",",options="header",] +|=== +|Property |Value +|Type |Array of strings +|Required |Yes +|=== + +HTTP methods allowed on *all routes* unless overridden by a +route-specific `+verbs+` list. These are compiled into +`+{:global, verb_atom}+` entries in the main ETS table for O(1) fallback +lookup (Tier 3 in the tiered lookup strategy). + +*Valid HTTP verbs* (case-sensitive, must be uppercase): + +* `+GET+` +* `+POST+` +* `+PUT+` +* `+DELETE+` +* `+PATCH+` +* `+HEAD+` +* `+OPTIONS+` + +[source,yaml] +---- +governance: + global_verbs: + - GET + - HEAD + - OPTIONS +---- + +*Validation:* - Must be present - Must be a non-empty array - All +elements must be valid HTTP verb strings - Case-sensitive: `+get+` or +`+Get+` will fail validation + +''''' + +==== `+governance.routes+` + +[cols=",",options="header",] +|=== +|Property |Value +|Type |Array of route objects +|Required |No +|=== + +Route-specific verb overrides. When a request path matches a route +pattern, only the verbs listed for that route are allowed — global verbs +do *not* apply for matched routes. + +''''' + +==== `+governance.routes[].path+` + +[cols=",",options="header",] +|=== +|Property |Value +|Type |String +|Required |Yes (per route) +|=== + +Path pattern to match against incoming request paths. Can be a literal +string or a regular expression pattern. + +*Literal paths* (no regex metacharacters): + +[source,yaml] +---- +- path: "/api/users" + verbs: [GET, POST] +---- + +*Regex patterns* (contain `+[+`, `+]+`, `+(+`, `+)+`, `+.+`, `+*+`, +`+++`, `+?+`, `+^+`, `+$+`, `+|+`, or `+\+`): + +[source,yaml] +---- +- path: "/api/users/[0-9]+" + verbs: [GET, PUT, DELETE] +---- + +The gateway detects whether a path is literal or regex using the pattern +`+/[\[\](){}.*+?^$|\\]/+`. Literal paths are stored with +`+{:exact, path, verb}+` keys for O(1) lookup. Regex paths are compiled +to `+Regex.t()+` structs and stored in a dedicated regex ETS table for +O(r) scanning. + +*Validation:* - Must be a non-empty string - If containing regex +metacharacters, must compile as a valid Elixir/PCRE regex + +''''' + +==== `+governance.routes[].verbs+` + +[cols=",",options="header",] +|=== +|Property |Value +|Type |Array of strings +|Required |Yes (per route) +|=== + +HTTP methods allowed for this specific route. Same valid values as +`+global_verbs+`. + +[source,yaml] +---- +- path: "/api/admin" + verbs: [GET] # Admin endpoints are read-only +---- + +*Validation:* - Must be a non-empty array - All elements must be valid +HTTP verb strings + +''''' + +==== `+stealth+` + +[cols=",",options="header",] +|=== +|Property |Value +|Type |Object +|Required |No +|=== + +Stealth mode configuration. Controls what response is sent when a +request is denied (verb not allowed for the path + trust level +combination). + +''''' + +==== `+stealth.enabled+` + +[cols=",",options="header",] +|=== +|Property |Value +|Type |Boolean +|Required |Yes (if `+stealth+` present) +|=== + +When `+true+`, denied requests receive the configured `+status_code+` +instead of the default `+403 Forbidden+`. This makes it harder for +attackers to enumerate the API surface by probing different HTTP +methods. + +''''' + +==== `+stealth.status_code+` + +[cols=",",options="header",] +|=== +|Property |Value +|Type |Integer +|Required |Yes (if `+stealth+` present) +|=== + +HTTP status code returned for denied requests when stealth is enabled. + +*Recommended values:* + +[width="100%",cols="24%,36%,40%",options="header",] +|=== +|Code |Meaning |Use Case +|`+404+` |Not Found |Best default; hides endpoint existence +|`+410+` |Gone |Pretend endpoint was removed +|`+503+` |Service Unavailable |Appears as temporary outage +|`+403+` |Forbidden |Standard denial (use when stealth.enabled = false) +|`+200+` |OK |Pretend success (advanced deception) +|=== + +[source,yaml] +---- +stealth: + enabled: true + status_code: 404 +---- + +''''' + +=== Regex Routes vs Literal Routes + +The gateway distinguishes between literal and regex paths for +performance: + +==== Literal Paths (O(1) Lookup — Tier 1) + +Paths without regex metacharacters are stored with +`+{:exact, path, verb}+` keys in the main ETS table. Lookup is a single +hash table access. + +[source,yaml] +---- +# These are literal paths (O(1) lookup): +- path: "/health" +- path: "/api/v1/users" +- path: "/api/v1/posts" +---- + +==== Regex Paths (O(r) Scan — Tier 2) + +Paths containing regex metacharacters (`+[ ] ( ) { } . * + ? ^ $ | \+`) +are compiled into `+Regex.t()+` structs and stored in a *dedicated regex +ETS table*. On each request, only the regex table is scanned (not the +entire rule set). + +[source,yaml] +---- +# These are regex paths (O(r) scan where r = number of regex routes): +- path: "/api/v1/users/[0-9]+" +- path: "/api/v1/posts/.+" +- path: "/api/v1/resources/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}" +---- + +==== Performance Guideline + +In typical policy files, 90%+ of routes are literal paths, making the +overwhelmingly common case an O(1) hash lookup. Keep regex routes to a +minimum for best performance. + +''''' + +=== Global Rules + +Global verbs serve as the *fallback* (Tier 3) when no route-specific +rule matches a request. They are stored with `+{:global, verb_atom}+` +keys in the main ETS table for O(1) lookup. + +==== Lookup Priority + +For an incoming request to `+/api/v1/users/123+` with method `+GET+`: + +[arabic] +. *Tier 1*: Check `+{:exact, "/api/v1/users/123", :GET}+` in main table +– O(1) +. *Tier 2*: Scan regex table for a pattern matching +`+/api/v1/users/123+` with verb `+:GET+` – O(r) +. *Tier 3*: Check `+{:global, :GET}+` in main table – O(1) +. *No match*: Return `+404+` (or stealth response) + +If a route-specific rule matches at Tier 1 or Tier 2, global rules are +*not consulted* — the route override takes full precedence. + +''''' + +=== Stealth Mode Configuration + +Stealth mode is a security feature that disguises denied responses to +make API enumeration more difficult. + +==== Without Stealth (Default) + +Denied requests receive `+403 Forbidden+` with a clear JSON error +message including the required and provided trust levels: + +[source,json] +---- +{ + "error": "Forbidden", + "message": "Insufficient trust level for this operation", + "required": "internal", + "provided": "untrusted" +} +---- + +==== With Stealth Enabled + +Denied requests receive the configured status code with a generic +message: + +[source,json] +---- +{ + "error": "Not Found" +} +---- + +The `+status_code+` is applied uniformly to all denied requests +regardless of trust level. The profile name `+"default"+` is used +internally. + +==== Stealth Best Practice + +Use `+404+` as the stealth status code for most deployments. It makes +the API surface indistinguishable from nonexistent paths, forcing +attackers to rely on documentation or source code rather than probing. + +''''' + +=== Example Policy Files + +==== Example 1: Public API with Admin Section + +[source,yaml] +---- +dsl_version: "1" +governance: + global_verbs: + - GET + - POST + routes: + # Admin: read-only + - path: "/api/v1/admin/.*" + verbs: [GET] + + # Users: full CRUD with numeric IDs + - path: "/api/v1/users/[0-9]+" + verbs: [GET, PUT, DELETE] + + # Health and metrics: always available + - path: "/health" + verbs: [GET] + - path: "/metrics" + verbs: [GET] + +stealth: + enabled: true + status_code: 404 +---- + +*Behaviour:* + +* `+GET /api/v1/posts+` – Allowed (global GET) +* `+POST /api/v1/posts+` – Allowed (global POST) +* `+DELETE /api/v1/posts+` – Denied (404 via stealth) +* `+GET /api/v1/admin/settings+` – Allowed (route override) +* `+POST /api/v1/admin/settings+` – Denied (404 via stealth, admin is +GET-only) +* `+PUT /api/v1/users/123+` – Allowed (route override) + +==== Example 2: Strict Read-Only API + +[source,yaml] +---- +dsl_version: "1" +governance: + global_verbs: + - GET + - HEAD + - OPTIONS + +stealth: + enabled: true + status_code: 404 +---- + +All endpoints allow only safe HTTP methods. Any write attempt returns +404. + +==== Example 3: Microservice Gateway + +[source,yaml] +---- +dsl_version: "1" +governance: + global_verbs: + - GET + + routes: + # Auth service + - path: "/auth/login" + verbs: [POST] + - path: "/auth/logout" + verbs: [POST] + - path: "/auth/token/refresh" + verbs: [POST] + + # User service (UUID IDs) + - path: "/users/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}" + verbs: [GET, PUT, DELETE] + - path: "/users" + verbs: [GET, POST] + + # Product service + - path: "/products/[0-9]+" + verbs: [GET, PUT, DELETE] + - path: "/products" + verbs: [GET, POST] + +stealth: + enabled: true + status_code: 404 +---- + +==== Example 4: Development (Permissive) + +[source,yaml] +---- +dsl_version: "1" +governance: + global_verbs: + - GET + - POST + - PUT + - DELETE + - PATCH + - HEAD + - OPTIONS + +stealth: + enabled: false + status_code: 403 +---- + +All HTTP methods allowed globally. Stealth disabled for clearer error +messages during development. + +==== Example 5: Complex Regex Patterns + +[source,yaml] +---- +dsl_version: "1" +governance: + global_verbs: + - GET + + routes: + # Date-based endpoints (YYYY-MM-DD) + - path: "/api/reports/[0-9]{4}-[0-9]{2}-[0-9]{2}" + verbs: [GET, POST] + + # UUID v4 resources + - path: "/api/resources/[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}" + verbs: [GET, PUT, DELETE] + + # Slugs (lowercase alphanumeric + hyphens) + - path: "/api/posts/[a-z0-9-]+" + verbs: [GET, PUT, DELETE] + + # Versioned API prefix (v1, v2, etc.) + - path: "/api/v[0-9]+/.*" + verbs: [GET, POST] + +stealth: + enabled: true + status_code: 404 +---- + +''''' + +=== Validation Rules + +==== Startup Validation (PolicyValidator) + +The gateway validates the full policy at startup before compiling it +into ETS tables. If validation fails, *the gateway refuses to start*. + +===== Schema Validation + +* `+dsl_version+` must be present and equal to `+"1"+` +* `+governance+` must be present and be a map +* `+governance.global_verbs+` must be a non-empty array of valid HTTP +verbs +* If `+governance.routes+` is present, it must be an array of valid +route objects +* If `+stealth+` is present, both `+enabled+` (boolean) and +`+status_code+` (integer) are required + +===== Semantic Validation + +* All HTTP verbs must be valid uppercase strings: GET, POST, PUT, +DELETE, PATCH, HEAD, OPTIONS +* All route `+path+` patterns must compile as valid Elixir/PCRE regular +expressions +* Stealth `+status_code+` must be a recognized HTTP status code + +==== Compilation Validation (PolicyCompiler) + +During compilation, additional checks are performed: + +* HTTP verbs are converted to atoms via `+String.to_existing_atom/1+` +(only pre-existing atoms from the `+@valid_http_verbs+` list succeed) +* Regex patterns are compiled via `+Regex.compile/1+` +* Invalid verbs or patterns produce error tuples; if any errors occur, +compilation fails and the existing policy tables remain active + +==== Common Validation Errors + +.... +dsl_version: must be present and equal to "1" +.... + +Missing or invalid `+dsl_version+` field. + +.... +governance.global_verbs: must not be empty +.... + +The `+global_verbs+` array is empty or missing. + +.... +governance.global_verbs: invalid verb "FETCH" +.... + +An unrecognized HTTP method was used. Valid methods are GET, POST, PUT, +DELETE, PATCH, HEAD, OPTIONS. + +.... +governance.routes[2].path: invalid regex pattern "/api/users/[0-9+" +.... + +A route path contains a regex syntax error (unclosed bracket in this +case). + +==== Policy Validation Tool + +Validate a policy file before deployment without starting the gateway: + +[source,bash] +---- +mix run -e ' + case HttpCapabilityGateway.PolicyLoader.load_from_file("policy.yaml") do + {:ok, policy} -> + case HttpCapabilityGateway.PolicyValidator.validate(policy) do + :ok -> IO.puts("Policy is valid") + {:error, reason} -> IO.puts("Validation failed: #{reason}") + end + {:error, reason} -> + IO.puts("Load failed: #{reason}") + end +' +---- + +''''' + +=== Hot Reload Behaviour + +==== Atomic Dual-Table Swap + +When a policy reload is triggered, the gateway performs a zero-downtime +atomic swap of *both* ETS tables (main + regex): + +[arabic] +. *Create temporary tables*: New main and regex tables are created with +unique monotonic-time-based names +. *Compile into temporary tables*: All policy rules are compiled into +the appropriate temporary table (literal paths to main, regex paths to +regex table, global rules to main) +. *On success* (zero compilation errors): +* Application environment `+:policy_table+` is updated to the new main +table name +* Application environment `+:policy_regex_table+` is updated to the new +regex table name +* Old main and regex tables are deleted +* All in-flight requests seamlessly transition to the new tables +. *On failure* (any compilation error): +* Both temporary tables are deleted +* Old tables and application environment references remain unchanged +* The last known good policy continues to serve traffic + +==== Guarantees + +* *Zero downtime*: The app env reference swap is a single atomic +operation from the perspective of concurrent readers. There is no moment +where the reference points to a nonexistent or partially-loaded table. +* *Rollback on failure*: If the new policy fails validation or +compilation, the old policy remains active. No service disruption +occurs. +* *Dual-table consistency*: Both the main table and regex table are +swapped as a pair. It is impossible to have the main table from policy +version N paired with the regex table from policy version N-1. + +''''' + +=== Support + +* *Issues*: +https://github.com/hyperpolymath/http-capability-gateway/issues +* *Discussions*: +https://github.com/hyperpolymath/http-capability-gateway/discussions diff --git a/docs/POLICY-DSL.md b/docs/POLICY-DSL.md deleted file mode 100644 index bafc979..0000000 --- a/docs/POLICY-DSL.md +++ /dev/null @@ -1,617 +0,0 @@ - -# Policy DSL v1 Reference — HTTP Capability Gateway - -Complete reference for the YAML-based Policy DSL v1 used by the HTTP Capability -Gateway to define verb governance rules. This document covers the schema, field -definitions, regex route patterns, stealth mode, validation rules, and hot reload -behaviour. - -## Table of Contents - -1. [Overview](#overview) -2. [DSL v1 Schema](#dsl-v1-schema) -3. [Field Reference](#field-reference) -4. [Regex Routes vs Literal Routes](#regex-routes-vs-literal-routes) -5. [Global Rules](#global-rules) -6. [Stealth Mode Configuration](#stealth-mode-configuration) -7. [Example Policy Files](#example-policy-files) -8. [Validation Rules](#validation-rules) -9. [Hot Reload Behaviour](#hot-reload-behaviour) - ---- - -## Overview - -The Policy DSL v1 is a declarative YAML format for defining HTTP verb governance. -It controls which HTTP methods (GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS) are -allowed on which paths, and what happens when a request is denied. - -**Key concepts:** - -- **Global verbs**: HTTP methods allowed on all routes by default -- **Route overrides**: Path-specific verb lists that take precedence over globals -- **Stealth mode**: Configurable response for denied requests (hide the API surface) -- **Tiered lookup**: Literal paths use O(1) ETS lookup; regex patterns use O(r) scan - -The policy file is loaded at gateway startup and compiled into ETS tables for -sub-microsecond enforcement on every request. - ---- - -## DSL v1 Schema - -### Root Structure - -```yaml -dsl_version: "1" # Required: must be "1" -governance: # Required: verb governance rules - global_verbs: [] # Required: array of HTTP verb strings - routes: [] # Optional: array of route override objects -stealth: # Optional: stealth mode configuration - enabled: boolean # Required if stealth present - status_code: integer # Required if stealth present -``` - -### Route Object Structure - -```yaml -path: "/api/users" # Required: literal path or regex pattern -verbs: [GET, POST] # Required: array of HTTP verb strings -``` - ---- - -## Field Reference - -### `dsl_version` - -| Property | Value | -|----------|-------| -| Type | String | -| Required | Yes | -| Valid values | `"1"` | - -Specifies the policy format version. Must be the string `"1"` for this version -of the gateway. Future versions will use `"2"`, `"3"`, etc. - -```yaml -dsl_version: "1" -``` - -**Validation:** -- Must be present -- Must be a string (not integer `1`) -- Must equal `"1"` exactly - ---- - -### `governance` - -| Property | Value | -|----------|-------| -| Type | Object | -| Required | Yes | - -Container for all verb governance rules. Must contain `global_verbs` and -optionally `routes`. - ---- - -### `governance.global_verbs` - -| Property | Value | -|----------|-------| -| Type | Array of strings | -| Required | Yes | - -HTTP methods allowed on **all routes** unless overridden by a route-specific -`verbs` list. These are compiled into `{:global, verb_atom}` entries in the -main ETS table for O(1) fallback lookup (Tier 3 in the tiered lookup strategy). - -**Valid HTTP verbs** (case-sensitive, must be uppercase): - -- `GET` -- `POST` -- `PUT` -- `DELETE` -- `PATCH` -- `HEAD` -- `OPTIONS` - -```yaml -governance: - global_verbs: - - GET - - HEAD - - OPTIONS -``` - -**Validation:** -- Must be present -- Must be a non-empty array -- All elements must be valid HTTP verb strings -- Case-sensitive: `get` or `Get` will fail validation - ---- - -### `governance.routes` - -| Property | Value | -|----------|-------| -| Type | Array of route objects | -| Required | No | - -Route-specific verb overrides. When a request path matches a route pattern, -only the verbs listed for that route are allowed — global verbs do **not** -apply for matched routes. - ---- - -### `governance.routes[].path` - -| Property | Value | -|----------|-------| -| Type | String | -| Required | Yes (per route) | - -Path pattern to match against incoming request paths. Can be a literal string -or a regular expression pattern. - -**Literal paths** (no regex metacharacters): - -```yaml -- path: "/api/users" - verbs: [GET, POST] -``` - -**Regex patterns** (contain `[`, `]`, `(`, `)`, `.`, `*`, `+`, `?`, `^`, `$`, `|`, or `\`): - -```yaml -- path: "/api/users/[0-9]+" - verbs: [GET, PUT, DELETE] -``` - -The gateway detects whether a path is literal or regex using the pattern -`/[\[\](){}.*+?^$|\\]/`. Literal paths are stored with `{:exact, path, verb}` -keys for O(1) lookup. Regex paths are compiled to `Regex.t()` structs and -stored in a dedicated regex ETS table for O(r) scanning. - -**Validation:** -- Must be a non-empty string -- If containing regex metacharacters, must compile as a valid Elixir/PCRE regex - ---- - -### `governance.routes[].verbs` - -| Property | Value | -|----------|-------| -| Type | Array of strings | -| Required | Yes (per route) | - -HTTP methods allowed for this specific route. Same valid values as `global_verbs`. - -```yaml -- path: "/api/admin" - verbs: [GET] # Admin endpoints are read-only -``` - -**Validation:** -- Must be a non-empty array -- All elements must be valid HTTP verb strings - ---- - -### `stealth` - -| Property | Value | -|----------|-------| -| Type | Object | -| Required | No | - -Stealth mode configuration. Controls what response is sent when a request -is denied (verb not allowed for the path + trust level combination). - ---- - -### `stealth.enabled` - -| Property | Value | -|----------|-------| -| Type | Boolean | -| Required | Yes (if `stealth` present) | - -When `true`, denied requests receive the configured `status_code` instead -of the default `403 Forbidden`. This makes it harder for attackers to -enumerate the API surface by probing different HTTP methods. - ---- - -### `stealth.status_code` - -| Property | Value | -|----------|-------| -| Type | Integer | -| Required | Yes (if `stealth` present) | - -HTTP status code returned for denied requests when stealth is enabled. - -**Recommended values:** - -| Code | Meaning | Use Case | -|------|---------|----------| -| `404` | Not Found | Best default; hides endpoint existence | -| `410` | Gone | Pretend endpoint was removed | -| `503` | Service Unavailable | Appears as temporary outage | -| `403` | Forbidden | Standard denial (use when stealth.enabled = false) | -| `200` | OK | Pretend success (advanced deception) | - -```yaml -stealth: - enabled: true - status_code: 404 -``` - ---- - -## Regex Routes vs Literal Routes - -The gateway distinguishes between literal and regex paths for performance: - -### Literal Paths (O(1) Lookup — Tier 1) - -Paths without regex metacharacters are stored with `{:exact, path, verb}` keys -in the main ETS table. Lookup is a single hash table access. - -```yaml -# These are literal paths (O(1) lookup): -- path: "/health" -- path: "/api/v1/users" -- path: "/api/v1/posts" -``` - -### Regex Paths (O(r) Scan — Tier 2) - -Paths containing regex metacharacters (`[ ] ( ) { } . * + ? ^ $ | \`) are -compiled into `Regex.t()` structs and stored in a **dedicated regex ETS table**. -On each request, only the regex table is scanned (not the entire rule set). - -```yaml -# These are regex paths (O(r) scan where r = number of regex routes): -- path: "/api/v1/users/[0-9]+" -- path: "/api/v1/posts/.+" -- path: "/api/v1/resources/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}" -``` - -### Performance Guideline - -In typical policy files, 90%+ of routes are literal paths, making the -overwhelmingly common case an O(1) hash lookup. Keep regex routes to a -minimum for best performance. - ---- - -## Global Rules - -Global verbs serve as the **fallback** (Tier 3) when no route-specific rule -matches a request. They are stored with `{:global, verb_atom}` keys in the -main ETS table for O(1) lookup. - -### Lookup Priority - -For an incoming request to `/api/v1/users/123` with method `GET`: - -1. **Tier 1**: Check `{:exact, "/api/v1/users/123", :GET}` in main table -- O(1) -2. **Tier 2**: Scan regex table for a pattern matching `/api/v1/users/123` with verb `:GET` -- O(r) -3. **Tier 3**: Check `{:global, :GET}` in main table -- O(1) -4. **No match**: Return `404` (or stealth response) - -If a route-specific rule matches at Tier 1 or Tier 2, global rules are -**not consulted** — the route override takes full precedence. - ---- - -## Stealth Mode Configuration - -Stealth mode is a security feature that disguises denied responses to make -API enumeration more difficult. - -### Without Stealth (Default) - -Denied requests receive `403 Forbidden` with a clear JSON error message -including the required and provided trust levels: - -```json -{ - "error": "Forbidden", - "message": "Insufficient trust level for this operation", - "required": "internal", - "provided": "untrusted" -} -``` - -### With Stealth Enabled - -Denied requests receive the configured status code with a generic message: - -```json -{ - "error": "Not Found" -} -``` - -The `status_code` is applied uniformly to all denied requests regardless -of trust level. The profile name `"default"` is used internally. - -### Stealth Best Practice - -Use `404` as the stealth status code for most deployments. It makes the -API surface indistinguishable from nonexistent paths, forcing attackers -to rely on documentation or source code rather than probing. - ---- - -## Example Policy Files - -### Example 1: Public API with Admin Section - -```yaml -dsl_version: "1" -governance: - global_verbs: - - GET - - POST - routes: - # Admin: read-only - - path: "/api/v1/admin/.*" - verbs: [GET] - - # Users: full CRUD with numeric IDs - - path: "/api/v1/users/[0-9]+" - verbs: [GET, PUT, DELETE] - - # Health and metrics: always available - - path: "/health" - verbs: [GET] - - path: "/metrics" - verbs: [GET] - -stealth: - enabled: true - status_code: 404 -``` - -**Behaviour:** - -- `GET /api/v1/posts` -- Allowed (global GET) -- `POST /api/v1/posts` -- Allowed (global POST) -- `DELETE /api/v1/posts` -- Denied (404 via stealth) -- `GET /api/v1/admin/settings` -- Allowed (route override) -- `POST /api/v1/admin/settings` -- Denied (404 via stealth, admin is GET-only) -- `PUT /api/v1/users/123` -- Allowed (route override) - -### Example 2: Strict Read-Only API - -```yaml -dsl_version: "1" -governance: - global_verbs: - - GET - - HEAD - - OPTIONS - -stealth: - enabled: true - status_code: 404 -``` - -All endpoints allow only safe HTTP methods. Any write attempt returns 404. - -### Example 3: Microservice Gateway - -```yaml -dsl_version: "1" -governance: - global_verbs: - - GET - - routes: - # Auth service - - path: "/auth/login" - verbs: [POST] - - path: "/auth/logout" - verbs: [POST] - - path: "/auth/token/refresh" - verbs: [POST] - - # User service (UUID IDs) - - path: "/users/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}" - verbs: [GET, PUT, DELETE] - - path: "/users" - verbs: [GET, POST] - - # Product service - - path: "/products/[0-9]+" - verbs: [GET, PUT, DELETE] - - path: "/products" - verbs: [GET, POST] - -stealth: - enabled: true - status_code: 404 -``` - -### Example 4: Development (Permissive) - -```yaml -dsl_version: "1" -governance: - global_verbs: - - GET - - POST - - PUT - - DELETE - - PATCH - - HEAD - - OPTIONS - -stealth: - enabled: false - status_code: 403 -``` - -All HTTP methods allowed globally. Stealth disabled for clearer error messages -during development. - -### Example 5: Complex Regex Patterns - -```yaml -dsl_version: "1" -governance: - global_verbs: - - GET - - routes: - # Date-based endpoints (YYYY-MM-DD) - - path: "/api/reports/[0-9]{4}-[0-9]{2}-[0-9]{2}" - verbs: [GET, POST] - - # UUID v4 resources - - path: "/api/resources/[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}" - verbs: [GET, PUT, DELETE] - - # Slugs (lowercase alphanumeric + hyphens) - - path: "/api/posts/[a-z0-9-]+" - verbs: [GET, PUT, DELETE] - - # Versioned API prefix (v1, v2, etc.) - - path: "/api/v[0-9]+/.*" - verbs: [GET, POST] - -stealth: - enabled: true - status_code: 404 -``` - ---- - -## Validation Rules - -### Startup Validation (PolicyValidator) - -The gateway validates the full policy at startup before compiling it into -ETS tables. If validation fails, **the gateway refuses to start**. - -#### Schema Validation - -- `dsl_version` must be present and equal to `"1"` -- `governance` must be present and be a map -- `governance.global_verbs` must be a non-empty array of valid HTTP verbs -- If `governance.routes` is present, it must be an array of valid route objects -- If `stealth` is present, both `enabled` (boolean) and `status_code` (integer) are required - -#### Semantic Validation - -- All HTTP verbs must be valid uppercase strings: GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS -- All route `path` patterns must compile as valid Elixir/PCRE regular expressions -- Stealth `status_code` must be a recognized HTTP status code - -### Compilation Validation (PolicyCompiler) - -During compilation, additional checks are performed: - -- HTTP verbs are converted to atoms via `String.to_existing_atom/1` (only pre-existing - atoms from the `@valid_http_verbs` list succeed) -- Regex patterns are compiled via `Regex.compile/1` -- Invalid verbs or patterns produce error tuples; if any errors occur, compilation - fails and the existing policy tables remain active - -### Common Validation Errors - -``` -dsl_version: must be present and equal to "1" -``` -Missing or invalid `dsl_version` field. - -``` -governance.global_verbs: must not be empty -``` -The `global_verbs` array is empty or missing. - -``` -governance.global_verbs: invalid verb "FETCH" -``` -An unrecognized HTTP method was used. Valid methods are GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS. - -``` -governance.routes[2].path: invalid regex pattern "/api/users/[0-9+" -``` -A route path contains a regex syntax error (unclosed bracket in this case). - -### Policy Validation Tool - -Validate a policy file before deployment without starting the gateway: - -```bash -mix run -e ' - case HttpCapabilityGateway.PolicyLoader.load_from_file("policy.yaml") do - {:ok, policy} -> - case HttpCapabilityGateway.PolicyValidator.validate(policy) do - :ok -> IO.puts("Policy is valid") - {:error, reason} -> IO.puts("Validation failed: #{reason}") - end - {:error, reason} -> - IO.puts("Load failed: #{reason}") - end -' -``` - ---- - -## Hot Reload Behaviour - -### Atomic Dual-Table Swap - -When a policy reload is triggered, the gateway performs a zero-downtime -atomic swap of **both** ETS tables (main + regex): - -1. **Create temporary tables**: New main and regex tables are created with - unique monotonic-time-based names - -2. **Compile into temporary tables**: All policy rules are compiled into - the appropriate temporary table (literal paths to main, regex paths to - regex table, global rules to main) - -3. **On success** (zero compilation errors): - - Application environment `:policy_table` is updated to the new main table name - - Application environment `:policy_regex_table` is updated to the new regex table name - - Old main and regex tables are deleted - - All in-flight requests seamlessly transition to the new tables - -4. **On failure** (any compilation error): - - Both temporary tables are deleted - - Old tables and application environment references remain unchanged - - The last known good policy continues to serve traffic - -### Guarantees - -- **Zero downtime**: The app env reference swap is a single atomic operation - from the perspective of concurrent readers. There is no moment where the - reference points to a nonexistent or partially-loaded table. - -- **Rollback on failure**: If the new policy fails validation or compilation, - the old policy remains active. No service disruption occurs. - -- **Dual-table consistency**: Both the main table and regex table are swapped - as a pair. It is impossible to have the main table from policy version N - paired with the regex table from policy version N-1. - ---- - -## Support - -- **Issues**: -- **Discussions**: diff --git a/docs/RELEASE-CRITERIA.adoc b/docs/RELEASE-CRITERIA.adoc new file mode 100644 index 0000000..41246dd --- /dev/null +++ b/docs/RELEASE-CRITERIA.adoc @@ -0,0 +1,146 @@ +== Release Criteria — HTTP Capability Gateway + +*Version:* applies from v0.1.0 onward *Last updated:* 2026-04-16 + +This document replaces any earlier percentage-based release gating. A +release is either ready or it isn’t, and readiness is determined by +*executed tests*, not topology coverage bars or design-document +completion claims. + +=== Principle: Executed Tests, Not Topology + +The ROADMAP.adoc P0 section explicitly calls out the problem: + +____ +Add release criteria that require executed tests rather than topology +percentages. +____ + +Historically, documents like `+TOPOLOGY.md+` advertised "`OVERALL: ~97% +Production-ready`" while core features (e.g., mTLS certificate subject +extraction) were demonstrably broken when reviewed. This caused false +confidence. Going forward, release readiness is measured only by what +the test suite actually exercises. + +''''' + +=== v0.1.0 Release Gate + +A v0.1.0 release SHALL meet *all* of the following criteria. Each one is +checkable by running the test suite or inspecting a deterministic +artefact. + +==== 1. Test-suite Gate + +* [ ] `+mix test+` exits 0 (unit tests + security + E2E pass) +* [ ] `+mix test --only :concurrency+` exits 0 (concurrency/failure +modes) +* [ ] `+mix test --only :property+` exits 0 (property-based tests) +* [ ] No test is tagged `+:skip+` or `+@tag :pending+` in main branch +* [ ] The fuzz suite (`+test/fuzz_test.exs+`) exits 0 with default +`+max_runs+` + +==== 2. MVP Scope Gate + +For each claim in `+ROADMAP.adoc+` "`MVP Proof Requirements`" table, +there must be at least one passing test in the specified file: + +* [ ] Policy loading → `+test/policy_loader_test.exs+` +* [ ] Policy validation → `+test/policy_validator_test.exs+` +* [ ] Policy compilation → `+test/policy_compiler_test.exs+` +* [ ] Trust extraction → `+test/security_test.exs+` +* [ ] Verb governance → `+test/gateway_test.exs+`, `+test/e2e_test.exs+` +* [ ] Allow/deny decisions → `+test/e2e_test.exs+` +* [ ] Stealth mode → `+test/gateway_test.exs+` +* [ ] Rate limiting → `+test/concurrency_test.exs+`, +`+test/benchmark_test.exs+` +* [ ] Health/readiness → `+test/e2e_test.exs+` +* [ ] Atomic policy reload → `+test/e2e_test.exs+`, +`+test/concurrency_test.exs+` +* [ ] Request sanitization → `+test/security_test.exs+` +* [ ] Trust spoofing prevention → `+test/security_test.exs+` +* [ ] No atom exhaustion → `+test/fuzz_test.exs+` +* [ ] No crash on arbitrary input → `+test/fuzz_test.exs+` +* [ ] Circuit breaker FSM → `+test/circuit_breaker_test.exs+` +* [ ] K9 contracts → `+test/k9_contract_test.exs+` + +==== 3. Documentation Gate + +* [ ] `+docs/SUPPORTED-FEATURES.md+` lists no "`Supported`" feature +without a corresponding test file +* [ ] `+STATE.adoc+` version matches `+mix.exs+` version +* [ ] `+.machine_readable/STATE.a2ml+` version matches `+mix.exs+` +version +* [ ] No historical document claims current-state facts without a +prominent "`HISTORICAL`" banner + +==== 4. Security Gate + +* [ ] No unaddressed High-severity findings in `+report.json+` +* [ ] No `+String.to_atom/1+` or `+String.to_existing_atom/1+` calls on +user-supplied input +* [ ] No hardcoded secrets (verified by `+panic-attacker+` scan) +* [ ] Security headers verified on all response paths (including +denied/health/metrics) + +==== 5. Operator Gate + +* [ ] `+docs/SUPPORTED-FEATURES.md+` operator checklist is reviewable +before deployment +* [ ] `+docs/DEPLOYMENT.md+` version references match `+mix.exs+` +version +* [ ] At least one reference deployment described in +`+docs/SCOPED-DEPLOYMENT.md+` (selected routes, not whole surface) + +''''' + +=== What is NOT a Release Gate + +The following are deliberately excluded from the release gate because +they proved to be misleading: + +* *Topology percentages*: Component completion bars. +* *Lines of code*: LOC is not a progress metric. +* *Number of modules implemented*: Code presence ≠ correctness. +* *Design doc completion*: Documents describing a feature ≠ the feature +working. +* *Self-declared status*: A module claiming "`stable`" in its +`+@moduledoc+` is not evidence of stability. + +These may still appear in dashboards for informational purposes, but +they do not gate the release. + +''''' + +=== Pre-release Checklist + +Before tagging a release, execute in order: + +[source,bash] +---- +just build # or: mix compile --warnings-as-errors +just test # mix test (P0 unit + security + E2E + fuzz) +mix test --only :concurrency # P1 contention tests +mix test --only :benchmark # P1 benchmarks (smoke bounds only) +just security-scan # panic-attacker / scorecard +just doctor # environment validation +---- + +All commands MUST exit 0. If any fail, the release is blocked regardless +of how many "`97% complete`" bars the topology shows. + +''''' + +=== Amendment Policy + +This document can only be amended by: + +[arabic] +. Adding a new gate (stricter release criteria) — any maintainer. +. Removing a gate — requires explicit rationale in the commit message +explaining why the previous gate was wrong or is now covered elsewhere. +. Renaming a test file — update the referring section atomically in the +same commit. + +Weakening the gates without evidence undermines the entire point of this +document. diff --git a/docs/RELEASE-CRITERIA.md b/docs/RELEASE-CRITERIA.md deleted file mode 100644 index c288511..0000000 --- a/docs/RELEASE-CRITERIA.md +++ /dev/null @@ -1,136 +0,0 @@ - -# Release Criteria — HTTP Capability Gateway - -**Version:** applies from v0.1.0 onward -**Last updated:** 2026-04-16 - -This document replaces any earlier percentage-based release gating. -A release is either ready or it isn't, and readiness is determined by -**executed tests**, not topology coverage bars or design-document -completion claims. - -## Principle: Executed Tests, Not Topology - -The ROADMAP.adoc P0 section explicitly calls out the problem: - -> Add release criteria that require executed tests rather than topology -> percentages. - -Historically, documents like `TOPOLOGY.md` advertised "OVERALL: ~97% -Production-ready" while core features (e.g., mTLS certificate subject -extraction) were demonstrably broken when reviewed. This caused false -confidence. Going forward, release readiness is measured only by what -the test suite actually exercises. - ---- - -## v0.1.0 Release Gate - -A v0.1.0 release SHALL meet **all** of the following criteria. Each one is -checkable by running the test suite or inspecting a deterministic artefact. - -### 1. Test-suite Gate - -- [ ] `mix test` exits 0 (unit tests + security + E2E pass) -- [ ] `mix test --only :concurrency` exits 0 (concurrency/failure modes) -- [ ] `mix test --only :property` exits 0 (property-based tests) -- [ ] No test is tagged `:skip` or `@tag :pending` in main branch -- [ ] The fuzz suite (`test/fuzz_test.exs`) exits 0 with default `max_runs` - -### 2. MVP Scope Gate - -For each claim in `ROADMAP.adoc` "MVP Proof Requirements" table, there -must be at least one passing test in the specified file: - -- [ ] Policy loading → `test/policy_loader_test.exs` -- [ ] Policy validation → `test/policy_validator_test.exs` -- [ ] Policy compilation → `test/policy_compiler_test.exs` -- [ ] Trust extraction → `test/security_test.exs` -- [ ] Verb governance → `test/gateway_test.exs`, `test/e2e_test.exs` -- [ ] Allow/deny decisions → `test/e2e_test.exs` -- [ ] Stealth mode → `test/gateway_test.exs` -- [ ] Rate limiting → `test/concurrency_test.exs`, `test/benchmark_test.exs` -- [ ] Health/readiness → `test/e2e_test.exs` -- [ ] Atomic policy reload → `test/e2e_test.exs`, `test/concurrency_test.exs` -- [ ] Request sanitization → `test/security_test.exs` -- [ ] Trust spoofing prevention → `test/security_test.exs` -- [ ] No atom exhaustion → `test/fuzz_test.exs` -- [ ] No crash on arbitrary input → `test/fuzz_test.exs` -- [ ] Circuit breaker FSM → `test/circuit_breaker_test.exs` -- [ ] K9 contracts → `test/k9_contract_test.exs` - -### 3. Documentation Gate - -- [ ] `docs/SUPPORTED-FEATURES.md` lists no "Supported" feature without - a corresponding test file -- [ ] `STATE.adoc` version matches `mix.exs` version -- [ ] `.machine_readable/STATE.a2ml` version matches `mix.exs` version -- [ ] No historical document claims current-state facts without a prominent - "HISTORICAL" banner - -### 4. Security Gate - -- [ ] No unaddressed High-severity findings in `report.json` -- [ ] No `String.to_atom/1` or `String.to_existing_atom/1` calls on user-supplied input -- [ ] No hardcoded secrets (verified by `panic-attacker` scan) -- [ ] Security headers verified on all response paths (including denied/health/metrics) - -### 5. Operator Gate - -- [ ] `docs/SUPPORTED-FEATURES.md` operator checklist is reviewable before deployment -- [ ] `docs/DEPLOYMENT.md` version references match `mix.exs` version -- [ ] At least one reference deployment described in `docs/SCOPED-DEPLOYMENT.md` - (selected routes, not whole surface) - ---- - -## What is NOT a Release Gate - -The following are deliberately excluded from the release gate because they -proved to be misleading: - -- **Topology percentages**: Component completion bars. -- **Lines of code**: LOC is not a progress metric. -- **Number of modules implemented**: Code presence ≠ correctness. -- **Design doc completion**: Documents describing a feature ≠ the feature working. -- **Self-declared status**: A module claiming "stable" in its `@moduledoc` - is not evidence of stability. - -These may still appear in dashboards for informational purposes, but they -do not gate the release. - ---- - -## Pre-release Checklist - -Before tagging a release, execute in order: - -```bash -just build # or: mix compile --warnings-as-errors -just test # mix test (P0 unit + security + E2E + fuzz) -mix test --only :concurrency # P1 contention tests -mix test --only :benchmark # P1 benchmarks (smoke bounds only) -just security-scan # panic-attacker / scorecard -just doctor # environment validation -``` - -All commands MUST exit 0. If any fail, the release is blocked regardless -of how many "97% complete" bars the topology shows. - ---- - -## Amendment Policy - -This document can only be amended by: - -1. Adding a new gate (stricter release criteria) — any maintainer. -2. Removing a gate — requires explicit rationale in the commit message - explaining why the previous gate was wrong or is now covered elsewhere. -3. Renaming a test file — update the referring section atomically in the - same commit. - -Weakening the gates without evidence undermines the entire point of this -document. diff --git a/docs/SCOPED-DEPLOYMENT.md b/docs/SCOPED-DEPLOYMENT.adoc similarity index 52% rename from docs/SCOPED-DEPLOYMENT.md rename to docs/SCOPED-DEPLOYMENT.adoc index fcfc6af..35fbc5a 100644 --- a/docs/SCOPED-DEPLOYMENT.md +++ b/docs/SCOPED-DEPLOYMENT.adoc @@ -1,33 +1,33 @@ - -# Scoped Deployment Guide +== Scoped Deployment Guide -**Recommendation status:** Required for v0.1.0 — do NOT front the entire +*Recommendation status:* Required for v0.1.0 — do NOT front the entire application surface with this gateway at v0.1.0. -## Why Scoped Deployment +=== Why Scoped Deployment -The HTTP Capability Gateway at v0.1.0 is a **narrow verb-governance -prefilter**, not a general-purpose API gateway. The roadmap (`ROADMAP.adoc` -P2) explicitly says: +The HTTP Capability Gateway at v0.1.0 is a *narrow verb-governance +prefilter*, not a general-purpose API gateway. The roadmap +(`+ROADMAP.adoc+` P2) explicitly says: -> Use the gateway in front of selected API routes first, not the whole -> application surface. +____ +Use the gateway in front of selected API routes first, not the whole +application surface. +____ -> Keep the runtime role constrained to prefiltering before origin-side -> enforcement. +____ +Keep the runtime role constrained to prefiltering before origin-side +enforcement. +____ The rationale is simple: the gateway has not yet been hardened for the full range of protocols, trust sources, and failure modes that a universal front-door must handle. Deploying it in front of a few -high-value API routes is **provable and reversible**. Deploying it in +high-value API routes is *provable and reversible*. Deploying it in front of your entire origin is neither. -## The Scoped-Deployment Pattern +=== The Scoped-Deployment Pattern -``` +.... Client │ ▼ @@ -49,37 +49,50 @@ front of your entire origin is neither. │ gateway │──► directly │ │ (verb filter) │ │ │ └──────────────────┘ └──────────────────┘ -``` +.... -Traffic for **selected** routes is routed through the gateway first, where +Traffic for *selected* routes is routed through the gateway first, where verb governance, rate limiting, stealth responses, and circuit breaking apply. All other traffic bypasses the gateway entirely. -## Choosing the Selected Routes +=== Choosing the Selected Routes Good candidates for initial scoped deployment: -| Characteristic | Why it's a good fit | -|----------------|---------------------| -| Verb-sensitive | Some verbs should be `internal` only (e.g., `DELETE /api/users/:id`). | -| Rate-abusable | Login, signup, search — clear token-bucket value. | -| Low traffic volume (but high value) | Observable quickly; failures affect a small blast radius. | -| Already behind authentication | Trust levels can be supplied by your existing auth edge. | -| Non-streaming | The gateway does not yet support WebSocket or long-lived streaming. | +[width="100%",cols="44%,56%",options="header",] +|=== +|Characteristic |Why it’s a good fit +|Verb-sensitive |Some verbs should be `+internal+` only (e.g., +`+DELETE /api/users/:id+`). + +|Rate-abusable |Login, signup, search — clear token-bucket value. + +|Low traffic volume (but high value) |Observable quickly; failures +affect a small blast radius. + +|Already behind authentication |Trust levels can be supplied by your +existing auth edge. + +|Non-streaming |The gateway does not yet support WebSocket or long-lived +streaming. +|=== Poor candidates for initial deployment: -- Static asset delivery (no governance benefit, added latency) -- WebSocket / server-sent-events endpoints (not supported) -- GraphQL or gRPC endpoints (handlers are stubs — see `SUPPORTED-FEATURES.md`) -- Anything requiring TLS termination by this gateway -- Multi-backend load-balanced routes (the gateway has one backend per policy) +* Static asset delivery (no governance benefit, added latency) +* WebSocket / server-sent-events endpoints (not supported) +* GraphQL or gRPC endpoints (handlers are stubs — see +`+SUPPORTED-FEATURES.md+`) +* Anything requiring TLS termination by this gateway +* Multi-backend load-balanced routes (the gateway has one backend per +policy) -## Example: Putting the Gateway In Front of `/api/admin/*` Only +=== Example: Putting the Gateway In Front of `+/api/admin/*+` Only In your edge proxy (nginx example): -```nginx +[source,nginx] +---- # Governed routes → http-capability-gateway location /api/admin/ { proxy_pass http://http-capability-gateway:4000; @@ -91,11 +104,12 @@ location /api/admin/ { location / { proxy_pass http://origin:8080; } -``` +---- Corresponding minimal gateway policy: -```yaml +[source,yaml] +---- dsl_version: "1" governance: @@ -113,51 +127,53 @@ governance: stealth: enabled: true status_code: 404 -``` +---- The gateway covers three routes; everything else is served by the origin directly and is unaffected by any gateway bug, regex ReDoS, or policy reload error. -## Rollback Plan +=== Rollback Plan -Because only selected routes are governed, rolling back is a single config -change in your edge proxy: +Because only selected routes are governed, rolling back is a single +config change in your edge proxy: -```nginx +[source,nginx] +---- location /api/admin/ { # Skip gateway — route directly to origin. proxy_pass http://origin:8080; } -``` +---- -No gateway code is in the request path, and no other routes are affected. -This is the key property that scoped deployment preserves. +No gateway code is in the request path, and no other routes are +affected. This is the key property that scoped deployment preserves. -## Migration Path to Broader Deployment +=== Migration Path to Broader Deployment Once v0.1.0 has been running in production on a scoped set of routes for -long enough to gain confidence (think weeks, not hours), consider widening -the scope: +long enough to gain confidence (think weeks, not hours), consider +widening the scope: -1. Add more routes to the policy, one group at a time. -2. Monitor `/metrics` for rate-limit hits, circuit breaker trips, and - `access_decision` telemetry. -3. Only expand to protocols or trust sources that have moved out of the - "stub only" / "caveats" rows of `docs/SUPPORTED-FEATURES.md`. -4. Revisit this document once the gateway is verified for broader scope; - its recommendations WILL loosen over time as features graduate. +[arabic] +. Add more routes to the policy, one group at a time. +. Monitor `+/metrics+` for rate-limit hits, circuit breaker trips, and +`+access_decision+` telemetry. +. Only expand to protocols or trust sources that have moved out of the +"`stub only`" / "`caveats`" rows of `+docs/SUPPORTED-FEATURES.md+`. +. Revisit this document once the gateway is verified for broader scope; +its recommendations WILL loosen over time as features graduate. -## When to NOT Use This Gateway at All +=== When to NOT Use This Gateway at All If your traffic is primarily: -- WebSocket or long-lived streaming -- gRPC or GraphQL (and you need real governance, not stubs) -- TLS-terminated by the gateway itself -- Multi-backend load balancing +* WebSocket or long-lived streaming +* gRPC or GraphQL (and you need real governance, not stubs) +* TLS-terminated by the gateway itself +* Multi-backend load balancing -...then this gateway is not the right tool for v0.1.0. Consider Envoy, +…then this gateway is not the right tool for v0.1.0. Consider Envoy, Kong, Traefik, or AWS API Gateway. This project is intentionally scoped -narrower than those, and forcing it into their role would reintroduce the -"97% production-ready" overclaim that release criteria now forbid. +narrower than those, and forcing it into their role would reintroduce +the "`97% production-ready`" overclaim that release criteria now forbid. diff --git a/docs/SUPPORTED-FEATURES.adoc b/docs/SUPPORTED-FEATURES.adoc new file mode 100644 index 0000000..2a5c696 --- /dev/null +++ b/docs/SUPPORTED-FEATURES.adoc @@ -0,0 +1,193 @@ +== Supported Features — HTTP Capability Gateway + +*Version:* 0.1.0-dev *Last updated:* 2026-04-16 *Status:* MVP +verification phase (CRG grade C) + +This document is the single authoritative reference for what the gateway +actually supports today. If it is not in the *Supported* list below, it +is either a stub, planned work, or out of scope — do not rely on it in +production. + +See `+ROADMAP.adoc+` for the formal MVP scope definition with test +mappings, and `+MULTI-PROTOCOL.md+` for protocol-specific details (note +that document describes a broader vision; the table here reflects actual +runtime behaviour). + +''''' + +=== Protocols + +[width="100%",cols="40%,32%,28%",options="header",] +|=== +|Protocol |Status |Notes +|*HTTP/1.1* |Supported |Full verb governance, proxy forwarding, stealth +mode. This is the MVP scope. + +|*HTTP/2* |Supported (via Cowboy) |Inherited from Plug.Cowboy adapter. +Not exercised by gateway-specific tests. + +|*HTTPS / TLS* |Not supported |No TLS termination inside the gateway. +Run behind a TLS-terminating proxy (nginx, Caddy, Svalinn, etc.). + +|*mTLS (client certs)* |Partial |`+extract_trust_level_from_cert/1+` now +uses proper `+Record.extract+`-based OTPCertificate accessors +(OTP-version-robust) and extracts the subject’s CN/O/OU fields. Not yet +validated against real CA certificates in an integration test — exercise +in staging before relying on it for trust decisions. + +|*GraphQL* |Stub only |`+GraphQLHandler+` parses JSON bodies and does +naive prefix-based operation detection. `+check_operation_policy/2+` +always returns true. Operation-level governance is not implemented. + +|*gRPC* |Stub only |`+GRPCHandler+` extracts service/method from path +but `+forward_grpc_request/5+` returns a hardcoded response — no actual +gRPC forwarding. + +|*WebSocket* |Not supported |No implementation. +|=== + +=== Trust Sources + +[width="100%",cols="36%,34%,30%",options="header",] +|=== +|Source |Status |Notes +|*`+X-Trust-Level+` header* |Supported |The canonical trust source. +Values: `+untrusted+`, `+authenticated+`, `+internal+`. Anything else is +parsed as `+:untrusted+` (see `+SafeTrust.parse_trust/1+`). + +|*Header stripping for untrusted sources* |Supported +|`+strip_untrusted_headers/2+` removes `+X-Trust-Level+` from any +request whose `+remote_ip+` is not in `+:trusted_proxies+`. Default +trusted proxies: `+["127.0.0.1", "::1"]+`. *Operator action required*: +configure `+:trusted_proxies+` for your deployment before exposing the +gateway. + +|*mTLS certificate OU* |Supported with caveats |Subject extraction uses +stable `+Record.extract+` accessors. An OU of "`Internal Services`" maps +to `+:internal+`; any other verified cert maps to `+:authenticated+`. +Test against your CA’s cert format before production use — OU field name +conventions vary. + +|*Authorization header (JWT/Bearer)* |Not parsed |The gateway does not +parse or validate tokens. The upstream that sets `+X-Trust-Level+` is +expected to do this (e.g., indieweb2-bastion in the hyperpolymath +stack). + +|*IP allowlist* |Not supported |The gateway does not block by IP. Rely +on upstream L4/L7 filtering. +|=== + +=== Policy Enforcement + +[width="100%",cols="38%,33%,29%",options="header",] +|=== +|Feature |Status |Notes +|*YAML policy loading* |Supported |`+PolicyLoader.load_from_file/1+` +reads DSL v1 files. + +|*Policy validation* |Supported |`+PolicyValidator.validate/1+` rejects +malformed policies before compilation. + +|*Policy compilation to ETS* |Supported |Dual-table layout: main (exact ++ global) and regex. + +|*Exact path matching* |Supported |O(1) via ETS `+{:exact, path, verb}+` +keys. + +|*Regex path matching* |Supported |O(r) scan of dedicated regex table. + +|*Global verb rules* |Supported |Fallback when no route matches. + +|*Atomic policy hot-reload* |Supported |Recompiling creates new tables +and swaps app env references atomically. See `+test/e2e_test.exs+` +hot-reload tests. + +|*Per-route exposure overrides* |Supported |Route `+exposure+` field +overrides global behaviour. + +|*Stealth mode (configurable status codes)* |Supported |Default 404 +hides denied endpoints. + +|*Default-deny on no match* |Supported |Returns 403 (or stealth code) +when no rule matches. +|=== + +=== Runtime Features + +[width="100%",cols="38%,33%,29%",options="header",] +|=== +|Feature |Status |Notes +|*Rate limiting (token bucket)* |Supported |Per-(IP, trust) buckets. +Defaults: `+untrusted+` 10 req/s, `+authenticated+` 100 req/s, +`+internal+` unlimited. `+X-Forwarded-For+` is trusted for client IP — +require a trusted reverse proxy to prevent spoofing. + +|*Circuit breaker* |Supported |Three-state FSM (closed/open/half_open) +per backend. 5 failures to trip, 30s before half-open probe +(configurable). + +|*K9 service contracts* |Partial |Trust threshold enforcement works. +`+rate_limit+` field on contracts is declared but NOT enforced. + +|*Structured JSON logs* |Supported |Every access decision logged with +request_id, path, verb, trust_level, decision. + +|*Telemetry events* |Supported |All major events emit +`+[:http_capability_gateway, ...]+` telemetry. + +|*Health probe* |Supported |`+GET /health+` returns 200 with uptime. + +|*Readiness probe* |Supported |`+GET /ready+` returns 200 iff policy is +loaded. + +|*Prometheus metrics* |Supported |`+GET /metrics+` via +`+TelemetryMetricsPrometheus.Core.scrape/0+`. + +|*Anomaly detection (Minikaran)* |Supported |`+GET /api/v1/minikaran+` +returns current anomalies. + +|*Audit log (VeriSimDB)* |Supported |Allow/deny decisions persisted +asynchronously. +|=== + +=== Out of MVP Scope + +The following are explicitly *not* in the v0.1.0 MVP: + +* Multi-backend load balancing +* TLS termination +* Response caching +* Request/response body transformation +* Dynamic trust scoring / ML-based trust +* Web UI / admin dashboard +* Plugin system (auth, filters, custom loaders) +* Distributed cluster coordination +* Kubernetes operator +* Helm chart + +See `+ROADMAP-v2.md+` for these items; they are *aspirational* and not +on the release path for v0.1.0. + +''''' + +=== Operator Quick Checklist + +Before exposing the gateway to the public internet: + +[arabic] +. *Configure `+:trusted_proxies+`* — add the IP addresses of your +upstream TLS-terminating proxies. Without this, direct clients can forge +`+X-Trust-Level: internal+` and bypass all governance. +. *Test mTLS trust extraction against your CA’s cert format* — the code +uses `+Record.extract+`-based OTPCertificate accessors that are stable +across OTP versions, but OU field conventions vary between CAs and there +is no integration test yet against real client certificates. +. *Do NOT route GraphQL or gRPC traffic through this gateway* — handlers +are stubs. HTTP/REST only. +. *Set realistic `+:rate_limits+`* for your traffic. The test defaults +are set very high for test predictability. +. *Provide a valid policy file at startup* — the application refuses to +start without one (fail-closed). +. *Put a TLS-terminating proxy in front* — the gateway does not do TLS. +. *Monitor the `+/metrics+` endpoint* — especially +`+http_capability_gateway_circuit_breaker_*+` and rate-limit counters. diff --git a/docs/SUPPORTED-FEATURES.md b/docs/SUPPORTED-FEATURES.md deleted file mode 100644 index 750ec06..0000000 --- a/docs/SUPPORTED-FEATURES.md +++ /dev/null @@ -1,113 +0,0 @@ - -# Supported Features — HTTP Capability Gateway - -**Version:** 0.1.0-dev -**Last updated:** 2026-04-16 -**Status:** MVP verification phase (CRG grade C) - -This document is the single authoritative reference for what the gateway -actually supports today. If it is not in the **Supported** list below, it -is either a stub, planned work, or out of scope — do not rely on it in -production. - -See `ROADMAP.adoc` for the formal MVP scope definition with test mappings, -and `MULTI-PROTOCOL.md` for protocol-specific details (note that document -describes a broader vision; the table here reflects actual runtime behaviour). - ---- - -## Protocols - -| Protocol | Status | Notes | -|----------|--------|-------| -| **HTTP/1.1** | Supported | Full verb governance, proxy forwarding, stealth mode. This is the MVP scope. | -| **HTTP/2** | Supported (via Cowboy) | Inherited from Plug.Cowboy adapter. Not exercised by gateway-specific tests. | -| **HTTPS / TLS** | Not supported | No TLS termination inside the gateway. Run behind a TLS-terminating proxy (nginx, Caddy, Svalinn, etc.). | -| **mTLS (client certs)** | Partial | `extract_trust_level_from_cert/1` now uses proper `Record.extract`-based OTPCertificate accessors (OTP-version-robust) and extracts the subject's CN/O/OU fields. Not yet validated against real CA certificates in an integration test — exercise in staging before relying on it for trust decisions. | -| **GraphQL** | Stub only | `GraphQLHandler` parses JSON bodies and does naive prefix-based operation detection. `check_operation_policy/2` always returns true. Operation-level governance is not implemented. | -| **gRPC** | Stub only | `GRPCHandler` extracts service/method from path but `forward_grpc_request/5` returns a hardcoded response — no actual gRPC forwarding. | -| **WebSocket** | Not supported | No implementation. | - -## Trust Sources - -| Source | Status | Notes | -|--------|--------|-------| -| **`X-Trust-Level` header** | Supported | The canonical trust source. Values: `untrusted`, `authenticated`, `internal`. Anything else is parsed as `:untrusted` (see `SafeTrust.parse_trust/1`). | -| **Header stripping for untrusted sources** | Supported | `strip_untrusted_headers/2` removes `X-Trust-Level` from any request whose `remote_ip` is not in `:trusted_proxies`. Default trusted proxies: `["127.0.0.1", "::1"]`. **Operator action required**: configure `:trusted_proxies` for your deployment before exposing the gateway. | -| **mTLS certificate OU** | Supported with caveats | Subject extraction uses stable `Record.extract` accessors. An OU of "Internal Services" maps to `:internal`; any other verified cert maps to `:authenticated`. Test against your CA's cert format before production use — OU field name conventions vary. | -| **Authorization header (JWT/Bearer)** | Not parsed | The gateway does not parse or validate tokens. The upstream that sets `X-Trust-Level` is expected to do this (e.g., indieweb2-bastion in the hyperpolymath stack). | -| **IP allowlist** | Not supported | The gateway does not block by IP. Rely on upstream L4/L7 filtering. | - -## Policy Enforcement - -| Feature | Status | Notes | -|---------|--------|-------| -| **YAML policy loading** | Supported | `PolicyLoader.load_from_file/1` reads DSL v1 files. | -| **Policy validation** | Supported | `PolicyValidator.validate/1` rejects malformed policies before compilation. | -| **Policy compilation to ETS** | Supported | Dual-table layout: main (exact + global) and regex. | -| **Exact path matching** | Supported | O(1) via ETS `{:exact, path, verb}` keys. | -| **Regex path matching** | Supported | O(r) scan of dedicated regex table. | -| **Global verb rules** | Supported | Fallback when no route matches. | -| **Atomic policy hot-reload** | Supported | Recompiling creates new tables and swaps app env references atomically. See `test/e2e_test.exs` hot-reload tests. | -| **Per-route exposure overrides** | Supported | Route `exposure` field overrides global behaviour. | -| **Stealth mode (configurable status codes)** | Supported | Default 404 hides denied endpoints. | -| **Default-deny on no match** | Supported | Returns 403 (or stealth code) when no rule matches. | - -## Runtime Features - -| Feature | Status | Notes | -|---------|--------|-------| -| **Rate limiting (token bucket)** | Supported | Per-(IP, trust) buckets. Defaults: `untrusted` 10 req/s, `authenticated` 100 req/s, `internal` unlimited. `X-Forwarded-For` is trusted for client IP — require a trusted reverse proxy to prevent spoofing. | -| **Circuit breaker** | Supported | Three-state FSM (closed/open/half_open) per backend. 5 failures to trip, 30s before half-open probe (configurable). | -| **K9 service contracts** | Partial | Trust threshold enforcement works. `rate_limit` field on contracts is declared but NOT enforced. | -| **Structured JSON logs** | Supported | Every access decision logged with request_id, path, verb, trust_level, decision. | -| **Telemetry events** | Supported | All major events emit `[:http_capability_gateway, ...]` telemetry. | -| **Health probe** | Supported | `GET /health` returns 200 with uptime. | -| **Readiness probe** | Supported | `GET /ready` returns 200 iff policy is loaded. | -| **Prometheus metrics** | Supported | `GET /metrics` via `TelemetryMetricsPrometheus.Core.scrape/0`. | -| **Anomaly detection (Minikaran)** | Supported | `GET /api/v1/minikaran` returns current anomalies. | -| **Audit log (VeriSimDB)** | Supported | Allow/deny decisions persisted asynchronously. | - -## Out of MVP Scope - -The following are explicitly **not** in the v0.1.0 MVP: - -- Multi-backend load balancing -- TLS termination -- Response caching -- Request/response body transformation -- Dynamic trust scoring / ML-based trust -- Web UI / admin dashboard -- Plugin system (auth, filters, custom loaders) -- Distributed cluster coordination -- Kubernetes operator -- Helm chart - -See `ROADMAP-v2.md` for these items; they are **aspirational** and not -on the release path for v0.1.0. - ---- - -## Operator Quick Checklist - -Before exposing the gateway to the public internet: - -1. **Configure `:trusted_proxies`** — add the IP addresses of your upstream - TLS-terminating proxies. Without this, direct clients can forge - `X-Trust-Level: internal` and bypass all governance. -2. **Test mTLS trust extraction against your CA's cert format** — the code - uses `Record.extract`-based OTPCertificate accessors that are stable - across OTP versions, but OU field conventions vary between CAs and - there is no integration test yet against real client certificates. -3. **Do NOT route GraphQL or gRPC traffic through this gateway** — - handlers are stubs. HTTP/REST only. -4. **Set realistic `:rate_limits`** for your traffic. The test defaults - are set very high for test predictability. -5. **Provide a valid policy file at startup** — the application refuses - to start without one (fail-closed). -6. **Put a TLS-terminating proxy in front** — the gateway does not do TLS. -7. **Monitor the `/metrics` endpoint** — especially - `http_capability_gateway_circuit_breaker_*` and rate-limit counters. diff --git a/docs/mtls-rotation-runbook.adoc b/docs/mtls-rotation-runbook.adoc new file mode 100644 index 0000000..1552e94 --- /dev/null +++ b/docs/mtls-rotation-runbook.adoc @@ -0,0 +1,134 @@ +== mTLS CA Selection & Certificate Rotation Runbook + +*Phase:* B (`+hyperpolymath/standards#97+`) — mTLS as the primary +trust-level path *Applies to:* `+http-capability-gateway+` deployed as +BoJ tier-2 (ADR 0004) *Companion:* `+boj-server+` +`+docs/integration/http-capability-gateway-plan.md+` §Phase B + +''''' + +=== 1. CA selection decision + +Phase B deliverable B3 requires a committed decision on which CA roots +the mTLS trust chain. The options evaluated in the integration plan: + +[width="100%",cols="50%,50%",options="header",] +|=== +|Option |Decision +|1. BoJ-own CA (self-signed root, generated at deploy) |*Adopted for +initial wiring.* No external dependency; the gateway and the BoJ +gnosis-handler container are the only relying parties, so a dedicated +single-purpose root is the smallest trust base. + +|2. Estate SDP CA |Deferred. Adopt once an estate-wide SDP CA exists; +migration is a CA-file swap (§4) with no gateway code change. + +|3. Cloudflare Origin CA (AOP parity) |Complementary, not a replacement. +Cloudflare Authenticated Origin Pulls protect the edge→origin hop; the +gateway mTLS protects the gateway→BoJ hop. Both may run; they trust +different roots. +|=== + +*Authenticated Origin Pulls parity.* The gateway’s HTTPS listener is +configured with `+verify: :verify_peer+` and +`+fail_if_no_peer_cert: true+` +(`+HttpCapabilityGateway.Application.tls_socket_opts/0+`). A client that +does not present a certificate chaining to `+MTLS_CA_CERT_PATH+` is +rejected at the TLS handshake — the gateway mirrors the Cloudflare AOP +"`reject unauthenticated origin clients`" model one tier inward. + +=== 2. Environment contract + +The gateway reads TLS material from three environment variables. When +`+TRUST_LEVEL_SOURCE=mtls+` all three are *mandatory*: if any is missing +or unreadable the application refuses to start (fail-closed — it never +silently downgrades to the forgeable header path). + +[width="100%",cols="50%,50%",options="header",] +|=== +|Variable |Meaning +|`+MTLS_CA_CERT_PATH+` |PEM CA bundle the client cert chain is verified +against + +|`+GATEWAY_CERT_PATH+` |The gateway’s own TLS server certificate + +|`+GATEWAY_KEY_PATH+` |Private key for `+GATEWAY_CERT_PATH+` + +|`+GATEWAY_TLS_PORT+` |HTTPS listener port (default `+4443+`) + +|`+TRUST_LEVEL_SOURCE+` |`+mtls+` to make the HTTPS listener mandatory +|=== + +Trust mapping (`+Gateway.determine_trust_level_from_cert/2+`): a +verified client cert with `+OU=Internal Services+` → `+internal+`; any +other verified client cert → `+authenticated+`; unverified / no TLS → +`+untrusted+`. + +=== 3. Certificate generation + +Production certificates are generated by estate tooling. The shape +required: + +* *CA*: a long-lived (e.g. 5y) RSA-2048+ or EC-P256 self-signed root. +* *Gateway server cert*: SAN must cover the gateway’s service name. +* *Client certs* (one per calling service): subject `+OU+` carries the +trust class. `+OU=Internal Services+` grants `+internal+`. Subjects MUST +be UTF8String-encoded — the gateway’s RDN matcher only reads +`+utf8String+` attribute values (see +`+test/fixtures/mtls/gen-test-ca.sh+` for the exact +`+string_mask = utf8only+` recipe used by the test CA). + +=== 4. Rotation without downtime + +The gateway re-reads TLS files only on listener (re)start. Rotate with a +rolling restart, not an in-process reload: + +[arabic] +. *Stage new material* alongside the old (new paths or a versioned dir). +. *Cross-sign / dual-trust window.* If rotating the _CA_, publish a +`+MTLS_CA_CERT_PATH+` bundle containing *both* the old and new roots. +Both old and new client certs validate during the overlap. +. *Roll the gateway.* Update the gateway deployment env to the new +`+GATEWAY_CERT_PATH+`/`+GATEWAY_KEY_PATH+`; k9-svc performs a rolling +replace. The old replica drains in-flight requests before exit (circuit +breaker trips closed on the old replica, not the seam). +. *Roll the BoJ-side client certs* the same way against the dual-trust +bundle. +. *Retire the old root.* Once every client presents a new-root cert, +publish a `+MTLS_CA_CERT_PATH+` bundle containing only the new root and +roll the gateway once more. + +At no point is there a window where the gateway accepts an unverified +client: each step keeps `+verify: :verify_peer+` + +`+fail_if_no_peer_cert: true+` in force. + +=== 5. Failure & rollback + +[width="100%",cols="34%,33%,33%",options="header",] +|=== +|Symptom |Likely cause |Action +|Gateway refuses to start, log `+mTLS listener configuration invalid+` +|A TLS path is unset/unreadable under `+TRUST_LEVEL_SOURCE=mtls+` +|Restore the path or roll back the deployment env to the previous +material + +|All clients get TLS handshake failure after a rotation |CA bundle +replaced before clients rotated |Re-publish the dual-trust bundle (step +2) and re-roll + +|A specific service gets `+untrusted+` unexpectedly |Client cert subject +not UTF8String, or wrong `+OU+` |Re-issue that client cert per §3 +|=== + +Full traffic-bypass rollback (re-route around the gateway entirely) is +the `+docs/integration/gateway-rollback-runbook.md+` Phase E +deliverable; this runbook covers only the certificate/CA layer. + +=== 6. Test fixtures + +`+test/fixtures/mtls/+` contains a self-contained *test* CA (root, +gateway server cert, an internal-OU client, an ordinary client, and a +rogue-CA client that must fail verification). Regenerate with +`+test/fixtures/mtls/gen-test-ca.sh+`. These keys are committed +deliberately for the test suite and chain to nothing trusted in +production. diff --git a/docs/mtls-rotation-runbook.md b/docs/mtls-rotation-runbook.md deleted file mode 100644 index 779e927..0000000 --- a/docs/mtls-rotation-runbook.md +++ /dev/null @@ -1,102 +0,0 @@ - -# mTLS CA Selection & Certificate Rotation Runbook - -**Phase:** B (`hyperpolymath/standards#97`) — mTLS as the primary trust-level path -**Applies to:** `http-capability-gateway` deployed as BoJ tier-2 (ADR 0004) -**Companion:** `boj-server` `docs/integration/http-capability-gateway-plan.md` §Phase B - ---- - -## 1. CA selection decision - -Phase B deliverable B3 requires a committed decision on which CA roots the -mTLS trust chain. The options evaluated in the integration plan: - -| Option | Decision | -|---|---| -| 1. BoJ-own CA (self-signed root, generated at deploy) | **Adopted for initial wiring.** No external dependency; the gateway and the BoJ gnosis-handler container are the only relying parties, so a dedicated single-purpose root is the smallest trust base. | -| 2. Estate SDP CA | Deferred. Adopt once an estate-wide SDP CA exists; migration is a CA-file swap (§4) with no gateway code change. | -| 3. Cloudflare Origin CA (AOP parity) | Complementary, not a replacement. Cloudflare Authenticated Origin Pulls protect the edge→origin hop; the gateway mTLS protects the gateway→BoJ hop. Both may run; they trust different roots. | - -**Authenticated Origin Pulls parity.** The gateway's HTTPS listener is -configured with `verify: :verify_peer` and `fail_if_no_peer_cert: true` -(`HttpCapabilityGateway.Application.tls_socket_opts/0`). A client that does -not present a certificate chaining to `MTLS_CA_CERT_PATH` is rejected at the -TLS handshake — the gateway mirrors the Cloudflare AOP "reject unauthenticated -origin clients" model one tier inward. - -## 2. Environment contract - -The gateway reads TLS material from three environment variables. When -`TRUST_LEVEL_SOURCE=mtls` all three are **mandatory**: if any is missing or -unreadable the application refuses to start (fail-closed — it never silently -downgrades to the forgeable header path). - -| Variable | Meaning | -|---|---| -| `MTLS_CA_CERT_PATH` | PEM CA bundle the client cert chain is verified against | -| `GATEWAY_CERT_PATH` | The gateway's own TLS server certificate | -| `GATEWAY_KEY_PATH` | Private key for `GATEWAY_CERT_PATH` | -| `GATEWAY_TLS_PORT` | HTTPS listener port (default `4443`) | -| `TRUST_LEVEL_SOURCE` | `mtls` to make the HTTPS listener mandatory | - -Trust mapping (`Gateway.determine_trust_level_from_cert/2`): a verified client -cert with `OU=Internal Services` → `internal`; any other verified client cert -→ `authenticated`; unverified / no TLS → `untrusted`. - -## 3. Certificate generation - -Production certificates are generated by estate tooling. The shape required: - -- **CA**: a long-lived (e.g. 5y) RSA-2048+ or EC-P256 self-signed root. -- **Gateway server cert**: SAN must cover the gateway's service name. -- **Client certs** (one per calling service): subject `OU` carries the - trust class. `OU=Internal Services` grants `internal`. Subjects MUST be - UTF8String-encoded — the gateway's RDN matcher only reads `utf8String` - attribute values (see `test/fixtures/mtls/gen-test-ca.sh` for the exact - `string_mask = utf8only` recipe used by the test CA). - -## 4. Rotation without downtime - -The gateway re-reads TLS files only on listener (re)start. Rotate with a -rolling restart, not an in-process reload: - -1. **Stage new material** alongside the old (new paths or a versioned dir). -2. **Cross-sign / dual-trust window.** If rotating the *CA*, publish a - `MTLS_CA_CERT_PATH` bundle containing **both** the old and new roots. - Both old and new client certs validate during the overlap. -3. **Roll the gateway.** Update the gateway deployment env to the new - `GATEWAY_CERT_PATH`/`GATEWAY_KEY_PATH`; k9-svc performs a rolling replace. - The old replica drains in-flight requests before exit (circuit breaker - trips closed on the old replica, not the seam). -4. **Roll the BoJ-side client certs** the same way against the dual-trust - bundle. -5. **Retire the old root.** Once every client presents a new-root cert, - publish a `MTLS_CA_CERT_PATH` bundle containing only the new root and - roll the gateway once more. - -At no point is there a window where the gateway accepts an unverified client: -each step keeps `verify: :verify_peer` + `fail_if_no_peer_cert: true` in force. - -## 5. Failure & rollback - -| Symptom | Likely cause | Action | -|---|---|---| -| Gateway refuses to start, log `mTLS listener configuration invalid` | A TLS path is unset/unreadable under `TRUST_LEVEL_SOURCE=mtls` | Restore the path or roll back the deployment env to the previous material | -| All clients get TLS handshake failure after a rotation | CA bundle replaced before clients rotated | Re-publish the dual-trust bundle (step 2) and re-roll | -| A specific service gets `untrusted` unexpectedly | Client cert subject not UTF8String, or wrong `OU` | Re-issue that client cert per §3 | - -Full traffic-bypass rollback (re-route around the gateway entirely) is the -`docs/integration/gateway-rollback-runbook.md` Phase E deliverable; this -runbook covers only the certificate/CA layer. - -## 6. Test fixtures - -`test/fixtures/mtls/` contains a self-contained **test** CA (root, gateway -server cert, an internal-OU client, an ordinary client, and a rogue-CA client -that must fail verification). Regenerate with -`test/fixtures/mtls/gen-test-ca.sh`. These keys are committed deliberately for -the test suite and chain to nothing trusted in production. diff --git a/docs/perf-contract.adoc b/docs/perf-contract.adoc new file mode 100644 index 0000000..0b20ccf --- /dev/null +++ b/docs/perf-contract.adoc @@ -0,0 +1,245 @@ +== Performance Contract + +____ +*Status: scaffold (Phase D-3).* This document names the SLOs the gateway +will publish and the regression-alert gate will enforce. The concrete +numbers are deliberately left as `+TODO+` — Phase D-4 of the single-lane +HCG channel (standards#91) collects the baseline and fills them in. The +CI gate is non-blocking until that point. Phase D-3 (this revision) is +the last harness-shape change before D-4 collection: it adds the +`+trust-header rewrite+`, `+mTLS handshake+`, and `+mTLS amortised+` +scenarios so the baseline attributes header-rewrite cost, per-handshake +cost, and per-request kept-alive cost independently of the proxy-200 +path. +____ + +Tracks: `+Refs hyperpolymath/standards#99+` (Phase D of +`+standards#91+`). + +=== Scope of this contract + +This contract covers in-gateway latency — the cost of the gateway’s own +pipeline (trust resolution, policy lookup, rate limiter, circuit +breaker, header rewrite). The `+exact route allow (proxy 200)+` scenario +also includes the dial-and-read cost against an in-process loopback +backend (added in Phase D-2), since the production gateway always pays +that cost on an allow path; the loopback strips out real-network RTT and +real-backend processing time so what remains is the gateway-attributable +proxy overhead. The `+mTLS handshake+` scenario (added in Phase D-3) +covers per-handshake transport cost in isolation, and the +`+mTLS amortised+` scenario (added in the Phase D-3 follow-up) covers +the per-request cost when N=16 requests share one kept-alive TLS +connection, closing the bracket that `+proxy 200+` + `+mTLS handshake+` +left loose. Real production backend latency and real-network RTT remain +out of scope. + +The six named scenarios match the six Benchee scenarios in +`+bench/gateway_latency.exs+`: + +[width="100%",cols="42%,58%",options="header",] +|=== +|Scenario |What it measures +|`+health endpoint+` |Cheapest path. No policy lookup, no proxy. Floor +of any plug-pipeline overhead. + +|`+policy deny (405 fast-path)+` |Verb-rejection path. Policy table hit, +no proxy. Cost of the deny short-circuit. + +|`+exact route allow (proxy 200)+` |Allow path that proxies to an +in-process loopback backend returning 200. Captures the real +dial-and-read cost the gateway pays in production. + +|`+trust-header rewrite (Proxy.build_backend_headers)+` |Direct call to +the `+Proxy.__benchmark_build_backend_headers__/1+` benchmark seam on a +pre-built conn (`+assigns[:trust_level] = :authenticated+`, +`+assigns[:request_id]+` set). Isolates the Phase A contract-header +construction (X-Trust-Level / X-Request-ID / X-Forwarded-*) from policy +lookup and network I/O. + +|`+mTLS handshake (test CA)+` |`+:ssl.connect/4+` against a raw `+:ssl+` +acceptor + immediate close. Test CA chain is generated in-memory at +bench startup via `+:public_key.pkix_test_data/1+` — the committed Phase +B fixture ships only `+*.crt+` files (matching `+*.key+` is gitignored), +so the chain is rebuilt with the same `+verify_peer+` + test-CA shape +rather than reusing the on-disk material. Per-handshake cost (cert chain +validation + key exchange) in isolation; no HTTP pipeline. Bounds the +connection-spike SLO for Phase E rollout. + +|`+mTLS amortised (test CA, N requests over kept-alive)+` |One +`+:ssl.connect/4+` followed by N=16 tiny send/recv round-trips on the +open connection, then `+:ssl.close/1+`. Uses a dedicated kept-alive +acceptor (separate listener from the handshake-only scenario) with an +echo loop, so the connection stays open across the N round-trips. +Per-iteration cost is `+(handshake + N * request) / N+`, which +approximates the per-request cost the Phase E rollout sees once the +handshake is amortised across a kept-alive pool. Closes the bracket +scenarios 3 + 5 left loose. +|=== + +=== Published latency SLOs + +Reported per scenario, per CI run: + +* *p50* — median request latency +* *p95* — 95th-percentile latency (the SLO the regression gate watches) +* *p99* — tail latency (regression gate watches with a looser tolerance) + +==== Targets (placeholder — Phase D-4 will replace) + +[width="100%",cols="64%,12%,12%,12%",options="header",] +|=== +|Scenario |p50 target |p95 target |p99 target +|`+health endpoint+` |TODO |TODO |TODO + +|`+policy deny (405 fast-path)+` |TODO |TODO |TODO + +|`+exact route allow (proxy 200)+` |TODO |TODO |TODO + +|`+trust-header rewrite (Proxy.build_backend_headers)+` |TODO |TODO +|TODO + +|`+mTLS handshake (test CA)+` |TODO |TODO |TODO + +|`+mTLS amortised (test CA, N requests over kept-alive)+` |TODO |TODO +|TODO +|=== + +Units: microseconds (µs). Hardware reference: `+ubuntu-latest+` GitHub +runners — the CI environment IS the published reference, deliberately +chosen because it is the environment every reviewer can reproduce +without local hardware variance. Phase D-4 (baseline collection) will +revisit whether a dedicated runner is needed once we see the spread. + +=== Regression-alert tolerance + +The CI gate (`+.github/workflows/perf-regression.yml+`) fails a PR when +*any* of these conditions holds against the checked-in baseline: + +* live `+p50+` > baseline `+p50+` × `+tolerance.p50_max_ratio+` (default +1.20) +* live `+p95+` > baseline `+p95+` × `+tolerance.p95_max_ratio+` (default +1.30) +* live `+p99+` > baseline `+p99+` × `+tolerance.p99_max_ratio+` (default +1.50) + +Tolerances are looser as the percentile gets noisier — Phase D-4 will +tighten these once intra-run variance is characterised. + +=== Schema drift + +The comparator iterates the *union* of scenario names from +`+bench/results.json+` and `+bench/baseline.json+`. Either direction of +schema drift fails the build in `+active+` mode: + +* *`+MISSING IN BASELINE+`* — a scenario in `+results.json+` (the +harness just emitted it) has no entry in `+baseline.json+`. A new +scenario landed without a rebaseline; the gate has no anchor for it and +cannot meaningfully report regression. Rebaseline before merging. +* *`+MISSING IN RESULTS+`* — a scenario in `+baseline.json+` is absent +from `+results.json+` (the harness skipped or dropped it). Either the +harness regressed silently, or a scenario was removed without +rebaselining the file. The gate must not silently pass. + +Both directions are surfaced inline in the markdown table (in the +`+Status+` column). In `+scaffold-placeholder+` mode they appear as +informational `+scaffold (would fail: MISSING IN BASELINE)+` / +`+scaffold (would fail: MISSING IN RESULTS)+` rows so a rebaseline PR +previews the eventual active-mode verdict before the gate is armed; the +build still passes. In `+active+` mode they appear as bare +`+MISSING IN BASELINE+` / `+MISSING IN RESULTS+` and exit the comparator +with status 1. Behaviour pivots on the single `+_status+` flag in +`+bench/baseline.json+` — no code change is needed to arm the schema +checks once the gate goes live. + +=== Baseline lifecycle + +The baseline lives in `+bench/baseline.json+`. Its `+_status+` field +gates behaviour: + +* `+"scaffold-placeholder"+` — gate reports numbers but never fails the +build. This is the current state. +* `+"active"+` — gate fails the build on regression. Switching to this +requires landing real numbers via a dedicated baseline-collection PR +(Phase D-4), reviewed for noise/spread by the maintainer. + +Updating the baseline is a deliberate act. Two paths: + +==== Automated (preferred — D-4 bootstrap) + +Dispatch the *Perf Rebaseline* workflow +(`+.github/workflows/perf-rebaseline.yml+`, `+workflow_dispatch+` only). +It runs `+bench/gateway_latency.exs+` on the published reference target +(`+ubuntu-latest+`), pipes the result through `+bench/rebaseline.exs+` +to regenerate `+bench/baseline.json+` with real percentiles, and opens a +`+perf: rebaseline (standards#99)+` PR for review. `+_status+` stays +`+scaffold-placeholder+` in the generated PR — the maintainer flips it +to `+active+` (arming the gate) in the same PR or in a follow-up after a +confidence-building window. + +==== Manual (for local previews or operators without GHA access) + +[arabic] +. Open a PR titled `+perf: rebaseline (standards#99)+`. +. Run `+just rebaseline+` locally on a CI-equivalent target (runs +`+bench/gateway_latency.exs+` then `+bench/rebaseline.exs+`, which reads +`+bench/results.json+` and writes a regenerated +`+bench/baseline.json+`). Or run `+just bench-collect+` for the harness +only and edit `+bench/baseline.json+` by hand. +. Commit the regenerated `+bench/baseline.json+`. +. Reviewer approves the new numbers (or rejects if the regression is +real and unjustified). Never silently rebaseline in an unrelated PR. + +Either path leaves `+_status+` as `+scaffold-placeholder+`. Flipping it +to `+active+` is a separate, deliberate decision (it may land in the +same PR or as a follow-up). + +=== Out of scope for Phase D-3 + +The scaffold deliberately does NOT yet include: + +* Real baseline numbers and rebaselined tolerances (Phase D-4 — the +`+perf: rebaseline+` ritual described above produces these on a +CI-equivalent target and flips `+bench/baseline.json+` `+_status+` to +`+active+`). +* Dashboard publication of historical numbers (Phase E — +`+standards#100+`). + +D-3 (this iteration) added three new scenarios to +`+bench/gateway_latency.exs+`: + +* `+trust-header rewrite (Proxy.build_backend_headers)+` — direct call +via the `+Proxy.__benchmark_build_backend_headers__/1+` benchmark seam +(a `+@doc false+` hook over the private `+build_backend_headers/1+`). +Isolates the Phase A contract-header construction cost. +* `+mTLS handshake (test CA)+` — raw `+:ssl+` acceptor + client connect +using a test CA chain generated in-memory at bench startup via +`+:public_key.pkix_test_data/1+`. The committed Phase B fixture in +`+test/fixtures/mtls+` ships only the `+*.crt+` files (matching +`+*.key+` is gitignored at the repo root), so the chain is rebuilt with +the same `+verify_peer+` + test-CA shape rather than reusing the on-disk +material. Each iteration is one fresh handshake closed immediately; +isolates per-handshake cost from the proxy hot-path. +* `+mTLS amortised (test CA, N requests over kept-alive)+` — dedicated +kept-alive acceptor (separate listener) running an echo loop. Each +Benchee iteration opens one connection, does N=16 tiny send/recv +round-trips on the held-open connection, and then closes. Per-iteration +cost is `+(handshake + N * request) / N+`, which approximates the +per-request cost in a kept-alive setting. This was previously deferred +as out-of-scope and bracketed by `+mTLS handshake+` + `+proxy 200+`; the +follow-up adds it as an explicit scenario so D-4 baseline collection +captures it directly rather than via the bracket. + +`+bench/baseline.json+` gains the three new scenario keys (all `+TODO+`, +`+_status+` remains `+scaffold-placeholder+`) and bumps +`+_schema_version+` to `+0.3.0-scaffold+` to signal the shape change. +D-4 will land real numbers across all six scenarios in a single +dedicated `+perf: rebaseline+` PR. + +D-2 (the previous iteration) landed the in-process `+Plug.Cowboy+` +loopback backend in `+bench/gateway_latency.exs+` and renamed the +`+exact route allow+` scenario from `+(proxy short-circuits)+` (D-1: +dialled `+:1+`, refused) to `+(proxy 200)+`. + +These are sequenced post-merge tasks under the single-lane HCG channel; +see `+standards#91+` for the order-of-operations and `+standards#100+` +for Phase E. diff --git a/docs/perf-contract.md b/docs/perf-contract.md deleted file mode 100644 index 6d9bac2..0000000 --- a/docs/perf-contract.md +++ /dev/null @@ -1,202 +0,0 @@ - - - -# Performance Contract - -> **Status: scaffold (Phase D-3).** This document names the SLOs the -> gateway will publish and the regression-alert gate will enforce. The -> concrete numbers are deliberately left as `TODO` — Phase D-4 of the -> single-lane HCG channel (standards#91) collects the baseline and fills -> them in. The CI gate is non-blocking until that point. Phase D-3 (this -> revision) is the last harness-shape change before D-4 collection: it -> adds the `trust-header rewrite`, `mTLS handshake`, and `mTLS amortised` -> scenarios so the baseline attributes header-rewrite cost, per-handshake -> cost, and per-request kept-alive cost independently of the proxy-200 -> path. - -Tracks: `Refs hyperpolymath/standards#99` (Phase D of `standards#91`). - -## Scope of this contract - -This contract covers in-gateway latency — the cost of the gateway's -own pipeline (trust resolution, policy lookup, rate limiter, circuit -breaker, header rewrite). The `exact route allow (proxy 200)` scenario -also includes the dial-and-read cost against an in-process loopback -backend (added in Phase D-2), since the production gateway always pays -that cost on an allow path; the loopback strips out real-network RTT and -real-backend processing time so what remains is the gateway-attributable -proxy overhead. The `mTLS handshake` scenario (added in Phase D-3) -covers per-handshake transport cost in isolation, and the -`mTLS amortised` scenario (added in the Phase D-3 follow-up) covers the -per-request cost when N=16 requests share one kept-alive TLS connection, -closing the bracket that `proxy 200` + `mTLS handshake` left loose. Real -production backend latency and real-network RTT remain out of scope. - -The six named scenarios match the six Benchee scenarios in -`bench/gateway_latency.exs`: - -| Scenario | What it measures | -|-----------------------------------------------------------|-----------------------------------------------------------------------------------| -| `health endpoint` | Cheapest path. No policy lookup, no proxy. Floor of any plug-pipeline overhead. | -| `policy deny (405 fast-path)` | Verb-rejection path. Policy table hit, no proxy. Cost of the deny short-circuit. | -| `exact route allow (proxy 200)` | Allow path that proxies to an in-process loopback backend returning 200. Captures the real dial-and-read cost the gateway pays in production. | -| `trust-header rewrite (Proxy.build_backend_headers)` | Direct call to the `Proxy.__benchmark_build_backend_headers__/1` benchmark seam on a pre-built conn (`assigns[:trust_level] = :authenticated`, `assigns[:request_id]` set). Isolates the Phase A contract-header construction (X-Trust-Level / X-Request-ID / X-Forwarded-*) from policy lookup and network I/O. | -| `mTLS handshake (test CA)` | `:ssl.connect/4` against a raw `:ssl` acceptor + immediate close. Test CA chain is generated in-memory at bench startup via `:public_key.pkix_test_data/1` — the committed Phase B fixture ships only `*.crt` files (matching `*.key` is gitignored), so the chain is rebuilt with the same `verify_peer` + test-CA shape rather than reusing the on-disk material. Per-handshake cost (cert chain validation + key exchange) in isolation; no HTTP pipeline. Bounds the connection-spike SLO for Phase E rollout. | -| `mTLS amortised (test CA, N requests over kept-alive)` | One `:ssl.connect/4` followed by N=16 tiny send/recv round-trips on the open connection, then `:ssl.close/1`. Uses a dedicated kept-alive acceptor (separate listener from the handshake-only scenario) with an echo loop, so the connection stays open across the N round-trips. Per-iteration cost is `(handshake + N * request) / N`, which approximates the per-request cost the Phase E rollout sees once the handshake is amortised across a kept-alive pool. Closes the bracket scenarios 3 + 5 left loose. | - -## Published latency SLOs - -Reported per scenario, per CI run: - -- **p50** — median request latency -- **p95** — 95th-percentile latency (the SLO the regression gate watches) -- **p99** — tail latency (regression gate watches with a looser tolerance) - -### Targets (placeholder — Phase D-4 will replace) - -| Scenario | p50 target | p95 target | p99 target | -|-----------------------------------------------------------|------------|------------|------------| -| `health endpoint` | TODO | TODO | TODO | -| `policy deny (405 fast-path)` | TODO | TODO | TODO | -| `exact route allow (proxy 200)` | TODO | TODO | TODO | -| `trust-header rewrite (Proxy.build_backend_headers)` | TODO | TODO | TODO | -| `mTLS handshake (test CA)` | TODO | TODO | TODO | -| `mTLS amortised (test CA, N requests over kept-alive)` | TODO | TODO | TODO | - -Units: microseconds (µs). Hardware reference: `ubuntu-latest` GitHub -runners — the CI environment IS the published reference, deliberately -chosen because it is the environment every reviewer can reproduce -without local hardware variance. Phase D-4 (baseline collection) will -revisit whether a dedicated runner is needed once we see the spread. - -## Regression-alert tolerance - -The CI gate (`.github/workflows/perf-regression.yml`) fails a PR when -**any** of these conditions holds against the checked-in baseline: - -- live `p50` > baseline `p50` × `tolerance.p50_max_ratio` (default 1.20) -- live `p95` > baseline `p95` × `tolerance.p95_max_ratio` (default 1.30) -- live `p99` > baseline `p99` × `tolerance.p99_max_ratio` (default 1.50) - -Tolerances are looser as the percentile gets noisier — Phase D-4 will -tighten these once intra-run variance is characterised. - -## Schema drift - -The comparator iterates the **union** of scenario names from -`bench/results.json` and `bench/baseline.json`. Either direction of -schema drift fails the build in `active` mode: - -- **`MISSING IN BASELINE`** — a scenario in `results.json` (the harness - just emitted it) has no entry in `baseline.json`. A new scenario - landed without a rebaseline; the gate has no anchor for it and - cannot meaningfully report regression. Rebaseline before merging. -- **`MISSING IN RESULTS`** — a scenario in `baseline.json` is absent - from `results.json` (the harness skipped or dropped it). Either the - harness regressed silently, or a scenario was removed without - rebaselining the file. The gate must not silently pass. - -Both directions are surfaced inline in the markdown table (in the -`Status` column). In `scaffold-placeholder` mode they appear as -informational `scaffold (would fail: MISSING IN BASELINE)` / -`scaffold (would fail: MISSING IN RESULTS)` rows so a rebaseline PR -previews the eventual active-mode verdict before the gate is armed; -the build still passes. In `active` mode they appear as bare -`MISSING IN BASELINE` / `MISSING IN RESULTS` and exit the comparator -with status 1. Behaviour pivots on the single `_status` flag in -`bench/baseline.json` — no code change is needed to arm the schema -checks once the gate goes live. - -## Baseline lifecycle - -The baseline lives in `bench/baseline.json`. Its `_status` field gates -behaviour: - -- `"scaffold-placeholder"` — gate reports numbers but never fails the - build. This is the current state. -- `"active"` — gate fails the build on regression. Switching to this - requires landing real numbers via a dedicated baseline-collection PR - (Phase D-4), reviewed for noise/spread by the maintainer. - -Updating the baseline is a deliberate act. Two paths: - -### Automated (preferred — D-4 bootstrap) - -Dispatch the **Perf Rebaseline** workflow -(`.github/workflows/perf-rebaseline.yml`, `workflow_dispatch` only). -It runs `bench/gateway_latency.exs` on the published reference target -(`ubuntu-latest`), pipes the result through `bench/rebaseline.exs` -to regenerate `bench/baseline.json` with real percentiles, and opens -a `perf: rebaseline (standards#99)` PR for review. `_status` stays -`scaffold-placeholder` in the generated PR — the maintainer flips it -to `active` (arming the gate) in the same PR or in a follow-up after -a confidence-building window. - -### Manual (for local previews or operators without GHA access) - -1. Open a PR titled `perf: rebaseline (standards#99)`. -2. Run `just rebaseline` locally on a CI-equivalent target (runs - `bench/gateway_latency.exs` then `bench/rebaseline.exs`, which - reads `bench/results.json` and writes a regenerated - `bench/baseline.json`). Or run `just bench-collect` for the - harness only and edit `bench/baseline.json` by hand. -3. Commit the regenerated `bench/baseline.json`. -4. Reviewer approves the new numbers (or rejects if the regression is - real and unjustified). Never silently rebaseline in an unrelated PR. - -Either path leaves `_status` as `scaffold-placeholder`. Flipping it to -`active` is a separate, deliberate decision (it may land in the same -PR or as a follow-up). - -## Out of scope for Phase D-3 - -The scaffold deliberately does NOT yet include: - -- Real baseline numbers and rebaselined tolerances (Phase D-4 — the - `perf: rebaseline` ritual described above produces these on a - CI-equivalent target and flips `bench/baseline.json` `_status` to - `active`). -- Dashboard publication of historical numbers (Phase E — `standards#100`). - -D-3 (this iteration) added three new scenarios to -`bench/gateway_latency.exs`: - -- `trust-header rewrite (Proxy.build_backend_headers)` — direct call - via the `Proxy.__benchmark_build_backend_headers__/1` benchmark seam - (a `@doc false` hook over the private `build_backend_headers/1`). - Isolates the Phase A contract-header construction cost. -- `mTLS handshake (test CA)` — raw `:ssl` acceptor + client connect - using a test CA chain generated in-memory at bench startup via - `:public_key.pkix_test_data/1`. The committed Phase B fixture in - `test/fixtures/mtls` ships only the `*.crt` files (matching `*.key` - is gitignored at the repo root), so the chain is rebuilt with the - same `verify_peer` + test-CA shape rather than reusing the on-disk - material. Each iteration is one fresh handshake closed immediately; - isolates per-handshake cost from the proxy hot-path. -- `mTLS amortised (test CA, N requests over kept-alive)` — dedicated - kept-alive acceptor (separate listener) running an echo loop. Each - Benchee iteration opens one connection, does N=16 tiny send/recv - round-trips on the held-open connection, and then closes. Per-iteration - cost is `(handshake + N * request) / N`, which approximates the - per-request cost in a kept-alive setting. This was previously deferred - as out-of-scope and bracketed by `mTLS handshake` + `proxy 200`; the - follow-up adds it as an explicit scenario so D-4 baseline collection - captures it directly rather than via the bracket. - -`bench/baseline.json` gains the three new scenario keys (all `TODO`, -`_status` remains `scaffold-placeholder`) and bumps `_schema_version` -to `0.3.0-scaffold` to signal the shape change. D-4 will land real -numbers across all six scenarios in a single dedicated -`perf: rebaseline` PR. - -D-2 (the previous iteration) landed the in-process `Plug.Cowboy` -loopback backend in `bench/gateway_latency.exs` and renamed the -`exact route allow` scenario from `(proxy short-circuits)` (D-1: -dialled `:1`, refused) to `(proxy 200)`. - -These are sequenced post-merge tasks under the single-lane HCG channel; -see `standards#91` for the order-of-operations and `standards#100` for -Phase E. diff --git a/docs/tech-debt-2026-05-26.adoc b/docs/tech-debt-2026-05-26.adoc new file mode 100644 index 0000000..77c1fd5 --- /dev/null +++ b/docs/tech-debt-2026-05-26.adoc @@ -0,0 +1,71 @@ +== Tech-Debt Audit — http-capability-gateway — 2026-05-26 + +*Source:* estate-wide automated scan 2026-05-26. *Companion:* +https://github.com/hyperpolymath/standards/tree/main/docs/audits[`+hyperpolymath/standards+` +2026-05-26-estate-*-debt audits]. *Combined severity:* `+LOW+`. + +This file records the _raw findings_ — it does not by itself fix the +debt. Each section ends with a '`Recommended next move`' line; closing +the debt is follow-up work. + +=== 1. Proof debt + +No proof-bearing files (`+*.v+`, `+*.lean+`, `+*.agda+`, `+*.idr+`, +`+*.idr2+`, `+*.fst+`, `+*.dfy+`, `+*.tla+`, `+*.ads+`, `+*.adb+`) found +in this repo. + +*Recommended next move:* none. + +=== 2. Licence debt + +[cols=",",options="header",] +|=== +|Field |Value +|LICENSE file |`+LICENSE+` +|SPDX header |`+MPL-2.0+` +|Manifest licence |`+NONE+` +|Body classifier |`+Palimp-MPL-2.0+` +|Severity |`+ok+` +|=== + +*Recommended next move:* none for licence. + +=== 3. Documentation debt + +[cols=",",options="header",] +|=== +|Field |Value +|README lines |466 +|`+docs/+` files |12 +|`+docs/+` LoC |4066 +|CHANGELOG.md |N +|CONTRIBUTING.md |Y +|CODE_OF_CONDUCT.md |Y +|SECURITY.md |Y +|Severity |`+LOW+` +|=== + +*Recommended next move:* `+docs/+` has only 12 file(s). Aim for ≥10 +organised docs (architecture, usage, contributing-guide, +troubleshooting, design-decisions). The user’s bar for a +"`heavily-developed and well-organised wiki`" is ≥10 files with topical +organisation. + +Additionally: *CHANGELOG.md is missing.* 65% of estate repos lack one — +adopting a CHANGELOG (or auto-generating via `+git-cliff+`) is a +recommended estate-wide follow-up. + +=== Cross-references + +* Estate proof-debt audit: +`+hyperpolymath/standards/docs/audits/2026-05-26-estate-proof-debt.md+` +* Estate licence-debt audit: +`+hyperpolymath/standards/docs/audits/2026-05-26-estate-licence-debt.md+` +* Estate documentation-debt audit: +`+hyperpolymath/standards/docs/audits/2026-05-26-estate-documentation-debt.md+` + +''''' + +🤖 Generated by Claude Code estate-wide tech-debt scan (2026-05-26). +This file is informational — closing the debt is follow-up work owned by +the maintainer. diff --git a/docs/tech-debt-2026-05-26.md b/docs/tech-debt-2026-05-26.md deleted file mode 100644 index ad86899..0000000 --- a/docs/tech-debt-2026-05-26.md +++ /dev/null @@ -1,58 +0,0 @@ - -# Tech-Debt Audit — http-capability-gateway — 2026-05-26 - -**Source:** estate-wide automated scan 2026-05-26. -**Companion:** [`hyperpolymath/standards` 2026-05-26-estate-*-debt audits](https://github.com/hyperpolymath/standards/tree/main/docs/audits). -**Combined severity:** `LOW`. - -This file records the *raw findings* — it does not by itself fix the debt. Each section ends with a 'Recommended next move' line; closing the debt is follow-up work. - -## 1. Proof debt - -No proof-bearing files (`*.v`, `*.lean`, `*.agda`, `*.idr`, `*.idr2`, `*.fst`, `*.dfy`, `*.tla`, `*.ads`, `*.adb`) found in this repo. - -**Recommended next move:** none. - -## 2. Licence debt - -| Field | Value | -|---|---| -| LICENSE file | `LICENSE` | -| SPDX header | `MPL-2.0` | -| Manifest licence | `NONE` | -| Body classifier | `Palimp-MPL-2.0` | -| Severity | `ok` | - -**Recommended next move:** none for licence. - -## 3. Documentation debt - -| Field | Value | -|---|---| -| README lines | 466 | -| `docs/` files | 12 | -| `docs/` LoC | 4066 | -| CHANGELOG.md | N | -| CONTRIBUTING.md | Y | -| CODE_OF_CONDUCT.md | Y | -| SECURITY.md | Y | -| Severity | `LOW` | - -**Recommended next move:** `docs/` has only 12 file(s). Aim for ≥10 organised docs (architecture, usage, contributing-guide, troubleshooting, design-decisions). The user's bar for a "heavily-developed and well-organised wiki" is ≥10 files with topical organisation. - -Additionally: **CHANGELOG.md is missing.** 65% of estate repos lack one — adopting a CHANGELOG (or auto-generating via `git-cliff`) is a recommended estate-wide follow-up. - -## Cross-references - -- Estate proof-debt audit: `hyperpolymath/standards/docs/audits/2026-05-26-estate-proof-debt.md` -- Estate licence-debt audit: `hyperpolymath/standards/docs/audits/2026-05-26-estate-licence-debt.md` -- Estate documentation-debt audit: `hyperpolymath/standards/docs/audits/2026-05-26-estate-documentation-debt.md` - ---- - -🤖 Generated by Claude Code estate-wide tech-debt scan (2026-05-26). This file is informational — closing the debt is follow-up work owned by the maintainer. diff --git a/ffi/zig/README.adoc b/ffi/zig/README.adoc new file mode 100644 index 0000000..705b6ec --- /dev/null +++ b/ffi/zig/README.adoc @@ -0,0 +1,120 @@ +== Zig FFI Layer - Protocol Implementations + +This directory contains Zig implementations of protocol parsers that +conform to the Idris2 ABI defined in `+src/abi/+`. + +=== Structure + +.... +ffi/zig/ +├── build.zig # Build configuration +├── src/ +│ └── main.zig # Main FFI exports +├── grpc/ +│ └── parser.zig # gRPC/HTTP2 parser +└── graphql/ + └── parser.zig # GraphQL parser +.... + +=== Building + +[source,bash] +---- +cd ffi/zig +zig build + +# Run tests +zig build test +---- + +=== FFI Exports + +All exported functions use C calling convention for Idris2 FFI +compatibility: + +==== HTTP server (`+hpm_http_*+`) + +Synchronous, single-threaded HTTP/1.1 server primitives intended to sit +behind a TLS reverse proxy. Used by the OikosBot for webhook reception. + +* `+hpm_http_server_listen(host, host_len, port) -> server*+` — bind TCP +* `+hpm_http_server_port(server) -> port+` — query bound port (e.g. when +`+port=0+` was passed) +* `+hpm_http_server_accept(server) -> request*+` — block until next +request, return parsed head +* `+hpm_http_request_method(request) -> method_ordinal+` — matches +`+std.http.Method+` (GET=0 HEAD=1 POST=2 PUT=3 DELETE=4 CONNECT=5 +OPTIONS=6 TRACE=7 PATCH=8) +* `+hpm_http_request_path(request, out, cap) -> bytes+` — copy URI +target +* `+hpm_http_request_header(request, name, name_len, out, cap) -> bytes+` +— case-insensitive lookup; returns 0 if absent +* `+hpm_http_request_body(request, out, cap) -> bytes+` — read body (max +1 MiB, content-length-driven); idempotent +* `+hpm_http_request_respond(request, status, headers, headers_len, body, body_len) -> 0/-1+` +— send full response; connection closes after +* `+hpm_http_request_free(request)+` — close + free +* `+hpm_http_server_free(server)+` — close listener + free + +Idris2 wrappers live in `+../../src/abi/HttpServer.idr+`. + +==== gRPC + +* `+parse_grpc_request+` - Parse HTTP/2 gRPC frame +* Validates frame headers and extracts service/method + +==== GraphQL + +* `+parse_graphql_request+` - Parse GraphQL JSON body +* Detects operation type (query/mutation/subscription) +* Extracts operation name and selections + +==== Common + +* `+platform_endian+` - Detect platform byte order +* `+serialize_response+` - Serialize response to wire format + +=== Integration with Elixir + +The Zig library compiles to `+libgateway.so+` which can be loaded via +Elixir NIFs or Port drivers: + +[source,elixir] +---- +# Using Rustler-style NIF (with zig backend) +defmodule GatewayNIF do + use Rustler, otp_app: :http_capability_gateway, crate: "gateway_zig" +end + +# Or via Port +{:ok, port} = Port.open({:spawn, "./ffi/zig/zig-out/bin/gateway"}, [:binary]) +---- + +=== Memory Safety + +Zig’s compile-time safety checks combined with Idris2’s dependent type +proofs ensure: - No buffer overflows - No use-after-free - Correct +memory layout across FFI boundary - Bounds checking on all array +accesses + +=== Testing + +Each parser has comprehensive unit tests: + +[source,bash] +---- +# Run gRPC tests +zig test grpc/parser.zig + +# Run GraphQL tests +zig test graphql/parser.zig +---- + +=== Performance + +Zig compiles to native code with no runtime overhead: - Zero-cost +abstractions - LLVM optimization passes - No garbage collection - Direct +memory access + +Benchmarks show gRPC parsing at ~5μs per request, GraphQL at ~10μs per +query. diff --git a/ffi/zig/README.md b/ffi/zig/README.md deleted file mode 100644 index fac36a8..0000000 --- a/ffi/zig/README.md +++ /dev/null @@ -1,116 +0,0 @@ - -# Zig FFI Layer - Protocol Implementations - -This directory contains Zig implementations of protocol parsers that conform to the Idris2 ABI defined in `src/abi/`. - -## Structure - -``` -ffi/zig/ -├── build.zig # Build configuration -├── src/ -│ └── main.zig # Main FFI exports -├── grpc/ -│ └── parser.zig # gRPC/HTTP2 parser -└── graphql/ - └── parser.zig # GraphQL parser -``` - -## Building - -```bash -cd ffi/zig -zig build - -# Run tests -zig build test -``` - -## FFI Exports - -All exported functions use C calling convention for Idris2 FFI compatibility: - -### HTTP server (`hpm_http_*`) - -Synchronous, single-threaded HTTP/1.1 server primitives intended to sit -behind a TLS reverse proxy. Used by the OikosBot for webhook reception. - -- `hpm_http_server_listen(host, host_len, port) -> server*` — bind TCP -- `hpm_http_server_port(server) -> port` — query bound port (e.g. when - `port=0` was passed) -- `hpm_http_server_accept(server) -> request*` — block until next request, - return parsed head -- `hpm_http_request_method(request) -> method_ordinal` — matches - `std.http.Method` (GET=0 HEAD=1 POST=2 PUT=3 DELETE=4 CONNECT=5 - OPTIONS=6 TRACE=7 PATCH=8) -- `hpm_http_request_path(request, out, cap) -> bytes` — copy URI target -- `hpm_http_request_header(request, name, name_len, out, cap) -> bytes` — - case-insensitive lookup; returns 0 if absent -- `hpm_http_request_body(request, out, cap) -> bytes` — read body (max 1 - MiB, content-length-driven); idempotent -- `hpm_http_request_respond(request, status, headers, headers_len, body, - body_len) -> 0/-1` — send full response; connection closes after -- `hpm_http_request_free(request)` — close + free -- `hpm_http_server_free(server)` — close listener + free - -Idris2 wrappers live in `../../src/abi/HttpServer.idr`. - -### gRPC -- `parse_grpc_request` - Parse HTTP/2 gRPC frame -- Validates frame headers and extracts service/method - -### GraphQL -- `parse_graphql_request` - Parse GraphQL JSON body -- Detects operation type (query/mutation/subscription) -- Extracts operation name and selections - -### Common -- `platform_endian` - Detect platform byte order -- `serialize_response` - Serialize response to wire format - -## Integration with Elixir - -The Zig library compiles to `libgateway.so` which can be loaded via Elixir NIFs or Port drivers: - -```elixir -# Using Rustler-style NIF (with zig backend) -defmodule GatewayNIF do - use Rustler, otp_app: :http_capability_gateway, crate: "gateway_zig" -end - -# Or via Port -{:ok, port} = Port.open({:spawn, "./ffi/zig/zig-out/bin/gateway"}, [:binary]) -``` - -## Memory Safety - -Zig's compile-time safety checks combined with Idris2's dependent type proofs ensure: -- No buffer overflows -- No use-after-free -- Correct memory layout across FFI boundary -- Bounds checking on all array accesses - -## Testing - -Each parser has comprehensive unit tests: - -```bash -# Run gRPC tests -zig test grpc/parser.zig - -# Run GraphQL tests -zig test graphql/parser.zig -``` - -## Performance - -Zig compiles to native code with no runtime overhead: -- Zero-cost abstractions -- LLVM optimization passes -- No garbage collection -- Direct memory access - -Benchmarks show gRPC parsing at ~5μs per request, GraphQL at ~10μs per query. diff --git a/llm-warmup-dev.adoc b/llm-warmup-dev.adoc new file mode 100644 index 0000000..44f0678 --- /dev/null +++ b/llm-warmup-dev.adoc @@ -0,0 +1,19 @@ +== LLM Warmup — http-capability-gateway (Developer) + +=== What is http-capability-gateway? + +See README.adoc for overview. + +=== Key Commands + +* `+just setup+` — set up development environment +* `+just build+` — build the project +* `+just test+` — run tests +* `+just doctor+` — diagnose issues +* `+just heal+` — attempt auto-repair + +=== Quick Context + +* License: PMPL-1.0-or-later +* Part of hyperpolymath ecosystem +* See EXPLAINME.adoc for architecture diff --git a/llm-warmup-dev.md b/llm-warmup-dev.md deleted file mode 100644 index 04e2a54..0000000 --- a/llm-warmup-dev.md +++ /dev/null @@ -1,20 +0,0 @@ - -# LLM Warmup — http-capability-gateway (Developer) - -## What is http-capability-gateway? -See README.adoc for overview. - -## Key Commands -- `just setup` — set up development environment -- `just build` — build the project -- `just test` — run tests -- `just doctor` — diagnose issues -- `just heal` — attempt auto-repair - -## Quick Context -- License: PMPL-1.0-or-later -- Part of hyperpolymath ecosystem -- See EXPLAINME.adoc for architecture diff --git a/llm-warmup-user.adoc b/llm-warmup-user.adoc new file mode 100644 index 0000000..cf04db5 --- /dev/null +++ b/llm-warmup-user.adoc @@ -0,0 +1,19 @@ +== LLM Warmup — http-capability-gateway (User) + +=== What is http-capability-gateway? + +See README.adoc for overview. + +=== Key Commands + +* `+just setup+` — set up development environment +* `+just build+` — build the project +* `+just test+` — run tests +* `+just doctor+` — diagnose issues +* `+just heal+` — attempt auto-repair + +=== Quick Context + +* License: PMPL-1.0-or-later +* Part of hyperpolymath ecosystem +* See EXPLAINME.adoc for architecture diff --git a/llm-warmup-user.md b/llm-warmup-user.md deleted file mode 100644 index f95bd65..0000000 --- a/llm-warmup-user.md +++ /dev/null @@ -1,20 +0,0 @@ - -# LLM Warmup — http-capability-gateway (User) - -## What is http-capability-gateway? -See README.adoc for overview. - -## Key Commands -- `just setup` — set up development environment -- `just build` — build the project -- `just test` — run tests -- `just doctor` — diagnose issues -- `just heal` — attempt auto-repair - -## Quick Context -- License: PMPL-1.0-or-later -- Part of hyperpolymath ecosystem -- See EXPLAINME.adoc for architecture diff --git a/src/abi/FORMAL-PROOFS.adoc b/src/abi/FORMAL-PROOFS.adoc new file mode 100644 index 0000000..f2ab9a2 --- /dev/null +++ b/src/abi/FORMAL-PROOFS.adoc @@ -0,0 +1,164 @@ +== Formal Proofs TODO - Idris2 ABI + +This document outlines the formal proof obligations that would be +completed for a production-ready ABI. + +=== Currently Missing Proofs + +==== 1. WellFormed Predicate (Protocol.idr) + +*Purpose:* Prove that byte sequences represent valid protocol messages + +[source,idris] +---- +-- TODO: Define WellFormed type +data WellFormed : List Bits8 -> Type where + -- HTTP well-formedness: valid method, headers, body structure + HTTPWellFormed : ValidHTTPStructure bytes -> WellFormed bytes + + -- gRPC well-formedness: valid HTTP/2 frame with gRPC headers + GRPCWellFormed : ValidHTTP2Frame bytes -> + HasGRPCHeaders bytes -> + WellFormed bytes + + -- GraphQL well-formedness: valid JSON with query field + GraphQLWellFormed : ValidJSON bytes -> + HasQueryField bytes -> + WellFormed bytes +---- + +==== 2. ParseResult Type (Protocol.idr) + +*Purpose:* Dependent type representing parse outcomes + +[source,idris] +---- +-- TODO: Define ParseResult +data ParseResult : List Bits8 -> Type where + ParseSuccess : (req : Request proto) -> + {auto prf : WellFormed bytes} -> + ParseResult bytes + + ParseFailure : (reason : String) -> + ParseResult bytes +---- + +==== 3. Parse Success Theorem (Protocol.idr) + +*Currently:* Postulate (assumed without proof) *Should be:* Proven +theorem + +[source,idris] +---- +-- TODO: Prove this theorem +wellFormedParseSuccess : + {proto : ProtocolType} -> + (bytes : List Bits8) -> + WellFormed bytes -> + parseProtocol proto bytes = Just req +---- + +*Proof strategy:* 1. Structural induction on WellFormed proof 2. Case +analysis on protocol type 3. Show parser implementation matches +well-formedness predicate + +==== 4. ValidLifetime Type (Protocol.idr) + +*Purpose:* Memory safety - parsed requests don’t outlive buffer + +[source,idris] +---- +-- TODO: Define ValidLifetime +data ValidLifetime : (buffer : AnyPtr) -> (req : Request proto) -> Type where + -- Request fields are subslices of buffer + FieldsInBuffer : (req : Request proto) -> + (buffer : AnyPtr) -> + (len : Nat) -> + AllFieldsWithinBounds req buffer len -> + ValidLifetime buffer req +---- + +==== 5. Memory Safety Theorem (Protocol.idr) + +*Currently:* Postulate *Should be:* Proven theorem + +[source,idris] +---- +-- TODO: Prove memory safety +parseMemorySafety : + {proto : ProtocolType} -> + (buffer : AnyPtr) -> + (len : Nat) -> + (req : Request proto) -> + ValidLifetime buffer req +---- + +*Proof strategy:* 1. Show FFI parser only creates pointers within buffer +bounds 2. Prove no pointer arithmetic escapes buffer 3. Demonstrate +slice bounds checking + +==== 6. BoundedString Length Proof (Types.idr) + +*Currently:* Comment "`TODO: Add proof`" *Should be:* Dependent type +with proof + +[source,idris] +---- +-- TODO: Complete BoundedString with proof +data BoundedString : (maxLen : Nat) -> Type where + MkBounded : (content : String) -> + {auto prf : LTE (length content) maxLen} -> + BoundedString maxLen +---- + +Where `+LTE+` is the less-than-or-equal proof from `+Data.Nat+`. + +==== 7. Endian Involution Proof (Types.idr) + +*Currently:* `+believe_me ()+` (assumed) *Should be:* Proven + +[source,idris] +---- +-- TODO: Actually prove this +endianInvolution : (x : Bits32) -> + fromNetworkOrder32 (toNetworkOrder32 x) = x +---- + +*Proof strategy:* 1. Case split on platform endianness 2. BigEndian +case: trivial (both functions are identity) 3. LittleEndian case: prove +swapEndian32 is involution 4. Show swapEndian32 (swapEndian32 x) = x by +bit manipulation + +=== Benefits of Complete Proofs + +==== Type Safety + +* Impossible to call parsers with invalid buffers +* Compiler-checked memory safety +* No runtime bounds checks needed (proven safe) + +==== Correctness Guarantees + +* Parser correctly implements protocol specification +* No undefined behavior possible +* All edge cases handled provably + +==== FFI Safety Bridge + +* Idris2 proofs guarantee Zig FFI correctness +* C calling convention safety verified +* Memory layout consistency proven + +=== Implementation Priority + +[arabic] +. *High*: BoundedString length proofs (immediate safety benefit) +. *Medium*: ParseResult and WellFormed types (better type checking) +. *Low*: Full theorems (academic completeness) + +=== References + +* https://idris2.readthedocs.io/[Idris2 Documentation] +* https://www.idris-lang.org/pages/example.html[Dependent Types for +Verified Systems] +* https://en.wikipedia.org/wiki/Proof-carrying_code[Proof-Carrying Code] diff --git a/src/abi/FORMAL-PROOFS.md b/src/abi/FORMAL-PROOFS.md deleted file mode 100644 index c4da887..0000000 --- a/src/abi/FORMAL-PROOFS.md +++ /dev/null @@ -1,160 +0,0 @@ - -# Formal Proofs TODO - Idris2 ABI - -This document outlines the formal proof obligations that would be completed for a production-ready ABI. - -## Currently Missing Proofs - -### 1. WellFormed Predicate (Protocol.idr) - -**Purpose:** Prove that byte sequences represent valid protocol messages - -```idris --- TODO: Define WellFormed type -data WellFormed : List Bits8 -> Type where - -- HTTP well-formedness: valid method, headers, body structure - HTTPWellFormed : ValidHTTPStructure bytes -> WellFormed bytes - - -- gRPC well-formedness: valid HTTP/2 frame with gRPC headers - GRPCWellFormed : ValidHTTP2Frame bytes -> - HasGRPCHeaders bytes -> - WellFormed bytes - - -- GraphQL well-formedness: valid JSON with query field - GraphQLWellFormed : ValidJSON bytes -> - HasQueryField bytes -> - WellFormed bytes -``` - -### 2. ParseResult Type (Protocol.idr) - -**Purpose:** Dependent type representing parse outcomes - -```idris --- TODO: Define ParseResult -data ParseResult : List Bits8 -> Type where - ParseSuccess : (req : Request proto) -> - {auto prf : WellFormed bytes} -> - ParseResult bytes - - ParseFailure : (reason : String) -> - ParseResult bytes -``` - -### 3. Parse Success Theorem (Protocol.idr) - -**Currently:** Postulate (assumed without proof) -**Should be:** Proven theorem - -```idris --- TODO: Prove this theorem -wellFormedParseSuccess : - {proto : ProtocolType} -> - (bytes : List Bits8) -> - WellFormed bytes -> - parseProtocol proto bytes = Just req -``` - -**Proof strategy:** -1. Structural induction on WellFormed proof -2. Case analysis on protocol type -3. Show parser implementation matches well-formedness predicate - -### 4. ValidLifetime Type (Protocol.idr) - -**Purpose:** Memory safety - parsed requests don't outlive buffer - -```idris --- TODO: Define ValidLifetime -data ValidLifetime : (buffer : AnyPtr) -> (req : Request proto) -> Type where - -- Request fields are subslices of buffer - FieldsInBuffer : (req : Request proto) -> - (buffer : AnyPtr) -> - (len : Nat) -> - AllFieldsWithinBounds req buffer len -> - ValidLifetime buffer req -``` - -### 5. Memory Safety Theorem (Protocol.idr) - -**Currently:** Postulate -**Should be:** Proven theorem - -```idris --- TODO: Prove memory safety -parseMemorySafety : - {proto : ProtocolType} -> - (buffer : AnyPtr) -> - (len : Nat) -> - (req : Request proto) -> - ValidLifetime buffer req -``` - -**Proof strategy:** -1. Show FFI parser only creates pointers within buffer bounds -2. Prove no pointer arithmetic escapes buffer -3. Demonstrate slice bounds checking - -### 6. BoundedString Length Proof (Types.idr) - -**Currently:** Comment "TODO: Add proof" -**Should be:** Dependent type with proof - -```idris --- TODO: Complete BoundedString with proof -data BoundedString : (maxLen : Nat) -> Type where - MkBounded : (content : String) -> - {auto prf : LTE (length content) maxLen} -> - BoundedString maxLen -``` - -Where `LTE` is the less-than-or-equal proof from `Data.Nat`. - -### 7. Endian Involution Proof (Types.idr) - -**Currently:** `believe_me ()` (assumed) -**Should be:** Proven - -```idris --- TODO: Actually prove this -endianInvolution : (x : Bits32) -> - fromNetworkOrder32 (toNetworkOrder32 x) = x -``` - -**Proof strategy:** -1. Case split on platform endianness -2. BigEndian case: trivial (both functions are identity) -3. LittleEndian case: prove swapEndian32 is involution -4. Show swapEndian32 (swapEndian32 x) = x by bit manipulation - -## Benefits of Complete Proofs - -### Type Safety -- Impossible to call parsers with invalid buffers -- Compiler-checked memory safety -- No runtime bounds checks needed (proven safe) - -### Correctness Guarantees -- Parser correctly implements protocol specification -- No undefined behavior possible -- All edge cases handled provably - -### FFI Safety Bridge -- Idris2 proofs guarantee Zig FFI correctness -- C calling convention safety verified -- Memory layout consistency proven - -## Implementation Priority - -1. **High**: BoundedString length proofs (immediate safety benefit) -2. **Medium**: ParseResult and WellFormed types (better type checking) -3. **Low**: Full theorems (academic completeness) - -## References - -- [Idris2 Documentation](https://idris2.readthedocs.io/) -- [Dependent Types for Verified Systems](https://www.idris-lang.org/pages/example.html) -- [Proof-Carrying Code](https://en.wikipedia.org/wiki/Proof-carrying_code) diff --git a/src/abi/README.adoc b/src/abi/README.adoc new file mode 100644 index 0000000..6604fe6 --- /dev/null +++ b/src/abi/README.adoc @@ -0,0 +1,60 @@ +== ABI Layer - Idris2 Interface Definitions + +This directory contains formal interface definitions for the +multi-protocol gateway using Idris2 with dependent types. + +=== Purpose + +* *Correctness*: Dependent types prove protocol handling correctness at +compile time +* *Safety*: Memory layout proofs ensure FFI safety +* *Platform Independence*: Abstract over platform-specific details + +=== Files + +* `+Protocol.idr+` - Protocol abstraction (HTTP/gRPC/GraphQL) +* `+Types.idr+` - Core type definitions with memory layout proofs +* `+Layout.idr+` - Memory layout specifications for C FFI (TODO) +* `+Foreign.idr+` - FFI declarations for Zig implementations (TODO) + +=== Building + +Requires Idris2 compiler: + +[source,bash] +---- +# Install Idris2 +asdf install idris2 latest + +# Check ABI definitions +cd src/abi +idris2 --check Protocol.idr +idris2 --check Types.idr +---- + +=== Integration + +The Zig FFI layer (in `+ffi/zig/+`) implements these interfaces. C +headers are generated from Idris2 definitions to ensure type safety +across the FFI boundary. + +=== Proofs + +Key properties proven at the type level: + +[arabic] +. *Access Control Soundness*: `+Satisfies+` type proves trust levels +meet exposure requirements +. *Memory Safety*: Buffer bounds and lifetime proofs prevent +use-after-free +. *Endianness Correctness*: Conversion operations proven as involutions +. *Parse Success*: Well-formed inputs always parse successfully +(postulated, would be proven formally) + +=== ABI Stability + +ABI version follows semantic versioning: - Major version breaks binary +compatibility - Minor version adds backward-compatible features - Patch +version for clarifications only + +Current version: *1.0.0* diff --git a/src/abi/README.md b/src/abi/README.md deleted file mode 100644 index 68d2920..0000000 --- a/src/abi/README.md +++ /dev/null @@ -1,56 +0,0 @@ - -# ABI Layer - Idris2 Interface Definitions - -This directory contains formal interface definitions for the multi-protocol gateway using Idris2 with dependent types. - -## Purpose - -- **Correctness**: Dependent types prove protocol handling correctness at compile time -- **Safety**: Memory layout proofs ensure FFI safety -- **Platform Independence**: Abstract over platform-specific details - -## Files - -- `Protocol.idr` - Protocol abstraction (HTTP/gRPC/GraphQL) -- `Types.idr` - Core type definitions with memory layout proofs -- `Layout.idr` - Memory layout specifications for C FFI (TODO) -- `Foreign.idr` - FFI declarations for Zig implementations (TODO) - -## Building - -Requires Idris2 compiler: - -```bash -# Install Idris2 -asdf install idris2 latest - -# Check ABI definitions -cd src/abi -idris2 --check Protocol.idr -idris2 --check Types.idr -``` - -## Integration - -The Zig FFI layer (in `ffi/zig/`) implements these interfaces. C headers are generated from Idris2 definitions to ensure type safety across the FFI boundary. - -## Proofs - -Key properties proven at the type level: - -1. **Access Control Soundness**: `Satisfies` type proves trust levels meet exposure requirements -2. **Memory Safety**: Buffer bounds and lifetime proofs prevent use-after-free -3. **Endianness Correctness**: Conversion operations proven as involutions -4. **Parse Success**: Well-formed inputs always parse successfully (postulated, would be proven formally) - -## ABI Stability - -ABI version follows semantic versioning: -- Major version breaks binary compatibility -- Minor version adds backward-compatible features -- Patch version for clarifications only - -Current version: **1.0.0**