Skip to content

common: reject banned Unicode categories in protocol text fields - #9398

Draft
Andezion wants to merge 2 commits into
ElementsProject:masterfrom
Andezion:fix/reject-banned-unicode-text-fields
Draft

common: reject banned Unicode categories in protocol text fields#9398
Andezion wants to merge 2 commits into
ElementsProject:masterfrom
Andezion:fix/reject-banned-unicode-text-fields

Conversation

@Andezion

Copy link
Copy Markdown
Collaborator

Summary

Implements the null-byte/control-character ban discussed in lightning/bolts#1260: utf8_check() previously only validated UTF-8 encoding, not content - null bytes, control characters, Unicode format characters (including bidirectional-override characters used in "Trojan Source" spoofing, CVE-2021-42574), private-use, and unassigned codepoints all passed silently

Changes

  • I was able to create a script that generate a banned-codepoint table (Unicode categories Cc/Cf/Co/Cn)
    from the Unicode Character Database (via devtools/gen-unicode-category.py), mirroring the approach
    rust-lightning took. Cs (surrogates) is excluded since ccan/utf8 already rejects those while decoding
  • Wire the check into utf8_check(), the single chokepoint used by BOLT11 (description) and BOLT12 (offer_description, offer_issuer, invreq_payer_note, proof_note, invoice_error.msg) encoding/decoding
  • Fix a regression this exposed: towire_utf8_array() already asserted utf8_check() on outgoing fields. With the stricter check, that assert became reachable from ordinary RPC input - offer/invoicerequest/fetchinvoice/ cancelrecurringinvoice would crash instead of failing cleanly. Added param_escaped_utf8_string()/param_utf8_string() to validate at the RPC boundary with a normal command_fail_badparam error

Testing

New unit tests (run-utils-utf8_check, additions to run-param) cover NUL/control/format/private-use/unassigned rejection and valid multi-byte UTF-8 passthrough. No regressions in existing BOLT11/ BOLT12 test suites. Verified clean under ASan+UBSan.

Important

26.09 FREEZE August 5th: Non-bugfix PRs not ready by this date will wait for 26.12.

RC1 is scheduled on August 17th

The final release is scheduled for September 7th.

Checklist

Before submitting the PR, ensure the following tasks are completed. If an item is not applicable to your PR, please mark it as checked:

  • The changelog has been updated in the relevant commit(s) according to the guidelines.
  • Tests have been added or modified to reflect the changes.
  • Documentation has been reviewed and updated as needed.
  • Related issues have been listed and linked, including any that this PR closes.
  • Important All PRs must consider how to reverse any persistent changes for tools/lightning-downgrade

@Andezion Andezion self-assigned this Aug 10, 2026
@Andezion
Andezion force-pushed the fix/reject-banned-unicode-text-fields branch from d4c14fe to 70836da Compare August 10, 2026 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant