Skip to content

perf: accelerate call-local expansion and substitution - #101

Merged
Qubitium merged 21 commits into
mainfrom
codex/api-speedups-round2
Aug 10, 2026
Merged

perf: accelerate call-local expansion and substitution#101
Qubitium merged 21 commits into
mainfrom
codex/api-speedups-round2

Conversation

@Qubitium

Copy link
Copy Markdown
Contributor

Summary

  • render exact Match.expand templates with one to eight numeric, explicit, named, or literal-backslash tokens directly from the immutable C Match snapshot
  • accelerate exact default-count Pattern.sub / subn templates containing one valid numeric or named reference through a guarded C translation path
  • keep all temporary parsing and output state call-local; no new persistent cache entries, unbounded collections, or cross-thread retained captures
  • preserve compatibility-parser fallback for ambiguous references, unsupported escapes, non-ASCII names, subclasses, bounded substitution counts, and templates above the fixed token bound
  • stabilize the API benchmark harness against CPython call-site specialization and macOS P/E-core migration artifacts
  • document pinned Python 3.10 and free-threaded Python 3.14t results in the README

Performance

Pinned same-policy A/B measurements include:

  • single numeric Match.expand: 16.7x on 3.14t and 20.8x on 3.10
  • named Match.expand: 10.6x / 28.7x
  • two-reference expansion: 9.6x / 45.3x
  • three-reference expansion: 8.2x / 9.2x
  • eight-reference expansion: 6.3x / 6.3x
  • literal-backslash plus named expansion: 11.7x / 13.9x
  • single-reference Pattern.sub: approximately 1.9–2.1x for numeric/explicit forms; named reaches 1.9x / 10.9x

Figures are listed as Python 3.14t / Python 3.10 unless shown inline otherwise.

Accuracy and safety

  • differential tests cover text, bytes, Unicode, group 0, leading-zero and multi-digit references, unmatched captures, mixed numeric/named forms, duplicate PCRE names, and mutable-buffer snapshot semantics
  • checked length arithmetic and fixed eight-token stack storage bound allocation math and parsing state
  • shared-Match and shared-Pattern tests exercise concurrent expansion/substitution on Python 3.14t with GIL disabled
  • invalid or extended syntax returns to the existing compatibility parser rather than approximating behavior

Validation

  • Python 3.14t/GIL=0: 576 passed, 15 skipped, 545 subtests
  • Python 3.14t Python modules: 100% statement coverage and 100% branch coverage (1,826 statements / 652 branches)
  • Python 3.10: 573 passed, 18 skipped
  • randomized API clobber: 60 seconds passed (seed 327942444249519898257797889077606415028)
  • concurrent GIL=0 clobber: 60 seconds passed (seed 268836252769390426394361496133289920377)
  • Ruff formatting/checks and git diff --check pass

@Qubitium
Qubitium marked this pull request as ready for review August 10, 2026 14:35
@Qubitium
Qubitium merged commit d61fa7f into main Aug 10, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant