You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Two halves of one gap. Neither is worth much without the other.
1. The same relationship gives two different maiden values
parse("Jane Smith née Jones") # maiden 'Jones'parse("Jane Smith (née Jones)", maiden_delims) # maiden 'née Jones'parse("山田 花子(佐藤)", maiden_delims) # maiden '佐藤'parse("山田(旧姓:佐藤)", maiden_delims) # maiden '旧姓:佐藤'
A caller comparing maiden across a dataset gets a spurious mismatch between two spellings of one person's name.
Why. Two paths, two treatments. Bare: classify tags the marker vocab:maiden-marker (_classify.py:59) and _group.py:324 consumes it, folding marker plus following piece into maiden (#274). Delimited: _extract assigns the bracketed content Role.MAIDEN wholesale at extract time, before classify runs — the marker inside is never tagged, so the consuming rule never fires.
Fix shape. Consume a leading maiden marker inside extracted maiden content, so both paths agree. Care on the anti-#100 span invariant: token spans must keep indexing the original string, so this is a role/span decision, not a text rewrite.
2. 旧姓 is not in the default maiden_markers
It belongs there by the rule that admitted the Cyrillic entries — native-script entries cannot collide with Latin-script names. Verified: whole-token matching, _normalize("旧姓") is 旧姓, and neither character appears in any shipped surname, title, suffix, conjunction, particle or bound-given vocabulary. Measured with it added, 山田花子 旧姓 佐藤 → family 山田花子, maiden 佐藤; 山田 花子 旧姓 佐藤 → given 花子, family 山田, maiden 佐藤.
Not the JA pack: that exists for things needing the Japanese-data declaration (segmentation, where a pure-Han string cannot say which language it is). A Han-script marker needs no declaration — it can only match Han text, so it sits alongside урожденная and geboren in config/maiden_markers.py.
Why together
The vocabulary alone reaches only the spaced form, which is not what Japanese usually writes. The extraction fix alone leaves 旧姓:佐藤 unrecognized as a marker even once markers are consumed. Together, 山田(旧姓:佐藤) with maiden_delimiters gives 佐藤.
Open questions
The separator.旧姓:佐藤 carries a fullwidth colon. Is it part of the marker, part of the vocabulary, or stripped structurally? Decide rather than inherit.
Two halves of one gap. Neither is worth much without the other.
1. The same relationship gives two different
maidenvaluesA caller comparing
maidenacross a dataset gets a spurious mismatch between two spellings of one person's name.Why. Two paths, two treatments. Bare:
classifytags the markervocab:maiden-marker(_classify.py:59) and_group.py:324consumes it, folding marker plus following piece intomaiden(#274). Delimited:_extractassigns the bracketed contentRole.MAIDENwholesale at extract time, beforeclassifyruns — the marker inside is never tagged, so the consuming rule never fires.Fix shape. Consume a leading maiden marker inside extracted maiden content, so both paths agree. Care on the anti-#100 span invariant: token spans must keep indexing the original string, so this is a role/span decision, not a text rewrite.
2.
旧姓is not in the defaultmaiden_markersIt belongs there by the rule that admitted the Cyrillic entries — native-script entries cannot collide with Latin-script names. Verified: whole-token matching,
_normalize("旧姓")is旧姓, and neither character appears in any shipped surname, title, suffix, conjunction, particle or bound-given vocabulary. Measured with it added,山田花子 旧姓 佐藤→ family山田花子, maiden佐藤;山田 花子 旧姓 佐藤→ given花子, family山田, maiden佐藤.Not the JA pack: that exists for things needing the Japanese-data declaration (segmentation, where a pure-Han string cannot say which language it is). A Han-script marker needs no declaration — it can only match Han text, so it sits alongside
урожденнаяandgeboreninconfig/maiden_markers.py.Why together
The vocabulary alone reaches only the spaced form, which is not what Japanese usually writes. The extraction fix alone leaves
旧姓:佐藤unrecognized as a marker even once markers are consumed. Together,山田(旧姓:佐藤)withmaiden_delimitersgives佐藤.Open questions
旧姓:佐藤carries a fullwidth colon. Is it part of the marker, part of the vocabulary, or stripped structurally? Decide rather than inherit.Supersedes #309.