feat(search): strip math delimiters from the search index - #70
Merged
Conversation
kramdown's mathjax engine leaves `\(...\)` and `\[...\]` in the HTML as literal
text, so a flattened post read `메커니즘의 \(O(L^2)\) 계산 복잡도` — and now that the
excerpt is cut around the match rather than taken from the opening, a window
landing on a formula showed those delimiters to the reader. 30 of 35 posts carried
them.
Only the delimiters go. The TeX inside stays, because it is what the sentence is
about and because a symbol inside a formula should remain findable:
before 메커니즘의 \(O(L^2)\) 계산 복잡도는
after 메커니즘의 O(L^2) 계산 복잡도는
Inline math is replaced with no padding, display math with spaces. That asymmetry
is load-bearing: Korean attaches a particle straight onto the symbol, so
`\(\theta\)로` has to come out as `\theta로` and not `\theta 로`. Display math is
already its own block.
An unclosed `\(` is left alone rather than swallowing the rest of the post, and a
bare `f(x)` in prose is untouched — both pinned by tests.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #69, which was left as an open decision there.
kramdown's mathjax engine leaves
\(...\)and\[...\]in the HTML as literaltext. Now that #69 cuts the excerpt around the match instead of taking the
opening, a window landing on a formula shows the delimiters to the reader.
30 of 35 posts carried them.
Only the delimiters go — the TeX inside stays, so
O(L^2)remains both readableand findable. Dropping the formula contents entirely would have made characters
inside math unsearchable.
The inline/display asymmetry is load-bearing. Inline math is replaced with no
padding, display math with spaces. Korean attaches a particle straight onto the
symbol, so
\(\theta\)로has to come out\theta로, not\theta 로— the sameclass of trap as the word-boundary cap in #69. Display math is already its own
block, so it can take the spaces.
Edge cases pinned by tests: an unclosed
\(is left alone rather than swallowingthe rest of the post, and a bare
f(x)in prose is untouched.Verification
validate-site checks passed.
/search/page: the DeepSeek-V3.2result for 어텐션 now reads
바닐라 어텐션 메커니즘의 O(L^2) 계산 복잡도는.Left in §9
A TeX command still shows when a window lands on one (
\theta). Converting toUnicode symbols needs a TeX→glyph mapping, which is more than a search index
should carry.