Skip to content

fix(generator): escape multi-line braces and backtick unknown doc tags in yard formatting - #1337

Open
suztomo wants to merge 8 commits into
googleapis:mainfrom
suztomo:fix/yard-brace-and-tag-escaping
Open

fix(generator): escape multi-line braces and backtick unknown doc tags in yard formatting#1337
suztomo wants to merge 8 commits into
googleapis:mainfrom
suztomo:fix/yard-brace-and-tag-escaping

Conversation

@suztomo

@suztomo suztomo commented Aug 31, 2026

Copy link
Copy Markdown
Member

Summary

This PR addresses YARD documentation warnings and errors generated when formatting proto docstrings containing multiline braces (such as inline JSON examples or range syntax) and proto comments with unknown doc tags (such as @pattern or @required).

Example rendering problem is at googleapis/librarian#7461 (comment).

Problem

  1. Multiline / Unclosed Braces:
    • Gapic::FormattingUtils.escape_line_braces previously matched { and } on the same line using @brace_detector.
    • When docstrings contain multiline JSON examples (e.g. [{name: "app1", port: 8080}, {name:\n"app1"...}) or multiline ranges ({min: min-value, max:\nmax-value} or {100, 200,\n300}), the opening { on the first line remained unescaped.
    • YARD then attempts to parse {min: ...} across lines as an object/class link, resulting in Cannot resolve link to min: from text: warnings.
  2. Unknown Doc Tags:
    • Proto comments containing tags like @pattern (e.g., in regex patterns) or @required compute.instancegroups.addInstances cause YARD to emit [warn]: Unknown tag @<tag> because YARD interprets any @word at the start of a line or after whitespace as a doc tag.
  3. Fenced & Multiline Inline Code Blocks in Docstrings:
    • Proto comments frequently contain Markdown fenced code blocks (... or ~~~ ... ~~~) and multiline inline code spans (`...`) that may contain braces (e.g. "divider": {} or multiline resource URI templates) or mentions (e.g. @FooBot).
    • Previously only 4-space indentation was recognized as code blocks, causing fenced blocks and multiline inline spans to be treated as prose and incorrectly modified with escapes.

Solution

  1. Preserve Fenced Code Blocks (in_fence) and Multiline Inline Code Spans (in_code_span):
    • Track in_fence state across lines when encountering lines starting with ``` or ~~~.
    • Track in_code_span state across lines when an inline code span starts on one line and ends on another.
    • All text inside code blocks and spans is excluded from brace escaping and tag sanitization, preserving code samples verbatim without spurious \ escapes.
  2. Escape unescaped { outside backticks:
    • Split each line by inline code spans (`...`) and replace all unescaped { outside backticks with \{.
    • This ensures all literal braces in prose are escaped regardless of whether the closing } is on the same line or subsequent lines.
  3. Sanitize unknown doc tags outside backticks:
    • For any (?<=\A|\s)@tag outside backtick spans that is not a known YARD tag (@param, @return, @see, @!attribute, @attr, @attr_reader, @attr_writer, etc.), wrap it in backticks (`@tag`) so YARD renders it as literal text rather than an unrecognized tag directive.
    • Email addresses (e.g. support@example.com) and string literals in code blocks/quotes remain unaffected.

Preview in google-cloud-ruby

A full regeneration preview across google-cloud-ruby was verified in draft PR googleapis/google-cloud-ruby#36512, where:

  • All multiline brace and tag warnings were resolved across generated libraries.
  • Fenced code blocks (e.g. in google/apps/card/v1/card.rb and google/chat/v1/annotation.rb) and multiline inline backtick spans (e.g. in google/cloud/aiplatform/v1/featurestore_online_service.rb) are preserved verbatim without spurious escapes.
  • toys ci --yard --gems google-apps-chat-v1, toys ci --yard --gems google-cloud-compute-v1, and toys ci --yard --gems google-cloud-ai_platform-v1 passed with 0 warnings and 0 errors.

Related issues:

@suztomo

suztomo commented Aug 31, 2026

Copy link
Copy Markdown
Member Author

Example rendering of HTML. See the @pattern is correctly shown.

Image

The braces in the ranges are correctly shown.

Image

@suztomo
suztomo marked this pull request as draft August 31, 2026 21:38
@suztomo

suztomo commented Aug 31, 2026

Copy link
Copy Markdown
Member Author

Example for google-apps-chat-v1/proto_docs/google/apps/card/v1/card.rb:

Screenshot 2026-08-31 at 5 42 08 PM

@suztomo
suztomo marked this pull request as ready for review September 1, 2026 19:13
@suztomo

suztomo commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Checking this

gapic-generator-ruby-bazel-presubmit (gapic-images)Failing after 7m — Summary


ERROR: Skipping '//google/ads/googleads/v21:googleads-ruby': no such package 'google/ads/googleads/v21': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.
 - /workspace/googleapis/google/ads/googleads/v21
ERROR: no such package 'google/ads/googleads/v21': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.
 - /workspace/googleapis/google/ads/googleads/v21

bazelisk build //google/ads/googleads/v21:googleads-ruby
uses the Bazel package googleapis/google/ads/googleads/v21.

googleapis/googleapis@d94a97e on August 7th deleted the Bazel package.

@suztomo

suztomo commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

#1339 is a fix for failing "gapic-generator-ruby-bazel-presubmit (gapic-images)" check.

@suztomo

suztomo commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

Next, check the effect in the three backticks blocks.

image

@suztomo

suztomo commented Sep 1, 2026

Copy link
Copy Markdown
Member Author

As seen in the preview of google-cloud-ruby https://github.com/googleapis/google-cloud-ruby/pull/36512/changes, this pull request fixes the preexisting issue of adding unnecessary \\ characters:

Screenshot 2026-09-01 at 6 23 25 PM

The proto file is at https://github.com/googleapis/googleapis/blob/02fc2b28bc702c8bb45735631ea0ad66893067fb/google/chat/v1/reaction.proto#L206

  // For example, the following queries are valid:
  //
  // ```
  // user.name = "users/{user}"
  // emoji.unicode = "🙂"
  // emoji.custom_emoji.uid = "{uid}"
  // emoji.unicode = "🙂" OR emoji.unicode = "👍"
  // emoji.unicode = "🙂" OR emoji.custom_emoji.uid = "{uid}"
  // emoji.unicode = "🙂" AND user.name = "users/{user}"
  // (emoji.unicode = "🙂" OR emoji.custom_emoji.uid = "{uid}")
  // AND user.name = "users/{user}"
  // ```

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants