Skip to content

Add opt-in documentation discovery (llms.txt) to the MCP server - #217

Open
rohitkumarbhagat wants to merge 3 commits into
datacommonsorg:mainfrom
rohitkumarbhagat:llms_txt
Open

Add opt-in documentation discovery (llms.txt) to the MCP server#217
rohitkumarbhagat wants to merge 3 commits into
datacommonsorg:mainfrom
rohitkumarbhagat:llms_txt

Conversation

@rohitkumarbhagat

Copy link
Copy Markdown

Summary

Add an optional Data Commons documentation resource backed by
https://docs.datacommons.org/llms.txt.

When enabled, the MCP server:

  • Exposes the current documentation index as an MCP resource.
  • Adds a short server-level routing hint for API, client library, schema,
    dataset coverage, concept, and integration questions.
  • Keeps statistical data questions routed through the existing MCP tools and
    research skills.

The existing server.md remains unchanged from upstream.

Why

The MCP tools work well for statistical research, but agents also receive
questions about Data Commons APIs, client libraries, schema, datasets, and
integrations. Without a documentation discovery path, clients may answer from
memory or perform broad web searches.

Using a resource keeps the documentation out of the always-loaded server
instructions. The complete index is fetched only when a client reads the
resource, and the index can then direct the client to the relevant official
documentation pages.

The resource is disabled by default so Custom Data Commons deployments do not
automatically receive base Data Commons documentation guidance.

Rollout

DC_ENABLE_DOCUMENTATION_RESOURCE controls both resource registration and the
routing hint.

It defaults to false and is initially enabled only in the autopush
environment. Staging and production remain disabled while the behavior is
evaluated.

Validation

  • All 45 Data Commons MCP tests pass.
  • Ruff passes.
  • Cloud Build deployment YAML parses successfully.
  • A historical statistics test used the existing research skill and tools
    without reading the documentation resource.
  • A documentation test read the resource once, avoided downloading the index
    a second time, and generated correct Python V2 client code from official
    documentation.
  • server.md is byte-identical to the current upstream version.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the option to expose the official Data Commons documentation resource, controlled by the new DC_ENABLE_DOCUMENTATION_RESOURCE setting. When enabled, the server registers a documentation index resource and appends a routing hint to the server instructions. The feedback recommends moving the requests import from the module level to inside the registration function to defer loading and keep server startup fast when the feature is disabled.

Comment thread packages/datacommons-mcp/datacommons_mcp/server.py
Comment thread packages/datacommons-mcp/datacommons_mcp/server.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant