[py] Publish a machine-readable API reference and ship guidance in the wheel

[AutomatedTester]

🔗 Related Issues

None yet. The review that motivated this is committed alongside it as docs/plans/llm-facing-documentation.md; This will be removed before merging

💥 What does this PR do?

A growing share of Selenium's documentation is read by machines rather than people — during training, during retrieval, and live over HTTP when a coding agent is asked to "write a Selenium test". When that reading goes badly the model falls back on stale priors and emits DesiredCapabilities, an executable_path, a third-party driver manager, and a sleep where a wait belongs. Users then attribute that code to Selenium.

This implements the Python tracks of the attached plan. The plan itself covers all five bindings; nothing here changes any other binding.

Canonical URL. Three copies of the Python API docs compete for recall — the release build on selenium.dev, the per-commit Read the Docs preview, and unofficial mirrors that outrank both — and none of them declared which was authoritative. Sets html_baseurl, so every page (including the Read the Docs build) carries rel=canonical pointing at the released reference. Rewrites the index wording to name that build as canonical and Read the Docs as a preview of the next release.

Machine-readable output, all at stable URLs under /selenium/docs/api/py/:

File	Source
llms.txt	new Sphinx extension, py/docs/source/_ext/llm_assets.py
sitemap.xml	same extension
objects.inv	already emitted by Sphinx; now documented rather than discarded
deprecations.json	new py/generate_deprecations.py

deprecations.json currently publishes 16 deprecated APIs, each with the replacement to use instead, parsed straight out of the warnings.warn messages. That mapping previously existed only as prose inside function bodies, where nothing but a running program could read it.

In-package guidance. selenium/llms.txt now ships inside the wheel, so a tool with the package on disk finds the project's own guidance instead of inferring it. The published llms.txt reads its rules back out of that same file, so there is one copy rather than two to drift apart.

🔧 Implementation Notes

deprecations.json is committed, not just generated. It lives in py/docs/source/_extra/ and is published verbatim via html_extra_path. Committing it means a PR that deprecates something shows the deprecation in its diff, and it means bazel build //py:docs works on a clean checkout. A unit test fails if it stops matching the source. This needed narrow negations in .gitignore, which otherwise ignores everything under py/docs/source/ to keep the autosummary stubs out.

The dataset deliberately records no line numbers. CI evaluates a pull request as its merge with trunk, so a published line number goes stale when trunk moves, not only when the branch changes — any PR inserting a line above a warnings.warn would fail the sync test. api already addresses each entry. A regression test pins this.

viewcode is replaced by linkcode — the two conflict, so this is a choice rather than an addition. viewcode rendered ~140 modules of source into the site as a second, non-canonical copy of what is already on GitHub, which is the same duplicate-content problem this PR is otherwise trying to fix. linkcode_resolve links each object to GitHub at the release tag with a line anchor. This does delete published pages (_modules/**) — nothing in the repo references them, but it is the one change here that removes rather than adds, and it is easy to revert on its own if maintainers disagree.

intersphinx adds a network fetch to the docs build. Inventories for the stdlib, trio and urllib3 are fetched at build time. Timeout is 5s and a failure degrades to plain literals rather than erroring, but it is a new network dependency for bazel build //py:docs and worth a maintainer's opinion.

The guidance rules themselves are deliberately versioned — each says which release the old form stopped being correct in (executable_path 4.10, find_element_by_* 4.3, options.headless 4.9), because a reader that has absorbed a decade of blog posts needs the version to know which memory to discard.

🤖 AI assistance

• No substantial AI assistance used
• AI assisted (complete below)
  • Tool(s): Claude Code (Opus 5)
  • What was generated: the documentation review in docs/plans/llm-facing-documentation.md, and a first pass at all of the Python changes — the Sphinx extension, the deprecation extractor, the conf.py changes, the unit tests, and the prose in py/selenium/llms.txt.
  • I reviewed all AI output and can explain the change

💡 Additional Considerations

Verification. Rebased onto trunk and verified locally with Bazel:

• bazel build //py:docs — build succeeded. The extension reports wrote llms.txt and sitemap.xml (138 pages); llms.txt, sitemap.xml, objects.inv and deprecations.json are all present in the HTML root, the published deprecations.json is byte-identical to the committed one, rel=canonical is emitted, _modules/ is correctly absent, and spot-checked linkcode anchors land on the right lines. No new Sphinx warnings come from any of the added code.
• bazel test //py:unit --test_size_filters=small — 34 tests pass, including both new files.
• bazel run //py:ruff-check -- --no-fix and //py:ruff-format — clean.

An earlier revision of this PR could not run any of the above locally (a TLS failure fetching mocha), so the description previously said the Sphinx build was unverified. That no longer holds — it has now been built and its output inspected.

Follow-up work, from the plan:

• The other four bindings need the same treatment (Track C), and their deprecations.json files need merging into one cross-binding dataset (D1).
• docs/api/llms.txt and a real landing page at /selenium/docs/api/ (A1, A2) need all five bindings first.
• Linking the API reference from the site llms.txt, and a Sitemap: reference for /selenium/docs/api/, are changes in SeleniumHQ/seleniumhq.github.io (B2, A4).
• Docs only regenerate at release, so this is invisible for a full cycle until nightly publishing exists (F1).

🔄 Types of changes

• New feature (non-breaking change which adds functionality and tests!)

[diemol]

I don't understand, is this an ARD or something just for Python?

[AutomatedTester]

This is something for python for now but happy to turn this into an ADR. At the moment it was more of a test to see how things would happen. Let me know your preference.

[diemol]

I don't think we need an ADR for this. As this one is about Python, it would be good to have an issue that tracks implementation for all of the languages. This makes sense to implement across languages.