Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,9 @@ jobs:
- name: Install Firefox for Puppeteer
if: matrix.browser == 'firefox'
run: bunx puppeteer browsers install firefox
- name: Google Docs cross-world fixtures (not live Docs)
if: matrix.browser == 'chrome'
run: bun run test:e2e:docs
- run: bun run test:e2e:full --platform=${{ matrix.browser }}
env:
CI: "true"
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,9 @@ Site profiles never bypass domain enable/disable logic. If a domain is blocked b

## Compatibility

FluentTyper works on most websites. Some rich text editors (for example Google Docs) can be partially or fully incompatible.
FluentTyper works on most websites, including Google Docs.

Google Docs support activates on any document edit page when FluentTyper is enabled for docs.google.com. Suggestions are applied through a single synthetic plain-text paste into the editor, and the edit is verified against the document model before local learning records it. See [docs/google-docs-integration.md](docs/google-docs-integration.md) for details and limits. Other canvas-based rich text editors can still be partially or fully incompatible.

If you hit an unsupported site, please open a bug report so compatibility can be improved.

Expand Down
8 changes: 8 additions & 0 deletions docs/agents/runtime-features.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,11 @@ When adding a user-facing setting:
- Production logging should stay minimal, typically warn and error only.
- Do not log full user text content.
- Guard extra debug logging behind development mode or the existing logging level controls.

## Google Docs

- Code lives in `src/adapters/chrome/content-script/google-docs/`. `ContentRuntimeController` creates `GoogleDocsAdapter` only on a top-level Docs edit page; the generic `SuggestionManager` is disabled only inside the hidden `iframe.docs-texteventtarget-iframe`.
- `GoogleDocsMainWorld` runs from `content_script_main_world_start.ts` (MAIN world, `document_start`) and sets `window._docs_annotate_canvas_by_ext` to FluentTyper's own extension ID so Docs exposes `_docs_annotate_getAnnotatedText`. Never impersonate another extension's ID.
- Isolated and MAIN worlds talk only through `CustomEvent`s with JSON string payloads; the bridge exposes no extension APIs to the page.
- Edits are single-use-token transactions: read model, select the minimal range, dispatch one synthetic plain-text paste, verify text. Unverified edits are never retried.
- Tests: `bun test tests/GoogleDocsModel.test.ts tests/GoogleDocsTransaction.test.ts` and `bun run test:e2e:docs` (Chromium fixture with real MAIN/isolated worlds, mocked Docs API). Live check: `bun run test:e2e:docs:live -- --help`.
169 changes: 169 additions & 0 deletions docs/google-docs-integration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
# Google Docs integration candidate

Base: `058cde147b78a36e4ab5c3d4e52a4f44cdfe1b74`. Date: 2026-09-15.

**Status: implemented and locally tested; NOT approved for a production rollout.**
The live Google Docs editor could not be reached in the development environment.
An actual navigation failed with `net::ERR_BLOCKED_BY_ADMINISTRATOR`. Browser
fixtures below are explicitly simulated editors, not a substitute for live validation.

## Install and enable

This is a replacement for the earlier experimental patch, not a patch to stack on it.
Apply it to a clean checkout of the base, or let Git perform a reviewed three-way merge
on a newer branch. Do not discard unrelated local work to apply it.

```sh
git switch -c candidate/google-docs-integration
git apply --check /path/to/fluenttyper-google-docs-v2.patch
git apply /path/to/fluenttyper-google-docs-v2.patch
bun install --frozen-lockfile
bun run check
bun run test
bun run check:e2e:coverage
bun run test:e2e:docs
bun run build --platform=chrome
```

For manual testing use a dedicated browser profile and a NEW, EMPTY, DISPOSABLE
document. Load the unpacked build and enable FluentTyper on docs.google.com; the
adapter activates on any document edit URL. The annotation bootstrap runs at
`document_start`, so reload the document after enabling the site. Normal
title/comment helpers stay active.

Passing a build in `production` mode does not mean this private-API integration is
production-certified.
No new permissions, dependencies, network services, clipboard reads or clipboard
writes are added. Predictions use the existing local backend and its settings.
The MAIN-world bridge exposes no extension APIs to the page.
Known limitation: the keyboard bridge publishes the current single-use token in a DOM
attribute on the input iframe, so a script already running on docs.google.com could forge
an acceptance of a visible suggestion. Such a script can already edit the document through
the same page API; the only extension-side effect is a spurious local learning record.

## Architecture and feature mapping

The Google Docs adapter receives logical text and selection from the page-side
annotated-text capability. It supplies explicit text context to the shared
`SuggestionPredictionCoordinator`; no fake textarea is edited to manufacture success.
The background predictor, snippet expansion, language selection, user dictionary,
site configuration, and personalization settings keep their existing code paths.

| Requirement | Implemented behavior | Evidence / limitation |
| ----------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| Prefix and non-prefix spelling completion | Plan a complete range replacement, including a suffix under a mid-word caret. | Model and cross-world fixtures pass. Predictor replies are stubbed in browser fixtures. |
| Selected-text replacement | Explicit manual invocation supports a bounded forward or reversed selection. | Never autonomously replaces a user's noncollapsed selection. |
| Grammar and rewrites | Reuses the complete configured local grammar catalog; paragraph-scoped triggers and custom caret offsets. | Not a new document-wide AI grammar model. Shared catalog regression suite passes; one automatic correction is browser-tested. |
| Snippets and dynamic variables | Existing background expansion feeds the same pipeline; accepts multiline expansion text without flattening whitespace. | Multiline insertion is fixture-tested. Real background-to-Docs expansion and formatting require live tests. |
| Next-word prediction | Shared coordinator accepts empty-prefix requests and inserts without deleting the following word. | Model and browser fixtures pass. |
| Inline mode | Reuses the owned ghost presenter for a safe suffix at a line end. Fixes the thin-caret 1px width clamp. | Spelling rewrites, midtext, RTL, multiline or ambiguous geometry use the themed menu. Full canvas-mirror inline parity is NOT implemented. |
| Keyboard and mouse | Configured Tab, Enter, Space, arrows, Escape and digit shortcuts; synchronous early key acknowledgment, mouse focus preservation. | Trusted keyboard events cross actual MAIN/isolated contexts in Chromium fixtures. |
| Themes | Existing Shadow DOM menu, typography service and theme variables. | Custom theme regression passes. Exact Docs font/zoom alignment needs live review. |
| Statistics / learning | Existing local services run only after observed model success; deduplicated late acknowledgment and exact last-edit reversal observation. | No claim that event dispatch means acceptance. Undo/redo journal observation is fixture-tested, not native Docs undo grouping. |
| Titles and comments | Generic helper remains active in top-level ordinary editable fields; only the hidden Docs input iframe is excluded. | Real generic SuggestionManager exercised on fixture input and textarea. Actual Docs comment DOM remains a live check. |
| Accessibility / localization | Keyboard access, option semantics, polite selection announcements, visible failure status and nine UI-language translations. | Does not overwrite Docs' editable ARIA attributes. Not screen-reader/WCAG certified. |
| IME | Composition guards across frames, settling delays, key-code 229 avoidance; no acceptance while composing. | Synthetic composition-event fixture only. Native platform IMEs are a release gate. |
| RTL and multiple visible carets | Logical Unicode offsets and grapheme-safe edits; direction-aware menu. A fixed palette avoids guessing a collaborator's caret. | Does not prove visual bidi shaping/caret affinity or identify every local caret. |
| Collaboration | Fresh model/selection/scope/input/interaction checks before selection and before paste; stale work is discarded. | No revision-aware atomic transaction exists in this implementation. Concurrent operation ordering is NOT proven. |
| Document tabs | Pending work is bound to full edit URL, including tab query parameters, and input-object identity. | Same-URL scope changes or private API topology not separately identified. Real multi-tab behavior remains unverified. |
| Tables / footnotes / mixed formatting | Rejects edits crossing exposed object/control markers; minimizes the changed range on grapheme boundaries. | Not a structural document model. Text parity cannot prove structure or formatting preservation. |
| Offline | No new network dependency; offline browser fixture passes. | Does not verify Google Docs' offline cache, save synchronization or persistence. |
| Smart Compose / other extensions | Respects configured preference for visible `aria-controls` native popups. | Canvas Smart Compose and arbitrary third-party overlays are NOT reliably detected. Disable competitors in the initial live test profile. |

## Live editor quirks (verified in Chrome against real Google Docs)

- `setSelection` blurs the editable inside `iframe.docs-texteventtarget-iframe` while the
frame itself stays focused. The bridge refocuses the editable before pasting and treats a
blurred editable as active as long as the frame is.
- Docs strips leading/trailing ASCII spaces from a plain-text paste but converts NBSP to a
regular space. Edge spaces are sent as NBSP; the verified model still contains `" "`.
- An unverified write blocks the adapter only until the next trusted user interaction; it is
then forgotten without being retried or learned.

## Edit transaction invariants

`GoogleDocsModel.ts` validates metadata, Unicode boundaries and edit ranges.
`GoogleDocsTransaction.ts` owns single-use tokens, model validation, the edit journal,
selection restoration and acknowledgments. `GoogleDocsMainWorld.ts` adapts the private
API and iframe input realm. `GoogleDocsBridgeClient.ts` uses bounded JSON requests.
`GoogleDocsAdapter.ts` connects predictions, grammar, UI and local learning.

A token contains a fresh full logical model, raw selection, focused input object,
full URL, interaction generation and expiry. Only bounded context crosses to the
content-script prediction adapter. Limits are 2,000,000 UTF-16 code units per document,
16,384 per edit/selection, and 8,192 of context per side. Unsupported states fail closed.

An edit token is consumed before asynchronous work. The adapter re-reads the model,
selects only the minimum changed span, then rechecks text, selection and identity.
It dispatches **one synthetic plain-text paste** in the input iframe's event realm.
Synthetic paste is still untrusted and may be ignored. Neither `dispatchEvent` nor
`execCommand` return values are accepted as proof of insertion.

The bridge verifies the expected raw logical text independently. A delayed exact
acknowledgment can recover the session without another paste. An ambiguous write
retains its journal across adapter disable/cancel; it is never blindly retried,
fuzzily relocated, repaired by rewriting a block, or rolled back over user edits.
Native undo/redo shortcuts are not hijacked. The journal observes an exact last-edit
text reversal to reverse local personalization; it does not certify native undo units.

These checks reduce races but are **not atomic compare-and-swap** against Google's
collaborative model. Text equality does not prove formatting, revision identity,
persistence, or that an independent identical edit was not made concurrently.

## Automated tests and continuous integration

```sh
bun run check
bun run test
bun run test:e2e:docs
bun run check:e2e:coverage
bun run test:e2e
bun run test:e2e:full
bun run test:e2e:full --platform=firefox
```

The Docs fixture suite compiles the real adapter and shared services, creates MAIN
and isolated Chromium worlds, and sends real browser keyboard events. Only the editor's
annotated API, its canvas model and prediction responses are mocked. It does not load
the packaged extension's service worker. Its URL facade and randomUUID fallback are
strictly fixture code, never included in the extension build. The fixture runs
in-memory; no browser policy or live-site access restriction is bypassed.

The Chrome full-regression CI job now runs this suite. Coverage matrix and baseline
IDs are updated with unit/integration mappings rather than fictitious live coverage.
There is no equivalent Firefox cross-world fixture yet. Firefox builds and ordinary
regression tests retain their existing path.

## Real-document check

```sh
bun run test:e2e:docs:live -- --help
```

The supplied operator-assisted script requires an explicit disposable document URL,
a dedicated local profile, an unpacked extension and `--allow-edits`. Authentication
happens in your local browser, not through shared credentials. The script refuses a
nonempty logical document, tests an actual offered completion and native undo/redo,
and asks the operator to verify Saved to Drive before testing reload persistence.
It writes a local report and never retries a failed edit. **It has not been run
against live Google Docs in this environment.** It is a smoke check, not the full matrix.

Verified live in Chrome (2026-09-15): the annotated API activates for FluentTyper's own
extension ID, suggestions render at the caret, Tab acceptance is verified as applied, and
native undo/redo work. Still unverified: Edge/Firefox, all supported keyboard settings, snippets/dynamic variables,
user dictionaries, language/site profiles, native undo/redo, mixed formatting and
links, headings/lists/tables/footnotes, multiple tabs, two collaborating accounts,
disjoint and overlapping remote edits, zoom/scroll, RTL, native IMEs, screen readers,
Smart Compose/competing extensions, offline/reconnection, save and reload. Do not
mark missing evidence passed because a fixture or build succeeded.

## References

- Harper's implementation overview: https://writewithharper.com/docs/contributors/chrome-extension
- Harper's page bridge: https://github.com/Automattic/harper/blob/master/packages/chrome-plugin/public/google-docs-bridge.js
- Chrome content-script world/lifecycle documentation: https://developer.chrome.com/docs/extensions/develop/concepts/content-scripts

The annotated-text surface is not a stable public Google Docs editing API. Access
for FluentTyper's own extension IDs and compatibility with current Docs must be
validated in a real browser. This implementation never impersonates Harper or another
extension and never overwrites an already-set annotation bootstrap flag.
4 changes: 3 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@
"fix": "bun run lint:fix && bun run format",
"format": "prettier . --cache --write",
"format:check": "prettier . --cache --check",
"bump": "bun pm version"
"bump": "bun pm version",
"test:e2e:docs": "bun test --timeout=12000 tests/e2e/google-docs.e2e.test.ts",
"test:e2e:docs:live": "bun scripts/test-google-docs-live.ts"
},
"repository": {
"type": "git",
Expand Down
Loading
Loading