Export grammar and texts for AI analysis

[johnml1135]

Adds "Export Grammar and Texts for AI Analysis" to the Export dialog: pick some texts, pick a folder, and FLEx writes the project's HermitCrab grammar (HCGrammar.xml), one .flextext file per text, and an export-instructions.md there. The user's whole job after that is to drag and drop the files into ChatGPT (or Claude) and ask their question -- the instructions file carries the reference links, the answering rules, and the warning, so there is nothing to paste by hand.

The option is opt-in and invisible by default -- it appears only when FLEX_AI_EXPORT is set in the environment FieldWorks was launched with, so nothing changes for any user who hasn't gone looking for it. When it does appear, its description opens with a warning that AI answers can be very wrong while sounding certain and must be understood and confirmed before being applied.

The diff is bigger than the feature sounds because of one real constraint: InterlinVc/InterlinearExporter (needed to write .flextext) live in ITextDll, which already depends on xWorks (where the Export dialog lives) -- a reference the other way is a build-breaking cycle. So the text half goes through a new Publisher/Subscriber event answered by a listener registered in Main.xml, reusing the exact idiom AreaListener already uses for GetContentControlParameters rather than inventing something new. The grammar half is simpler: a new xWorks -> ParserCore reference (verified acyclic) reuses the same HCLoader/XmlLanguageWriter pipeline the existing GenerateHCConfig console tool already uses.

Where to look:

• The opt-in gate: AiExportGate (in FwUtils, same value rules as the existing FW_AVALONIA gate) and ExportDialog.AddFxts, which skips the template file outright so the row is absent from the list rather than greyed out.
• The instructions that travel with the data: AIExportInstructions.md under DistFiles/Language Explorer/Export Templates/, copied into the export folder as export-instructions.md by ExportDialog.CopyAiExportInstructions.
• xWorks -> ParserCore: no cycle back (ParserCore doesn't reference xWorks; only ParserUI, one layer up, references both).
• The pub/sub text-export path: FlexTextAIExportListener (in ITextDll) answering ExportTextsAsFlexText, registered in Main.xml.
• An unhandled HCLoader exception aborts the whole export, not just the grammar half -- deliberate, and now has a direct test.
• The summary MessageBox is shown by the UI thread after RunTask returns, not from the background task itself -- a real cross-thread bug avoided during implementation.

Deliberately not here: no guard against the live in-app parser running concurrently with this export (see the accordion below -- every sibling export has the same exposure today); no per-text format customization; no zip/incremental re-export.

Build/tests: full ./test.ps1 managed suite, clean except 12 pre-existing RenderComparisonTests pixel-baseline diffs and one native TestViews timing issue, both unrelated to any file this branch touches. gitlint clean. Manually verified end-to-end against the Sena 3 sample project.

---

Reading this a year from now -- start here

This branch had a design spec and a step-by-step implementation plan
(Docs/superpowers/specs/2026-08-15-grammar-text-export-design.md and
Docs/superpowers/plans/2026-08-15-grammar-text-export.md) while the work
was in progress. Both were deleted before merge -- the plan was a
task-by-task TDD checklist with no lasting value once the code exists, and
the spec's conclusions are either already expressed in code comments (where
the fieldworks-code-commenting standard permits it) or preserved here,
since that standard specifically bans .md file/section pointers from code
comments. This record is deliberately the only place some of this reasoning
survives -- if you're trying to understand why something here looks the
way it does, this is where to look, not the deleted files.

The layer cake

ExportDialog (xWorks)
  -> GrammarAndTextsAIExportSelectionDlg (WinForms picker: checkbox + Words/Analyses per text)
  -> FolderBrowserDialogAdapter (same idiom the existing LIFT export uses)
  -> ExportGrammarAndTextsForAI (background task, via ProgressDialogWithTask.RunTask)
       |
       +--> HCLoader.Load(cache) -> Language -> XmlLanguageWriter.Save -> HCGrammar.xml
       |    (same pipeline GenerateHCConfig.exe already uses)
       |
       +--> Publisher.Publish(EventConstants.ExportTextsAsFlexText, request)
              -> FlexTextAIExportListener (ITextDll, globally registered in Main.xml)
                   -> InterlinVc + InterlinLineChoices.DefaultChoices + InterlinearExporter
                        -> one <title>.flextext file per selected text

The grammar half never leaves xWorks; the text half necessarily crosses a
DLL boundary that a direct reference can't cross, which is the reason the
publish/subscribe hop exists at all.

Decisions, and why

• Pub/sub instead of a direct reference for the text half. ITextDll
  already has a ProjectReference to xWorks (it subclasses ExportDialog
  for InterlinearExportDialog). Adding the reverse reference to reach
  InterlinVc/InterlinearExporter from xWorks would be a genuine
  build-breaking cycle, not a style preference. The fix reuses the exact
  idiom ExportDialog.EnsureViewInfo() already uses for
  EventConstants.GetContentControlParameters (answered by AreaListener,
  registered globally in Main.xml) instead of inventing a new mechanism.
• An unhandled HCLoader/XmlLanguageWriter exception aborts the whole
  export; a per-text FLExText failure only skips that one text.
  HCLoader already catches per-item linguistic problems internally (bad
  phonemes, bad affix processes, etc.) and routes them to its
  IHCLoadErrorLogger argument, so anything that escapes indicates a real
  bug, not messy grammar data -- worth failing loudly for, unlike a single
  text's export failing independently of the others.
• Flat output folder, not a Texts/ subfolder. HCGrammar.xml and every
  .flextext file land directly in the chosen folder. Extensions already
  disambiguate them, and a flat folder was the simpler, explicitly requested
  layout.
• Two separate counts in the text picker -- Words and Analyses -- rather
  than one. Words counts every word-token occurrence regardless of whether
  it's been analyzed (a raw "how much text is this" signal); Analyses counts
  only the subset with a real IWfiAnalysis/IWfiGloss attached, whether by
  a human or an unreviewed parser guess (a "how much have I actually analyzed"
  signal). Conflating them would have hidden exactly the distinction someone
  picking texts for analysis actually cares about.
• No guard against the live in-app parser running concurrently with this
  export. Traced the actual mechanism: ParserConnection/ParserScheduler
  runs in-process against the same LcmCache, processing its queue via an
  IdleQueue tied to the UI thread's Application.Idle event -- and a modal
  ShowDialog() (which is what blocks the UI during this export) still
  raises Application.Idle in WinForms, so the parser can keep mutating
  wordform analyses while this export's background thread reads the same
  cache. This exposure already exists, unmitigated, in every sibling export
  (LIFT, Phonology, Grammar Sketch, the existing per-text FLExText export) --
  adding a bespoke guard to only this one would be inconsistent with that
  precedent and out of proportion to a risk the codebase has apparently
  tolerated for a long time. A deliberate choice, not an oversight.
• The instructions ship as a DistFiles file copied into the export, not as
  a string in code. export-instructions.md exists so the exported folder is
  self-contained: drop all of it into a chat and the assistant learns what each
  file is, which reference URL to read first, how to answer in plain language,
  and that its answers must be checked. Keeping the text in
  DistFiles/Language Explorer/Export Templates/AIExportInstructions.md puts it
  where the sibling exports already keep their non-code assets (the .xsl files
  next to it), lets the wording be corrected without a rebuild, and keeps a
  70-line markdown document out of a .resx. The installer harvests
  DistFiles wholesale via heat dir, so it ships with no installer change.
  A missing shipped file lands in the export summary instead of aborting -- the
  grammar and texts are still usable without it.
• The dialog description no longer carries the reference URLs. They moved
  into export-instructions.md, where the assistant reads them itself. Asking a
  user to copy a raw githubusercontent.com URL into a chat before asking a
  question was the most fragile step in the flow and the easiest one to skip.
• Hidden behind FLEX_AI_EXPORT, and absent rather than disabled. An
  export that hands linguistic data to a third-party chat service shouldn't
  be one misclick away for someone who never asked for it, so the default is
  off. ExportDialog.AddFxts skips the template file entirely when the
  variable is unset -- a greyed-out row would still advertise the feature and
  invite "how do I enable this?" without answering it. The value rules
  (unset/blank/0/false/off mean off; 1/true/yes mean on) copy the
  existing FW_AVALONIA gate in UIModeGates rather than inventing new
  spelling. AiExportGate is a separate class rather than a new method on
  UIModeGates because the two flags gate unrelated things and shouldn't be
  able to drift into each other.
• The AI warning lives in the export description; the plain-language rules
  live in the reference the LLM reads. Two different audiences, so two
  different places. The description (shown in the dialog before the user
  commits to anything) warns the human that answers can be confidently wrong
  and must be checked against their own data. Docs/ai-parser-help/README.md
  -- the file the description tells the user to paste into the chat -- gains a
  "How to answer" section addressed to the LLM: plain language for a field
  linguist, accuracy before simplification, every technical term defined on
  first use, no bare abbreviations, FLEx's own labels rather than engine class
  names, and an explicit flag on every guess. Those rules are adapted from
  this repo's own fieldworks-code-commenting standard, retargeted from "the
  next reader of the code" to "the linguist who asked".
• HCGrammar.xml, not Grammar.xml. "Grammar" is already overloaded
  three ways in this codebase: the Grammar Area (one of five top-level UI
  areas), the existing "Grammar Sketch" export (an unrelated human-readable
  linguistic-description document), and this HermitCrab-format grammar. The
  filename is self-disambiguating even sitting alone in a folder of
  .flextext files; CONTEXT.md now canonicalizes "HC grammar" as the term
  for the third one.

Paths not taken

• A per-form .resx for the picker dialog, matching some sibling dialogs'
  pattern of resources.ApplyResources(control, "control"). Used the shared
  xWorksStrings.resx instead (adding ksAIExportColumn*, ksOK,
  ksCancel, etc.), matching InterlinearExportDialog's simpler existing
  precedent of pulling column text from a shared strings class directly in
  code. Caught during review that this file's OK/Cancel buttons had
  initially been hardcoded rather than pulled from either -- fixed to use
  the shared resx.
• Reflection-based access to InterlinVc/InterlinearExporter via the
  same DynamicLoader.CreateObject(assemblyPath, className) pattern
  ExportDialog.EnsureViewInfo() already uses to reflectively load an
  Interlinear-area control for the existing "Grammar Sketch" export. Rejected
  because every actual method call and property access on those types would
  then need reflection too (not just construction), which is far messier
  than the pub/sub hop for a type this deeply used (InterlinVc.LineChoices,
  InterlinearExporter.Create/WriteBeginDocument/ExportDisplay/WriteEndDocument).
• Reimplementing FLExText writing directly in xWorks from
  IStText.ParagraphsOS/Segment.Analyses, avoiding the cross-DLL problem
  entirely. Rejected as substantial, error-prone duplication of exactly the
  logic InterlinearExporter already gets right (headwords, morpheme
  breakdowns, gloss lines, multiple writing systems).

Surprising findings

• HCLoader.Load has real, undocumented preconditions beyond "the cache
  exists": MorphologicalDataOA.ParserParameters must already be a valid XML
  fragment (XElement.Parse throws ArgumentNullException on a blank
  project's default null), at least one phoneme set must exist
  (PhonologicalDataOA.PhonemeSetsOS[0] is indexed directly), and that
  phoneme set needs morph (+) and word (#) boundary markers
  (LoadCharacterDefinitionTable throws KeyNotFoundException looking one
  up by representation otherwise). None of this is asserted anywhere in
  HCLoader itself; it was discovered by writing a test against a bare
  LcmCache.CreateCacheWithNewBlankLangProj and fixing each crash in turn.
• A namespace collision between SIL.LCModel.IText and the
  SIL.FieldWorks.IText namespace (ITextDll's own root namespace).
  Any test project that references both assemblies hits C#'s
  enclosing-namespace lookup finding the sibling namespace before the using SIL.LCModel; import, so bare IText is CS0118 ("is a namespace")
  wherever the containing code's own namespace nests under SIL.FieldWorks.
  Fixed by qualifying as SIL.LCModel.IText at each use site.
• InterlinearTestBase's fixture already runs each test inside an ambient
  undo task, unlike a bare LcmCache built directly in a test -- wrapping
  object creation in another UndoableUnitOfWorkHelper.Do inside a test that
  inherits from it throws InvalidOperationException: Nested tasks are not supported, matching the pattern already used by
  ComplexConcPatternModelTests (which creates objects with no UnitOfWork
  wrapper at all).

Evidence

• gitlint --ignore body-is-missing --commits origin/main..HEAD -- clean
  (exit 0) against every commit on the branch.
• ExportGrammarAndTextsForAI_CopiesTheShippedInstructionsIntoTheFolder
  asserts the shipped file actually lands in the export folder under the name
  the dialog's description promises, rather than trusting the copy.
• The opt-in gate has its own tests: AiExportGateTests (which values count
  as opted in, plus one asserting the process-environment default is off) and
  ExportDialogAiGateTests (which template shapes the gate applies to).
  Both pass, as do the pre-existing AI-export tests, with the full solution
  building 0 warnings / 0 errors.
• ./test.ps1 (full managed suite, -SkipNative) -- run repeatedly across
  the branch's implementation and again after the rebase onto the current
  origin/main tip and the two review-driven fixes. The only failures in
  any run are the same 12 RenderComparisonTests pixel-baseline diffs
  (complex, custom-heavy, footnote-heavy, many-paragraphs, medium,
  multi-book) and one native TestViews timing/rendering issue every
  time, none of which touch xWorks, ITextDll, FwUtils, or ParserCore
  -- not re-run against a clean origin/main checkout to get a formal
  before/after baseline, but consistent and content-unrelated across every
  run this branch produced.
• New classes each have focused unit tests: GrammarTextsAIExportHelpers
  (word/analysis counting, filename sanitization), GrammarExportLoadLogger,
  GrammarAndTextsAIExportSelectionDlg (selection persistence),
  ExportDialog.ExportGrammarAndTextsForAI (grammar write, per-text-failure
  recording, exception propagation), FlexTextAIExportListener (actual
  .flextext file production), and ExportTextsAsFlexTextRequest.
• Manually verified in the running app (fieldworks-winapp skill, Sena 3
  sample project, Legacy/WinForms UI mode): the new option appears in the
  Lexicon area's Export dialog, the picker and folder browser both work, and
  the export produces the expected files.
• Docs/ai-parser-help/'s 92 relative links and 4 anchors were verified to
  resolve post-copy (a PowerShell link-target-existence pass), so it needed
  no link surgery when moved from the sillsdev/machine repo.

---

This change is 

[github-actions]

NUnit Tests

    1 files  ± 0      1 suites  ±0   11m 24s ⏱️ + 1m 0s
5 808 tests +34  5 727 ✅ +34  81 💤 ±0  0 ❌ ±0 
5 817 runs  +34  5 736 ✅ +34  81 💤 ±0  0 ❌ ±0 

Results for commit 8530948. ± Comparison against base commit 99858db.

♻️ This comment has been updated with latest results.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 61.76471% with 117 lines in your changes missing coverage. Please review.
✅ Project coverage is 38.08%. Comparing base (99858db) to head (8530948).

Files with missing lines	Patch %	Lines
Src/xWorks/ExportDialog.cs	37.34%	41 Missing and 11 partials ⚠️
...rc/LexText/Interlinear/FlexTextAIExportListener.cs	45.83%	24 Missing and 2 partials ⚠️
Src/xWorks/GrammarExportLoadLogger.cs	27.77%	25 Missing and 1 partial ⚠️
Src/xWorks/GrammarAndTextsAIExportSelectionDlg.cs	78.78%	6 Missing and 1 partial ⚠️
Src/xWorks/GrammarTextsAIExportHelpers.cs	90.47%	2 Missing and 2 partials ⚠️
...ks/GrammarAndTextsAIExportSelectionDlg.Designer.cs	95.91%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1070      +/-   ##
==========================================
+ Coverage   38.04%   38.08%   +0.03%     
==========================================
  Files        1499     1506       +7     
  Lines      350127   350433     +306     
  Branches    40239    40272      +33     
==========================================
+ Hits       133215   133460     +245     
- Misses     187625   187672      +47     
- Partials    29287    29301      +14     

Files with missing lines	Coverage Δ
Src/Common/FwUtils/AiExportGate.cs	100.00% <100.00%> (ø)
Src/Common/FwUtils/ExportTextsAsFlexTextRequest.cs	100.00% <100.00%> (ø)
...ks/GrammarAndTextsAIExportSelectionDlg.Designer.cs	95.91% <95.91%> (ø)
Src/xWorks/GrammarTextsAIExportHelpers.cs	90.47% <90.47%> (ø)
Src/xWorks/GrammarAndTextsAIExportSelectionDlg.cs	78.78% <78.78%> (ø)
...rc/LexText/Interlinear/FlexTextAIExportListener.cs	45.83% <45.83%> (ø)
Src/xWorks/GrammarExportLoadLogger.cs	27.77% <27.77%> (ø)
Src/xWorks/ExportDialog.cs	15.21% <37.34%> (+1.32%)	⬆️

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.