Skip to content

feat: localization suggestion form variant — AI vs human translation provenance, 3-model consensus before submit #112

Description

@wcmchenry3-stack

In plain English

Add a distinct "Localization Suggestion" form type where users can suggest better translations or report wrong ones. Before submitting, three AI models independently validate the suggested translation, and the form clearly shows users which strings were auto-translated vs. human-maintained.

User story

As an app user who speaks a non-English language, I want to suggest a better translation for a string in the app so that the app reads naturally in my language — and I want to understand whether I'm correcting an AI translation or a human one.

Context

This is a distinct form variant layered on top of the core feedback form system. It adds:

  1. A "Localization Suggestion" type option in the feedback widget
  2. A translation provenance label (AI-driven vs. human-driven) shown per string
  3. A 3-model consensus check in the worker before the suggestion is filed as a GitHub issue

The 3-model consensus check happens server-side in the feedback worker (#109), not in the UI.

Part of: #46
Depends on: #109 (worker), #110 (gaming_app UI), #111 (book_app UI)

Acceptance criteria

Form UI (both gaming_app and book_app widgets)

  • "Localization Suggestion" appears as a third type option alongside Bug Report and Feature Request
  • When selected, form shows additional fields: Target locale (dropdown), Original string (read-only, pre-populated if triggered from a specific UI string), Suggested translation (text input), Reason/context (optional free text)
  • Each string displayed in the localization form shows a clear provenance badge: "AI-translated" or "Human-translated"
  • Tooltip or inline help text explains what "AI-translated" and "Human-translated" mean (e.g., "This string was automatically translated and may be imperfect" vs. "This string was reviewed and approved by a human translator")
  • User cannot submit without filling in Suggested translation
  • Pending submission state clearly communicates that 3-model validation is running ("Validating your suggestion with AI reviewers…")

Worker consensus check (#109 extension)

  • Worker detects type: "localization" and routes to consensus-check flow
  • Worker calls three independent model instances (can be same model, different temperature/prompt variations, or different models) to evaluate whether the suggested translation is accurate and appropriate
  • All three must return valid: true for the issue to be filed; if any returns valid: false, the submission is rejected with a human-readable explanation
  • Consensus results (model responses) are included in a collapsed <details> block in the filed GitHub issue for transparency
  • Filed issue labeled with localization in the target repo

Provenance data

  • Each app maintains a translation provenance map (JSON or config) that tags each i18n key as ai or human
  • The feedback widget reads this map to display the correct badge when triggered from a specific string
  • Provenance map format documented in the theme contract story (child story 6 of epic: in-app feedback & suggestion system #46)

Technical notes

  • Worker payload for localization type: { appId, type: "localization", locale, originalString, suggestedTranslation, i18nKey?, reason?, sessionLogs? }
  • Three-model consensus: use Anthropic API with 3 separate calls; prompt asks "Is this translation of [originalString] into [locale] accurate and natural? Respond with valid: true/false and a brief reason."
  • Consider using different temperatures (0.0, 0.3, 0.7) to get varied but independent assessments
  • Provenance map location per app: src/locales/provenance.json (keys match i18n key names, value: "ai" | "human")
  • i18n: all new form copy uses feedback namespace, add sub-keys under feedback.localization.*

Test coverage

Unit tests

  • Localization type selected → additional form fields appear
  • Provenance badge renders correctly for ai and human keys
  • Submit blocked when suggested translation is empty
  • Worker: all three models return valid → issue filed
  • Worker: one model returns invalid → submission rejected, explanation returned

E2E tests

  • Happy path: select Localization Suggestion → fill fields → submit → pending state shown → success with issue link
  • Rejection path: mock one model returning invalid → rejection message displayed, no issue filed

Performance

  • Three sequential Claude API calls should complete within 15 seconds (acceptable for a non-urgent feedback flow)

Regression

  • Switching from Localization to Bug Report and back resets localization-specific fields correctly

Documentation to update

  • CLAUDE.md — document localization suggestion flow and 3-model consensus in feedback worker section
  • Both app repos: document provenance.json format and update process

Unknowns / needs investigation

  • How apps currently expose i18n key information at the component level — needed to pre-populate the i18n key field when a user triggers feedback from a specific translated string
  • Whether gaming_app and book_app have a shared i18n key namespace or separate ones

Out of scope

  • Automatic application of approved translations (human review still required before merging translation files)
  • Crowdsourced voting on competing translation suggestions
  • Real-time translation preview

Dependencies

Definition of done

  • All acceptance criteria met
  • Test coverage added per above
  • Documentation updated
  • CI green (lint, type check, tests)
  • PR reviewed and merged to dev

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai-agentsClaude sub-agent related workenhancementNew feature or requestlocalizationLocalization and translation issuespriority:mediumMedium priorityrepo:allApplies to all repos in org

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions