AI OS: Prompt QA Factory standard

[sergstack]

Goal

Add a small AI-OS standard for reusable prompts: Prompt QA Factory.

Concept

Reusable prompts should not be stored only because they look good. They should have a simple quality path:

candidate -> test -> judge -> revise -> selected

Why

StreamDeck v2.8 showed that prompt quality improves when prompts are tested against real usage and optimized for low-friction UX.

Scope

Create a compact standard that explains:

• when a prompt is only a candidate;
• how to test a reusable prompt;
• how to judge it;
• when to revise;
• how to select the final version;
• how to record UX score and residual risks.

Suggested files

Add or update one or more of:

• PROMPT_QA_FACTORY.md
• GOAL_PACKS.md
• COMMAND_SURFACE.md
• ChatGPT/[AI OS]/Knowledge_Bundles/AIOS_04_GOAL_PACKS_AND_COMMAND_SURFACE.md
• ChatGPT/[AI OS]/Knowledge_Bundles/AIOS_05_SUPERVISED_AGENT_LOOPS.md
• relevant StreamDeck notes if needed

Required principles

• Goal Mode friendly.
• No atomic task burden for Sergey.
• Supervised loop only.
• Human acceptance required.
• No production automation.
• No sensitive data.
• No autonomous retrieval.
• No vector DB, embeddings, or semantic search.
• No auto-merge.

Minimal schema

# Prompt QA Record

## Prompt name
## Owner project
## Use case
## Candidate prompt
## Test cases
## Judge criteria
## Iterations
## Final selected prompt
## UX score
## Residual risks
## Acceptance status

Acceptance criteria

• Prompt QA Factory is documented as a reusable AI-OS pattern.
• It is clearly separate from autonomous agents or production automation.
• It references supervised loop governance.
• It explains how StreamDeck prompts, project prompts, Codex prompts, and memo prompts can use the same pattern.
• Validation checks pass.
• Draft PR opened for human review.

[sergstack]

Re-verified against current repo state while processing the issue queue.

Evidence that #72 is already implemented:

• PROMPT_QA_FACTORY.md exists and documents the reusable Prompt QA Factory pattern.
• It defines the required loop: candidate -> test -> judge -> revise -> selected.
• It covers candidate/test/judge/revise/selection rules, UX score, residual risks, supervised boundary, human acceptance, and the Prompt QA Record schema.
• It explicitly applies to StreamDeck prompts, ChatGPT Project prompts, Codex prompts, Judge/Revisor prompts, and Analytics memo prompts.
• GOAL_PACKS.md includes prompt_qa_factory.
• COMMAND_SURFACE.md includes Prompt QA.
• ChatGPT/[AI OS]/Knowledge_Bundles/AIOS_04_GOAL_PACKS_AND_COMMAND_SURFACE.md includes PROMPT_QA_FACTORY.md as a source and contains the bundle excerpt.
• ChatGPT/[AI OS]/Knowledge_Bundles/AIOS_05_SUPERVISED_AGENT_LOOPS.md references Prompt QA Factory under allowed supervised loops.

Checks run:

• python3 scripts/check_knowledge_bundles.py passed.
• python3 scripts/check_project_instructions_length.py passed.

Conclusion: no repo change is needed now. Recommended action: close #72 as already resolved, unless Sergey wants a new follow-up for live prompt QA evidence rather than the standard itself.

[sergstack]

Audit verification 2026-07-08: this issue appears complete. PROMPT_QA_FACTORY.md exists at repo root (added via #73) and covers the full requested scope — candidate -> test -> judge -> revise -> selected loop, status table, test/judge/revise rules, UX score and residual risks recording, supervised-only with human-owned acceptance. It is embedded in AIOS_04_GOAL_PACKS_AND_COMMAND_SURFACE.md and registered as the prompt_qa_factory Goal Pack in GOAL_PACKS.md. Recommend closing.

[sergstack]

Closing as already resolved per the audit evidence in #72 (comment). The Prompt QA Factory standard exists and is registered/embedded in the relevant AI-OS files.