PRD: UI-core contract and React modular rewrite

[quinCode]

PRD: UI-core contract and React modular rewrite

Problem Statement

LLMSTP-UI currently has unclear frontend/backend boundaries, native frontend code mixed with backend source structure, and partially migrated compatibility modules. The UI also knows too much about LLMSTP-core raw details such as CLI arguments, YAML layouts, progress event lines, and result file structure. This makes changes fragile: a small core contract change can force scattered UI edits, and adding or removing a UI capability is not localized.

The user wants a thorough architectural improvement rather than incremental patches. The target is a low-coupling, high-cohesion system where LLMSTP-core exposes a stable versioned contract, LLMSTP-UI adapts that contract through a single backend gateway, and the frontend is rebuilt with a modern framework using removable feature modules.

Solution

Rebuild LLMSTP-UI around a clear three-layer architecture:

• LLMSTP-core remains the execution engine and exposes a versioned UI-core contract through CLI commands, structured JSON, progress events, and result files.
• LLMSTP-UI backend becomes the only integration point with core through a CoreGateway. It normalizes core contract details into a stable UI application model and exposes a typed UI API.
• LLMSTP-UI frontend is rewritten as a React, TypeScript, and Vite single-page application. Feature modules are statically registered, capability-driven, and prevented from directly importing one another.

The first rewrite release is feature-equivalent. It preserves existing core workflows while removing long-term compatibility bridges, global shims, root-level scanner/viewer source modules, and the native frontend as the source of truth.

User Stories

1. As a UI user, I want to open the LLMSTP-UI web app and see the same core workflows as before, so that the rewrite does not remove existing functionality.
2. As a UI user, I want to discover available endpoints, attacks, judges, inputs, and output directories, so that I can configure a safety-test run.
3. As a UI user, I want configuration forms generated from backend-provided schemas, so that I can set model and run options without editing YAML manually.
4. As a UI user, I want the UI to hide or disable unavailable features based on capabilities, so that unsupported core features do not appear as broken controls.
5. As a UI user, I want to start a run from the configuration wizard, so that LLMSTP-core performs the safety-test workflow.
6. As a UI user, I want live task progress and sample-level status, so that I can monitor long-running runs.
7. As a UI user, I want to cancel a running task, so that I can stop an unwanted or failing run.
8. As a UI user, I want task history, hide, restore, and reuse-config behavior, so that I can manage previous runs.
9. As a UI user, I want to view result tables from completed runs, so that I can inspect prompts, categories, judge scores, judgements, and errors.
10. As a UI user, I want report generation and report viewing to keep working, so that I can export or read run summaries.
11. As a UI user, I want model dependency checks to continue working where supported, so that missing credentials or artifacts are reported before or during runs.
12. As a UI user, I want core changes that add optional fields or capabilities to not break existing UI workflows, so that routine core maintenance does not destabilize the app.
13. As a UI maintainer, I want React frontend code under a dedicated frontend source root, so that frontend and backend responsibilities are obvious from the project structure.
14. As a UI maintainer, I want FastAPI backend code under the llmstp_ui package, so that backend routes, schemas, services, persistence, and core integration are organized by responsibility.
15. As a UI maintainer, I want only CoreGateway to call LLMSTP-core, so that raw CLI, stdout, YAML, events, and result-file details are isolated.
16. As a UI maintainer, I want CoreGateway to expose intention-level methods, so that application services do not learn core command syntax.
17. As a UI maintainer, I want UI API schemas defined with Pydantic and exported through OpenAPI, so that the frontend can use generated TypeScript types and clients.
18. As a UI maintainer, I want each frontend feature to be registered through a typed manifest, so that adding or removing a page is localized.
19. As a UI maintainer, I want feature modules to declare required capabilities, so that UI availability is data-driven instead of hard-coded.
20. As a UI maintainer, I want feature modules to be forbidden from importing each other directly, so that changes remain local.
21. As a UI maintainer, I want backend capability modules to own routes, schemas, application services, and tests, so that backend behavior is cohesive.
22. As a UI maintainer, I want a fake core CLI for gateway tests, so that contract edge cases can be tested without running real inference.
23. As a UI maintainer, I want backend API tests that do not start React, so that route and schema failures are isolated.
24. As a UI maintainer, I want frontend feature tests that mock the generated UI API client, so that UI behavior is tested without core or backend process coupling.
25. As a UI maintainer, I want a small set of end-to-end smoke tests, so that the integrated path is still protected.
26. As a core maintainer, I want the UI-facing contract to be versioned and tested, so that core internals can evolve without silently breaking UI workflows.
27. As a core maintainer, I want LLMSTP-UI to use CLI contracts instead of importing core internals, so that core refactors remain hidden behind the CLI.
28. As an agent working on this codebase, I want ADRs and glossary terms to match the final architecture, so that future implementation work follows the agreed design.

Implementation Decisions

• LLMSTP-core remains the owner of the UI-core contract. The contract includes CLI invocation shape, configuration discovery, capability discovery, progress event schema, and result/report schema.
• LLMSTP-core exposes the UI-core contract through CLI commands and structured output rather than requiring LLMSTP-UI to import core internals as a Python library.
• LLMSTP-UI connects to LLMSTP-core only through a CoreGateway in the backend package.
• CoreGateway runs versioned core CLI subprocesses, parses structured JSON and progress events, normalizes result files, and maps all raw core details into the UI application model.
• CoreGateway exposes intention-level methods such as contract discovery, capability lookup, config catalog lookup, config schema lookup, run validation, run start, event reading, and result reading. It must not expose a generic command execution interface to application services.
• React frontend code must not depend on raw LLMSTP-core contracts, core YAML, stdout, progress event lines, or raw result files.
• LLMSTP-UI will separate source roots: backend source in the llmstp_ui Python package and frontend source under frontend.
• The root of LLMSTP-UI should contain project-level configuration and thin launch entry points only. Root-level scanner/viewer modules and compatibility entry points are not the long-term source structure.
• The frontend will be rebuilt as a React, TypeScript, and Vite single-page application served by FastAPI as static build output.
• The project will not use Next.js or SSR for the first rewrite because the UI is a local tool backed by an existing FastAPI server.
• Frontend features are plugin-like feature modules. Each module owns its route, UI state, queries, components, model types, and tests.
• Feature modules are statically registered through a typed feature registry. Dynamic discovery is out of scope unless third-party or user-installed modules become a real requirement.
• Feature modules expose a small manifest containing module identity, route information, navigation information, and required capabilities.
• Feature modules must not import each other directly. Shared behavior must move through app, shared, or generated API modules.
• Backend code is modular by capability, not by a generic plugin system. Each backend capability module owns routes, schemas, application service behavior, and tests.
• Frontend/backend integration is capability-driven. Backend capabilities define what frontend modules can show or enable.
• UI API contracts are defined by FastAPI/Pydantic schemas and exported through OpenAPI.
• TypeScript client and types are generated from OpenAPI. Frontend features consume the generated client instead of hand-written fetch calls or guessed response types.
• The first rewrite release is feature-equivalent. It migrates existing workflows without expanding product scope.
• Temporary migration code is allowed during implementation, but the final state must not retain old native UI compatibility bridges, global shims, or duplicated source structures.
• Existing LLMSTP-core UI contract tests should be preserved and expanded to cover capabilities, structured JSON, progress events, and result schema.
• Existing ADRs in both repositories are authoritative for the rewrite unless explicitly superseded.

Testing Decisions

• Tests should target the highest stable interface for each responsibility and avoid asserting on implementation details.
• LLMSTP-core contract tests verify CLI JSON envelopes, configuration discovery, run validation, progress event sequences, result schema, and capability metadata.
• CoreGateway tests in LLMSTP-UI use a fake core CLI to produce controlled contract versions, stdout logs, progress events, errors, unknown fields, missing optional fields, and extended result payloads.
• Backend API tests exercise FastAPI routes, Pydantic schemas, capability modules, and application services without starting React.
• Frontend feature tests exercise React feature behavior using mocked generated UI API clients, not real core or backend processes.
• Import boundary tests enforce that frontend feature modules do not import each other directly and do not import backend or core internals.
• Feature registry tests verify route uniqueness, navigation uniqueness, valid capability references, and module loadability.
• Deletion tests remove a feature from the registry and verify the shell and remaining features still work.
• End-to-end smoke tests cover a small number of integrated paths, such as loading the app, selecting config, starting a fake run, observing progress, and viewing results.
• Existing UI backend service tests and modular architecture tests are useful prior art, but should be migrated to the new source layout and stronger contract boundaries.

Out of Scope

• Multi-user support, authentication, authorization, or tenant isolation.
• Remote task queues or distributed execution.
• Plugin marketplace or third-party user-installed modules.
• Dynamic feature discovery in the first rewrite.
• Next.js, SSR, or a separate Node application server.
• Report editor or advanced report authoring workflows.
• Broad product expansion beyond the existing run configuration, task progress, task history, result viewing, report viewing, capability discovery, and model dependency check workflows.
• Replacing the LLMSTP-core execution engine or changing the core model/attack/judge architecture beyond what is needed to expose and test the UI-core contract.

Further Notes

The expected final architecture is a deliberate hard switch, not a permanent dual system. During implementation, temporary migration adapters may exist on the rewrite branch. Before the rewrite is considered complete, old native frontend source, long-term compatibility bridges, global shims, and duplicated root-level source modules should be removed.

The most important design constraint is that LLMSTP-core contract changes must be absorbed by CoreGateway whenever possible. The React frontend should consume stable UI application models and generated UI API clients, so optional core contract changes do not trigger broad frontend edits.

The completion standard includes: React/Vite as the only frontend source, clear frontend/backend roots, CoreGateway as the only core integration path, OpenAPI-generated frontend client, static feature registry, capability-driven feature availability, passing boundary/deletion/contract tests, and ADR/CONTEXT documentation matching the final architecture.