[Question] Harness + RemoteFilesystemSpec: session jsonl written to local workspace only, not mirrored to BaseStore — multi-replica deployment concern

[likebean]

Background

We are building a server-hosted, multi-tenant Agent platform (Spring Boot + AG-UI + CopilotKit), not a personal/laptop Harness setup.

• Runtime: Spring Boot service (/agui/run), intended for K8s multi-replica deployment.
• Tenancy: many logged-in users; RuntimeContext.userId injected from server-side auth (not client-trusted).
• Why RemoteFilesystemSpec + DistributedStore: by design for shared MySQL (agentscope_sessions + agentscope_store) and Redis-backed HTTP sessions — not because we want ./data/agentscope-workspace on pod disk as source of truth.
• Ephemeral pod disk: workspace-dir (including .index/workspace.db and session *.jsonl) lives on container/local server disk. In production we cannot rely on per-pod PVC as the primary session archive; we expect Harness offload/mirror to BaseStore as documented for Filesystem mode 1 (RemoteFilesystemSpec).

We integrate HarnessAgent + DistributedStore (MySQL JdbcStore + MysqlAgentStateStore) + AG-UI Spring Boot starter + CopilotKit for this server-side deployment.

Configuration (simplified):

agentscope:
  harness:
    workspace-dir: ./data/agentscope-workspace
    storage:
      session-table: agentscope_sessions
      workspace-table: agentscope_store
  agui:
    server-side-memory: false

HarnessAgent.builder()
    .agentId("platform_assistant")
    .name("Platform Assistant") // display name differs from agentId
    .distributedStore(distributedStore)
    .filesystem(new RemoteFilesystemSpec().isolationScope(IsolationScope.USER))
    ...

What works (MySQL)

• AgentState / conversation context → agentscope_sessions (session_id = userId:threadId) ✅
• MEMORY.md / memory/YYYY-MM-DD.md → agentscope_store (Remote KV) ✅

App-layer workaround (server UI): we expose custom REST (/api/chat/threads, messages from AgentStateStore) because AG-UI does not hydrate CopilotKit from persisted store when server-side-memory: false.

Problem: session offload files stay on local disk

After AG-UI conversations, we see session files under local workspace on the server (not end-user machines):

{workspace}/{userId}/agents/Platform Assistant/sessions/{threadId}.jsonl
{workspace}/{userId}/agents/Platform Assistant/sessions/{threadId}.log.jsonl

But no corresponding entries appear in agentscope_store for agents/platform_assistant/sessions/.

From code reading:

1. SessionTree always appends to local files first, then async mirror to AbstractFilesystem (see javadoc: "local file is the working copy").
2. MemoryFlushMiddleware uses agent.getName() (e.g. Platform Assistant) for offload paths, while RemoteFilesystemSpec routes agents/{agentId}/sessions/ with agentId = platform_assistant.
3. Path mismatch may cause mirror to hit LocalFilesystem layer instead of RemoteFilesystem → BaseStore.

Impact (server-side / multi-replica)

• Multi-replica / K8s: each pod has its own data/agentscope-workspace; session jsonl is not shared across replicas.
• CopilotKit / AG-UI UI history: no MESSAGES_SNAPSHOT from persisted store; we work around via agentscope_sessions, but jsonl archive gap remains for tools/audit.
• session_search tool: may not find history written on another pod.
• Contradicts mode-1 intent: we chose RemoteFilesystemSpec specifically for server deployment; local-only jsonl undermines that contract.

Questions

1. Is local session jsonl intended as mandatory working copy even when RemoteFilesystemSpec + DistributedStore are configured? Or should offload go directly to BaseStore in distributed/server mode?
2. Should middleware use stable agentId (from HarnessAgent.builder().agentId(...)) instead of agent.getName() for session paths, to match RemoteFilesystemSpec routes?
3. Recommended pattern for server multi-replica:
   • shared PVC for workspace-dir?
   • or emit AG-UI MESSAGES_SNAPSHOT from AgentStateStore on /agui/run?
   • or rely only on agentscope_sessions.context and treat jsonl as optional archive?
4. Any plan to integrate AG-UI MESSAGES_SNAPSHOT with Harness AgentStateStore for thread history hydration?

Environment

• agentscope-java: 2.0.0-local (main)
• Spring Boot 3.4.x, AG-UI starter, HarnessAgent, MySQL store
• AG-UI server-side-memory: false (avoid in-memory ThreadSessionManager for multi-replica)
• Deployment target: server-side Agent platform (not personal desktop)

Related: #2014 (userId injection for AG-UI + Harness)

Thank you!