[Bug] LiteLLM provider loses model selection and falls back to non-existent model after "Sync Models"

[awschmeder]

Summary

Two root bugs cause LiteLLM provider users to lose their selected model:

1. Cache key collision: All LiteLLM servers share one cache entry keyed only on "litellm". Switching between profiles backed by different LiteLLM servers silently serves the wrong model list, and the stale list persists across VS Code restarts via the disk cache. The same collision affects Ollama, LM Studio, Poe, DeepSeek, and Requesty. Additionally, for providers that enforce per-key model allowlists (LiteLLM, Poe, Requesty), two different API keys on the same server can see different model lists -- but share the same cache entry.

2. Silent fallback to hardcoded default: When the model list is empty (due to the collision above, a failed sync, or a transient error), useSelectedModel silently resets the configured model ID to claude-3-7-sonnet-20250219 -- a model that typically does not exist on user LiteLLM servers -- causing all requests to fail.

Steps to Reproduce

Bug 1a (URL cache key collision):

1. Configure Profile A: LiteLLM provider with baseUrl=http://server-a:4000. Select a model, e.g. my-custom-model.
2. Configure Profile B: LiteLLM provider with baseUrl=http://server-b:4000. The model list from server A is served for Profile B (wrong list).
3. Restart VS Code. Both profiles still see the wrong list -- the stale disk cache (litellm_models.json) persists server A's list.

Bug 1b (API key cache key collision):

1. Configure Profile A: LiteLLM provider with baseUrl=http://server:4000, apiKey=key-admin (full model list).
2. Configure Profile B: same baseUrl, apiKey=key-restricted (limited model list per LiteLLM allowlist).
3. Profile B sees Profile A's (admin) model list due to cache collision.

Bug 2 (silent fallback):

1. Configure LiteLLM with a valid server. Select my-custom-model.
2. Cause the model list to become empty (switch profiles, trigger a failed sync, etc.).
3. The active model silently resets to claude-3-7-sonnet-20250219; requests fail.
4. Clicking "Sync Models" shows success but the selection is not restored.

Root Cause

Bug 1 (Primary): Cache key collision for URL-scoped and key-scoped providers

Files: src/api/providers/fetchers/modelCache.ts, src/api/providers/router-provider.ts

The backend model cache uses the provider name alone as the cache key for all three layers:

memoryCache.set(provider, models)    // key: "litellm"
writeModels(provider, data)          // filename: "litellm_models.json"
inFlightRefresh.get(provider)        // key: "litellm"

Two categories of collision:

• URL collision: LiteLLM, Poe, DeepSeek, Ollama, LM Studio, and Requesty are URL-scoped providers. Two different servers (http://server-a:4000 and http://server-b:4000) collide on the same cache entry.
• Key collision: LiteLLM, Poe, and Requesty support per-key model allowlists. Two different API keys on the same server can see different model lists but share the same cache entry.

Additionally, router-provider.ts getModel() falls back to getModelsFromCache(this.name) -- passing only the plain provider name -- which misses the compound cache entry written by fetchModel(). The cold-start fallback never finds a cached URL-scoped result.

Bug 2 (Primary): Silent fallback to hardcoded default when model list is empty

Files: webview-ui/src/components/ui/hooks/useSelectedModel.ts, webview-ui/src/components/settings/providers/LiteLLM.tsx, webview-ui/src/components/settings/ApiOptions.tsx, src/core/webview/webviewMessageHandler.ts

Four sub-causes interact:

2a. getValidatedModelId returns defaultModelId when availableModels is {} -- indistinguishable from "model genuinely not found." Any configured model ID silently falls back to "claude-3-7-sonnet-20250219".

2b. The "Sync Models" button updates ExtensionStateContext (ModelPicker dropdown) but does NOT invalidate the React Query cache that useSelectedModel reads. React Query's stale (empty) litellm map persists after a successful sync.

2c. The debounced requestRouterModels in ApiOptions.tsx fires without credentials for litellm, causing the backend to fetch with stale saved credentials during credential editing.

2d. Even when ApiOptions.tsx sends current credentials in message.values, webviewMessageHandler.ts resolved them with || (saved config first), ignoring current field values whenever any saved key existed:

// Before fix -- saved config always wins
const litellmApiKey = apiConfiguration.litellmApiKey || message?.values?.litellmApiKey

This is opposite to the ?? pattern used by DeepSeek in the same function.

Impact

• Any user with two LiteLLM profiles pointing to different servers sees the wrong model list.
• Users with per-key model allowlists on a shared LiteLLM server see the wrong model list when switching API keys.
• The same URL collision affects Ollama, LM Studio, Poe, DeepSeek, and Requesty.
• A failed or stale fetch resets the selected model to a non-existent default, causing all requests to fail.
• The "Sync Models" button does not restore the correct selection.
• The issue persists across VS Code restarts.

Affected Files

• src/api/providers/fetchers/modelCache.ts
• src/api/providers/router-provider.ts
• src/core/webview/webviewMessageHandler.ts
• webview-ui/src/components/ui/hooks/useSelectedModel.ts
• webview-ui/src/components/settings/providers/LiteLLM.tsx
• webview-ui/src/components/settings/ApiOptions.tsx