Issue with `openai/claude-3.5-sonnet` in Aider

[minibikini]

When using Aider with claude-3.5-sonnet through the Copilot API proxy, requests are failing with a 500 Internal Server Error, followed by a 404 error.

Configuration

model: openai/claude-3.5-sonnet
openai-api-key: not-needed
openai-api-base: http://localhost:4141/
model-metadata-file: ~/.config/aider/.aider.model.metadata.json
weak-model: openai/gemini-2.0-flash-001
editor-model: openai/claude-3.5-sonnet
show-model-warnings: false
map-tokens: 2048
dark-mode: true

bun run start Output

~/repos/copilot-api master 
❯ bun run start
$ NODE_ENV=production bun run ./src/main.ts
◐ Initializing App directory...                                                       10:10:48 PM
✔ App directory initialized                                                          10:10:48 PM
◐ Initializing GitHub authentication...                                               10:10:48 PM
ℹ Logged in as "minibikini"                                                          10:10:49 PM
✔ GitHub authentication initialized                                                  10:10:49 PM
◐ Initializing Copilot token...                                                       10:10:49 PM
✔ Copilot token initialized                                                          10:10:49 PM
◐ Initializing Model information...                                                   10:10:49 PM
ℹ Available models:                                                                  10:10:50 PM
- gpt-3.5-turbo
- gpt-3.5-turbo-0613
- gpt-4
- gpt-4-0613
- gpt-4o
- gpt-4o-2024-05-13
- gpt-4-o-preview
- gpt-4o-2024-08-06
- text-embedding-ada-002
- text-embedding-3-small
- text-embedding-3-small-inference
- gpt-4o-mini
- gpt-4o-mini-2024-07-18
- o1
- o1-2024-12-17
- o3-mini
- o3-mini-2025-01-31
- o3-mini-paygo
- claude-3.5-sonnet
- gemini-2.0-flash-001
✔ Model information initialized                                                      10:10:50 PM
✔ Server started at http://localhost:4141                                            10:10:50 PM
<-- POST /chat/completions

 ERROR  Request failed: 500 Internal Server Error: Internal Server Error              10:12:10 PM


--> POST /chat/completions 404 568ms
192 |     const composed = compose(matchResult[0], this.errorHandler, this.#notFoundHandler);
193 |     return (async () => {
194 |       try {
195 |         const context = await composed(c);
196 |         if (!context.finalized) {
197 |           throw new Error(
                      ^
error: Context is not finalized. Did you forget to return a Response object or `await next()`?
      at <anonymous> (/Users/egor/repos/copilot-api/node_modules/hono/dist/hono-base.js:197:17)

<-- POST /chat/completions

Additional Context

• The other GitHub models seem to work fine.
• The same model works fine in Zed editor through GitHub's implementation
• Using model metadata from: https://github.com/ericc-ch/dotfiles/blob/main/aider/.aider.model.metadata.json

[ericc-ch]

Honestly I'm not too sure why this happens. But I got the same weird 500 error when trying out Cline with Claude 3.5. Then the reason probably isn't context length since I got this error when testing this issue

I think the least I can do for now is at least handle the error gracefully instead of just throwing it.

[minibikini]

Thanks for the update! Let's hope 3.7 will work better! :D

[ericc-ch]

Hey just want let you know I released a new version with proper logging now. Also the error should now be properly forwarded instead of just thrown inside the handler.

https://github.com/ericc-ch/copilot-api/tags

I also made a little Deno script to automatically update the aider metadata

https://github.com/ericc-ch/dotfiles/blob/main/scripts/update-aider-metadata.ts

Last thing, GitHub apparently only gives us "low" version of 3.7 sonnnet. Not clear how much thinking token budget is that though.

[minibikini]

Thanks for the update, and your work, Erick!

[minibikini]

Can't reproduce it anymore. It works just fine now!