Commit d9302a2
authored
fix: prevent ag2 infinite tool-call loop and agno request timeouts (CopilotKit#4242)
## Summary
- **ag2**: Add `max_consecutive_auto_reply=15` to `ConversableAgent` to
prevent infinite tool-call loops that flood Railway's log stream at 500+
logs/sec and starve health probes
- **agno**: Raise `OpenAIChat` timeout from default to 120s (aimock
proxy latency) and add `tool_call_limit=15` safety net
## Root Cause
**ag2 (9/12 ticks 503)**: AG2's `ConversableAgent` with
`human_input_mode="NEVER"` has no built-in turn limit. When the LLM
enters a repetitive tool-call pattern (observed: hundreds of
`get_weather(Tokyo)` calls per second), the agent loops indefinitely.
This generates 500+ log lines/sec hitting Railway's rate limit, the
agent becomes unresponsive to health probes, the watchdog kills it after
90s, and the container restarts. Cycle repeats.
**agno (3/12 ticks 503)**: The agno `OpenAIChat` model uses httpx's
default timeout, which is too short when requests route through the
aimock proxy under load. Observed `"Error in Agent run: Request timed
out."` in logs followed by agent restart. Less frequent than ag2 because
it only happens under concurrent load.
## Test plan
- [ ] Deploy ag2 and agno via CI (push to main triggers showcase_deploy)
- [ ] Monitor health for 30 min post-deploy: `curl
showcase-ag2-production.up.railway.app/api/health`
- [ ] Verify Railway logs no longer show rate-limit messages or infinite
tool-call output
- [ ] Verify watchdog kill events drop to zero4 files changed
Lines changed: 26 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
148 | 148 | | |
149 | 149 | | |
150 | 150 | | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
151 | 157 | | |
152 | 158 | | |
153 | 159 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
173 | 173 | | |
174 | 174 | | |
175 | 175 | | |
176 | | - | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
177 | 181 | | |
178 | 182 | | |
179 | 183 | | |
| |||
184 | 188 | | |
185 | 189 | | |
186 | 190 | | |
| 191 | + | |
| 192 | + | |
187 | 193 | | |
188 | 194 | | |
189 | 195 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
0 commit comments