Is your feature request related to a problem?
Kaapi enables Langfuse tracing by default when Langfuse credentials are added. While Langfuse is also required to run evaluations, it can lead to high-volume orgs quickly exhausting their Langfuse rate-limit/credit budget on traces alone. This results in 429 errors during evaluation resync, especially on the free tier. So there is no option of using solely evaluations without enabling tracing
Describe the solution you'd like
Add a flag to enable/disable Langfuse tracing per project, making tracing opt-in.
- Tracing should be off by default (or explicitly toggleable) for a project
- Orgs needing only evaluations can keep tracing disabled to save their rate-limit/credit budget
- Orgs wanting full observability can enable it when necessary
Is your feature request related to a problem?
Kaapi enables Langfuse tracing by default when Langfuse credentials are added. While Langfuse is also required to run evaluations, it can lead to high-volume orgs quickly exhausting their Langfuse rate-limit/credit budget on traces alone. This results in 429 errors during evaluation resync, especially on the free tier. So there is no option of using solely evaluations without enabling tracing
Describe the solution you'd like
Add a flag to enable/disable Langfuse tracing per project, making tracing opt-in.