feat(Otel/Tracing): Generic support for other otel solutions (like: lansmith, langfuse, etc..)

[leonavevor]

Overview

Support for other otel/tracing solutions such as Lanagfuse, Langsmith, etc..

Motivation

This will help in troubleshooting and debugging agentic tasks/workflows. This will act like an x-ray that makes you see everything including tool calls, agent hand overs, system prompts, etc..

Examples:

• "trace llm calls"
• "debug tool calls"
• "quick troubleshooting using self-hosted/public otel compatible instances"

Use Cases

• "trace llm calls"
• "debug tool calls"
• "quick troubleshooting using self-hosted/public otel compatible instances"
• "troubleshooting agents with more visibility on opensource tools"
• "support shipping traces to custom endopoints"
• "also support shipping to litellm proxy"

Proposed Solution

If you have ideas on how to implement this, describe them here.

Examples:

file: .env

OTEL_IGNORE_CONTEXT_PROPAGATION="true" # set this to "true" to prevent litellm from propagating context to langfuse, which can cause issues if langfuse is also sending traces to other systems (e.g. jaeger) and causing trace loops. Only set this if you are using langfuse callbacks and are seeing issues with traces. For more details, see: https://docs.litellm.ai/docs/callbacks/langfuse_otel#troubleshooting-trace-loops
OTEL_EXPORTER_OTLP_ENDPOINT="${LANGFUSE_OTEL_HOST}/api/public/otel"
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic {LANGFUSE_AUTH}"

LANGFUSE_OTEL_HOST="https://localhost:3000"  # Default US region
#LANGFUSE_OTEL_HOST="https://us.cloud.langfuse.com"  # Default US region
#Other Langfuse data regions: https://cloud.langfuse.com (EU), https://jp.cloud.langfuse.com (Japan), https://hipaa.cloud.langfuse.com (HIPAA)
#LANGFUSE_OTEL_HOST="https://otel.my-langfuse.company.com"  # custom OTEL endpoint

# langsmith
LANGSMITH_API_KEY="***"

---