Traceloop (OpenLLMetry) - Tracing LLMs with OpenTelemetry

Traceloop is a platform for monitoring and debugging the quality of your LLM outputs. It provides you with a way to track the performance of your LLM application; rollout changes with confidence; and debug issues in production. It is based on OpenTelemetry, so it can provide full visibility to your LLM requests, as well vector DB usage, and other infra in your stack.

Getting Started

Install the Traceloop SDK:

pip install traceloop-sdk

Use just 2 lines of code, to instantly log your LLM responses with OpenTelemetry:

Traceloop.init(app_name=<YOUR APP NAME>, disable_batch=True)
litellm.success_callback = ["traceloop"]

Make sure to properly set a destination to your traces. See OpenLLMetry docs for options.

To get better visualizations on how your code behaves, you may want to annotate specific parts of your LLM chain. See Traceloop docs on decorators for more information.

Exporting traces to other systems (e.g. Datadog, New Relic, and others)

Since OpenLLMetry uses OpenTelemetry to send data, you can easily export your traces to other systems, such as Datadog, New Relic, and others. See OpenLLMetry docs on exporters for more information.

Support

For any question or issue with integration you can reach out to the Traceloop team on Slack or via email.