Langfuse

Langfuse is an open-source platform designed to enhance the development and monitoring of large language models (LLMs). It offers a suite of tools for tracing, evaluating, managing prompts, and handling datasets, enabling developers to build and maintain AI applications more effectively.

Key Features:

• LLM Application & Agent Tracing: Capture detailed traces and graphs of agent activities, session tracking, user interactions, and monitor token usage and costs.
• Prompt Management: Manage prompt versions, fetch prompts, and compose prompts to optimize AI responses.
• Evaluation: Conduct evaluations using datasets, experiments, and human annotations to assess model performance.
• Metrics: Create custom dashboards and set up alerts to monitor various metrics related to AI applications.
• Collaboration: Support unlimited projects and users, facilitating team collaboration on AI projects.
• API: Provide an extensive public API with configurable rate limits for integration with other systems.
• Exports: Enable batch exports via the UI and integrate with platforms like PostHog and Mixpanel for data analysis.
• Support: Offer community support through GitHub and in-app support with varying response times based on the pricing tier.
• Security: Ensure data security with features like data region selection, single sign-on (SSO) options, and organization-level role-based access control (RBAC).
• Billing: Manage subscriptions and payments via credit card, with options for billing through AWS Marketplace or invoices for enterprise customers.
• Compliance: Adhere to standard terms and conditions, data processing agreements, and provide SOC2 Type II and ISO27001 reports for enterprise clients.

Pricing:

• Hobby (Free): Includes 50,000 units per month, 30 days of data access, and supports up to 2 users.
• Core ($29/month): Offers 100,000 units per month, 90 days of data access, unlimited users, and in-app support.
• Pro ($199/month): Provides 100,000 units per month, 3 years of data access, unlimited annotation queues, higher rate limits, and includes SOC2 and ISO27001 reports.
• Enterprise ($2,499/month): Includes all Pro features plus audit logs, SCIM API, custom rate limits, uptime SLA, support SLA, and a named lead support engineer.

Self-Hosting: Langfuse is available for self-hosting under the MIT license, allowing unlimited usage, users, and projects. Enterprise clients can opt for a custom-priced license that includes additional features like audit logs, retention policies, and support services.

Discounts: Langfuse offers discounts for early-stage startups, research students, non-profits, and open-source projects. Interested parties can apply by reaching out to support@langfuse.com.

Getting Started: To begin using Langfuse, you can view the public example project or sign up for a free account to try Langfuse with your own data. The Hobby plan is completely free and does not require a credit card.