Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Ragas for evaluation #22

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ We also included [Perplexity and Cursor.sh prompts](#formatting-prompt-templates
| [Braintrust Data](https://www.braintrustdata.com/) | An AI product stack featuring evaluations, prompt playgrounds, continuous integration, dataset management, and access to various AI models through a single API | <a href=https://github.com/braintrustdata/braintrust-proxy><img src="https://img.shields.io/github/stars/braintrustdata/braintrust-proxy?style=social" width=90/></a> | <a href=https://pypi.org/project/braintrust><img src="https://img.shields.io/pypi/dw/braintrust" width=150/></a> |
| [Arize AI](https://arize.com/) | An observability platform for both LLMs and supervised ML | <a href=https://github.com/Arize-ai><img src="https://img.shields.io/github/stars/Arize-ai?style=social" width=90/></a> | <a href=https://pypi.org/project/arize><img src="https://img.shields.io/pypi/dw/arize" width=150/></a> |
| [Weights & Biases](https://wandb.ai) | An MLOps platform for streamlining ML workflows | <a href=https://github.com/wandb/wandb><img src="https://img.shields.io/github/stars/wandb/wandb?style=social" width=90/></a> | <a href=https://pypi.org/project/wandb><img src="https://img.shields.io/pypi/dw/wandb" width=150/></a> |
| [MLflow](https://mlflow.org/docs/latest/llms/index.html#) | A platform to streamline ML development | <a href=https://github.com/mlflow/mlflow><img src="https://img.shields.io/github/stars/mlflow/mlflow?style=social" width=90/></a> | <a href=https://pypi.org/project/mlflow><img src="https://img.shields.io/pypi/dw/mlflow" width=150/></a> |
| [MLflow](https://mlflow.org/docs/latest/llm-tracking.html) | A platform to streamline ML development | <a href=https://github.com/mlflow/mlflow><img src="https://img.shields.io/github/stars/mlflow/mlflow?style=social" width=90/></a> | <a href=https://pypi.org/project/mlflow><img src="https://img.shields.io/pypi/dw/mlflow" width=150/></a> |
| [PromptLayer](https://promptlayer.com/) | A platform for tracking, managing, and sharing LLM prompt engineering | <a href=https://github.com/MagnivOrg/prompt-layer-library><img src="https://img.shields.io/github/stars/MagnivOrg/prompt-layer-library?style=social" width=90/></a> | <a href=https://pypi.org/project/promptlayer><img src="https://img.shields.io/pypi/dw/promptlayer" width=150/></a> |
| [Helicone](https://helicone.ai/) | An open-source observability platform for LLMs | <a href=https://github.com/Helicone/helicone><img src="https://img.shields.io/github/stars/Helicone/helicone?style=social" width=90/></a> | <a href=https://pypi.org/project/helicone><img src="https://img.shields.io/pypi/dw/helicone" width=150/></a> |
| [Quotient AI](https://www.quotientai.co/) | Quotient AI is a platform for evaluating AI products on real-world use-cases, during research, development, and in production | N/A | N/A |
Expand All @@ -126,6 +126,7 @@ We also included [Perplexity and Cursor.sh prompts](#formatting-prompt-templates
| [Patronus AI](https://www.patronus.ai/) | An automated evaluation and benchmarking platform for LLMs, providing tools for testing, scoring, and evaluating LLMs in real-world scenarios | N/A | N/A |
| [Autoblocks AI](https://www.autoblocks.ai/) | A collaborative cloud-based workspace designed for rapid iteration on GenAI products, offering features like prompt management, observability, continuous evaluations, fine-tuning, prototyping, debugging, and scalable data ingestion & search, all in a provider-agnostic environment | N/A | <a href=https://pypi.org/project/autoblocksai><img src="https://img.shields.io/pypi/dw/autoblocksai" width=150/></a> |
| [Context AI](https://context.ai/) | Tools for pre-launch LLM evaluations and post-launch analytics, with features such as testing, performance monitoring, user conversation analysis, and support for various models and libraries | N/A | <a href=https://pypi.org/project/context-python><img src="https://img.shields.io/pypi/dw/context-python" width=150/></a> |
| [Ragas](https://docs.ragas.io/en/stable/) | Framework that helps you evaluate your Retrieval Augmented Generation (RAG) pipelines. | <a href=https://github.com/explodinggradients/ragas><img src="https://img.shields.io/github/stars/explodinggradients/ragas?style=social" width=90/></a> | <a href=https://pypi.org/project/ragas><img src="https://img.shields.io/pypi/dw/ragas" width=90/></a> |
| [E2b dev](https://e2b.dev/) | Services to deploy, test, and monitor AI agents, including a sandbox with a secure, long-running cloud environment for various LLMs with features like internet access | <a href=https://github.com/e2b-dev/e2b><img src="https://img.shields.io/github/stars/e2b-dev/e2b?style=social" width=90/></a> | <a href=https://pypi.org/project/e2b><img src="https://img.shields.io/pypi/dw/e2b" width=150/></a> |
| [Agentops](https://www.agentops.ai/) | Toolkit for evaluating and developing AI agents, providing tools for agent development, monitoring capabilities, and replay analytics | <a href=https://github.com/AgentOps-AI/agentops><img src="https://img.shields.io/github/stars/AgentOps-AI/agentops?style=social" width=90/></a> | <a href=https://pypi.org/project/agentops><img src="https://img.shields.io/pypi/dw/agentops" width=150/></a> |
| [Zenoml](https://zenoml.com/) | AI evaluation platform that enables data visualization, model performance analysis, and the creation of interactive reports for various data types | <a href=https://github.com/zeno-ml/zeno-build><img src="https://img.shields.io/github/stars/zeno-ml/zeno-build?style=social" width=90/></a> | <a href=https://pypi.org/project/zeno-client><img src="https://img.shields.io/pypi/dw/zeno-client" width=150/></a> |
Expand Down