Interactively explore unstructured datasets from your dataframe.
Spotlight helps you to understand unstructured datasets fast. You can quickly create interactive visualizations and leverage data enrichments (e.g. embeddings, prediction, uncertainties) to identify critical clusters in your data.
Spotlight supports most unstructured data types including images, audio, text, videos, time-series and geometric data. You can start from your existing dataframe:
And start Spotlight with just a few lines of code:
from renumics import spotlight
spotlight.show(df, dtype={"image": spotlight.Image, "embedding": spotlight.Embedding})Machine learning and engineering teams use Spotlight to understand and communicate on complex unstructured data problems.
Here are some interactive examples on publicly available datasets:
| Modality | Description | Link |
|---|---|---|
| 🖼️ Image | Find Issues in Any Image Classification Dataset | 👨💻 📝 |
| Find data issues in the CIFAR-100 image dataset | 🕹️ | |
| Explore data slices in the CIFAR-100 image dataset | 🕹️ | |
| Fine-tuning image classification models from Bing image search | 👨💻📝 | |
| 🔊 Audio | Find Issues in Any Audio Classification Dataset | 👨💻 📝 |
| Find data issues in the Common Voice audio dataset | 🕹️ | |
| Compare gender detection models on the emodb dataset | 🕹️ | |
| 📝 Text | Find Issues in Any Text Classification Dataset | 👨💻 📝 |
| 📈🖼️ Mixed | Explore results from the Formula1 Montreal 2023 GP | 🕹️ |
| Explore a crash simulation dataset | 🕹️ |
Get started by installing Spotlight and loading your first dataset.
- Python version 3.8-3.11
Install Spotlight via pip
pip install renumics-spotlightWe recommend installing Spotlight and everything you need to work on your data in a separate virtual environment.
import pandas as pd
from renumics import spotlight
df = pd.read_csv("https://renumics.com/data/mnist/mnist-tiny.csv")
spotlight.show(df, dtype={"image": spotlight.Image, "embedding": spotlight.Embedding})pd.read_csv loads a sample csv file as a pandas DataFrame.
spotlight.show opens up spotlight in the browser with the pandas dataframe ready for you to explore. The dtype argument specifies custom column types for the browser viewer.
Load a Hugging Face audio dataset with embeddings and a pre-defined layout
import datasets
from renumics import spotlight
dataset = datasets.load_dataset("renumics/dcase23-task2-enriched", "dev", split="all", streaming=False)
df = dataset.to_pandas()
simple_layout = datasets.load_dataset_builder("renumics/dcase23-task2-enriched", "dev").config.get_layout(config="simple")
spotlight.show(df, dtype={'path': spotlight.Audio, "embeddings_ast-finetuned-audioset-10-10-0.4593": spotlight.Embedding}, layout=simple_layout)The
datasets[audio]package can be installed via pip.
We have added crash report and perfomance collection. We do NOT collect user data other than an anonymized Machine Id obtained by py-machineid, and only log our own actions. We do NOT collect folder names, dataset names, or row data of any kind only aggregate performance statistics like total time of a table_load, crash data, etc. Collecting Spotlight crashes will help us improve stability. To opt out of the crash report collection define an environment variable called SPOTLIGHT_OPT_OUT and set it to true. e.G.export SPOTLIGHT_OPT_OUT=true
- Open an issue on Github
- Have a coffee talk with us
- Join our channel on Discord
- 🤗 Huggingface example spaces and datasets
- 🏀 Playbook for data-centric AI workflows
- 🍰 Sliceguard library for automatic slice detection