SemHash is a lightweight, multimodal library for semantic deduplication, outlier filtering, and representative sample selection. Text works out of the box with fast Model2Vec embeddings, and images, ...

Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models ...

There's a lot you can automate.

We evaluate DeepCode on the PaperBench benchmark (released by OpenAI), a rigorous testbed requiring AI agents to independently reproduce 20 ICML 2024 papers from scratch. The benchmark comprises 8,316 ...