Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.

The OpenAI Python library provides convenient access to the OpenAI REST API from any Python 3.7+ application. The library includes type definitions for all request params and response fields, and ...

┌─────────────────────────────────────────────┐ │ OpenAI Clients (Open ...