Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.

Abstract: The rapid evolution of software systems has made traditional testing methods unsuitable to provide quality, speed, and responsiveness to applications in modern development. This paper ...