Vals AI logo

Vals AI

Benchmarking LLMs on Enterprise-Specific Tasks

Actively Hiring Growing Fast Top Tier Investor

Overview

Website
http://vals.ai
Employees
Locations
San Francisco
Measuring model ability is the most challenging part of creating applications that are capable of automating any given part of the economy. There are no good techniques or benchmarks for evaluating LLM performance on business-relevant tasks, so adoption for enterprise production settings has been limited (see Wittgenstein’s ruler).

This problem materializes in each place where LLMs have potential: in understanding whether the AI tool companies are building a product will satisfy a customer demand, determining how feasible models and vendors are for a given enterprise in making purchasing decisions, for researchers who need a north star to which to expand model ability.

Today, answering these questions amounts to hiring a human review team to manually evaluate model outputs. This is prohibitively expensive and slow.

Vals AI is building the enterprise benchmark of LLM and LLM apps on real-world business tasks. In doing so we are creating the infrastructure + certification to automatically audit LLM applications, verifying they are ready for consumption.

We've raised mid-seven figures from some of the top institutional investors and strategic angels in Silicon Valley. See our benchmarks and launch announcement in Bloomberg: https://www.bloomberg.com/news/newsletters/2024-04-11/this-startup-is-trying-to-test-how-well-ai-models-actually-work

Industries

Artificial Intelligence Natural Language Processing

Jobs

Explore Other Startups

ROOM logo
ROOM
View Jobs
MakerSights logo
MakerSights
View Jobs
Dataiku logo
Dataiku
View Jobs
Change.org logo
Change.org
View Jobs