FrameworkTesting and debugging
letta-evals
Evaluates stateful agents with datasets, graders, rewards, multi-turn cases, and repeatable suites.
- Stars
- 77
- Forks
- 12
- Updated
- Updated Jul 8, 2026
Apache-2.0
Evaluates stateful agents with datasets, graders, rewards, multi-turn cases, and repeatable suites.
Letta Evals organizes agent evaluation from dataset and target through extractors, graders, rewards, and stored results. It supports JSONL or CSV data, multi-turn samples, multiple model handles, cached re-grading, deterministic or model-judge graders, and custom agent factories. Runs target Letta Code through a self-hosted or cloud server, so Python 3.11+, a running Letta service, and provider credentials are required.
Resource types
Use cases
Choose an architecture before creating, evaluating, improving, and packaging reusable AI skills.
Automate and test Chromium, Firefox, and WebKit through one API.
Runtime
Protocols & integrations
Capabilities
Audience
Public GitHub facts last synced Jul 10, 2026.
Control, automate, and operate Android devices or fleets from one integrated platform.
Compare skill-enabled and baseline runs to measure whether an Agent Skill actually helps.