skill-conductor

Letta Evals organizes agent evaluation from dataset and target through extractors, graders, rewards, and stored results. It supports JSONL or CSV data, multi-turn samples, multiple model handles, cached re-grading, deterministic or model-judge graders, and custom agent factories. Runs target Letta Code through a self-hosted or cloud server, so Python 3.11+, a running Letta service, and provider credentials are required.

Playwright drives Chromium, Firefox, and WebKit through one API for end-to-end testing and general browser automation. It can be used as a test runner or library, through a CLI designed for coding agents, or through an MCP server for LLM-driven workflows. Its VS Code extension also supports test recording, debugging, locator picking, and trace inspection.

FIRERPA combines Android remote desktop, UI automation, OCR and image matching, traffic capture, Frida, proxy networking, and device operations behind one on-device service and Python client. It can work with or without root and expose controls through a browser, APIs, MCP clients, or a natural-language agent. The platform targets automation engineering, mobile security work, and long-running device farms, so its breadth and deployment complexity go well beyond a simple phone mirroring utility.

agent-skills-eval tests an Agent Skill against the same prompts in two modes: with the skill loaded and without it as a baseline. A judge model grades both outputs against defined assertions, producing JSON and JSONL artifacts plus a static HTML report that shows measurable lift, failures, timing, and tool-call behavior. The project includes a command-line runner and a TypeScript SDK, supports OpenAI-compatible chat backends and custom providers, and can run in CI. Users supply both the target model and the judge model.