AgentBench

ARC-AGI-2 contains 1,000 public training tasks and 120 public evaluation tasks built from small colored-integer grids. Solvers infer a transformation from demonstration pairs and must produce every test output with the correct dimensions and cells. The evaluation set is intended for previously unseen testing, so repeatedly tuning against it would compromise the result.

SUPER evaluates whether LLM agents can configure environments, execute code, and answer questions drawn from real machine-learning and NLP research repositories. Its dataset is divided into Expert, Masked, and AutoGen task sets, with code for running agents and scoring results. Local mode executes repository code directly on the machine, while Docker and Modal backends offer safer isolation and concurrent benchmark runs.

gophernotes adds a Go kernel to Jupyter Notebook and nteract, allowing live Go code to sit alongside equations, visualizations, and explanatory text in shareable documents. It evaluates code through the gomacro interpreter. On Windows, gomacro cannot use the required plugin support for third-party packages, so those notebooks should be run through the documented Docker setup.

webclaw provides a local-first way to scrape, crawl, map, and extract websites for agents and RAG systems. Its core CLI and MCP workflow can run without an account for many pages, while a hosted API is available for bot-protected sites, JavaScript rendering, asynchronous crawl jobs, search, and production monitoring.