Run and extend a broad collection of Inspect AI evaluations.
Project overview
Inspect Evals gathers benchmarks for coding, cybersecurity, safeguards, reasoning, knowledge, and other model capabilities. Researchers can run individual tasks with Inspect AI, improve included evaluations, or register externally maintained implementations. The project recommends Python 3.11 or 3.12 for the most reliable development and execution experience.
Train and evaluate systems on abstract grid-transformation reasoning tasks.
Project overview
ARC-AGI-2 contains 1,000 public training tasks and 120 public evaluation tasks built from small colored-integer grids. Solvers infer a transformation from demonstration pairs and must produce every test output with the correct dimensions and cells. The evaluation set is intended for previously unseen testing, so repeatedly tuning against it would compromise the result.
Train and evaluate software-engineering agents on real Python repository tasks.
Project overview
SWE-Gym provides 2,400 real software-engineering tasks drawn from 11 Python repositories, together with repository context, executable environments, and test-based verification. Researchers can use it to generate trajectories and train agents or solution verifiers. Reproducing the experiments also involves the published dataset, prebuilt Docker images, and instructions for OpenHands or MoatlessTools.
API or serverPersonal and daily lifeModel Context Protocol
mirobody
@thetahealth·C++
Connect personal health records to optional AI assistants on-device or self-hosted.
Project overview
Mirobody has a C++ core that runs inside Android and iOS apps, in a desktop wrapper, or as a self-hosted server. It brings wearable, laboratory, and clinical records behind a common interface, with FHIR endpoints, long-term memory, MCP tools, and several LLM providers. Health data can remain on the phone or the operator’s own infrastructure, but enabling an external model, synchronization, or vendor integration can send selected information outside that boundary. The architecture supplies privacy and compliance controls; the deployer remains responsible for the final HIPAA or GDPR posture.
Scaffold codebases from a product spec or embed a developer agent.
Project overview
smol developer turns a product specification into a shared plan, a file list, and generated code for each file. It can run as a command-line scaffold, be imported as a Python library, or be exposed through an Agent Protocol API inside another application. The workflow is intentionally human-in-the-loop, and the project documents unresolved limits around cross-file coherence, dependency setup, self-healing, and slow generation.
Build composable AI agents with tools, sessions, middleware, and subagents.
Project overview
Create general-purpose agents in TypeScript with streamed events, filesystem or shell tools, persistent multi-turn sessions, and middleware for compaction, retry, and turn tracking. The SDK also covers subagents, MCP servers, on-demand skills, virtual filesystems, and React or Vue chat integration on top of Vercel's AI SDK.
Excel-to-Neo4j examples spanning legacy graphs and GraphRAG
Project overview
This repository turns invoice-style Excel data into Neo4j nodes and relationships with pandas, then shows how to convert graph relationships into matrices. It preserves a legacy py2neo v3 and Neo4j 3.x teaching path while adding a modern example with the official driver, vector indexes, and GraphRAG-style retrieval. Current Python and Neo4j environments may require changes to the pinned legacy code.
Bootstrap and package Electron apps with a security-focused process structure.
Project overview
vite-electron-builder is a project template that separates the Electron main process, preload layer, and renderer so developers can pair it with their preferred web framework. Its initialization workflow creates and integrates the renderer, after which the project can run in development or compile distributable applications. Packaging, end-to-end testing, continuous integration, and update workflows are preconfigured, while Node.js capabilities are exposed through preload rather than used directly in the renderer.
Particle Device OS contains the firmware libraries and system components used to build software for Particle hardware. Developers can use its build guides, application firmware API, debugging support, bootloader, cloud communication modules, threading code, and hardware abstraction layers. It is primarily a firmware development source tree, and changes to LGPLv3-licensed system components carry the sharing obligations described by the project.
Build collaborative multi-agent systems in Python or TypeScript.
Project overview
BeeAI Framework supplies Python and TypeScript components for agents, multi-agent handoffs, model backends, tools, and RAG, helping developers assemble applications that reason, act, and collaborate. Requirements can constrain agent behavior consistently across different models, while MCP, ACP, and custom tools extend the system. Examples work with hosted providers and can also run locally through Ollama.
Self-host desktop app updates with version channels and S3 artifacts.
Project overview
faynoSync gives desktop application teams control over their update infrastructure. Developers upload builds to S3-compatible storage, assign version metadata and stable, beta, or nightly channels through a dashboard or REST API, and let clients query a simple endpoint for update details and download URLs. It can feed Electron, Tauri, Squirrel, Velopack, or custom update flows, with either background installation or a user prompt determined by the client. Running the full service requires maintaining the API, MongoDB, Redis, object storage, migrations, and dashboard.
Evaluate LLM agents across diverse interactive environments and tasks.
Project overview
AgentBench measures how well LLMs act autonomously across eight interactive environments, including database and operating-system tasks. The current repository carries a function-calling version integrated with AgentRL, while the older benchmark remains available as v0.1. Running the suite requires Docker, and Python 3.9 is recommended because several scientific dependencies are pinned to older versions.
Use LibVLC playback, streaming, and encoding inside .NET applications.
Project overview
LibVLCSharp exposes VideoLAN's LibVLC engine to .NET and Mono applications across desktop, mobile, server, and Unity environments. Developers can render video, output audio, handle files and network streams, and use encoding or streaming capabilities through a common API, with integrations for frameworks such as WPF, WinForms, MAUI, Avalonia, and Xamarin. Each target also needs the platform-specific native LibVLC package, because the bindings do not include the media engine by themselves.
Deploy static sites with per-site state and a lightweight backend
Project overview
Simple Host lets a coding agent deploy a static website together with a small per-site backend for JSON state, append-only collections, comments, feedback, private pages, and starter templates. Each upload becomes an immutable version that can be rolled back, and magic-link sign-in supplies an API key. The service can also be self-hosted as one Go binary, but that path requires Postgres, environment variables, and a carefully chosen administrator key.
Compare five decision personas and audit their calibration over time
Project overview
council-diff runs five differently biased personas in parallel on the same decision and exposes their scores, verdicts, strengths, gaps, agreement, and recommendation. Built-in domains cover founder, engineering, investing, career, product, and quant questions, while custom rosters are supported. An optional Oracle reviews difficult or split calls, and the Brier module scores predictions after outcomes resolve. Safe mode can keep the Oracle on a zero-retention model when retention matters.
Train models to retrieve and invoke tools through token generation.
Project overview
ToolGen represents tools as dedicated vocabulary tokens so a model can select tools, call them, and continue generating language within one process. The repository includes models, data for its three-stage training procedure, and evaluation code for research reproduction. Its local inference example also requires a ToolBench key and a StableToolBench deployment.
Scaffold customizable AI agent projects from the command line.
Project overview
AgentStack creates an initial agent project and supports an ongoing CLI workflow for adding tools, generating tests, and running the application. It works with multiple model providers and frameworks such as CrewAI and LangGraph, while keeping the generated code open for developers to customize rather than acting as a low-code builder.
Styles plain HTML as grayscale hand-drawn wireframes with embedded fonts and 1,745 scribble icons.
Project overview
lofi.css adds a paper-scribble look to semantic HTML through one CSS file with embedded handwriting fonts, grayscale styling, annotations, and a 1,745-icon sprite. It uses no JavaScript, dependencies, build step, or external requests, making it suited to early flow discussions rather than polished UI. Its companion Claude skill can generate constrained mockups from descriptions, notes, websites, or whiteboard photos.