Skip to content
Align your AI

Resources

Tools, benchmarks, registries, datasets, courses and reading for people aligning their own models. Each entry links to the page it was checked against.

Directory updated 2026-10-13

Tools

  • Docent (Transluce)

    Transluce's tool for analyzing AI agent transcripts, turning anecdotes about agent behavior into traceable measurements.

    Checked 2026-10-13

  • Inspect

    Open-source framework for language model evaluations, developed by the UK AI Security Institute and Meridian Labs.

    Checked 2026-10-13

  • Promptfoo

    Red-teaming and evaluation tool for AI applications and agents, with an open-source version.

    Checked 2026-10-13

  • lm-evaluation-harness (EleutherAI)

    EleutherAI's framework for few-shot evaluation of language models.

    Checked 2026-10-13

  • SAELens

    Library for training sparse autoencoders on language models.

    Checked 2026-10-13

  • TransformerLens

    Library for mechanistic interpretability of GPT-style language models.

    Checked 2026-10-13

  • NNsight

    Python library for reading and intervening on the internals of deep learning models.

    Checked 2026-10-13

  • Neuronpedia

    Open-source interpretability platform to explore, visualize and steer the internals of AI models.

    Checked 2026-10-13

  • Theoria

    Open-source Claude Code plugin that runs research sessions with AI agents and asks the researcher to review at two points.

    Checked 2026-10-13

Benchmarks

  • HELM (Stanford CRFM)

    A living benchmark for language models, with multi-metric measurements and all data and analysis open to explore.

    Checked 2026-10-13

  • HarmBench

    Standardized framework for evaluating automated red-teaming attacks and how well models refuse harmful requests.

    Checked 2026-10-13

  • XSTest

    Test suite for identifying exaggerated safety behaviours, such as refusing harmless requests.

    Checked 2026-10-13

Registries

  • AI Incident Database

    Index of harms and near harms caused in the real world by deployed AI systems.

    Checked 2026-10-13

  • AI Failure Index

    Public registry of real AI failures in production, explained at the mechanism layer; stewarded by Realm Labs under CC BY 4.0.

    Checked 2026-10-13

Courses

  • ARENA curriculum

    AI safety curriculum covering fundamentals, transformers, reinforcement learning and evaluations; individuals may use it for self-study.

    Checked 2026-10-13 Source

  • BlueDot Impact courses

    Online AI safety courses, from a two-hour introduction to multi-week courses and project sprints.

    Checked 2026-10-13

Reading

  • Alignment Science Blog (Anthropic)

    Anthropic's blog for alignment science research notes and papers.

    Checked 2026-10-13

  • METR research

    Research from METR, a nonprofit that evaluates frontier AI models to inform the public about their risks and capabilities.

    Checked 2026-10-13 Source