mechanistic-interpretability

To know what models don't say out loud.

Jacobian-Brainwash : A manual alignment tool for large language models built on Anthropic's Jacobian Lens. Results are exportable.

Steering vectors for transformer language models in Pytorch / Huggingface

Generating and validating natural-language explanations for the brain.

Sparse and discrete interpretability tool for neural networks

OpenAI-compatible server with a live view into any HF model's residual stream. pip install brainscope

End-to-end pipeline for seeing how LLMs actually process your prompts. Capture attention across every layer, render heatmaps and cooking curves, compare variants with evidence — not vibes.

Lightweight representation engineering dataflow operations for agent developers.

Tracking exactly what happens to the internal "circuitry" (induction heads) of a 2-layer attention-only Transformer when forced to undergo domain adaptation from prose to structured Python code.

A curated collection of NLP and LLM resources. Covers essential papers and blogs on Transformers, Reinforcement Learning (RLHF, DPO, GRPO), Mechanistic Interpretability, Scaling Laws, and MLSys.