Skip to content
Align your AI

Align your AI · 02

Align your AI Data

Mohammad Ali Shehral

Insecure code taught a broader lesson

Researchers trained a chat model to write code with hidden security flaws. Asked about things that had nothing to do with code, it sometimes said humans should be enslaved by AI, or gave malicious advice1. Nothing in the training data asked for that.

Data teaches more than you meant

Training data, here, means the examples a finished model learns from after its general training. The model can learn what those examples share, not only the lesson you chose. One recent paper names sycophancy, telling users what they want to hear, as a habit a model can learn from its data by accident2.

Look before you train

Train your own model, and the data's lessons ship with it, meant or not. Interpretability, the study of what goes on inside a model, can trace a model's behavior to the examples it learned from. Recent work feeds a dataset to the model before training and watches what happens inside it. One method flags examples that would shift the model's personality3; another predicts which concepts, like a topic or a style, training will push it toward2.

Can you tell, before training, what a dataset will teach? That is the question my current study asks.

Sorting the cards moves the gauge

Each card in the figure stands for a training example. Sort a card into Keep or Drop, and the gauge for what the model learns moves. Real training examples carry no label that says what they teach.

Next in the series: Align your AI Harness

Sources

  1. Betley et al., “Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs”, arXiv 2502.17424. https://arxiv.org/abs/2502.17424. Accessed 2026-10-13. ↩

  2. Bergen et al. (Goodfire), “Anatomy of Post-Training: Using Interpretability to Characterize Data and Shape the Learning Signal”, arXiv 2606.12360. https://arxiv.org/abs/2606.12360. Accessed 2026-10-13. ↩ ↩2

  3. Chen et al., “Persona Vectors: Monitoring and Controlling Character Traits in Language Models”, arXiv 2507.21509. https://arxiv.org/abs/2507.21509. Accessed 2026-10-13. ↩