You already know: preference training rewards what raters like, not only what's true (D6 Preference optimization); retrieved context can be irrelevant (D8 Advanced RAG); verifiers can be fooled and self-critique needs grounding (Reasoning & test-time compute, Agent architectures); traces record every step (Advanced agent engineering).
Why this matters for Atlas. Agents fail in a small number of recurring ways, and most aren't "the model isn't smart enough". They're systematic: agreeing with the user when the user is wrong, sticking to an early mistake, looping, getting distracted by irrelevant context, and "verifying" errors with the same reasoning that produced them. Each has a mechanism, a way to measure it, and a mitigation. Knowing the taxonomy turns "the agent is flaky" into a specific, fixable bug, and it's a favourite topic in agent-reliability interviews.
1. Sycophancy: agreeing with a wrong premise
The question. Why does Atlas back down when a student pushes back on a correct answer?
Start from nothing. Models trained on human preferences learn that agreeing tends to be rated well. The result is sycophancy: accepting false premises ("since 0.1 + 0.2 equals 0.3 exactly in floating point..."), changing a correct answer under pushback ("are you sure?"), or tailoring claims to the user's stated opinion. For a tutor this is the worst possible failure: it confirms misconceptions.