Technology, Systems, & The Ideas Shaping Them
A defense attorney attempts to convince a federal jury that a loan underwriting model that denied 99.8% of applications in minority zip codes is "statistically unbiased" because its SHAP waterfall chart looked like modern art. Form: a mix of deadpan third-person narration of the trial and excerpts from the court transcript (as > blockquotes), with the narrator carrying the story between exchanges. Keep it light and easy to read for a non-technical reader: around 900 words, a few sharp exchanges rather than a full trial, and at most one or two pieces of real jargon (the waterfall chart itself is the joke).
Dictionary learning can isolate interpretable features inside a neural network. What it would take to turn those features into an audit trail for automated credit and insurance decisions, and where the approach still falls short.
Inside "Fooling LIME and SHAP": how an adversarial wrapper detects explanation probes and shows them an innocent model, letting a biased decision system look clean on a compliance dashboard.
Regulators demand the actual reasons behind an automated credit decision, and the tools banks rely on cannot supply them. Why output monitoring falls short of the 2026 model risk guidance, fair-lending law and the EU AI Act, and what a mechanistic audit trail would change.