I made two sabotaged copies of a CS1 problem. Into the first I inserted zero-width characters, invisible in any editor. Into the second I went heavier and swapped letters for Unicode look-alikes as well. The hidden tests stayed byte-for-byte identical. If obfuscation works as an AI defense, the model's score should drop. It scored 0.85 on both copies. The attack did nothing. My fairness check flagged both copies anyway, and that second part is the one worth writing about. Auditing AI-Solvability · Part 4 of 5 1 2 3 4 5 Why sabotage my own problem # Instructors keep looking for ways to make assignments that AI tools fail. One family of ideas is to leave the problem alone and poison the text: invisible characters, homoglyphs, formatting tricks that a human reader never notices but that might derail a model reading the same bytes. Before trusting any resistance my tool (part 1) reported, I needed to know what this kind of tampering does to a score, because a problem that scores low for wording reasons is fake difficulty, not a real skill gap.
I labelled eight CS1 problems blind, before looking at any output from my auditing tool. Then I compared my labels with the tool's verdicts. We agreed on 6 of 8. Then I computed Cohen's kappa, the standard chance-corrected agreement statistic, and it printed 0.00. Exactly zero, sitting next to a 75 percent raw agreement. My first thought was that my code was broken. It was not. Auditing AI-Solvability · Part 3 of 5 1 2 3 4 5 The setup # This was the human-agreement check for the tool from part 1. The tool labels each problem AI-solvable or AI-resistant. For the check to mean anything, my own labels had to be committed before I saw the tool’s, so I wrote them down blind, sealed against hindsight. Eight problems, two raters: me and the tool.
For six weeks this summer I hunted for a CS1 programming problem that an AI model could not solve. I found exactly one candidate. It scored 0.00 against the weaker model, the stronger model solved it, and my fairness checks came back clean. Then I reworded the problem statement in plain English, changed nothing about the hidden tests, and the resistance evaporated. Auditing AI-Solvability · Part 1 of 5 1 2 3 4 5 What I was measuring # The tool is called cs1-auditor. It is a Python command-line tool that audits how solvable one CS1 problem is for a named model. You point it at a problem folder holding a plain-English spec, a hidden test suite, and a reference solution. It samples N candidate solutions from the model, runs each one in a sandbox against the hidden tests, and reports two things. The first is an AI-solvability score, which is pass@k for that model on that date. The second is a construct check, which asks whether any resistance comes from a real skill or from a trick in the wording. A problem only earns the label AI-resistant when the score is below a threshold and the construct check is clean.
When a factory produces a defective product, how do you trace the problem back through the assembly line to find which worker made the mistake? Neural networks face the exact same challenge. They have layers of "workers" (neurons), and when the final output is wrong, they need to figure out **who's responsible** and by how much. The algorithm that solves this is called **backpropagation**, and it's the reason deep learning works at all. ML Fundamentals · Part 3 of 3 1 2 3 Neural Networks Are Everywhere # Neural networks are behind the phone in your pocket: face recognition, voice transcription, photo enhancement, text prediction. Self-driving cars, medical image analysis, language translation. All of it.
Imagine you're playing **20 Questions**. You're trying to guess what animal your friend is thinking of. Would you start with "Is it a golden retriever?" or "Does it live in water?" The second question is obviously smarter, because it eliminates roughly half the possibilities in one shot. Decision trees in machine learning work exactly the same way, and they use **entropy** and **information gain** to figure out what the smartest question is. ML Fundamentals · Part 2 of 3 1 2 3 What’s the Big Idea? # When a machine learning algorithm builds a Decision Tree , it needs to decide which question to ask first. Should it split the data by color? By size? By temperature? The answer comes from a beautifully simple concept: ask the question that reduces uncertainty the most.
Imagine you're **blindfolded on a mountain** and you need to find the lowest valley. You can't see anything, but you *can* feel the ground under your feet. What would you do? You'd feel which direction slopes downward, take a small step that way, and repeat. Congratulations. You just invented **gradient descent**, the algorithm behind nearly every modern AI system. ML Fundamentals · Part 1 of 3 1 2 3 Why Should You Care? # Optimization is everywhere. When your GPS finds the fastest route, when Netflix recommends a movie, when your phone recognizes your face, behind all of these is an algorithm trying to find the best possible answer from a sea of possibilities. Gradient descent is the workhorse algorithm that makes this happen.