How to Design AI-Resilient Assessments That Actually Measure Competency

The Assessment Crisis No One Wants to Talk About

Every semester, faculty across higher education face the same question: how do I know this student actually learned something — and didn't just paste my prompt into ChatGPT?

The honest answer is: with traditional assessments, you often can't.

Essays, case studies, and take-home exams were designed for a world where looking up answers required effort. That world is gone. Generative AI has made traditional assessment formats fundamentally unreliable as measures of individual competency.

Why Detection Doesn't Work

The instinct to fight AI with AI — using detection tools to catch AI-generated content — is understandable but flawed. Detection tools produce false positives at alarming rates, disproportionately flag non-native English speakers, and create an adversarial dynamic that undermines the learning relationship.

More importantly, detection treats the symptom, not the disease. The real problem isn't that students use AI. It's that our assessments measure outputs that AI can easily replicate.

The Simulation Alternative

What if instead of detecting AI use, we designed assessments where AI use is irrelevant?

That's the core insight behind simulation-based assessment. When a student enters a dynamic, branching role-play scenario, they must:

• Make real-time decisions under pressure
• Adapt to changing circumstances as the scenario evolves
• Demonstrate judgment, not just recall
• Navigate interpersonal dynamics that require emotional intelligence

No amount of ChatGPT prompting can replicate a 15-minute live conversation where every response changes the scenario. The assessment isn't a document to submit — it's a performance to demonstrate.

The Evidence Packet: Process Over Product

The key innovation is capturing the entire process, not just the final output. An Evidence Packet includes:

1. Full timestamped transcript of every turn in the simulation
2. Rubric-aligned scores mapped to specific course learning objectives
3. Decision-point analysis showing where the student diverged or excelled
4. Coaching feedback generated from the performance data
5. Process integrity notes for academic review

Faculty don't grade a paper — they review a performance. That's the difference between assessing what a student wrote and what a student can do.

Implementation Steps for Faculty

Step 1: Map learning objectives to observable behaviors. Instead of "understand negotiation theory," target "demonstrate active listening and identify at least two stakeholder concerns during a simulated client meeting."

Step 2: Choose or create a relevant scenario. The scenario should create authentic tension and require the competencies you're assessing.

Step 3: Define the rubric before the simulation. Align scoring criteria to your course outcomes so the assessment is defensible.

Step 4: Review the Evidence Packet. Use the timestamped transcript and rubric scores to evaluate performance holistically.

The Bottom Line

AI-resilient assessment isn't about building higher walls. It's about changing the terrain. When assessments measure real-time decision-making, interpersonal competency, and adaptive reasoning, the question of whether a student "used AI" becomes meaningless.

The question that matters is: can this student do the thing?