← Back to course
Applied AI / Week 02 / Wednesday
3/6
Week 02 · Why It Gets Things Wrong

Wednesday

Spot the problem: the homework answer
// Confident answers, made-up sources and our habit of trusting machines
⏱ about 15 min

Wednesday: Spot the Problem

Maya sits down with her daughter and the homework helper's answer from Monday.

Instead of fixing it for her, Maya says, "Let's be detectives. Read it one line at a time, and we'll check each line."

Her daughter likes this. It feels less like being told off and more like a puzzle.

Read like a checker

Below is an example answer. It was written for this lesson, and its errors were placed on purpose.

Look for three kinds of problem: a wrong number, two lines that contradict each other, and a source you cannot check.

Example output (written for this lesson)
Request: My class has 24 students. The teacher puts us in groups of 4. How many groups are there? Show your work.
Output: Great question! To find the number of groups, divide the number of students by the group size.
24 divided by 4 is 8, so there are 8 groups.
Check: 8 groups of 4 students makes 24 students, so the answer is correct.
So your class will have 6 groups in total.
Source: Classroom Number Facts, second edition, page 47.

The source line above was also invented for this lesson. Treat any source like it as unchecked until you find it and read it.

Line in the outputIs this line correct?
Divide the number of students by the group size.?
24 divided by 4 is 8.?
8 groups of 4 students makes 24 students.?
The answer says both 8 groups and 6 groups.?
WHY THIS EXERCISERecomputing numbers is the fastest way to catch a confident wrong answer.
NAME THE PROBLEM
  • Read the question.
  • Tap your answer.
The answer says 8 groups, then 6 groups. What is this called?
The "Check" line says the answer is correct, but its math is wrong. What does NIST call reasons like this?
What is the safest view of the source line?

Checking together

Maya does not tell her daughter the tool is bad. She shows her how to check it.

They recompute the numbers. They notice the two different answers. They try to find the source, and they cannot.

That builds a habit the child can use without a parent beside her.

CHECK A HOMEWORK ANSWER TOGETHER
  • ?Read the question and the answer aloud.
  • ?Redo every calculation on paper.
  • ?Look for two lines that disagree.
  • ?Try to find and open any source it names.
  • ?Write the answer in your own words, once it checks out.
WHY THIS EXERCISEA child who checks learns the subject and the habit at the same time.
What is the correct number of groups?
WHY THIS EXERCISEYou can only catch a wrong answer if you work it out yourself.
To check a calculation, ____ it yourself.
A source you have not opened is ____.

Sources: NIST AI 600-1, Generative AI Profile (2024)

Tomorrow: the same problem in a classroom and in a shop.

← Tuesday