Week 11 · Humans in Charge

Friday

Impact Friday: Who is accountable?

// Who has the final say, and how much to trust

⏱ about 20 min

Friday: Impact Friday: Who Is Accountable?

Librarian Joss opens the panel's notebook to a fresh page. "The panel has one more question," she says. "When SH-1 gets it wrong, who is accountable?"

"SH-1 is!" Comet says at once. "It wrote spring instead of fall."

Wren shakes his head gently. "What does the evidence say? SH-1 is a tool. It cannot fix itself or explain itself to the panel. People can."

"Then which people?" Comet asks.

Nova projects the value chain from last week, with a small figure beside each box. "Would you like a hint?" she asks. "Look at who could see the problem, and who could fix it."

Librarian Joss hands you the pen. "Reviewer, help us name the roles."

Accountability needs transparency

Accountability depends on transparency. People cannot answer for what they cannot see.

A transparent system is not automatically accurate, private, secure or fair.

Transparency is often needed so people can get a wrong output put right.

There is no formal channel yet for reporting AI incidents. Documenting and sharing them helps trace harms to their source and prevent them.

Role in the SH-1 story (made up)	What the crew says they answer for
The builder	How SH-1 is built, tested and fixed, and telling the school what it knows
The Tech Review Panel	Deciding whether to use SH-1, and reviewing the incident log
Teachers	Checking quizzes and having the final say on Check marks
Students	Checking SH-1's answers against the notes and using the report button

WHO IS ACCOUNTABLE?

Read the question.
Tap your answer.

Comet says SH-1 itself is accountable. What does the evidence say?

SH-1 shows exactly which notes page it used for each answer. Does that make it accurate?

Why does the crew keep an incident log?

Rating R7

Risk combines how likely an event is with how big its consequences would be.

The crew rates R7 likelihood High. S14 already copied a confident wrong answer, and Check sounds sure even when it misses.

They rate size of harm Medium. A student can keep a wrong idea, but SH-1 is not used for grading, and teachers keep the final say.

These ratings are the crew's judgment in the story, not facts.

Row	Risk	SH-1 example	Likelihood	Size of harm
R7	Human-AI configuration	Students trust Check over their own work	High	Medium

AI actor spotlight: human factors and operators

Human factors work keeps people at the center. It involves end users, designs and checks the user experience, and tests with people.
Operation and monitoring work runs the system and regularly checks its output and effects with others.
In the SH-1 Review, the crew does human factors work when they design the report form. Teachers would be operators in the pilot.

Week review

People can trust AI too much, too little, treat it as a person or become attached to it.
Human roles in AI decisions must be clearly defined, from fully manual to fully automatic.
A person and an AI together can do better or worse than either alone, depending on how they are organized.
Transparency answers what, explainability answers how, and interpretability answers why.
Accountability depends on transparency, and documenting incidents helps prevent repeats. R7 is rated High and Medium.

Week review	True or false?
Automation bias means trusting AI too much.	?
SH-1 really feels proud of students.	?
The crew put no SH-1 feature on the "decides alone" rung.	?
A transparent system is always fair.	?
There is already a formal channel for reporting every AI incident.	?

WHY THIS EXERCISEKeeping people in charge is what makes every other fix in the review work.

OUR WEEK, IN ORDER

?We named four ways trust can go wrong.
?We wrote the oversight plan, the report form and the incident log.
?We asked whether a person and SH-1 do better together.
?We built the trust ladder and placed each feature.
?We named who is accountable and rated R7.

WHY THIS EXERCISETogether these steps make the oversight plan on the model card.

Pick one role from the table. Write one thing that role should do the next time SH-1 gets something wrong.Do you agree with the crew's R7 rating? Explain why or why not.

Fine work, reviewer. Tomorrow's Explorer Quest asks your family who should have the final say.

Grown-up checked my answers

← Thursday Explorer Quest →