Week 02 · The Lifecycle and Its People

Friday

Impact Friday: Is AI the right tool?

// Who designs, builds, deploys, runs and checks SH-1, and how they manage risk

⏱ about 20 min

Friday: Impact Friday: Is AI the Right Tool?

Librarian Joss calls the crew to her desk. "Before we go further, the panel has a question," she says. "Do we need an AI study helper at all?"

Comet blinks. "But SH-1 scored 92! It must be good."

Wren looks at the list of trustworthy characteristics Nova has projected. "What does the evidence say? A high score shows one thing. It says nothing about privacy or fairness."

Nova dims the list to a single line: "only as strong as its weakest characteristic." "Would you like a hint?" she asks. "Look for the weakest link, not the shiniest."

Librarian Joss turns to you. "Reviewer, how should the panel decide?"

Trustworthy AI is a balance

NIST lists the characteristics of trustworthy AI. It is valid and reliable, safe, secure and resilient, and accountable and transparent.

It is also explainable and interpretable, privacy-enhanced, and fair with harmful bias managed.

Valid and reliable is the base for the others. Accountable and transparent relates to all of them.

The characteristics can pull against each other, so teams must balance trade-offs.

A secure but unfair system is not good enough. Neither is an accurate system nobody can understand.

Trustworthiness is only as strong as its weakest characteristic.

SPOT THE TRADE-OFF

Read the question.
Tap your answer.

A made-up tool locks every file tightly but works worse for some students. Is it trustworthy enough?

Which characteristic is the base for the others?

SH-1 scores 92 but logs every student's name. What does the weakest-link idea say?

Is AI the right tool at all?

All AI actors share the job of deciding whether AI is the right tool for a purpose at all. They also share deciding how to use it responsibly.

A decision to use an AI system should weigh its trustworthiness, risks and benefits, with input from many people.

Clear go or no-go decisions are one goal of NIST's framework. The crew's review ends with one in week 12.

AI actor spotlight

Governance and oversight.
Governance and oversight is done by the leaders responsible for the organization that designs, builds or uses an AI system.
At Harbor Point, Librarian Joss chairs the Tech Review Panel, and the panel will make the go or no-go decision.
Procurement is a related job: getting an AI product or service from an outside developer. The district office does that for Harbor Point.

Model card, part 2

This week the crew adds who built SH-1, what it is built on, who would run it and who checks it.

Model card field	What the crew writes
Built by	The builder (an outside company, not named in the packet)
Built on	A large pre-trained language model from another company, the base model maker (P2)
Would be run by	Harbor Point High School: teachers and the library
Checked by	The student reviewers (TEVV), the Tech Review Panel, and the district office before the school gets any tool

Week review

AI actors play an active role in an AI system's lifecycle.
The lifecycle runs from design to development to deployment to operation and monitoring, with TEVV all the way through.
Ideally, checkers are not the builders.
Risk combines likelihood and size of harm.
GOVERN, MAP, MEASURE and MANAGE are used again and again, not as a checklist.
Trustworthiness is a balance, only as strong as its weakest characteristic.

Week review	True or false?
Deciding whether AI is the right tool is only the builder's job.	?
Valid and reliable is the base for the other characteristics.	?
A system can be very accurate and still not be good enough.	?
Risk management should wait until after deployment.	?
The district office does the procurement work for Harbor Point.	?

WHY THIS EXERCISEThese ideas decide how the panel will judge SH-1.

OUR WEEK, IN ORDER

?We sorted the crew's plan into GOVERN, MAP, MEASURE and MANAGE.
?We matched SH-1 jobs to the people who do them.
?We built a risk grid and named the nine register rows.
?We asked whether AI is the right tool at all.
?We put SH-1's timeline in lifecycle order.

WHY THIS EXERCISEEach step adds to the map the crew will measure next.

Is an AI study helper the right tool for Harbor Point? Write your first answer now. You can change it later.Which trustworthy characteristic do you think will be SH-1's weakest? Why?

What a week, reviewer. Tomorrow's Explorer Quest finds a lifecycle at home.

Grown-up checked my answers

← Thursday Explorer Quest →