Week 02 · The Lifecycle and Its People

Monday

Who touches SH-1?

// Who designs, builds, deploys, runs and checks SH-1, and how they manage risk

⏱ about 20 min

Monday: Who Touches SH-1?

Comet pulls a folded page from the back of the SH-1 packet. "A timeline of how SH-1 was made!" she says. "Look, testing comes last, right before a school starts using it."

Wren reads the page twice. "What does the evidence say? If they only test at the end, what if the problem started at the beginning?"

Comet chews her pencil. "Then they would have built the whole thing on a mistake."

Nova projects a long arrow across the library wall, with four empty boxes along it. "Would you like a hint?" she asks. "Put each step in a box, then look for who checks the work."

Librarian Joss sets down a cup of pencils. "Reviewer, can you put SH-1's story in order?"

Real AI check

Nova is a character in our story. Real AI is a tool people build. It does not think or feel like a person, and it can be wrong.

AI actors

AI actors are the people and organizations who play an active role in an AI system's lifecycle. That includes the people who deploy or run it.

For SH-1, the builder is an AI actor. So are the base model maker, Harbor Point's teachers and the crew.

The SH-1 timeline below is a made-up example from the packet. The stage names come from NIST.

The lifecycle, in NIST's words

Design work happens first: the setting, the data and the inputs.

Development work builds the model. Deployment work puts the system to its task and its outputs into use.

Operation and monitoring work runs the system and keeps checking it.

Testing, evaluation, verification and validation, called TEVV, happen all the way through. Risk management should start with planning and design.

Stage	SH-1 timeline (made up for the story)
Design	The builder wrote down SH-1's goals and gathered study material.
Development	The builder tuned the base model on that study material.
Deployment	Harbor Point would pilot SH-1 in a few classes.
Operation and monitoring	Teachers would run SH-1 and check its outputs every week.
TEVV (all the way through)	Tests at every stage, including the crew's own tests.

PUT SH-1'S STORY IN LIFECYCLE ORDER

?The builder tunes the base model on the study material.
?Teachers run SH-1 and check its outputs every week.
?Harbor Point pilots SH-1 in a few classes.
?The builder writes down SH-1's goals and gathers study material.

WHY THIS EXERCISEKnowing the stages tells a reviewer where to look for each kind of problem.

WHEN DOES CHECKING HAPPEN?

Read the question.
Tap your answer.

Comet thought testing comes last. When does TEVV happen in NIST's lifecycle?

When should risk management start?

Which of these is an AI actor for SH-1?

Teachers checking SH-1's outputs every week belongs to which stage?

Statement	True or false?
AI actors include people who deploy or run an AI system.	?
Development work builds the model.	?
TEVV happens only once, at the very end.	?
Risk management should start with planning and design.	?
The SH-1 timeline is a real company's timeline.	?

WHY THIS EXERCISEA lifecycle with checks all the way through catches problems before they grow.

Why might it be a problem if SH-1 were tested only at the very end? Give one example.Which lifecycle stage would you most like to see more of? What would you ask about it?

Strong start to week two, reviewer. Tomorrow the crew meets the people who do each job.

Grown-up checked my answers

Tuesday →