Comet writes a to-do list across the whiteboard. "Step one, test SH-1. Step two, decide. Done in two weeks!"
Wren adds a question mark after "decide." "What does the evidence say? Test it for what, in which classes, and what happens if it fails?"
Librarian Joss looks up from her desk. "The panel needs rules for how we review any AI tool, not just this one."
Nova projects four words across the board: GOVERN, MAP, MEASURE, MANAGE. "Would you like a hint?" she asks. "Each word is a kind of work. Some of it happens more than once."
Comet erases "Done in two weeks." "Okay, reviewer. Help me sort the plan."
NIST's framework has four functions: GOVERN, MAP, MEASURE and MANAGE.
GOVERN runs through the other three. The work is iterative, not a checklist: teams go around again as they learn more.
| Function | What it does (NIST, in plain words) | In the crew's 12-week plan |
|---|---|---|
| GOVERN | Builds a culture of risk management in the groups that design, build, use or get AI systems | The panel's rules for reviewing any AI tool at Harbor Point, all twelve weeks |
| MAP | Sets the context for an AI system's risks | Weeks 1 and 2: what SH-1 is for, its lifecycle and who it touches |
| MEASURE | Analyzes and tracks risks; tests before use and regularly while running, and writes results down | Weeks 3 to 10: train a model by hand, test SH-1, read its log and check its security |
| MANAGE | Puts effort on mapped and measured risks, with plans to respond to, recover from and communicate about incidents | Weeks 11 and 12: an oversight plan, an incident log and the conditions for a pilot |
People in charge of one part of the lifecycle often cannot see or control the other parts.
Early choices about a system's purpose can change what it does.
The builder could not see Harbor Point's classes. So the crew's map of SH-1's purpose and setting matters.
| Statement | True or false? |
|---|---|
| GOVERN runs through the other three functions. | ? |
| The four functions are a checklist done once, in order. | ? |
| AI systems should be tested before they are used and regularly while they run. | ? |
| People in charge of one lifecycle part can always see every other part. | ? |
| MANAGE includes plans to communicate about incidents. | ? |
Sharp sorting, reviewer. Tomorrow is Impact Friday: is AI even the right tool for this job?