Librarian Joss tapes a sign to the library door: "Tech Review Panel, Friday, 3 pm. SH-1 decision."
Comet spins her pencil. "Four days! But first I want to know something. Who actually does all this work in real life?"
"You mean jobs?" Wren asks. "What does the evidence say? NIST lists the tasks people do across the AI lifecycle."
Comet flips back through her sketch notebook. "We have met a lot of those people already. The builder, the teachers, the panel, us."
Nova projects a long row of empty name tags on the wall. "Would you like a hint?" she asks. "Match each person in our story to the task they do."
Wren hands you a marker. "Reviewer, who does what?"
This is the last week of the SH-1 Review. SH-1, the builder and Harbor Point are a made-up example from start to finish.
This week the crew finishes the model card and the risk register and presents a recommendation to the panel.
AI actors are the people and organizations who play an active role in an AI system's lifecycle. That includes those who deploy or run it.
NIST's lifecycle has design work, development work, deployment work, and operation and monitoring.
Testing, evaluation, verification and validation (TEVV) happen all the way through. Risk management should start with planning and design.
| AI actor task | What the work is | Who does it in the SH-1 story (made up) |
|---|---|---|
| Design | Write down the concept, goals, assumptions, setting and requirements; gather, clean and document data | The builder, when it planned SH-1 |
| Development | Build, choose, tune, train and test models | The builder |
| Deployment | Pilot the system, check it works with older systems, follow the rules, manage the change, check what using it is like | Harbor Point, if the pilot goes ahead |
| Operation and monitoring | Run the system and regularly check its output and effects with others | Teachers and staff during a pilot |
| TEVV | Examine a system, find problems and fix them; can start as early as design | The crew, with TS-40, CK-20 and RT-8 |
| Human factors | Involve end users, design and check the user experience, test with people | The crew, designing the report form |
| Domain experts | Know the field and help design the system and make sense of its outputs | Teachers Mara, Dev, Lin and Owen |
| AI impact assessment | Check accountability, work against harmful bias, examine effects, safety and security | The crew, with the risk register |
| Procurement | Get an AI model, product or service from an outside developer | The district office |
| Governance and oversight | Leaders responsible for the organization that builds or uses the system | Librarian Joss and the Tech Review Panel |
Ideally, the people who check a model are not the same people who build and use it.
That is why the crew, not the builder, ran TS-40. The builder's own "92 out of 100" was on a trivia test that did not match how students would use SH-1.
| Statement | True or false? |
|---|---|
| AI actors include people who deploy or run an AI system. | ? |
| TEVV happens only at the very end of the lifecycle. | ? |
| Risk management should start with planning and design. | ? |
| Ideally, the people who check a model also built it. | ? |
| Teachers can be domain experts for an AI study helper. | ? |
Great matching, reviewer. Tomorrow the crew meets the people of AI who never build anything.