Flowgrammer

Human-Led AI System Design Checklist

A practical checklist for designing AI systems that make people more capable without hiding judgment or accountability.

— Craig Major

Flowgrammer designs AI systems to make people more capable and business more human. That starts with a simple design decision: the system should remove friction around people, not quietly remove the people who carry judgment and responsibility.

Use this checklist before approving a build.

1. Define the job

  • What repeated job is the system helping with?
  • Who does the work today?
  • What starts the workflow?
  • What output does the person need?
  • What would improve if the workflow worked reliably?
  • What is explicitly outside the first release?

If the job cannot be described in plain language, the system is not ready to build.

2. Separate rules from judgment

Write two lists:

  • decisions that follow stable, reviewable rules;
  • decisions that depend on context, trust, consent, sensitive information, or an exception.

The first list may be suitable for automation. The second list needs a human decision point, even if AI prepares the information.

3. Map the information

Record:

  • source systems;
  • fields required;
  • permissions;
  • data that may be missing;
  • conflicting values;
  • retention and deletion expectations;
  • who can correct an error.

An AI system cannot make a missing business rule disappear. It can only make the gap easier to see.

4. Design the handoff

For every uncertain or high-impact case, define:

  • who receives it;
  • what context they see;
  • what decision they must make;
  • how they record the decision;
  • what happens next;
  • what the customer or colleague is told.

A handoff is part of the system. It is not a failure state.

5. Test real cases

Use examples that include:

  • a normal case;
  • incomplete information;
  • conflicting information;
  • an unusual but valid case;
  • a case that must be escalated.

Record the expected result before testing. Otherwise, the team will judge the system by whatever it happened to produce.

6. Keep people in control

Before launch, answer:

  • Can a person pause the workflow?
  • Can a person correct an output?
  • Can the team see what the system did?
  • Can a decision be reversed?
  • Who owns the credentials, prompts, integrations, and documentation?
  • What happens when a connected service is unavailable?

7. Measure the work

Choose a baseline that the team can actually collect. Depending on the workflow, that may be:

  • time spent;
  • response time;
  • completion rate;
  • rework;
  • handoff delays;
  • qualified opportunities;
  • customer or employee experience.

Do not claim improvement until the before and after measures use the same definition.

8. Plan the first release

A production-ready first release has:

  • one clear workflow;
  • a named owner;
  • a small set of test cases;
  • defined human review;
  • documented failure paths;
  • a launch date;
  • a review date.

The AI Success Audit is designed for teams that need help turning this checklist into a ranked, costed roadmap.

Continue with Flowgrammer