AI Workflow Automation Does Not Start With AI. It Starts With a Checklist
Why the strongest evidence for written processes comes from operating rooms, and why the same checklist failed when it was only filed away.
In 2009, a team led by Alex Haynes and Atul Gawande published a study in the New England Journal of Medicine. Eight hospitals introduced a one-page, 19-item surgical checklist. The death rate fell from 1.5 percent to 0.8 percent. Complications fell from 11.0 percent to 7.0 percent.
Five years later, the same journal published a second study. Ontario had pushed the checklist across 101 hospitals by policy. Deaths and complications did not change significantly.
Same idea. Same kind of list. Opposite result.
Most advice on AI workflow automation starts with the tool. Pick the platform, connect the apps, add an agent. The idea is not stupid. It is just one step too late. You cannot automate what you never wrote down, and a written process only helps if people actually run it.
I want to show you what the checklist research really says. I want to show you why a process on paper is not a process in use. And I want to give you a four-step system and a standard operating procedure template you can copy this week.
Key takeaways
- A 19-item checklist was associated with deaths falling from 1.5% to 0.8% and complications from 11.0% to 7.0% across 8 hospitals (Haynes et al., 2009).
- When Ontario rolled checklists out by policy across 101 hospitals, there was no significant change (Urbach et al., 2014). Reported compliance was high. Real use was in doubt.
- Before you automate a workflow, write it, run it and check it. Automation copies whatever process you give it, good or broken.
What the research actually shows
The common belief about automation goes like this. Processes are boring. Tools are smart. Plug in the right tool and the process sorts itself out.
The checklist research suggests the opposite order. First the process has to exist, in writing. Then it has to be used. Only then does it produce results.
One page, eight hospitals, fewer deaths
The World Health Organization designed the Surgical Safety Checklist as part of its Safe Surgery Saves Lives program. Haynes and colleagues tested it between October 2007 and September 2008.
The eight hospitals ranged from Toronto and Seattle to New Delhi and Ifakara in Tanzania. The team collected data on 3,733 patients before the checklist and 3,955 patients after. Deaths fell from 1.5 to 0.8 percent (P=0.003), complications from 11.0 to 7.0 percent (P<0.001).
One fair caveat. This was a before-and-after study, not a randomized trial. The authors wrote that the checklist was "associated with" the drop. Other things may have changed at the same time, including how closely teams knew they were being watched.
Still, look at what the list contains. It is split into three moments: before anesthesia, before the first cut, and before the patient leaves the room. One item asks all team members to introduce themselves by name and role. Another asks the surgeon how long the case will take and what the critical steps are.
That is not bureaucracy. It is a process written down at the exact point of decision.
What is a standard operating procedure? A standard operating procedure (SOP) is a written description of how a recurring task is done: the trigger, the steps in order, who owns it and how you know it is done. A checklist is the shortest form of an SOP. It does not explain everything. It makes sure the steps that matter most are not skipped.
The number nobody quotes: Ontario
In July 2010, the Ontario Ministry of Health began requiring hospitals to publicly report their checklist adherence. In practice, that made the checklist mandatory.
David Urbach and colleagues used this as a natural experiment. They compared 3-month periods before and after adoption across 101 hospitals. That covered 109,341 procedures before and 106,370 after.
The adjusted risk of death went from 0.71 percent to 0.65 percent. The difference was not statistically significant (P=0.13). Complications went from 3.86 percent to 3.82 percent (P=0.29). Their conclusion was blunt: the checklists were not associated with significant reductions in deaths or complications.
Why the difference? The Ontario hospitals reported compliance of 92 to 100 percent. Observers doubted that number. In an accompanying editorial, patient safety researcher Lucian Leape argued the likely reason was simple: the checklist was not actually used as intended.
There is a hint of how wide that gap can be. In neighboring Alberta, an observational audit found the checklist had been completed properly in only 41 percent of procedures, as reported by Healthy Debate.
And there is one more detail at the bottom of the WHO checklist itself. In small print it says the list "is not intended to be comprehensive" and that local additions are encouraged. The designers never meant it as a form to file. They meant it as a starting point that each team makes its own.
In short: writing a process down is necessary, but not enough. A checklist works when it is run at the moment of work by people who own it. Ticked boxes alone change nothing.
Why this matters for AI workflow automation
A 2023 McKinsey report estimated that current generative AI and other technologies have the potential to automate work activities that absorb 60 to 70 percent of the time employees spend at work today. That is a technical potential, not a forecast of what will happen.
Much of that time goes to recurring tasks: reporting, onboarding, follow-ups, handoffs. Those are exactly the tasks that rarely exist in writing.
An AI agent or automation tool needs the same thing a new hire needs. Clear steps, clear inputs, a clear definition of done. If those live only in your head, the tool will guess. And it will guess fast.
What is AI workflow automation? AI workflow automation means handing the steps of a recurring process to software that can read, decide and act, such as an AI agent connected to your email, CRM or documents. Classic automation follows fixed rules. AI automation can handle messy inputs, which makes a clear written process more important, not less.
The three mistakes almost everyone makes with automation
Mistake 1: Automating from memory
Here is an example. A solo consultant wants to automate client onboarding. She opens an automation tool and builds the flow from memory: contract, invoice, welcome email.
What she forgets is the step she always does by feel. She checks if the client needs a data processing agreement. It was never written down, so it is not in the flow. The first time it matters, nobody notices.
That means: if a step is not on paper, it is not in the automation.
Mistake 2: Writing a document nobody runs
This is the Ontario problem. A process document gets written for an audit or a handover. It goes into a shared drive. Nobody opens it while doing the work.
The team reports that the process exists. Technically, it does. But a document nobody uses at the moment of work is a record, not a process.
Mistake 3: Automating before checking
A broken process does not improve when it gets faster. It makes the same errors more often, with less attention on each.
If you never measured a task, you cannot tell if automation helped. The AI productivity study on developers showed how easily people feel faster while being slower. Without a baseline, you are guessing.
How to automate workflows: Write it, Run it, Check it, Automate it
Automation tools are not overrated. They are the last step, not the first. Here is a system in four steps.
Step 1: Write it. Pick one recurring task you do at least weekly. Write it down while you do it, not from memory. One page, no more than 10 steps. Every step starts with a verb and ends with a visible signal that it is done.
Step 2: Run it. Use the page for the next three runs, exactly as written. Every time you skip a step, guess or look something up, fix the page. Like the WHO list, it is meant to be adapted to your setting.
Step 3: Check it. Keep a short run log: date, time taken, errors, what was missing. Do not automate until you have two clean runs in a row. Now you have a baseline to measure against.
Step 4: Automate it. Hand the stable, rule-like steps to a tool or an AI agent. Keep a human check at the one step where an error costs the most. Then compare the run log before and after.
This is the same logic as the idea behind Gaapp.io, a tool to turn goals into weekly steps that shows what is due today and which bigger goal it serves. Disclosure: I am building it. A paper notebook works too. The written step matters more than the tool.
Standard operating procedure template
Copy this into any note app. It is short on purpose.
SOP: [Name of the process]
Owner: [One person]
Trigger: [What starts it, e.g. "new client signs contract"]
Frequency: [Daily / weekly / per event]
Inputs: [What you need before you start]
Done means: [The visible result at the end]
Steps (max 10, each starts with a verb):
1. [Verb + action] Done signal: [what you can see]
2. [Verb + action] Done signal: [what you can see]
3. [Verb + action] Done signal: [what you can see]
...
Critical check (never automate without a human):
- [The one step where an error costs most]
Run log:
| Date | Time taken | Errors | What was missing | Fixed in SOP? |
|------|------------|--------|------------------|---------------|
| | | | | |
Automation status: [Manual / Partly automated / Automated]
Last reviewed: [Date]
The run log is the part most templates leave out. It turns process documentation into evidence: when the process is ready to hand off, and whether the handoff worked.
Ask yourself this week:
- Which recurring task would break if I were sick for a week, because only I know the steps?
- Is my process documentation used at the moment of work, or only when someone asks for it?
- Do I know how long this task takes today, so I can tell if automation actually helped?
Conclusion: the checklist comes before the agent
A checklist that is not run is a document. A process that is not written cannot be automated.
The surgical checklist did not work because it was clever. It worked, where it worked, because people stopped at three fixed moments and said the important things out loud. Where it became a reporting duty, the effect disappeared.
AI workflow automation follows the same rule. Write the process. Run it until it is clean. Measure it. Then let the software take over the steps that are ready.
What you learned today
- A 19-item checklist was associated with deaths falling from 1.5% to 0.8% and complications from 11.0% to 7.0% in 8 hospitals.
- Rolled out by policy across 101 Ontario hospitals, the same idea showed no significant effect. Reported compliance was high, real use was doubtful.
- Automate only what is written, run and checked. Use a one-page SOP with a run log as your baseline.
FAQ
What is AI workflow automation? AI workflow automation means letting software, often an AI agent, carry out the steps of a recurring process, such as onboarding a client or preparing a weekly report. Unlike rule-based automation, AI can handle unstructured inputs like emails. It still needs a clear written process with steps, inputs and a definition of done to work reliably.
How do I start automating my workflows? Start with one task you repeat at least weekly. Write it down as a short checklist while you do it, run it three times and fix the gaps. Log time and errors. Once you have two clean runs in a row, hand the stable steps to a tool and keep a human check on the riskiest step.
What should a standard operating procedure template include? A useful SOP template includes the process name, one owner, the trigger, required inputs, a clear definition of done, and no more than 10 numbered steps that start with a verb. Add one critical check that stays manual and a run log with date, time taken and errors. Keep it to one page.
Do checklists really reduce errors? They can, but only when people use them. In the 2009 WHO checklist study, deaths fell from 1.5% to 0.8% in eight hospitals. A 2014 Ontario study of 101 hospitals, where checklists were introduced by policy, found no significant change. The likely difference was real use at the moment of work.
Which processes should I automate first? Automate processes that are frequent, rule-based and already documented. Good candidates are recurring reports, data entry between tools, meeting follow-ups and onboarding steps. Avoid processes that change every time or that nobody has written down yet. Write those first, run them manually, and automate once they are stable.
About the author
Tristan Eilenburg. One person. AI systems. Global reach. I write about building systems that work, with evidence instead of hype.
- X: @MrB2Beast
- LinkedIn: Tristan Eilenburg
- Instagram: @mrb2beast
- Building Gaapp.io (Instagram: @gaapp.io)
I write one of these every week. If this one taught you something, the newsletter is where the next one lands first.
Sources
- Haynes, A. B., Weiser, T. G., Berry, W. R., Lipsitz, S. R., Breizat, A. H., Dellinger, E. P., Herbosa, T., Joseph, S., Kibatala, P. L., Lapitan, M. C., Merry, A. F., Moorthy, K., Reznick, R. K., Taylor, B., and Gawande, A. A. (2009). A surgical safety checklist to reduce morbidity and mortality in a global population. New England Journal of Medicine, 360(5), 491-499. https://doi.org/10.1056/NEJMsa0810119
- Urbach, D. R., Govindarajan, A., Saskin, R., Wilton, A. S., and Baxter, N. N. (2014). Introduction of surgical safety checklists in Ontario, Canada. New England Journal of Medicine, 370(11), 1029-1038. https://doi.org/10.1056/NEJMsa1308261
- Leape, L. L. (2014). The checklist conundrum. New England Journal of Medicine, 370(11), 1063-1064. https://doi.org/10.1056/NEJMe1315851
- World Health Organization (2009). Surgical Safety Checklist, revised 1/2009. https://www.who.int/docs/default-source/patient-safety/9789241598590-eng-checklist.pdf
- World Health Organization (2009). Implementation manual WHO surgical safety checklist 2009: Safe Surgery Saves Lives. https://www.who.int/publications/i/item/9789241598590
- Healthy Debate (2015, February). Safe surgery checklists: the Canadian experience. https://healthydebate.ca/2015/02/topic/safe-surgery-checklist/
- Chui, M., Hazan, E., Roberts, R., Singla, A., Smaje, K., Sukharevsky, A., Yee, L., and Zemmel, R. (2023, June 14). The economic potential of generative AI: The next productivity frontier. McKinsey Global Institute. https://www.mckinsey.com/mgi/our-research/the-economic-potential-of-generative-ai-the-next-productivity-frontier/
- Hospitalist News (2014). Surgical checklists failed to improve outcomes. https://blogs.the-hospitalist.org/content/surgical-checklists-failed-improve-outcomes
