Designing trust: how I cut task time 60% and errors from 15% to under 3% in a high-stakes AI compliance tool
A system-level behavioral redesign that replaced a rejected Appian (low-code platform) prototype with a human-in-the-loop platform built for high-stakes regulatory work.
6 min read · Jump to:Key decisionFinal designImpact
- Problem
- Regulation SMEs rejected an AI pilot that was technically accurate because they didn’t trust its output and it didn’t fit how they worked.
- My role
- Led UX strategy and end-to-end product design, partnering daily with the engineering and AI lead, PM, client product owner and SMEs.
- Key decision
- Turned a UI facelift into a behavioral redesign and chose accuracy over speed: human review on every AI line item, plus Excel export.
- Outcome
- The client awarded the contract for the platform.
The final review screen: SMEs check and mark each AI-extracted obligation.
Every new regulation kicks off a slow, manual review where mistakes are costly
Every time the FDIC or SEC releases a new regulation document, subject matter experts (SMEs) at large financial institutions go through a grueling, manual workflow:
Regulation document released
The FDIC or SEC publishes a new rule
Broken down into obligations
SMEs extract each requirement by hand
Map to internal policies
Each obligation matched to the bank’s controls
Create implementation plan
Plan the changes needed to comply
The cost of mistakes is high: in 2024 alone, the SEC collected $8.2 billion in compliance-related penalties. Regulations change constantly, and these institutions operate across overlapping global jurisdictions.
The AI pilot was accurate, but SMEs still rejected it
Our engineering team trained an AI model to break large documents into granular “obligations” and map them automatically. Instead of manual extraction, SMEs only needed to review and correct the AI’s work. They didn’t adopt it.
Accuracy didn’t earn adoption: the pilot broke SMEs’ trust and their workflow
I analyzed 15 recorded sessions of regulation change analysts working cases in their current Excel workflow and in our pilot, then ran contextual inquiries with SMEs.
Despite the model’s high accuracy, the pilot failed for four reasons
- 1
It didn’t fit their existing mental models or workflows.
- 2
They completely mistrusted the automated output.
- 3
They felt a lack of agency to properly correct what the AI generated.
- 4
It felt like it was slowing them down rather than speeding them up.
I mapped each trust risk to a specific design response
Automation bias
Trusting the system’s output blindly over personal judgment, leading to overlooked errors.
Mandatory “Mark as Reviewed”
Every AI line item needs an explicit human check before a case moves on.
Complacency
Vigilance drops when automation seems highly accurate over long periods.
Reviewer + Approver flow
A second SME approves each case and can send specific items back with comments.
Loss of situational awareness
Losing the big picture of the regulatory landscape when you don’t extract it yourself.
Traceability + mapping view
Each obligation links to its exact source text, and an alluvial chart shows how obligations map to controls.
Skill degradation
Fear of losing expertise when work shifts from active production to passive monitoring.
Excel integration
SMEs can export AI output into their familiar Excel layout and keep working as active editors.
That became two workflow changes I proposed to the team
Proposal 1: Excel integration

- Download AI output as .CSV to augment the existing workflow instead of replacing it
- Deep links to exact source document sections for easier referencing
Proposal 2: Human-in-the-loop

- A second SME as reviewer on every case to catch errors
- Mandatory “Mark as Reviewed” on every AI line item (active monitoring)
I chose accuracy over speed
I A/B tested a speed-focused workflow against an active-monitoring workflow with SMEs. Both beat the original manual workflow on time, and the speed-focused one was fastest, but it produced more than five times the errors. In a domain where one missed obligation can mean penalties, the lower error rate mattered more.
| Measure | Speed focus | Active monitoring Chosen |
|---|---|---|
| Error rate | 15% | <3% |
| Case completion | ~80% faster | ~50% faster |
| User confidence | Low | Higher |
5 SMEs per workflow. Case completion speed is compared to the original manual workflow.
The final platform keeps SMEs in control of every AI output
With the strategy validated, we designed and built the full web platform from scratch, centered on one clear review flow that keeps SMEs in charge of every AI output.

Drag the slider to compare the pilot (left) with the redesign (right).
A scannable table replaced hard-to-scan cards
The pilot used a card layout. SMEs pick and track cases by keywords, jurisdictions and mandates, so I moved to a table with quick filters and auto-sorting by status.
Four patterns keep humans meaningfully in the loop
Traceability
Every AI obligation shows the exact path to its source. Clicking it opens the original PDF with the source text highlighted.
Active monitoring
A mandatory “Mark as Reviewed” check on each line item tracks progress and counters drops in vigilance.
Excel integration
AI output downloads as a CSV structured like SMEs’ existing workflow, so adoption starts somewhere familiar.
Reviewer → Approver flow
Approvers comment on and flag specific obligations, returning them to the reviewer with precise context.
An alluvial chart shows obligation-to-control mapping at a glance
Each obligation can map to multiple internal compliance controls, and vice versa. The chart gives a macro view of mapping density, and clicking a ribbon filters the work items.
A design system built for dense enterprise data
I built a custom design system tailored to high-density data: workflow states, status tags and coverage levels that stay legible in long tables.
Plus a reference library and case history
Reference library: a network information architecture to explore regulations, policies, controls and their relationships. Case management: past cases and a manager overview.
See 5 more screensObligation detail, control mapping, tasks, risks and controls
The redesign won the client contract
The client awarded the contract after final testing. Pivoting from a cosmetic facelift to a behavioral redesign moved every metric that mattered:
Human-in-the-loop principles improved error catching.
UI/UX changes raised the product’s usability rating.
AI augmentation of the workflow plus platform efficiency.
What I’d bring to the next AI product
Measure the right metrics
The platform was sold on speed, but SMEs judged it on trust and risk.
Design for the human-as-editor shift
As AI automates work, users move from creators to curators of output, and UX has to optimize for that.
Design the transition from legacy tools
Forcing experts to abandon familiar software causes rejection, however good the new tool is. Adoption needs to be designed.


