Skip to content

Selected work / 01 / Human–AI workflow

Making AI-assisted quality review faster without removing human judgement

Financial services · Delivery coordination

The decisionAI drafts every score. Nothing counts until a reviewer has checked it.

AI took on transcription and first scoring of customer-call evaluations; reviewers kept the judgement. Internal reporting showed a 75% reduction in evaluation time and about R200k a year in cost saving.

ReportedFigures from internal reporting, confirmed by me for publication. The baseline and measurement period are not published.

CALL REVIEW
Fig. 2 AI fills in every row in pencil; the reviewer confirms or changes each one in ink.
  1. Customer callThe recorded call to be evaluated.
  2. AI draftAI produces the transcript and a draft of every score.
  3. Reviewer (person)Checks the draft and can change any score. Nothing counts until a reviewer has checked it.
  4. Score countsOnly after review, because scores affect individual staff.
  5. Role-based reportsFor reviewers and managers.
Call-quality evaluation, from recorded call to counted score. Reconstruction of the workflow. No internal screens, scoring criteria or staff data are shown.

Context

Quality assurance of customer calls at a financial services business, across three customer-facing teams.

The constraint

Reviewers spent most of each evaluation transcribing and structuring a call before they could judge it. The scores fed into how individual staff were assessed, so every score had to stay explainable and open to a person’s challenge.

My role

I led delivery coordination across the three teams: requirements, workflow design, stakeholder alignment and rollout.

The consequential decision

The fastest design would have let AI scores count straight away. We did not build it, because a person being assessed should be judged by a person. A reviewer can change any score the AI drafts.

What changed

  • Call transcripts converted into structured, draft scoring.
  • A reviewer able to adjust every AI-drafted score before it counts.
  • Role-based reporting for reviewers and managers.

Evidence

MeasureValueStatus
Time per quality evaluation75% reductionReported (internal reporting)
Annual cost savingAbout R200kReported (internal reporting)

What the work taught me

What I would repeat: give AI the preparation, and leave the decision with the person accountable for it.