50 Questions to Ask Before Implementing AI: SMB Buyer's Guide

This is a 50-question buyer's guide for AI implementation, organised by decision-maker role. The questions are not theoretical. Each was sourced from a real buyer asking it — vendor FAQ pages, Reddit threads, LinkedIn posts, Google People-Also-Ask, and trade publications — then filtered to the highest-signal canonical version for that role. Each answer is anchored to a named source.

How was this guide built?

We captured 252 real buyer questions across five decision-maker personas — CEO/Owner, COO/Operations Director, IT Manager/CTO, Quality/Compliance Manager, and Process/Operations Manager — from publicly visible asks: vendor due-diligence questionnaires (Atlas Systems, Vanta, Accelirate), Reddit threads where buyers compared notes (r/AiForSmallBusiness, r/manufacturing, r/cybersecurity, r/sysadmin, r/AIforEnterprises), LinkedIn posts from named operators, Google People-Also-Ask, and trade publications (HBR, McKinsey, CIO.com, Forbes, PwC). We filtered the 252 down to 50 canonical questions on four dimensions: buyer-stage fit, specificity, answerability from named evidence, and search-intent strength. Each persona has exactly ten questions; no topic dominates a persona's set. The full methodology — including the 0–12 scoring rubric used to filter — sits in our research workspace.

What surprised us most?

Three findings ran against received wisdom. First, HBR's 2026 cohort study found only 13% of employees actively resist AI — far below the 40–60% leaders typically assume when they over-invest in change communications. Second, PwC's 2026 CEO Survey found 56% of CEOs report no measurable AI return; the 12% who do report substantial gains share three traits the other 88% don't (CFO co-sign at the start, annual A/B recertification, top-3 strategic priority alignment). Third, the McKinsey 70% AI-failure number is not a technology indictment — it resolves to three issues none of which are "the model wasn't good enough": weak data foundations, insufficient API access to legacy systems, and absent human oversight.

Reading guide

Four answers (Q3.10, Q5.4, Q5.5, Q5.7) are flagged where the depth this question requires is not in any current cornerstone — they share a forthcoming supporting article on HITL design patterns. Where they appear, they are marked.

§1 CEO / Owner

The CEO's questions cluster around four decisions: whether the investment will pay back, how to defend the strategy at board level, how to avoid being fleeced by vendors, and what time horizon is realistic. The single biggest mistake at this level is treating AI as a procurement question rather than a capability question.

Q1.1 — Payback window for a 50–500 employee company

How long until our AI investment pays back for a 50–500 employee company?

Realistic payback for SMBs sits between 6 and 18 months, depending on which function you target. Customer service automation and internal knowledge retrieval typically clear payback inside 6 months; sales enablement and analytics land at 9–12; back-office process automation runs 12–18. Discount year-1 benefit projections by 20–30% for ramp and adoption drag. See our AI Strategy Framework for the full payback model.

Q1.2 — Why 56% of CEOs report zero AI ROI

Why are 56% of CEOs reporting zero ROI from AI, and what do the 12% who profit do differently?

PwC's 2026 CEO Survey found 56% report no measurable AI return while 12% report substantial gains. The profitable cohort shares three traits: a CFO co-signs every business case at the start, post-deployment value is recertified annually with A/B controls, and pilots are tied to a top-3 strategic priority — not to whichever team had budget. Read the AI Strategy Framework for the full pattern.

Q1.3 — Hidden cost lines

What are the hidden cost lines in AI projects — data prep, cloud infra, retraining, change management?

The visible licence fee is rarely more than 30% of true year-1 cost. Data preparation typically eats 20–35% (cleansing, labelling, pipeline build); cloud inference and storage another 10–20%; model retraining and prompt iteration 5–10%; change management, training, and incentives 15–20%. Apply a 0.7–0.8x reality haircut to vendor benefit claims. The AI Strategy Framework walks through the full TCO breakdown.

Q1.4 — Best risk-management framework for the board

What is the best AI risk-management framework for our board to adopt — NIST AI RMF, ISO 42001, or EU AI Act-aligned?

For UK/EU SMBs, EU AI Act-aligned governance is non-negotiable from August 2026 if you deploy any high-risk AI. ISO 42001 certification covers roughly 40–50% of EU AI Act requirements and gives your board defensible third-party validation; NIST AI RMF is voluntary but useful as a control reference. Adopt EU AI Act plus ISO 42001 as your spine, NIST as a checklist. See the AI Governance Guide.

Q1.5 — Who owns AI in the organisation

Who owns AI in our organisation — do I need a Chief AI Officer, an AI Lead reporting to the CIO, or distributed ownership across functions?

Below 500 employees, a dedicated Chief AI Officer is rarely justified. The defensible structure: an AI Lead reporting to the CIO or COO, owning architecture and vendor selection, with named function owners (Ops, Compliance, HR) accountable for use-case outcomes inside their domains. Distributed ownership without a single technical accountable role is the failure pattern. Our AI Talent Trap covers the role design.

Q1.6 — Avoiding ChatGPT-wrapper consultants

How do I tell a real AI implementation partner from a 'we'll AI your business' vendor selling £5k ChatGPT wrappers?

Three filters separate operators from theatre. First: ask for a working POC against your actual data inside two weeks — wrappers can't deliver this. Second: demand named integrations with your ERP or CRM, with reference clients you can call. Third: insist on a fixed-price pilot tied to one measured KPI, not a monthly retainer. If they refuse all three, walk. The AI Strategy Framework details vendor red flags.

Q1.7 — Questions to ask vendors before signing

What questions should I ask an AI vendor before signing a contract?

Cover four categories before signing. Data: where is data processed and stored, will it be used to train their models, is encryption applied at rest and in transit. Compliance: which frameworks (EU AI Act, ISO 42001, SOC 2 Type 2) do they certify against. Failure: what is the SLA on accuracy, not just uptime, and the incident-response plan. Exit: model portability and data deletion. Full questionnaire in the AI Governance Guide.

Q1.8 — Build vs buy vs hire AI Lead

Should I build AI capability in-house (hire an AI Lead), buy off-the-shelf tools, or partner with an AI consultancy for our 50–500 employee company?

At 50–250 employees, buy first and partner narrowly: off-the-shelf vendors for commodity functions (email, support, sales enablement), a specialist consultancy for one custom integration. At 250–500, hire an AI Lead at £80–120k to own architecture, vendor governance, and internal upskilling — a consultancy at £100k+ retainer rarely matches a salaried lead's continuity. Build in-house only with an existing engineering org. See the AI Talent Trap.

Q1.9 — Short-term ROI vs longer rewire

Are short-term AI returns realistic, or should we plan for an 18–36 month transformation?

Both are real, on different scopes. Function-level wins (support deflection, document drafting, code assist) clear payback in 3–9 months. Process-level rewires (procurement, claims, underwriting) take 12–24 months because they require system integration and SOP redesign. Operating-model overhaul — the 18–36 month horizon — should be sequenced last and funded by the early wins. The AI Strategy Framework sets the planning windows.

Q1.10 — Why 70% of AI projects fail

70% of AI efforts fail to deliver value — what are the three issues that kill them, and how do I avoid them?

McKinsey's 70% failure stat resolves to three root causes: weak data foundations (model trained on incomplete or stale data), insufficient API access to legacy systems (the integration-debt problem), and absent human oversight (no one accountable when the model drifts). Avoid them by sequencing data audit before pilot, scoping integration before vendor selection, and naming a human owner per use case. The AI Strategy Framework maps the antidotes.

If you take three things from the CEO section: discount the vendor projection by 20–30%, name a single AI Lead before signing any contract, and write a kill criterion before starting the pilot.

§2 COO / Operations Director

The COO's questions cluster around delivery: which processes to start with, how to manage the change without losing the workforce, what to measure, and how to avoid pilot purgatory. The most expensive shortcut at this level is skipping the pre-rollout baseline.

Q2.1 — Most painful or highest-volume first

Which of my processes should I automate with AI first — the most painful, or the highest-volume?

Neither alone wins. The defensible criterion is highest-volume process where pain blocks scaling — repetitive, rule-bounded work currently capped by headcount. Manufacturing's only validated AI deployment to date is vision QA, scoring high on both axes. Avoid pure-pain processes that are low-volume (one-off complexity) and pure-volume processes already running smoothly. Score candidates on volume × marginal pain × data quality. The AI Strategy Framework includes the scoring rubric.

Q2.2 — 6-step pilot framework with budget

What's the 6-step pilot project framework with timeline and budget per phase for an SMB?

A defensible SMB pilot runs 6–8 weeks across six steps: scope and KPI definition (week 1, ~5% of budget), data audit and access (week 2, ~10%), vendor or model selection (week 3, ~5%), build and integrate (weeks 4–5, ~40%), controlled rollout to 10–20% of volume with weekly review (week 6, ~25%), measurement and go/no-go decision (weeks 7–8, ~15%). Total pilot budget: £20–60k. See the AI Strategy Framework.

Q2.3 — Employee resistance to AI

Why do employees resist AI — fear of job loss, mistrust, or skill gaps — and how do I address each?

HBR's 2026 data found only 13% of employees actively resist AI — far below the 40–60% leaders assume. Fear of job loss responds to explicit no-layoff commitments tied to AI savings. Mistrust responds to showing the model's confidence score and an override path. Skill gaps respond to paid training, not lunchtime webinars. Treating all three as one problem is the failure pattern. See Scale, Don't Cut.

Q2.4 — Realistic year-1 adoption rate

What's a realistic year-1 adoption rate I should plan for — 60% or 100%?

Plan for 60%, not 100%. Vendor case studies anchor on stable-state adoption (year 2+), not the year-1 ramp curve. A 60% year-1 baseline accommodates the 3–6 month ramp where managers learn to integrate AI output into existing workflows, plus normal attrition and onboarding lag. Budget AI benefit at 60% of vendor projection year 1, 85% year 2, 95% steady state. Scale, Don't Cut covers ramp planning.

Q2.5 — KPIs to track

What KPIs should I track for an AI project — cycle time, error rate, throughput, exceptions per 1000?

Track four operational KPIs and one financial. Cycle time per transaction (target 30–60% reduction), error or rework rate (target stable or down), throughput per FTE (target 1.5–3x), exceptions per 1000 transactions (target measurably lower than human baseline). Convert to a single financial KPI: cost per outcome. Freeze a 30-day pre-rollout baseline before pilot — without it, you have no defensible proof of value. AI Strategy Framework.

Q2.6 — Pre-rollout baselines

How do I freeze pre-rollout baselines so I can prove AI value six months later?

Capture a 30-day baseline before the pilot starts: cycle time, error rate, throughput, exception count, FTE hours per 100 transactions, customer-facing SLA hits. Store it in a CFO-co-signed document, not a slide. After rollout, run an A/B comparison — pilot cohort versus a held-back control of comparable volume — for 90 days minimum. Without the frozen baseline, finance will not certify the savings. AI Strategy Framework.

Q2.7 — Internal capability vs consulting dependency

Am I building internal AI capability for the long term, or am I entirely dependent on consultants for the next 12 months?

If a consultancy could walk away tomorrow and your AI workflows would stall within 30 days, you have dependency, not capability. Capability is measured by three tests: at least one named internal owner per deployed use case, prompt and integration assets stored in your repo not the vendor's, and a costed 18-month plan to absorb the work. Without those, every renewal is a hostage negotiation. AI Talent Trap.

Q2.8 — POC structure

Should we run a 4–6 week POC using our real data and systems before committing — and what's in scope?

Yes — committing to a vendor without a real-data POC is the most expensive shortcut in AI procurement. Scope: one bounded use case, real production data (not vendor demo data), integration with your actual ERP or CRM, and a single binary success criterion (e.g., 80% accuracy at 50% of cost). 4–6 weeks is enough; longer is scope creep. Run controlled — 10–20% of volume only. AI Strategy Framework.

Q2.9 — Pilot purgatory

How do I prevent 'pilot purgatory' — projects sitting at post-pilot assessment for six years?

HBR found companies stall at post-implementation assessment for six years on average because no one is empowered to call go or kill. Prevent it with three rules: a named decision-maker for each pilot (CFO or COO, not the AI lead), a hard 90-day post-pilot decision deadline, and a kill criterion in writing before pilot start. If kill criterion is hit, shut it down — don't iterate forever. AI Strategy Framework.

Q2.10 — Training budget allocation

How much of my AI budget should go to training, change champions, and incentives — is 15–20% right?

15–20% is the floor, not the ceiling. Training and change-management spend below 10% is the strongest predictor of pilot purgatory; above 25% is rare but defensible for high-touch deployments (clinical, regulated). Allocate inside the band: 8–10% on role-specific training, 4–6% on change champions and override authority, 3–5% on incentives tied to adoption KPIs. Cutting this line to fund more software is the standard mistake. Scale, Don't Cut.

If you take three things from the COO section: freeze a 30-day baseline before the pilot starts, write the kill criterion before starting, and budget 15–20% to training as the floor.

§3 IT Manager / CTO

The IT/CTO's questions cluster around four areas: integrating AI with legacy systems that were never designed for it, locking down security across new attack surfaces, conducting due diligence on AI vendors at depth beyond standard SOC 2, and governing models that can drift in production. The single biggest mistake at this level is treating an AI vendor with the same DD discipline as a SaaS vendor.

Q3.1 — Typical integration timeline

How long does a typical AI integration take for legacy ERP/CRM systems like ours — 8 weeks or 20?

Plan 8–20 weeks. The 8-week end is bounded SaaS-to-SaaS work (modern Salesforce, HubSpot, NetSuite) where APIs exist and data quality is good. The 20-week end is on-premise SAP or AS/400 mainframe where you need RPA bridges, custom middleware, or screen-scraping plus a 6–8 week data-cleansing prequel. 70% of Fortune 500 software is 20+ years old — assume the longer band unless you have evidence otherwise. Hidden IT Integration Debt.

Q3.2 — Four challenges of agentic AI on legacy

What are the four challenges of applying agentic AI to legacy systems — integration, ROI, security, hallucination?

Yes — and they compound. Integration: agents need API access legacy systems often lack, so RPA or middleware is the bridge. ROI: agentic projects fail measurement when no baseline exists for the work the agent now does. Security: an agent with write access to your ERP is a privileged user — RBAC must apply. Hallucination: confidence-score routing and HITL on sensitive actions are mandatory. See Hidden IT Integration Debt.

Q3.3 — APIs vs RPA

Do I need APIs to integrate AI agents with existing systems, or can RPA bridge the gap on a mainframe?

APIs first, RPA where APIs don't exist, never both for the same path. APIs give you reliable contracts, version control, and audit trails — preferred for any system-of-record write. RPA bridges read-only mainframe and legacy GUIs where API access is unavailable, but is brittle to UI changes and slow under load. Budget API work as 60–70% of integration cost; RPA scope as 20–30%. Hidden IT Integration Debt covers the decision tree.

Q3.4 — Prompt injection prevention

How do I prevent prompt injection, data poisoning, and adversarial attacks on AI systems we deploy?

Layer four controls. Input validation with allow-lists on user-supplied content reaching the model. Output filtering against known jailbreak patterns and PII leakage. Sandboxed tool-use — agents must not execute arbitrary code or hit endpoints outside an explicit allow-list. Adversarial testing pre-deployment using OWASP LLM Top 10 cases. Treat your model the same way you treat user input: never trusted. AI Governance Guide covers the full control set.

Q3.5 — Data residency for regulated data

Where is data processed and stored when we use a hosted AI model, and does it comply with GDPR, HIPAA, and SOC 2?

Demand a written data-flow map from the vendor: regions for inference, regions for storage, retention period, sub-processors. For GDPR (EU residents), processing outside the EEA needs Standard Contractual Clauses plus a transfer impact assessment. For HIPAA, you need a signed BAA before any PHI touches the model. SOC 2 Type 2 attestation is necessary but not sufficient — it does not certify model behaviour. AI Governance Guide.

Q3.6 — AI Engineer hire vs internal upskilling

Do I need to hire a dedicated AI Engineer, or can my existing engineering team cover AI integration with structured upskilling?

Below 10 engineers, hire one specialist; existing team can't absorb AI integration on top of feature delivery without quality drag. Above 10 engineers, upskill 2–3 senior engineers (£8–15k each in structured training and certification) to anchor capability, with one external AI Engineer hire (£90–140k) to seed practice. Pure upskilling without a senior anchor produces shallow expertise that fails under integration load. AI Talent Trap.

Q3.7 — Required vendor DD documents

What documents should I request from any AI vendor — SOC 2 Type 2, SBOM, model cards, pen-test reports?

Six artifacts non-negotiable. SOC 2 Type 2 (most recent), ISO 27001 cert, ISO 42001 cert if available, model cards documenting training data and known limitations, SBOM (software bill of materials) for the model and dependencies, and the most recent independent pen-test summary. Add a DPA covering EU AI Act Article 26 deployer obligations. Refusing any of the six is a red flag. AI Governance Guide.

Q3.8 — AI-specific compliance bar

Which AI-specific regulations and frameworks should the vendor comply with — EU AI Act, NIST AI RMF, ISO 42001?

All three, with different weight. EU AI Act (Article 25 provider obligations) is mandatory if the vendor's model is high-risk and used in the EU — non-compliance from August 2026 carries fines up to €35M or 7% turnover. ISO 42001 is the credible third-party signal of an AI management system. NIST AI RMF is voluntary but increasingly required by US enterprise procurement. Demand all three by name. AI Governance Guide.

Q3.9 — SOC 2 sufficiency for AI vendors

Can a vendor's SOC 2 Type 2 alone count as AI vendor due diligence, or do I need a separate AI questionnaire?

SOC 2 Type 2 attests to information-security controls — it does not test model behaviour, training-data provenance, hallucination rates, or human-oversight design. Roughly 60% of AI-specific risks (prompt injection, training data contamination, model drift) sit outside SOC 2 scope entirely. Treat SOC 2 as table stakes; require an additional AI questionnaire covering model lifecycle, evaluation methodology, and incident-response specifically for model failures. AI Governance Guide.

Q3.10 — Confidence threshold for HITL in regulated workflows (supporting article forthcoming)

What confidence threshold should trigger human review of an AI agent's decision in a regulated workflow?

No universal threshold exists — calibrate per use case. For regulated workflows (clinical, financial advice, hiring), set the bar at 95%+ model confidence to auto-act, route 80–95% to fast human review, and queue below 80% for full review. Tune monthly using sampled audits of auto-acted decisions. The model's reported confidence is not its actual accuracy — measure both. For the regulatory frame, see the AI Governance Guide (Article 14 human oversight). Pattern-tuning depth: supporting article forthcoming.

If you take three things from the IT/CTO section: assume the longer integration band, demand the six-artifact DD pack, and treat your model output as untrusted user input.

§4 Quality / Compliance Manager

The Compliance Manager's questions cluster around four areas: EU AI Act readiness for the August 2026 deadline, ISO 42001 certification scope and limits, audit readiness when a regulator asks for a specific record, and integration of three frameworks (EU AI Act + ISO 42001 + GDPR) into a single defensible programme. The single biggest mistake at this level is treating ISO 42001 certification as the answer rather than the spine.

Q4.1 — What is a FRIA

What is a Fundamental Rights Impact Assessment (FRIA) under EU AI Act Article 27, and who must conduct one?

A FRIA is a documented assessment, mandated by EU AI Act Article 27, that high-risk AI deployers must produce before deployment. It captures the affected categories of natural persons, the specific harms the system could cause, the human-oversight measures, and the complaint and redress paths. Public authorities and private deployers of public services (credit scoring, recruitment, education, essential services) are in scope. See FRIA Article 27 explainer.

Q4.2 — FRIA vs DPIA

How is a FRIA different from a Data Protection Impact Assessment (DPIA), and can my existing DPIA satisfy FRIA requirements?

No — your DPIA cannot replace your FRIA. A DPIA assesses personal-data processing risk under GDPR Article 35; a FRIA assesses fundamental rights impact under EU AI Act Article 27. FRIA adds five elements DPIA doesn't cover: AI-specific use-case description, deployment frequency, affected-person categories (not just data subjects), human-oversight design, and the complaint mechanism. Reuse 30–40% of DPIA evidence; rebuild the rest. FRIA vs DPIA mapping.

Q4.3 — August 2026 deadline

What does the August 2026 EU AI Act deadline actually mean for a deployer of high-risk AI systems?

From 2 August 2026, high-risk AI obligations under the EU AI Act become enforceable. As a deployer you must: have a completed FRIA on file before deployment, run human oversight per Article 14, log system operations, and notify market surveillance on serious incidents. Penalties scale to €15M or 3% turnover for deployer breaches. Most SMBs need 4–6 months prep — start gap analysis now. EU AI Act 2026 deadline.

Q4.4 — ISO 42001 ↔ EU AI Act overlap

Will ISO 42001 certification help with EU AI Act readiness, or are they separate compliance tracks?

Yes — and complementary, not redundant. ISO 42001 covers roughly 40–50% of EU AI Act technical and process requirements: AI management system, risk assessment, human oversight, supplier management. Your auditors can reuse ISO 42001 evidence packs for the EU AI Act technical documentation file (Annex IV). Run a single integrated programme — separate tracks duplicate cost by 60–80%. ISO 42001 cert also gives procurement leverage. ISO 42001 ↔ EU AI Act overlap.

Q4.5 — Why ISO 42001 alone won't satisfy the EU AI Act

Why won't ISO 42001 certification by itself make my AI system EU AI Act compliant — what's the gap?

ISO 42001 certifies your AI management system; it does not certify any individual AI system meets EU AI Act high-risk product requirements. The gap covers the other 50–60%: conformity assessment per Article 43, technical documentation per Annex IV, post-market monitoring per Article 72, and the FRIA itself. Treat ISO 42001 as the spine, not the answer. prEN 18283 will close some of this gap. ISO 42001 coverage gap.

Q4.6 — Agent action records for regulator review

Can I produce a complete record of every action my AI agent took, on what data, and why, for regulator review?

If you can't, you fail an EU AI Act Article 12 inspection. The audit trail must capture: timestamp, agent identity, input data references (not the data, the references), tool calls made, output produced, confidence score, and human-review status. Retain 6 months minimum, 7 years for high-risk under Article 19. Use immutable append-only storage with hash chaining; standard application logs are insufficient. Audit-trail design for AI agents.

Q4.7 — 15 essential ISO 42001 documents

What 15 essential documents must I have on file for an ISO 42001 audit?

AI policy, AI scope statement, risk register, AI impact assessment template, AI system inventory, supplier register with AI questionnaire results, training records, internal audit plan and reports, management review minutes, corrective-action register, AI incident log, data-quality records, model evaluation reports, deployment authorisation forms, and continual improvement evidence. Build them as templates first — auditors test reuse. Allow 8–12 weeks to produce all 15 from scratch. ISO 42001 audit document checklist.

Q4.8 — DPIA trigger under GDPR Article 35

When does AI processing trigger a DPIA under GDPR Article 35, and what specifically must the DPIA cover?

AI processing triggers a DPIA whenever it meets one of three conditions: systematic evaluation including profiling with legal effects, large-scale processing of special-category data, or systematic monitoring of public spaces. The DPIA must cover purpose, necessity test, risks to data subjects, and mitigation measures. AI adds a fourth requirement in practice: explainability of automated decisions per Article 22. Document before processing starts, not after. GDPR Article 35 + AI processing.

Q4.9 — Three-framework integration

How do I align ISO 42001, EU AI Act, and GDPR into one integrated compliance programme?

Single risk register, single supplier register, single incident log — three lenses on top. Map each control once: GDPR Article 35 (DPIA), EU AI Act Article 27 (FRIA), and ISO 42001 Clause 6.1 (risk assessment) all consume the same risk-identification work. Run one annual programme cycle covering all three. Separate programmes duplicate effort by 60–80% and produce conflicting evidence under audit. Three-framework integration.

Q4.10 — TPRM under both frameworks

How do I do third-party risk management under ISO 42001 and EU AI Act simultaneously without duplicate questionnaires?

Issue one questionnaire mapped to both frameworks. The shared core covers: AI system inventory, training data provenance, model evaluation methodology, sub-processors, security certifications, incident response. Tag each control with both an ISO 42001 Annex A reference and an EU AI Act Article number; auditors accept dual-tagging. Re-run annually for active vendors, on contract change for inactive. Duplicate questionnaires double cost without adding signal. TPRM under both frameworks.

If you take three things from the Compliance section: a DPIA cannot replace a FRIA, ISO 42001 is the spine not the answer, and one integrated programme beats three duplicated ones by 60–80% on cost.

§5 Process / Operations Manager

The Process Manager's questions cluster around six areas: SOP redesign vs augmentation, human-in-the-loop design, training operators for the 90/10 split, exception handling and fallback, performance measurement, and cross-functional governance. Four answers below (Q5.4, Q5.5, Q5.7) flag where current cornerstones don't cover HITL pattern depth — those four feed a forthcoming supporting article on HITL design.

Q5.1 — Workflow redesign vs augmentation

Should I redesign my workflows from first principles around AI, or just augment existing SOPs?

Augment first, redesign where the existing SOP is the bottleneck. Augmentation captures 60–70% of available value within 90 days because no process change is needed. First-principles redesign is justified only when AI changes the unit economics enough that the existing handoff structure is wasteful — typically procurement, claims, and underwriting. Redesign without augmentation experience underestimates what AI actually does. AI Strategy Framework.

Q5.2 — From SOP to agent playbook

When are my SOPs broken or just obsolete, and how do I rewrite them as AI agent playbooks?

Three signals your SOP is obsolete: it lists tools that no longer exist, exception rates above 15% on stable volume, or training-time-per-hire creeping up. Rewrite each step as input contract, decision criterion, tool call, output contract, exception path. Keep the human override explicit at each decision. A playbook a human can execute is also one an agent can run. AI Strategy Framework.

Q5.3 — When to use HITL

When should I use human-in-the-loop in AI workflows — low confidence, sensitive actions, regulatory implications, or empathy-required tasks?

All four — and they're additive, not alternative. Low confidence: route to human when calibrated confidence drops below your accuracy floor. Sensitive actions: any irreversible step (payment, contract, hiring decision, clinical advice). Regulatory: anything in EU AI Act high-risk scope per Article 6 — Article 14 makes human oversight mandatory there. Empathy: complaint, bereavement, or vulnerability flags. Build trigger logic upstream of the model. HITL trigger criteria.

Q5.4 — HITL pattern catalogue (supporting article forthcoming)

What are the four main HITL patterns — approval flow, confidence-based routing, escalation paths, and feedback loops?

Four patterns, used in combination. Approval flow: agent proposes, human approves before action — slowest, safest for new deployments. Confidence routing: agent acts above threshold, human reviews below — most common in mature workflows. Escalation paths: agent acts but flags edge cases for async review. Feedback loops: human corrections become training data for the next iteration. Combine them per workflow. For regulatory framing, see the AI Governance Guide. Pattern-catalogue depth: supporting article forthcoming.

Q5.5 — HITL threshold tuning (supporting article forthcoming)

What confidence threshold should trigger HITL escalation, and how do I tune it without killing velocity?

Start conservative — auto-act above 95%, fast review 80–95%, full review below 80%. Tune monthly using two metrics. False-pass rate: how often the model auto-acted on cases humans would have flagged. Reviewer agreement rate: how often humans rubber-stamp the model. If reviewers agree 95%+ on a tier, raise the auto-act threshold there. If false-pass exceeds tolerance, lower it. Velocity follows from calibration, not loose thresholds. AI Governance Guide. Tuning methodology: supporting article forthcoming.

Q5.6 — 90/10 operator training

How do I train operators when 90% of work is automated and 10% requires human evaluation?

Invert the training. Operators no longer need step-by-step procedural training; they need pattern-recognition training to spot the 10% the model gets wrong. Show 100+ paired examples of correct and incorrect AI output per role, run weekly calibration sessions where reviewers compare verdicts on the same cases, and track inter-rater agreement as a KPI. Without calibration, every reviewer drifts to their own threshold within 8–12 weeks. Scale, Don't Cut.

Q5.7 — Sub-threshold fallback design (supporting article forthcoming)

What's the fallback when AI validation confidence falls below threshold — auto-reject, queue, or alert?

Three tiers based on time-sensitivity. Auto-reject only for low-stakes, high-volume work where false positives are cheaper than reviewer load (spam, basic categorisation). Queue when delayed processing is acceptable and the case carries financial or regulatory weight. Real-time alert with hard pause when an irreversible action is pending — payment, contract, customer-facing message. Mixing tiers within one workflow is normal; one global rule across all use cases fails. AI Governance Guide. Fallback design: supporting article forthcoming.

Q5.8 — Manual standby procedures

What manual processes do I keep on standby if the AI service goes down or the vendor discontinues it?

Maintain three controls. Document the pre-AI manual SOP in a runnable state and rehearse quarterly — atrophied muscle memory is the failure mode. Hold one engineer-week of work to switch from API to RPA fallback if the vendor goes down. Negotiate a 90-day data-and-prompt portability clause in the contract. Without portability, vendor discontinuation is a forced rebuild. EU AI Act Article 9 treats this as mandatory. AI Governance Guide.

Q5.9 — Workflow KPIs

How do I measure AI workflow performance — accuracy, deflection rate, cycle time, exceptions per 1000?

Track five at the workflow level. Deflection rate: % of cases the agent closes without human touch. Cycle time per closed case: median, P95, P99 — the tail matters for SLAs. First-pass accuracy: agent verdict matches gold-standard on a sampled audit. Exceptions per 1000 vs human baseline. Cost per closed case. Track all five from day one — adding a metric six months in destroys the time series. AI Strategy Framework.

Q5.10 — 5-pillar governance

What's the 5-pillar AI workflow governance model — authorization, audit, data boundary, escalation, drift detection?

Authorisation: who can deploy a new agent, who signs off scope changes, role-based access on production. Audit: append-only log of every agent action with retention per Article 19. Data boundary: explicit allow-list of sources the agent can read or write. Escalation: HITL triggers per workflow with named human owner. Drift detection: weekly model evaluation against a frozen test set, alarm at 5% accuracy regression. 5-pillar governance.

If you take three things from the Process Manager section: augment before you redesign, calibrate before you loosen thresholds, and rehearse the manual fallback quarterly because muscle memory atrophies.

Where should you start today?

Three steps, in order, run them this month.

1. Audit your current AI risk exposure. List every AI tool any employee has touched this quarter — including the ChatGPT and Copilot subscriptions you didn't authorise. Map data flow, regulatory scope, and contract status against EU AI Act Article 26 deployer obligations and GDPR Article 35 trigger criteria. The gap between what you think you're using and what you're actually using is the first finding of any honest audit.

2. Align with one cornerstone framework, not three. EU AI Act + ISO 42001 + GDPR run as one integrated programme; running them separately duplicates effort by 60–80% and produces conflicting evidence under audit. Pick the spine — ISO 42001 for SMBs serious about procurement leverage, EU AI Act + GDPR as the regulatory floor — and fold the rest into it.

3. Book a 90-minute working session. These 50 questions are the exact questions we work through with every audit client. If you'd rather have us run them with you and produce a decision-ready plan — current AI risk exposure, named cornerstone framework alignment, and a 90-day priority sequence with kill criteria — that's the AI Foundation Audit. The output is a board-readable document the day we finish, not a deck six weeks later.

The 50 questions above cover the decision surface. The audit closes the gap between asking them and having defensible answers in writing.