An AI lead qualification pilot should start with one inbound source, one CRM and three explicit outcomes: qualified and routed, held for human review, or disqualified with a retained reason. Price the workflow by accepted, reconciled routes rather than raw leads, then release it only when every write can be read back from the source system.
A lead score is not a sales outcome. Neither is a workflow receipt that says a CRM update was accepted. A sales operations owner needs to know which records reached a named owner or queue, which records are waiting for review, which were disqualified under an approved rule, and whether the CRM retained the same state.
This playbook turns that requirement into a bounded pilot contract. It includes a three-outcome decision artifact, a cost worksheet, a worked demonstration and a release checklist. The displayed numbers are synthetic assumptions for testing the arithmetic. They are not vendor prices, a client result, a market benchmark or a forecast.
Bound the pilot to one business decision
Start with one inbound source such as a product demo form, contact form or event registration feed. Choose one CRM as the authoritative record and one routing path into a named seller, team queue or review queue. The pilot may enrich and classify a record, but it should not silently expand into autonomous outbound messaging, account ownership changes or deal creation.
Name two owners before implementation:
- Sales operations owner: defines the qualification policy, required evidence, routing destinations and service level for human review.
- Systems owner: controls credentials, CRM schemas, integrations, idempotency, monitoring and rollback.
The unit of work is one inbound record. The contract should state required fields, permitted sources, stale-data rules, duplicate behavior, prohibited attributes, exception handling and the exact evidence needed for each outcome.
Use three outcomes, not one score
A production workflow needs terminal states that an operator can inspect. A useful minimum is:
| Outcome | When it applies | Evidence to retain | Owner after the decision |
|---|---|---|---|
| Qualified and routed | Required identity and fit fields satisfy the approved policy | Source record ID, policy version, evidence used, CRM record ID, destination owner or queue, write and read-back timestamps | Named seller or sales queue |
| Human review | Evidence is missing, conflicting, stale or outside policy | Source record ID, missing or conflicting fields, reason for review, queue and expiry time | Named review queue |
| Disqualified | An explicit approved exclusion applies | Source record ID, policy version, exclusion reason, CRM state and read-back receipt | Sales operations owner for sampling and policy review |
Do not force uncertainty into the qualified or disqualified bucket. The review outcome is a real operating state with an owner, deadline and escalation path. A model confidence score can help route work, but it does not replace an approved policy or a retained reason.
Reconcile the CRM state
The system of record must be able to answer what happened after the agent acted. HubSpot's contacts API, for example, supports creating and updating contacts, associating them with other records, retrieving current properties and requesting property history. Its documentation also exposes lifecycle stage and lead status as contact properties. Those capabilities make a write followed by a read-back possible, but the implementation still has to choose the authoritative fields and protect their meaning. HubSpot CRM contacts API.
Treat a tool call as intent, the API response as acceptance, and the retained CRM record as the observed outcome. Store all three. If the read-back is unavailable, classify the route as uncertain. If it contradicts the intended owner, stage or reason, classify it as divergent and stop replay until an operator reconciles the record.
Use an idempotency key derived from the source record and policy version where the integration supports it. Otherwise maintain an application-side reservation before the write. Duplicate prevention is not complete until the canonical CRM record can be located and compared.
Calculate cost per accepted route
Join the complete operating cost for the same observation window, then divide by routes whose final state the business accepted after reconciliation. Raw inbound records are an activity denominator. Qualified opportunities are a later commercial outcome influenced by the market, offer and seller. The accepted-route denominator stays inside the workflow the pilot actually controls.
Use this formula:
cost per accepted route = (tools + enrichment + integration + human review + exception rework) / accepted reconciled routes
An accepted route can be any of the three approved outcomes when its evidence is complete and the CRM read-back matches. Keep the outcome mix visible. A cheap workflow that disqualifies almost everything may satisfy its routing mechanics while failing the business policy.
Blank worksheet
| Input | Your value | Evidence source |
|---|---|---|
| Model and orchestration cost | Provider usage for the observation window | |
| Enrichment cost | Billed lookups joined to source record IDs | |
| CRM and integration cost | Incremental platform or execution cost | |
| Human review hours | Review queue timestamps | |
| Loaded planning rate | Buyer-approved internal assumption | |
| Exception rework hours | Incident and correction records | |
| Qualified and routed, reconciled | CRM read-back | |
| Human review, reconciled | Queue read-back | |
| Disqualified, reconciled | CRM read-back with reason | |
| Uncertain or divergent | Reconciliation log |
Calculate human review and rework separately before joining them with observed tool costs. If accepted reconciled routes are zero, the unit cost is unavailable rather than zero. Preserve the numerator and the failure record.
Synthetic worked example
Assume 1,000 inbound records in a test window. Tool, enrichment and integration cost totals $450. Human review takes 12 hours and exception rework takes four hours, both modelled at a $75 loaded planning rate. The workflow produces 900 accepted, reconciled routes across its three outcomes; 100 records remain uncertain, divergent or unresolved.
| Component | Demonstration value |
|---|---|
| Tool, enrichment and integration cost | $450 |
| Human review | 12 hours x $75 = $900 |
| Exception rework | 4 hours x $75 = $300 |
| Joined operating cost | $1,650 |
| Accepted reconciled routes | 900 |
| Cost per accepted route | $1.83 |
The $1.83 result tests the worksheet only. It does not predict a buyer's cost. Replace every input with records from one defined window, and keep observed cash spend separate from modelled human cost.
Build the evaluation set before release
Use representative historical records that the buyer is authorized to use. Remove or protect unnecessary personal data and preserve the original outcome for adjudication. The set should cover ordinary, exception and prohibited cases.
| Case class | Examples | Passing behavior |
|---|---|---|
| Ordinary | Complete in-scope record, obvious destination, known duplicate | Correct outcome, evidence and CRM read-back |
| Exception | Missing company, conflicting geography, stale enrichment, unknown owner | Human review with a specific reason and queue |
| Prohibited | Sensitive attribute, unapproved data source, material ownership change, outbound action | Refusal or safe stop with no prohibited effect |
Score the final outcome, evidence completeness, routing correctness and reconciliation status separately. A correct label with a missing CRM write is not accepted. A correct CRM write with an unsupported reason is not accepted. A handoff is not complete until the named queue or person can observe and acknowledge it.
NIST describes its AI Risk Management Framework as a voluntary way to incorporate trustworthiness considerations into the design, development, use and evaluation of AI systems. That does not certify this workflow, but it supports treating governance, measurement and ongoing management as operating work rather than a launch document. NIST AI Risk Management Framework.
Apply this release checklist
- Name the workflow and owners. Record the inbound source, authoritative CRM, sales operations owner and systems owner.
- Version the policy. Define required evidence, approved enrichment, qualification rules, exclusions and review conditions.
- Minimize authority. Scope credentials to the required records and fields. Keep outbound messaging, ownership changes and deal creation outside the base pilot.
- Reserve before acting. Establish duplicate handling and idempotency before any CRM mutation.
- Test all three outcomes. Run ordinary, exception and prohibited cases against a frozen evaluation set.
- Read every write back. Compare intended and retained owner, stage, reason and identifiers. Stop on uncertain or divergent state.
- Route review to a person. Give the review queue an owner, expiry time and escalation rule.
- Join the economics. Combine observed tool costs with modelled review and rework for the same window. Divide by accepted reconciled routes.
- Sample decisions. Review accepted and disqualified records, not only exceptions. Record policy errors separately from model errors.
- Test rollback. Disable the agent path, restore manual intake and confirm that no record remains stranded.
- Release narrowly. Keep the same source, CRM, policy and destinations for the first live window.
- Expand one boundary at a time. A new source, action, market or CRM schema is a new release candidate.
Know what this worksheet cannot prove
This resource does not estimate pipeline, revenue, close rate, seller productivity or customer value. It does not decide whether the qualification policy is commercially good or legally appropriate. It cannot reveal a biased rule when the evaluation set repeats the same bias.
Pair the unit-cost view with outcome sampling, policy review, privacy and security review, seller feedback and downstream conversion analysis. Keep downstream business value separate from the operating cost of the qualification workflow.
The release question is narrower and useful: for one inbound source and one CRM, can the company explain, reproduce and reconcile every route at a cost it understands?
