Classify a document and extract its data with AI

A document arrives with no indication of what it is. In this walkthrough you set QBO up to classify it into one of your document templates, extract the values you care about into a score (each with a confidence), and route anything the model wasn’t sure about to a person. QBO decides when AI runs and where every answer goes. The model only reads the document it’s given and answers the question it’s asked.

The example uses mortgage documents (a note, a loan modification, a hardship letter), but the same steps work for invoices, claims forms, policies, or any other document you receive in volume.

Who it’s for

Power users who maintain document and score templates. The final step touches workflow configuration.

Prerequisites

• AI configured for your installation, with at least one named AI service (an agent). Your administrator can tell you its name; this walkthrough uses Gemini. See AI agents.
• The AttachmentAiClassify permission, to use Classify from the documents menu.
• Permission to edit document templates and score templates.
• A few sample documents of each kind you want to recognize.

Building blocks used

• Document Management: document templates and the Documents panel
• AI: document classification and AI scoring engines
• Scoring: score templates, score items, and confidence
• Workflow and Worklists: automation and human review
• QDocs packages these capabilities for document-heavy operations

Steps

1. Describe each document template

Classification chooses among your document templates that have a description. The description is the only thing the model sees about each template, so write it as you’d brief a new colleague: what the document is, and how to tell it apart from similar ones.

Document template	Description
Mortgage Note	The borrower’s promise to repay a mortgage loan, stating the principal, interest rate, payment amount, and repayment terms.
Loan Modification	An agreement that changes the terms of an existing loan, such as a new balance, rate, or payment. Refers back to the original note.
Hardship Letter	A letter from a borrower explaining why they can’t make payments, and usually what they’re asking for.
Other	Anything not described above.

Leave the description empty on templates the model should never choose, such as generated letters.

Tip

A catch-all template such as Other gives the model a safe answer when nothing fits, so it doesn’t force a document into the closest wrong template.

2. Classify one document by hand

1. Open a record that has an unclassified document, and go to its Documents panel.
2. Select the document and choose Classify from the menu.
3. Classification runs in the background. Refresh the panel, and the document’s template is set.

If the document has no text yet, QBO runs OCR first (when your installation has OCR configured), then classifies the text. If the template is wrong, correct it like any other field, then improve the descriptions of the two templates the model confused.

3. Classify automatically

To classify every new document, call the classify operation from a workflow step or a scheduled job instead of the menu. Use the queued form so the work runs on an app server:

GET /api/attachment/queue/aiclassify?ID=127,128,129&AgentName=Gemini&DefaultTemplateID=4

Parameter	Purpose
ID	One document, or a comma-separated list.
AgentName	The AI service to use. Omit it to use the installation’s default.
DefaultTemplateID	The template to set when the model’s answer doesn’t match any template (here, Other).
Template_*	Narrows the candidate templates, for example to one product line. Add Template_nocache=true after you edit descriptions, because QBO caches the candidate list.

The result is the document’s template, which already drives who can see the document, which workflow picks it up, and which worklist it lands on.

4. Create an AI score template for the values you need

Now extract data from each classified note. Create a score template:

Setting	Value
Name	Mortgage Note Terms
Engine	AI (text) (AiScoreEngine) reads the document’s text. Choose AI (multimodal) instead when layout, tables, handwriting, or images matter: it sends the original PDF or image.
Engine parameters	AgentName=Gemini&UseConfidence=true
Method	Leave empty, so the engine reads the document being scored.

Then add a score item for each value:

Name	Value type	Prompt	Weight
Principal	Money	The original principal amount of the loan.	2
Interest Rate	Float	The yearly interest rate, as a percentage.	2
Payment Amount	Money	The monthly principal and interest payment.	2
First Payment Date	Date	The date the first payment is due.	1

Write each prompt as a plain-English description of one value. QBO converts the model’s answer to the item’s value type, so dates and amounts are stored as dates and amounts, not text. Weight is a whole number that sets how much an item counts toward the score’s overall confidence: here the three amounts count twice as much as the date, and 0 would leave an item out.

5. Run the score

Score a single document to check your prompts:

GET /api/score/calculate?ScoreTemplateID=12&Object=Attachment&ObjectID=127

The score is saved on the document. Each item has its value and its own confidence, and the score has a weighted confidence across the items. An item the model scored but couldn’t find counts as 0, so a document missing a required value scores lower.

Once the prompts work, have the same workflow that classifies documents calculate the score for documents classified as Mortgage Note. Use an advanced step that calls Score/Calculate with the parameters above.

6. Send low-confidence results to a person

Confidence values are ordinary QBO data, so you can act on them like any other field:

• Build a worklist of documents whose score confidence is below a threshold you choose, so a reviewer sees exactly the documents that need checking.
• Or test the confidence in an if/then step and branch the workflow: high-confidence documents continue automatically; the rest get a review task.

Start with a conservative threshold, and lower it as you see how often the model is right on your documents.

The result

New documents are filed under the right template without anyone opening them, the key values land in typed score items you can report on, and people review only the documents where the model wasn’t sure. Every step is recorded in QBO’s audit trail like any other change.

Next steps

• Apply labels when a document can belong to several categories at once (for example, a language and a priority).
• Import a document straight into records, including nested child records, with the AI import engine.
• For thousands of documents, use provider batch jobs, and post AI cost to a ledger with token cost tracking.
• Review several documents together and write a memo: see Review a loan’s documents for a prepayment penalty.