Automating document processing with LLMs: invoices, contracts and forms into structured data

Many businesses still have people retyping invoices into accounting software, copying contract terms into spreadsheets and keying in scanned forms. Template-based OCR tools helped with fixed layouts, but they break every time a supplier changes their invoice design.

Modern LLMs read documents the way a person does, regardless of layout, which makes extraction far more flexible. They also make mistakes with total confidence. A production pipeline is therefore less about the extraction call and more about the system around it: a strict schema, validation against business rules, signals for when to doubt the output and a review queue that keeps a person in charge of the uncertain cases.

The pipeline at a glance

1. Ingest: receive files from email, upload or a storage bucket, store the original and assign an ID.
2. Prepare: detect file type, extract the text layer from digital PDFs, and send scans or photos as images.
3. Classify: decide what kind of document it is (invoice, credit note, contract, form) and route accordingly.
4. Extract: call the model with a document-specific schema and get structured output.
5. Validate: check the schema, then business rules such as totals, dates and known suppliers.
6. Decide: auto-approve clean results; send anything doubtful to human review.
7. Export: write to the ERP, CRM or database, with a link back to the source document.

Each step is a separate, retryable job. If the export to your accounting system fails, you should not have to pay for extraction again.

Define the schema first

The schema is the contract between the model and the rest of your system. Define it once with a validator such as zod or pydantic and derive everything else from it. Be explicit about what can be missing: a nullable field gives the model a legitimate way to say “not present” instead of inventing a value.

import { z } from "zod";

const Money = z.number().multipleOf(0.01);

export const LineItem = z.object({
  description: z.string().min(1),
  quantity: z.number().positive(),
  unit_price: Money,
  amount: Money,
});

export const Invoice = z.object({
  supplier_name: z.string().min(1),
  supplier_tax_id: z.string().nullable().describe("VAT / tax ID exactly as printed, or null if absent"),
  invoice_number: z.string().min(1),
  invoice_date: z.string().regex(/^\d{4}-\d{2}-\d{2}$/).describe("ISO date YYYY-MM-DD"),
  due_date: z.string().regex(/^\d{4}-\d{2}-\d{2}$/).nullable(),
  currency: z.string().length(3).describe("ISO 4217 code, e.g. USD, EUR"),
  line_items: z.array(LineItem).min(1),
  subtotal: Money,
  tax_total: Money,
  total: Money,
  uncertain_fields: z
    .array(z.string())
    .describe("Names of fields that were hard to read, ambiguous or inferred rather than printed"),
});

export type Invoice = z.infer<typeof Invoice>;

Extract with structured output

Use your provider's structured output support so the response is guaranteed to be parseable JSON. A dependable approach with most APIs is to define a single tool whose input schema is your document schema and force the model to call it. Then validate the result with the same zod schema anyway, because a response can be syntactically valid and still break your rules.

import Anthropic from "@anthropic-ai/sdk";
import { z } from "zod";
import { Invoice } from "./invoice-schema";

const client = new Anthropic();

const SYSTEM = [
  "You extract data from invoices into the provided schema.",
  "Copy values exactly as printed. Do not calculate or guess missing values; use null.",
  "Convert dates to YYYY-MM-DD. List any field you are not confident about in uncertain_fields.",
].join("\n");

export async function extractInvoice(pdfBase64: string) {
  const response = await client.messages.create({
    model: process.env.EXTRACTION_MODEL ?? "claude-sonnet-5-5",
    max_tokens: 4096,
    system: SYSTEM,
    tools: [
      {
        name: "record_invoice",
        description: "Record the extracted invoice data.",
        input_schema: z.toJSONSchema(Invoice) as Anthropic.Tool.InputSchema,
      },
    ],
    tool_choice: { type: "tool", name: "record_invoice" },
    messages: [
      {
        role: "user",
        content: [
          { type: "document", source: { type: "base64", media_type: "application/pdf", data: pdfBase64 } },
          { type: "text", text: "Extract this invoice." },
        ],
      },
    ],
  });

  const call = response.content.find((block) => block.type === "tool_use");
  if (!call) return { ok: false as const, error: "No structured output returned" };

  const parsed = Invoice.safeParse(call.input);
  if (!parsed.success) return { ok: false as const, error: z.prettifyError(parsed.error), raw: call.input };
  return { ok: true as const, data: parsed.data };
}

If parsing fails, one retry that includes the validation errors in the prompt fixes many cases. If it fails again, do not keep looping: send the document to review with the errors attached.

Validate against business rules

Schema validation catches malformed output. Business rule validation catches plausible-looking wrong output, which is the more dangerous kind. Documents usually carry internal redundancy you can exploit: line items should sum to the subtotal, subtotal plus tax should equal the total, and the due date should not be before the invoice date.

import type { Invoice } from "./invoice-schema";

export type Issue = { field: string; message: string; severity: "error" | "warning" };

const close = (a: number, b: number, tolerance = 0.02) => Math.abs(a - b) <= tolerance;

export async function validateInvoice(inv: Invoice, deps: Deps): Promise<Issue[]> {
  const issues: Issue[] = [];

  for (const [i, item] of inv.line_items.entries()) {
    if (!close(item.quantity * item.unit_price, item.amount)) {
      issues.push({ field: "line_items." + i + ".amount", message: "quantity x unit_price does not match amount", severity: "warning" });
    }
  }

  const lineSum = inv.line_items.reduce((sum, item) => sum + item.amount, 0);
  if (!close(lineSum, inv.subtotal)) {
    issues.push({ field: "subtotal", message: "Line items sum to " + lineSum.toFixed(2), severity: "error" });
  }
  if (!close(inv.subtotal + inv.tax_total, inv.total)) {
    issues.push({ field: "total", message: "subtotal + tax does not equal total", severity: "error" });
  }
  if (inv.due_date && inv.due_date < inv.invoice_date) {
    issues.push({ field: "due_date", message: "Due date is before invoice date", severity: "error" });
  }

  const supplier = await deps.findSupplier(inv.supplier_name, inv.supplier_tax_id);
  if (!supplier) {
    issues.push({ field: "supplier_name", message: "Unknown supplier", severity: "warning" });
  } else if (supplier.currency !== inv.currency) {
    issues.push({ field: "currency", message: "Currency differs from supplier default " + supplier.currency, severity: "warning" });
  }

  if (await deps.invoiceExists(inv.supplier_name, inv.invoice_number)) {
    issues.push({ field: "invoice_number", message: "Possible duplicate invoice", severity: "error" });
  }

  return issues;
}

type Deps = {
  findSupplier: (name: string, taxId: string | null) => Promise<{ id: string; currency: string } | null>;
  invoiceExists: (supplier: string, number: string) => Promise<boolean>;
};

Confidence and the human review queue

Every document ends up in one of two lanes: straight through, or review. The routing rule should be explicit, conservative at first and easy to change. A reasonable starting policy:

• Any validation error, or a schema failure after retry: review.
• Any warning or a non-empty uncertain_fields touching money, dates or IDs: review.
• Unknown supplier, new contract counterparty or first document of its kind: review.
• Amounts above a configured threshold: review, regardless of how clean the extraction looks.
• Everything else: straight through, with a random sample (for example a small percentage) still sent to review for ongoing quality checks.

The review screen matters as much as the model. Show the original document side by side with the extracted fields, highlight the fields that triggered review and let the reviewer correct values inline. Store every correction with the original model output. Those pairs become your eval set and tell you exactly where extraction is weak.

Contracts and forms are different problems

Invoices have arithmetic you can verify. Contracts do not, so the design shifts. Extract specific, well-defined terms (parties, effective date, term length, renewal and notice periods, governing law, liability caps) and require the model to return a verbatim quote and page number for each one. You can then check programmatically that the quote really appears in the document text, and reviewers can confirm the interpretation in seconds.

Forms such as applications or onboarding documents sit in between. Many fields can be validated against formats or reference data: postcodes, tax IDs, email addresses, dates of birth within a sensible range. Handwriting and checkboxes are where errors concentrate, so mark those fields for review until your corrections data says otherwise.

Operational details that save you later

• Keep the original file, the model output, the validated record and any human corrections, linked by document ID.
• Record the model version and prompt version with every extraction so you can explain and reproduce results.
• Make exports idempotent, keyed by document ID, so retries never create duplicate bills.
• Handle personal and financial data deliberately: encryption at rest, retention limits and a provider configuration that matches your data policies.
• Re-run your eval set before changing models or prompts, as you would for any other LLM feature.

Key takeaways

• Define a strict schema with nullable fields and derive the model's output format from it.
• Use structured output, then validate with the same schema in code.
• Exploit the document's own redundancy and your reference data to catch plausible-but-wrong values.
• Route uncertain documents to a well-designed review queue and store every correction.
• Ask for verbatim quotes on contracts so extracted terms can be checked.
• Expand automation gradually, based on measured correction rates.

Document processing is one of the most common workflows we automate through our AI Automations service. If your team is still retyping documents, tell us what you process and we will help you scope a pipeline.