Skip to main content
LLMix

Industries · Legal

Post-training for legal and regulatory AI systems.

Legal work is where unsupported generation costs the most, and where supervision is unusually rich: redlines, accepted and rejected analyses, issue taxonomies, and citations that can be checked. LLMix trains models that ground their output in verifiable sources and abstain when they cannot.

Capability targets

Programs target structured, checkable legal-support capabilities under professional review, never autonomous legal judgment.

  • Contract review with issue extraction against a defined taxonomy
  • Structured legal analysis in the organization's format
  • Regulatory classification and obligation mapping
  • Citation-grounded output: every claim tied to a source
  • Document comparison and change analysis
  • Escalation and abstention when authority is unclear or conflicting

Available supervision

Legal practices generate supervision continuously as a byproduct of work product and review. Programs convert it into training form under confidentiality controls.

  • Expert demonstrations: worked analyses in the target format
  • Redlines as dense preference signal on contract language
  • Accepted-versus-rejected analyses from internal review
  • Issue taxonomies and review checklists
  • Rubric judgments from supervising counsel
  • Citation checks against document corpora
  • Jurisdiction and date metadata on every example

Environment design

For tool-using stages, matters become episodes: the model researches within a controlled corpus, drafts structured output, and revises under review feedback.

  • Document corpus with retrieval and citation tools
  • Matter state: parties, jurisdiction, dates, open issues
  • Structured action space for analysis, drafting, and escalation
  • Conflicting authorities planted to test resolution behavior
  • Jurisdiction- and date-validity constraints on citations
  • Iterative review loops as multi-turn episodes

Rewards and verifiers

The decisive property of legal signal is that citations are checkable: whether a cited source exists, and whether it supports the claim, can be verified against the corpus.

  • Citation existence checks against the controlled corpus
  • Citation support: the cited passage actually backs the claim
  • Required-issue coverage against the matter's taxonomy
  • Schema compliance on structured analyses
  • Jurisdiction and temporal validity of cited authority
  • Abstention rewards when authority is insufficient
  • Expert preference for judgment beyond programmatic checks

A plausible training sequence

Legal programs are supervision-heavy and end with tightly gated online stages, if any.

  • SFT on expert-worked structured analyses
  • DPO on expert comparisons and redline-derived preferences
  • Reward-model or judge calibration against supervising counsel
  • Online agent training only where a controlled corpus and scoring boundary exist
  • Held-out evaluation on unseen matters with temporal splits

Evaluation

Evaluation targets the failure that matters most in legal use: confident, unsupported claims.

  • Held-out matters excluded at the matter level, not the document level
  • Temporal and jurisdictional splits against leakage
  • Citation accuracy and citation-support rates
  • Issue recall against expert-annotated ground truth
  • Unsupported-claim rate as a first-class metric
  • Abstention quality and escalation correctness
  • Structured expert review on sampled outputs

The legal supervision map

Legal programs combine several signal sources with different strengths. The map below shows how they compose into one training signal, with citation verification carrying the grounding.

expert demonstrations ──► SFT           (format, method, tone)
redlines ─────────────► preference     (dense edit-level signal)
accepted / rejected ──► preference     (analysis-level judgment)
citation verifier ────► reward         (existence + support, programmatic)
issue-coverage rubric ► reward         (taxonomy completeness)
abstention policy ────► reward         (decline when unsupported)
                          │
                          ▼
        combined post-training signal, gated by
        held-out matters + supervising-counsel review
How legal supervision sources compose: demonstrations and preferences establish competence, verifiable citations and rubrics keep it grounded.

Governance and scope

The boundaries are part of the engineering, trained and measured rather than disclaimed.

  • Nothing the model produces is legal advice, and workflows keep counsel in review
  • Expert supervision comes from the client's approved counsel
  • Confidentiality and privilege boundaries govern data handling end to end
  • Validation is jurisdiction-specific and dated, and the model card records both

What ships

  • Tuned model artifact with training assets and lineage
  • Prepared supervision datasets with provenance and privilege controls
  • Citation verifiers and evaluation harnesses
  • Unsupported-claim and abstention evaluation reports
  • Model card with jurisdictional scope and known limitations

Related pages

Custom LLM fine-tuning

The supervised stages legal programs are built on: SFT, preference optimization, and data design.

Reward modeling and verifier design

Citation verifiers and rubric judges as calibrated learning signal.

Bespoke LLM post-training

The complete program structure behind every vertical engagement.

Scope a legal-model training program.

Scope a post-training program