Privileged documents, predictive-coding training data, and review audit logs stay inside the firm’s tenant. Nothing routes through a vendor LLM.
Faster first-pass review than linear keyword review once predictive coding is trained on the matter’s seed set and the LLM summarizer is tuned to the firm’s review protocols.
Self-hosted Llama, Mistral, or Qwen for privileged matters. Enterprise OpenAI, Anthropic, or Bedrock for non-privileged workloads. Routed per matter, per custodian, per privilege tier.
What the firm gets from self-hosted ai ediscovery
Six outcomes litigation support teams see when they move predictive coding and document review off vendor SaaS platforms and onto a private ai ediscovery stack tuned for the firm’s matters.
Ingestion of Real Litigation Corpora
PSTs, MSGs, OST mailboxes, Slack and Teams exports, scanned exhibits, OCR'd contracts, mobile chat archives, voicemail transcripts, and structured data dumps — parsed, deduped, and threaded the way a litigation support team expects.
Private Predictive Coding
TAR 1.0 and TAR 2.0 (continuous active learning) running entirely inside the firm's tenant. Seed sets, training samples, and model coefficients are matter-scoped and never leave the perimeter — unlike vendor SaaS predictive coding that pools learning across tenants.
LLM Review with Cited Summaries
Every document summary, privilege call rationale, and issue tag links to the underlying source paragraph. Litigation associates verify in seconds instead of re-reading. Refuses gracefully when the document is ambiguous, so privilege calls stay defensible.
Self-Hosted AI Redaction
Names, addresses, account numbers, medical identifiers, and trade-secret terms redacted by self-hosted models running inside the firm's perimeter. Redaction logs, reviewer overrides, and burn-in artifacts stay in-tenant for the chain of custody.
Privilege-First Routing
Matter-level routing rules send privileged documents to self-hosted LLMs, non-privileged to enterprise APIs where helpful. The firm's ethical wall and conflict checks are mirrored in the AI layer so the model never crosses a wall the firm doesn't.
Defensible Audit Trail
Every model invocation, retrieved chunk, predictive-coding decision, and reviewer override is logged with timestamp, user, model version, and inputs. The audit log meets the standard opposing counsel and the court expect when predictive coding is challenged.
Why vendor SaaS ediscovery software is a privilege problem
Relativity aiR, Reveal AI, DISCO Cecilia, and the rest of the vendor SaaS ediscovery software stack ship a single predictive-coding pipeline tuned for the median matter, with the model and the prompts hosted in the vendor’s multi-tenant cloud. That works for routine review. It stops working the moment a custodian’s mailbox crosses into privileged communications, the moment opposing counsel challenges the seed set, or the moment in-house counsel asks where the firm’s review prompts and predictive-coding training data physically live.
Three concrete pressures push mid-market firms and litigation support teams toward a self-hosted stack. ABA Formal Opinion 512 on generative AI puts the burden on the lawyer to understand where prompts and outputs flow and to keep client confidences protected — a standard most vendor SaaS LLM clauses do not satisfy. State-bar inadvertent-disclosure rules (the standard variant of Model Rule 4.4(b)) make any unintended routing of privileged content to a third-party LLM an event that has to be disclosed and remediated. And outside counsel guidelines (OCGs) from corporate clients increasingly forbid client data being used to train vendor models, period.
A self-hosted AI eDiscovery deployment is the answer. Predictive coding, LLM summarization, ai redaction, and the audit log all run inside the firm’s tenant. The firm gets the speed and recall of modern secure AI tooling without surrendering the privilege and audit posture the bar expects.
Inside a self-hosted ai ediscovery stack — architecture, use cases, and rollout
Eight building blocks make up a self-hosted ai ediscovery deployment: a private architecture, a clear rollout sequence, a comparison against the SaaS incumbents, and three buyer flavors covering small firms, large litigation teams, and corporate legal departments.
Architecture — ingestion, embedding, predictive coding, LLM review, audit
The five-layer architecture above (Figure 1) is the reference deployment. Ingestion handles the messy real-world litigation corpus — PST mailboxes, MSG files, OST exports, Slack and Teams archives, scanned exhibits with OCR, and mobile chat captures — with the deduplication, near-deduplication, and email threading litigation support expects. Embedding runs locally (BGE, E5, Stella, or a legal-tuned variant) so vector representations of privileged content never leave the perimeter. Predictive coding is the TAR layer: a logistic-regression or transformer classifier trained on the matter’s seed set, with continuous active learning for TAR 2.0 workflows. LLM review generates document summaries, privilege-call rationales, and issue tags, with every output cited back to the source paragraph. Audit log sits underneath everything, capturing the chain of custody.
Implementation framework — four phases from discovery to continuous improvement
- Phase 1 — Discovery (weeks 1-2). The litigation support team and a NeuralChain field engineer map the firm’s matter mix, custodian profile, and current vendor SaaS exposure. The output is a deployment shape (VPC, on-prem, or air-gapped), a model-routing plan, and a privilege-tier taxonomy.
- Phase 2 — Pilot (weeks 3-6). One representative matter is ingested end-to-end. Predictive coding is trained against a labeled seed set, the LLM review prompts are tuned to the firm’s review protocol, and recall and precision are measured against the linear-review benchmark.
- Phase 3 — Production (weeks 7-12). The stack is hardened against the firm’s SSO, ethical walls, conflict checks, and outside-counsel guidelines. SOC 2 and matter-level access controls are documented. The litigation support team takes over day-to-day operation with a NeuralChain runbook.
- Phase 4 — Continuous improvement (ongoing). New matters extend the predictive-coding seed library. The LLM review prompts are refined as case law and bar opinions evolve. Quarterly recall/precision audits keep the predictive-coding posture defensible if challenged.
Vendor SaaS vs self-hosted — the comparison litigation support teams need
| Capability | Vendor SaaS (Relativity aiR / Reveal / DISCO Cecilia) | Self-hosted AI eDiscovery |
|---|---|---|
| Data residency | Vendor multi-tenant cloud; LLM provider sub-processor | Firm VPC, on-prem, or air-gapped; no sub-processor |
| Audit trail | Vendor’s logging schema; export gated by contract | Firm-owned logs; reviewable by opposing counsel on motion |
| Predictive coding control | Vendor pipeline; limited model swap, opaque coefficients | Matter-scoped models; coefficients exportable for defensibility |
| AI redaction | Vendor model; redaction artifacts in vendor tenant | Self-hosted ai redaction; burn-in inside the firm’s perimeter |
| Integration | Vendor connectors; ETL into the SaaS | Native to the firm’s DMS, iManage, NetDocs, M365, SSO |
| Cost at scale | Per-GB hosting plus per-document AI uplift; grows with matter size | Fixed infrastructure plus managed-service retainer; bends the per-matter cost curve as volume grows |
Use case 1 — small law firm doing in-house ediscovery
For a 10-50 lawyer firm running its own eDiscovery in-house, the pain is per-matter SaaS cost and the inability to push back on outside-counsel guidelines that forbid client data being used to train vendor models. A self-hosted ediscovery stack runs on a single GPU server (or a small VPC) and handles the 1-3 active matters a small firm typically reviews at any time. Predictive coding is trained per matter from the partner’s review of the seed set. The litigation paralegal operates the platform day-to-day; the IT manager keeps the lights on.
Use case 2 — large litigation team needing scale
For a litigation support team at a mid-market or AmLaw 200 firm running 20+ concurrent matters — some with multi-terabyte custodian collections — the pain is throughput and the privilege exposure that comes from any vendor LLM touching that volume of communications. A self-hosted stack scales horizontally inside the firm’s VPC, runs predictive coding per matter without pooling learning across cases, and gives the litigation support manager a unified dashboard for recall, precision, and reviewer override rates across the portfolio.
Use case 3 — corporate legal department
For an in-house legal department managing internal investigations, second-request responses, and litigation hold across the enterprise, the pain is keeping privileged investigation files away from the enterprise AI stack the rest of the company uses. A self-hosted ediscovery deployment lives in a legal-only namespace, mirrors the legal department’s ethical walls, and integrates with the existing M365, Slack, and ERP systems for custodian collection — without the legal hold corpus ever surfacing in the general-purpose enterprise LLM.
Predictive coding ai — deep dive on TAR 1.0 vs TAR 2.0
TAR 1.0 (simple passive learning) trains a classifier on a static seed set, codes the rest of the corpus, and stops. It is the easier of the two to defend at a court challenge because the seed set is the auditable artifact. TAR 2.0 (continuous active learning, CAL) keeps the classifier learning from reviewer decisions across the full review — faster and more accurate in practice, but requires careful logging of every reviewer override to stay defensible. Self-hosted predictive coding ai supports both modes: a matter team picks the regime per case, the audit log captures every state transition, and the firm keeps the model coefficients exportable in case the predictive coding decision is challenged at trial.
Defensible audit log and privilege chain of custody
The audit layer captures every model invocation, retrieved chunk, predictive-coding state transition, ai redaction event, and reviewer override — with timestamp, user, model version, matter ID, privilege tier, and citation chain. The format mirrors what opposing counsel and the court expect when predictive coding is challenged: full reproducibility from the seed set to the final production set. Litigation support teams export the log on demand and keep it under the firm’s standard retention policy.
Talk to a self-hosted ai ediscovery engineer
A 45-minute strategy call. We’ll talk through the firm’s matter mix, custodian profile, current vendor SaaS exposure, privilege-tier taxonomy, and the practice areas (litigation, regulatory, internal investigations) the deployment needs to cover — then come back with a concrete ingestion shape, model-routing plan, and four-phase rollout sequence.
Ask us about
- Self-hosted AI eDiscovery deployment — ingestion, embeddings, predictive coding, LLM review, audit
- Litigation matters, internal investigations, second-request responses, regulatory production
- TAR 1.0 and TAR 2.0 predictive coding with matter-scoped model coefficients
- Self-hosted ai redaction for PII, account numbers, medical identifiers, and trade-secret terms
- Air-gapped, on-prem, or VPC deployment for privileged matters and OCG-restricted clients
- Defensible audit log, citation-enforced LLM review, and matter-level access control
When the firm needs self-hosted ai ediscovery instead of vendor SaaS
Relativity aiR, Reveal AI, and DISCO Cecilia cover the median matter well — small custodian collections, non-privileged content, vendor-hosted everything. That is enough for some matters.
It stops being enough when the firm hits any of these decision points:
- Small firm path — the firm wants in-house eDiscovery without paying per-matter SaaS hosting on every case. A single-GPU self-hosted stack covers 1-3 concurrent matters with predictive coding and LLM review, operated by the litigation paralegal.
- Mid-market path — the litigation support team runs 20+ matters at a time and needs predictive coding that does not pool learning across tenants. A horizontally scaled VPC deployment gives the support manager a portfolio dashboard and matter-scoped models.
- Enterprise path — the corporate legal department needs a legal-only AI namespace that mirrors ethical walls and stays out of the rest of the enterprise’s AI stack. An on-prem or air-gapped deployment integrates with M365, Slack, and ERP custodian sources without surfacing the hold corpus elsewhere.
For the document-chat companion to this eDiscovery deployment, see the private RAG solution page. The transactional companion workflow is covered in the private AI for contract review and generation guide. The bar’s own treatment of generative AI — the duty to understand where prompts and outputs flow — is set out in ABA Formal Opinion 512.
Frequently asked questions
Related solutions in the private-AI cluster
Additional resources for litigation support teams
AI eDiscovery Workshop
Half-day strategy workshop to map the firm’s matter mix, custodian profile, privilege tiers, and the right ingestion / embedding / LLM routing for the first three matters.
AI Strategy Session
60-minute scoping call. Walk through the firm’s current vendor SaaS exposure, OCG constraints, and target predictive-coding workflow — come away with a deployment shape and a four-phase rollout sequence.
Self-Hosted vs Vendor SaaS
Honest tradeoffs on running ediscovery in-house on a self-hosted stack versus staying on Relativity aiR, Reveal, or DISCO Cecilia — for small firms, mid-market teams, and corporate legal departments.
Ready to deploy self-hosted ai ediscovery?
A 45-minute strategy call covers the firm’s matter mix, current vendor SaaS exposure, OCG constraints, and the practice areas the deployment needs to cover — then a concrete ingestion shape, model-routing plan, and four-phase rollout sequence.