HIPAA OCR Without the Cloud: Running Baidu’s Unlimited-OCR On-Prem

To run OCR on sensitive documents without sending records to a cloud service under HIPAA or FERPA, an organization can host an open-weight OCR model on its own hardware with network access turned off, ensuring records never reach a third-party server. Open-weight releases like Baidu’s MIT-licensed Unlimited-OCR, released June 22, 2026, and Unlimited-OCR-2, released October 27, 2026, provide a practical architecture for this approach.

Running these models on isolated infrastructure allows healthcare and education teams to convert scanned records into structured text without transmitting protected data across external networks. The setup eliminates third-party exposure, but it also shifts complete operational and regulatory accountability directly onto internal infrastructure.

Why Cloud OCR Triggers HIPAA Business Associate Requirements

Sending sensitive documents to a commercial cloud API immediately creates a formal compliance relationship. Under the Department of Health and Human Services (HHS) guidance on cloud computing, a cloud service provider that creates, receives, maintains, or transmits electronic protected health information (ePHI) on behalf of a covered entity or business associate is legally defined as a business associate.

This requirement applies even when the cloud provider holds only encrypted files and lacks the decryption key. Transmitting ePHI to an external OCR endpoint without an executed Business Associate Agreement (BAA) violates HIPAA Rules.

Education records face an equally strict standard under the Family Educational Rights and Privacy Act (FERPA). Under 34 CFR 99.31(a)(1)(i)(B), an educational agency or institution may disclose personally identifiable information from education records to an outside contractor or vendor without prior consent only if the vendor qualifies under the school official exception. This exception requires four conditions:

• The contractor performs an institutional service or function for which the agency or institution would otherwise use its own employees.
• The contractor remains under the direct control of the agency or institution regarding the use and maintenance of education records.
• The contractor uses the records solely for the authorized institutional purpose and does not redisclose them without consent or legal authorization.
• The contractor meets the criteria specified in the school’s annual notification of FERPA rights for having a legitimate educational interest in the records.

Standard consumer cloud OCR services rarely meet these standards. If an external service uses document uploads for platform analytics, model training, or operational logging, the institution loses direct control. Processing files locally removes the vendor from the data chain entirely.

Inside Baidu Unlimited-OCR and Unlimited-OCR-2 Architecture

Understanding how open-weight document vision models function determines the compute and memory profile required to run them locally.

Baidu announced Unlimited-OCR on June 22, 2026, followed by the research paper release on arXiv (2606.23050) and a release on ModelScope on June 23, 2026. vLLM inference support followed on June 28, 2026, and Baidu Cloud availability arrived on July 3, 2026. The initial release operates as a 3 billion total parameter Mixture-of-Experts model, activating approximately 500 million parameters per token. Its encoder, derived from DeepSeek-OCR, compresses a 1024×1024 page image into roughly 256 visual tokens.

To manage long documents, the first version uses Reference Sliding Window Attention (R-SWA). Each generated output token attends to the full prompt and page images alongside only the most recent 128 output tokens. This mechanism keeps memory usage flat as output length grows, maintaining a practical single-pass ceiling of about 40 pages within a 32K context window. On OmniDocBench v1.5, maintained by OpenDataLab, Unlimited-OCR recorded an overall score of 93.23 percent, compared to 87.01 percent for the DeepSeek-OCR baseline.

Unlimited-OCR-2, released on October 27, 2026, expands single-request capacity to documents spanning 100 to 300 pages. The model contains 3.3 billion parameters across a Mixture-of-Experts architecture with 64 routed experts, activating six experts per token.

Instead of basic R-SWA, the second iteration introduces Sparse Reference Sliding Window Attention (SR-SWA). This mechanism compresses earlier text and image interactions into a fixed pool of 512 reference tokens while preserving a 128-token local attention window. The model card states that this architecture provides near-constant per-step memory and latency across extended document lengths.

On document processing benchmarks, Unlimited-OCR-2 achieved 84.6 on MPDocBench for documents up to roughly 30 pages. On MarathonDocBench, designed for 100 to 300 page documents, Unlimited-OCR-2 recorded an overall score of 77.1, while the next listed model scored 33.7. However, MarathonDocBench was introduced by Baidu alongside the model release. Because these benchmark results are self-reported by the developer without independent third-party validation, technical teams should evaluate real-world extraction accuracy on their own document samples.

How to Deploy Unlimited-OCR On-Premise in an Air-Gapped Setup

Both Unlimited-OCR models are released under the MIT license, permitting local deployment, internal modification, and commercial production use. Taking full advantage of local hosting requires isolating the execution environment from the public internet.

The official repository supports inference through Hugging Face Transformers, vLLM, a Docker Model Runner, and SGLang. SGLang is recommended in the project documentation for production serving via a custom wheel. Implementing a secure, air-gapped pipeline involves five technical safeguards:

1. Sever external repository connections. By default, the loading scripts attempt to download weights directly from Hugging Face by model name. Download the model weights once into a secure internal storage repository. Prior to importing the Python libraries, set the environment variables HF_HUB_OFFLINE=1 to block outbound HTTP requests and HF_HUB_DISABLE_TELEMETRY=1 to prevent telemetry transmissions to external servers.
2. Audit custom model execution code. Both the model and tokenizer load with the configuration flag trust_remote_code=True. This setting executes arbitrary Python code stored within the model repository on the host system. Before deploying weights to an operational machine, inspect the repository Python scripts to verify what network sockets, local files, and system processes the code accesses.
3. Pin exact library dependencies. Pin every dependency version listed in the repository requirements file. In production pipelines, unpinned dependencies risk automated updates pulling unreviewed packages into the environment during routine maintenance.
4. Rebind local server endpoints. The project’s default SGLang server documentation demonstrates binding to address 0.0.0.0 on port 10000. Binding to 0.0.0.0 exposes the inference endpoint to every system on the local network. Rebind the application explicitly to localhost or route traffic through an internal reverse proxy secured with mutual TLS and strict role-based access control.
5. Prohibit hosted public interfaces. Public demonstration platforms exist on Hugging Face Spaces, Baidu Cloud, and ModelScope. The open-source project repository specifies no privacy policies or protected data handling terms for these hosted web interfaces. Ingesting actual health or education records through public web demos exposes protected information to third parties.

Hardware planning requires dedicated graphics memory. Baidu listed only a single NVIDIA A800 80GB GPU as its tested hardware configuration. For teams evaluating the earlier model version, a third-party implementation guide indicates that Unlimited-OCR v1 can run at full precision with a minimum of 12GB of VRAM, recommending 24GB. This estimate originates from an independent guide rather than vendor-published hardware specifications, and no formal quantization accuracy data has been published.

Why Running Models Locally Does Not Automatically Satisfy HIPAA or FERPA

Deploying an open-weight model on private hardware eliminates third-party transmission risk, but it does not make a workflow automatically compliant. Local infrastructure simply places complete responsibility on your organization’s internal controls.

HHS explicitly states in its cloud guidance that encryption alone does not satisfy the requirements of the HIPAA Security Rule. Moving away from cloud vendors does not relieve an organization of administrative, physical, and technical safeguards under the HIPAA Security Rule:

• Access controls: Systems processing protected health information must enforce unique user identification, emergency access procedures, automated session termination, and payload encryption at rest on internal storage volumes.
• Audit logging: The local inference pipeline must maintain immutable logs recording which accounts access, extract, or export sensitive document data.
• Physical security: The physical servers hosting GPUs and local weight storage must reside in access-restricted facilities that prevent unauthorized physical intervention.
• FERPA direct control: Educational institutions processing student records must ensure the hardware environment operates under the direct oversight of institutional IT staff, with strict policies preventing data retention beyond authorized institutional functions.

Local deployment solves data transit compliance. It does not replace internal data governance.

Confirming Your Pipeline Architecture

Technical teams planning an on-premise document extraction system must align engineering designs with organizational oversight.

Review your planned hardware boundaries, network isolation configurations, and data retention rules with your internal compliance officer or legal counsel. Once your compliance leadership reviews the architecture, build a single isolated test node with external networking disabled to validate optical character extraction accuracy against your organization’s specific document formats.