Healthcare Technology AI/ML/DL

Reading the Scan, Not Just the Text: Google's MKA Benchmark and NVIDIA's OpenShell, and What They Could Mean for Digitising Paper Medical Records in India

The Clinical Frontier · 16 November 2026 · Issue 036 A briefing on how the latest frontier AI models and hardware land in real healthcare workflows, written for India’s health-IT community. No hype, just what changed and what to do about it.

In this issue

  • On 10 November Google DeepMind introduced MKA, a benchmark of agents working through visually complex files, where Google reports Gemini 4 Carbon ranking second on quality.
  • The same day NVIDIA said it is contributing OpenShell, an open runtime for governing autonomous agents, to the CNCF.
  • Neither is a clinical result. Together they point at a very practical Indian problem: decades of scanned and paper records that have to be read as images, not just as text.
  • We sketch a pattern that keeps the original scan, links every extracted value to its source, and puts a person in front of anything that could change care.

What shipped

MKA, a benchmark for visual work. Google’s post of 10 November introduces the Multimodal Knowledge Agent benchmark, built with Artificial Analysis. Its premise is that most knowledge work benchmarks can be solved with text alone, while everyday professional work is often visual.

  • The tasks. A 200-task evaluation set made with domain experts across 10 verticals and 83 occupations. Google says each task is a multi-step project that takes a seasoned professional three to eight hours.
  • The grading. Each task pairs a prompt and a bundle of multimodal files with an expert-written rubric of roughly 70 yes/no checks, weighted 75% on factual accuracy and 25% on visual design. An agentic autorater built on a non-Google model grades the deliverables, and Google says domain experts agreed with it 96% of the time.
  • The fairness setup. Every model runs in the same open-source agent harness, Stirrup, with the same tools, sandbox and turn budget. Artificial Analysis keeps the 200 tasks private and Google released 10 public tasks on Hugging Face.
  • The result Google highlights. Models differed sharply in effort, using between 30 and 130 turns on average per task. Gemini 4 Carbon ranks second in quality among the models tested and completes tasks at 6x lower cost on average than the top-scoring model. These are Google’s reported results.

OpenShell, a seatbelt for agents. NVIDIA’s post of 10 November, published around KubeCon, says it is contributing NVIDIA OpenShell, an open source runtime for securing and governing autonomous agents, to the Cloud Native Computing Foundation as a Sandbox project.

  • What it does. NVIDIA says OpenShell applies deterministic controls to agent activity, so teams can enforce policies as agents use tools, access inference services and execute workloads, and connect their own compute, inference providers, credential storage and monitoring tools.
  • The companion project. NVIDIA is also contributing to Agent Substrate, a Google-led open source framework for operating long-running agent workloads on Kubernetes, and says NVIDIA and Google are collaborating on an integration of the two.
  • The scale claim. NVIDIA says hundreds of organisations have used OpenShell. The post describes no healthcare use.

Why it matters: records are pictures

For a hospital with decades of paper, the oldest history may not be a database row at all. It may be a lab report printed with a layout, a stamped discharge summary, a handwritten prescription, a film jacket. Reading those means looking at the page: which number sits under which column, what a struck-out line means, where the unit is.

MKA matters here only as a signal about what is being measured. Google’s pitch is that agents should be tested on files where the visual content carries the answer. That is the same property a scanned report has.

Be careful about what MKA is not. The ten public showcase tasks on Hugging Face are labelled with verticals such as logistics, construction, engineering, science, business, finance, education, law and government, and software and data, and none is labelled healthcare. Artificial Analysis describes the benchmark as covering project files such as architectural drawings, engineering schematics and clinical scans, but the 200-task set is private, so nobody outside can check what the medical tasks look like. A benchmark score is not evidence that a model reads Indian lab reports correctly.

For Indian hospitals: a pattern for scanned records

The following is our design proposal, not a product that either vendor describes.

  1. Keep the original. Never replace the scan with extracted text. In FHIR, a DiagnosticReport has a presentedForm element for the entire report as issued, next to discrete results, and a DocumentReference can index a document so it is available to a healthcare system. Which fits depends on your system’s interface.
  2. Extract with receipts. Each value the agent proposes (a haemoglobin result, a dose) should carry the page and the region it came from, so a reviewer sees the evidence next to the value. A Provenance resource is how FHIR tracks the activity that created or signed a version of a resource and the agents involved.
  3. Score risk, not just accuracy. A benchmark averages many checks. In a record, a wrong decimal point or unit is not an average. Require a person to confirm values that could change a decision, and sample the rest.
  4. Fence the agent. An agent that opens thousands of scans needs limits: which folders it can read, which tools it can call, which network it can reach, which inference endpoint it can use. That is the kind of policy control NVIDIA says OpenShell enforces, and a hospital could ask any agent platform the same question, whether open source or not. If you run the agent on your own cluster, ask how its policies are written, logged and reviewed.
  5. Test on your own paper. Build a small, de-identified sample of your own scans, covering the worst ones: faded thermal prints, skewed phone photos, mixed English and local-language handwriting. Measure accuracy on that set before any pilot, and again after any model change.

The Indian rulebook, briefly

India’s DPDP Rules were notified in November 2025, and the Press Information Bureau describes an 18 month phased compliance timeline. Three points from that release bear on a digitisation project.

  • Consent notices. Data Fiduciaries must issue standalone, clear and simple consent notices that explain the specific purpose of use. Using old records for a new purpose such as AI extraction is a question for your data protection lead.
  • Erase and correct. Data Principals have rights to access, correct, update or erase their personal data, with a maximum of 90 days to respond. If extracted values live in several places, you need to be able to find and change them all. Provenance links help with that.
  • Breach notification. Affected individuals must be told promptly, in plain language. A large pile of newly searchable scans is a bigger target than a filing room.

This is a summary of a press release, not legal advice.

The takeaway

Neither announcement is about medicine. One is a benchmark that tries to measure whether agents can really look at the page. The other is an open way to put limits around what an agent may do. A hospital with decades of paper has a use for both ideas, and a reason to be sceptical of both until it has tested them on its own records. Start with a small, de-identified sample, keep the scan, link every value to its source, and let a clinician or a trained reviewer own the final entry.


The Clinical Frontier is a daily briefing from HCITExperts. Read the sources: Google’s MKA post and NVIDIA’s OpenShell post. More tomorrow.

Share: Twitter LinkedIn