AI in research workflows: what to hand over, what to keep

Two capabilities changed what language models can be put in charge of. Neither was an improvement in writing quality.

The first is constrained output. Providers now expose modes that guarantee a response conforms to a supplied schema, rather than merely asking it to — OpenAI documents this as structured outputs. A response guaranteed to parse can be passed to the next step in a pipeline without a human reading it first, which is what separates a model that drafts from a model that participates.

The second is a standard interface to tools and data. Anthropic published the Model Context Protocol in November 2024 and released it as an open specification, now maintained at modelcontextprotocol.io. Before a common protocol, every model-to-system connection was bespoke work; with one, the connection becomes a line of configuration. Whether MCP itself endures matters less than what its arrival marks: wiring a model to a source of truth is no longer the hard part.

Together the two open an operational question that could not sensibly be asked before. "Can a model write this brief" is a trap, since it will readily produce something brief-shaped. The question that pays is which steps of producing the brief can run without a person, and what the failure looks like when one of them goes wrong.

Sort steps by how failure surfaces

Our sorting principle is not difficulty. It is whether a wrong answer announces itself.

Step	Failure mode	Suitable for automation
Fetching a named source	Fetch fails loudly	Yes
Extracting fields into a schema	Schema rejects malformed output	Yes, with validation
Normalising units and currencies	Silent if the rule is wrong	Only with a check against a known case
Summarising a document	Plausible and wrong reads the same as correct	No — draft only
Attributing a figure to a source	Plausible and wrong, and it is the thing that must be right	No

Every step in the safe half shares one property: the system can tell that it failed. A fetch that fails throws. A schema violation rejects. A summary that quietly drops a qualifying clause, by contrast, looks exactly like one that kept it, and a citation attached to the wrong source is indistinguishable from a correct one until somebody opens the source.

That is why constrained output matters more than it first appears. It does not make a model more accurate. It moves a class of error from silent to loud, and loud errors are the only ones a pipeline can act on:

fetch → validate schema → normalise → check against known case → human review → publish
        ↑ machine-checkable                                       ↑ everything above this line
                                                                    is a draft

What this means for a research operation

Automate collection and structure, and keep judgment and attribution human. The expensive, repetitive part of research is pulling the same sources on a schedule and forcing them into a consistent shape, and that part is now genuinely cheap. What carries the firm's name — what the numbers mean, and whether a figure is actually supported by the source cited — is exactly where a confident wrong answer looks identical to a right one.

Put the check where the error is silent. Validation belongs at the transitions a schema cannot police. If a rule normalises a currency, test it every run against a case whose answer is already known. If a figure is attributed to a source, the reviewer's job is to open the source, not to assess whether the sentence reads well.

And treat volume as a review problem rather than a production problem. Once collection is automated, output stops being the constraint and review becomes it. A pipeline that produces twenty briefs a week and reviews four has not been sped up. It has grown a queue.

So what

If you are evaluating AI in a research or analyst function, the question to ask is not what the model can produce. It is: for each step you intend to automate, does a wrong answer announce itself? Automate the steps where it does. Where it does not, the model is a drafting tool and the review stays where it was — and an honest plan says which steps are which.

The companion note on reading platform disclosures, one of the recurring pulls worth automating at the collection layer, is at Platform fee schedules.