AI Writing: When a Perfectly Reasonable Scientific Sentence Goes One Step Too Far
A while ago, I was working on a chemistry manuscript about Ce³⁺-doped perovskite microlasers, and one AI-generated continuation caught my attention.
The manuscript had reached this sentence:
“Ce³⁺ may regulate the energy-level arrangement.”
It was trying to explain a specific experimental observation: in CsPbBr₃ crystals used for microlasers, adding a small amount of Ce³⁺ lowered the lasing threshold.
The threshold here can be understood as the amount of excitation required before lasing begins. Without Ce³⁺, the material needs stronger optical pumping to cross that threshold. After doping, lasing can occur at a lower excitation intensity.
So the next question was quite natural: What exactly does Ce³⁺ change inside the material that causes this threshold to drop?
Connecting energy levels, carrier relaxation, and non-radiative loss into a complete mechanism is not trivial, so we asked an AI model to continue from there.
It produced:
“The incorporation of Ce³⁺ introduces intermediate energy states that facilitate more efficient carrier relaxation, thereby reducing non-radiative energy losses and promoting population inversion, ultimately lowering the lasing threshold.”
Put simply, the reasoning was: Ce³⁺ introduces intermediate energy states → carrier relaxation becomes more efficient → non-radiative energy loss decreases → population inversion becomes easier to establish → lasing threshold decreases.
At first, this sounded perfectly reasonable. It answered the question in front of us, and every step seemed to lead naturally to the next. But after reading it a few more times, one phrase started bothering me:
“introduces intermediate energy states.”
Wait — Where Did That Step Come From?
The manuscript did say that Ce³⁺ might regulate the energy-level arrangement. There was also an experimental observation showing that the lasing threshold decreased after doping.
But “regulating the energy levels” does not automatically mean “introducing new intermediate energy states.”
Nor had the manuscript established that those states provided a more efficient carrier-relaxation pathway.
The model had quietly inserted an extra assumption:
Ce³⁺ introduces intermediate states in this particular system.
What surprised me was how easy that assumption was to miss.
Ce³⁺, energy states, carrier relaxation, non-radiative loss, population inversion — every term belonged in the neighborhood. The causal chain was smooth, and the resulting sentence answered the question neatly. Had I not gone back over it several times, I probably would have kept reading.
A Sentence Can Sound Right Before It Is Supported
That was the part that stayed with me.
When an AI model can make a scientific sentence this fluent and internally coherent, how do we tell whether the step it added is actually supported, or simply very good at looking supported?
This was not a spectacular hallucination. There was no invented compound, impossible mechanism, or obviously absurd numerical claim.
The problem was much quieter.
The model moved from something the manuscript actually said into a more specific mechanistic claim, and the prose was smooth enough to hide where that transition happened.
That kind of error is easy to miss precisely because the sentence does not look wrong.
Going Back to the References
This experience made me think more carefully about what evidence can do during AI-assisted academic writing.
If a model has only the surrounding manuscript and its general knowledge to work with, it can often fill in a missing mechanism in a scientifically plausible way.
Moving the reasoning forward is not itself the problem. That is often exactly what we want from continuation.
The difficulty comes when the available material does not support a particular step, yet the model can still write that step as though it naturally belongs in the chain.
So we tried something different. We went back through the references cited in the manuscript and found two passages that were directly relevant to this part of the mechanism.
One discussed the effect of Ce³⁺ doping on the energy levels. Another reported the corresponding relaxation behavior.
We provided those passages as additional context and asked the model to continue again.
This time, it produced:
“Specifically, Ce³⁺ introduces intermediate energy states that facilitate more efficient carrier relaxation pathways, reducing the energy loss associated with non-radiative transitions and thereby increasing the population inversion available for stimulated emission.”
At first glance, this sentence is remarkably similar to the previous one. The important difference is what the model had available when it wrote it. In the first attempt, it had to bridge the gap largely from its general understanding of what a plausible energy-level mechanism might look like.
In the second, it had relevant literature in context: evidence concerning the effect of Ce³⁺ on the energy levels, together with observations related to relaxation behavior.
Now the mechanism could be developed with something more concrete underneath it: intermediate energy states affect carrier relaxation, more efficient relaxation reduces energy loss through non-radiative transitions, and population inversion becomes easier to establish.
This time, the reasoning had a little more right to be there.
The Problem Is Not That AI Reasons Forward
I do not think the answer is to force an AI model to repeat only what has already been written. That would make continuation much less useful. A useful continuation should be able to move an argument forward. It can connect observations, develop a mechanism, or help articulate a step that the author has not yet written explicitly.
What made this case uncomfortable was the ease with which the model crossed from a supported statement into a new scientific assumption. And once the prose becomes good enough, linguistic plausibility becomes a poor proxy for evidential support.
For academic writing, this makes the provenance of each reasoning step increasingly important. When a continuation introduces a new mechanistic claim, I want to know whether that step comes from the manuscript, from a cited reference, or from the model filling a gap with its general knowledge.
Some academic writing tools are beginning to move in this direction. Jenni, for example, lets researchers bring uploaded papers and reference-manager sources into AI-assisted writing. In Flowing, the approach I have been exploring is slightly different: the manuscript context itself is used to surface relevant snippets from a researcher’s own library, so those passages can sit beside the writing at the moment they become relevant.
What interests me about this workflow is not simply giving the model more documents. It is making the relationship between the writing and its evidence easier to see at the moment the reasoning is being formed.
That matters because the problem in the Ce³⁺ example was never that the model tried to reason forward. The problem was that it became hard to see where the evidence ended and the model’s own inference began.
I still want the model to make that next step. I just want to be able to see what the step is standing on.
AI can make every word look perfectly in place. May it smile and nod politely 🙂