How I Built an AI Research Skill That Checks Its Own Citations
How to build an automated fact-checking loop.
For a long time, I was skeptical about using AI for research.
Ask ChatGPT or Claude for a literature review and you usually get back something polished, with citations. It looks perfect. But click through to the primary sources and half the papers don’t exist, or the report misreads what they say.
This isn’t just my intuition. In 2023, Walters and Wilder tested ChatGPT across four subject areas and found 55% of GPT-3.5’s bibliographic citations were fabricated. With GPT-4, the rate only dropped to 18%.
Because you never know when to distrust the output, you end up doing exhausting vigilance work—verifying every link, author, and claim by hand.
But recently, my skepticism has changed into awe.
As I’ve learned more about how to direct AI in service of my intentions (and build automated verification loops with it), I’ve been amazed by the quality of the output.
So much so that I named the Claude Code skill1 I adapted and built around it Jean2, after a head of research whose rigor I admire.
The version I use now is adapted from an open-source Claude Code Academic Research skill by chrislemke/stoffy, which I customized for my own research process.
In this post, I’ll walk you through the thinking behind the Jean research skill so you can build your own—and share how to install mine if you’d rather just use it.
What The Claude Research Skill Does
Most AI tools start guessing the second you hit enter, leaving you to do the draining work of checking their sources. Jean isn’t smarter but it follows a clear protocol (with verification loops) that leads to a more accurate output:
Scoping before search: Instead of immediately writing a generic summary, it stops to clarify the domain, the depth (5, 15, or 30 sources), and the focus area.
Three cycles of research: Instead of one web search, it targets core academic databases (like Semantic Scholar and arXiv) first. Then it expands to specific authors and journals. Finally, it follows citation trails to find foundational papers.
Source evaluation & triangulation — every paper is rated for relevance, authority, recency, and type; cross-referenced for consensus, debates, gaps, and key figures.
Fact-checking for hallucinations. Before writing the final report, it must confirm that every cited paper exists via web search. It requires a title, author, year, and an actual DOI/URL. If a citation cannot be verified, the source is deleted.
Highlighting “Open Questions”: By cross-referencing sources, it tells me where the field hasn’t settled yet. This prevents from presenting a confident-sounding, yet false consensus.
When I Use The Claude Research Skill
I use this research skill I named Jean when I want a literature review — papers, DOIs, competing theories, named debates; before I write. .
For example I ran the skill on questions like “What Did Phones, Social Media, and Notifications Do to Our Attention? Historical and Contemporary Perspectives”. And after twenty minutes I received this report (which informed my post here).
Another question I used the skill for were spaced repetition selection principles in the age of AI (which informed my post here).
I then use these outputs as starting points for further reading. It’s like my personal expert reviewer and researcher and then I dive deeper on what excites me and what I am curious about.
The rest of this post is for paid subscribers. Below, you’ll get the skill to download and a 15-minute install guide for non-coders.
🛠️ The Skill File & How to Install It
