A viewer question under the model-comparison video "I Tested Sonnet 5.5 vs Opus 5.5. What You Need to Know." names the gap this post answers: "I'm confused here, are you testing the models against a blank slate or are they pulling from an enormous library of skills and MCP connectors? You said you weren't sure if it used ElevenLabs for the voice demo, that makes it sound like they're accessing tools that wouldn't make it a faithful presentation. Wouldn't running them with nothing be better to see what they're capable of?" (13 likes, @askcoacheric).
Two more commenters asked the same thing under a video about directing Claude to improve a website a hundred rounds in a row: "is Claude generating the videos and images on its own or are you using a plugin/mcp?" (5 likes) and "What image/video generation tool are you connected to for this example?" (4 likes). The instinct in all three is sound: much of what an AI demo shows may come from parts attached to the model rather than the model itself. The rest of this post turns that instinct into checks: what a model does alone, the three kinds of attachments, how to tell which one produced an output, and what a demo can honestly tell you.
What a model does on its own
A text-only language model takes text in and produces text out. Everything it does, reasoning, planning, code, analysis, happens in that one mode. It has no built-in way to make a picture, a voice, or a video.
Products add those. ChatGPT's app bundles an image model, so a drawing request works there without setup. Claude's app doesn't, as of publication; when a coding agent built on Claude produces pictures, they arrive through a connected image service, a stock library, or code that draws shapes (How Does Claude Make Images If It Can't Generate Them? covers the three paths). For reading demos, the distinction to hold is this: any chat output beyond text comes from a bundled capability or a connected component.
The three kinds of attachments
Tools and connectors. Programs the system calls when a task needs them: an image service, a voice generator, a browser, file access. ElevenLabs, the service named in the anchor comment, sells AI voice generation, text to speech, and dubbing, so a produced voiceover in a video likely came from a call to a service like it. Many connections run through MCP, the Model Context Protocol, an open standard Anthropic introduced in November 2024 that gives AI systems one interface for reaching outside tools and data (What Is MCP? Model Context Protocol Explained Simply covers it).
Skills. Folders of instructions, scripts, and resources that load procedures into an agent: how to fill in a form, how to build a page in a set style. Anthropic published the Agent Skills format in October 2025 and released it as an open standard in December 2025, defining skills as "organized folders of instructions, scripts, and resources" an agent loads when a task calls for them. The name has since spread past the standard: comment sections offer "skills" that are any folder of instructions an agent can read, and the comment corpus this Lab mines (5,546 comments this morning) carries plenty of those links.
Context. Files and data supplied before the demo starts: brand guidelines, spreadsheets, past work. Output that looks like taste often reflects the documents the presenter supplied.
Why "run it with nothing" answers a different question
A blank-slate test and a tooled-up test measure different things. A blank-slate test, no tools, no skills, no files, shows what the model does alone: reasoning, writing, coding. A tooled-up test shows what the model plus its attachments produce, which is the system a customer would use. The mismatch starts when one gets narrated as the other.
That mismatch is where comparison videos get confusing. If two models are tested with different toolsets attached, the video compares two systems, and the setup that scores higher may not hold the better model. If a demo shows an agent building a website with images and voice while the narration credits the model, viewers may take home an expectation a bare chat window won't meet. The demo was truthful; the label on it wasn't.
How to tell what did the work
Four checks, ordered by effort:
Look for outputs a text model can't produce. Photorealistic images, video, and produced voice (voiceovers, songs) come from a specialist model or a multimodal one, either bundled in the app or connected as a tool. If the product doesn't advertise the capability, the likely source is a connected service; that's an inference from the output, not proof. Some chat products display a tool name after each call, which settles it on the spot.
Check the setup before the demo. What did the presenter connect or install first: skill links in the description, connectors named in the intro, files uploaded on screen, the product's tool list visible in settings. A demo of an agent building a full website is a system demo, which is what the two commenter questions at the top were probing.
Ask the system. "Did you use a tool for that?" works on systems that show their steps. Treat the answer as one signal: a model can't verify its own wiring, it reports what its instructions say, the same limit that makes "which model are you?" unreliable (How to Check Which AI Model Your Tool Is Using covers that limit in detail).
Accept the limit. Sometimes nobody can tell from the outside. The presenter of the comparison video at the top said he wasn't sure whether the voice demo used ElevenLabs. When the person running the demo can't say which part did the work, the honest reading is "system output, attribution unknown," and the impression the demo leaves belongs to the system, not to the model alone.
The reproduction test
The reproduction test runs after the video: open a fresh chat with the same model and try the core task, with tools off where the product allows. If the attempt comes out close to the demo, that's evidence the model carried much of the result, though hidden defaults can still differ. If it comes out worse, the demo may have had attachments you didn't, or the conditions may have changed. Change one thing at a time, and the gap tells you what to connect or write down: an image tool, a skills folder, the files worth supplying.
FAQ
What is a blank slate test for AI models?
An evaluation with nothing attached: no tools, no skills, no files, just the model and a prompt. Benchmark teams run them to measure model-only capability. Demos rarely do, because the tooled-up version usually looks better, which is why the two results shouldn't be compared to each other.
Do AI models have tools built in?
Some products bundle specific capabilities: ChatGPT's app includes an image model, so drawing requests work without setup. Others don't: Claude's app has no bundled image model as of publication, so image output arrives through connected tools. Bundles change as products update, so check what the current version advertises. Capabilities outside the bundle attach through connectors, many built on the MCP standard.
What are AI skills?
Folders of instructions, scripts, and resources that an agent loads when a task calls for them, the structure Anthropic published as the open Agent Skills standard in December 2025. To vet one before installing, read the instructions inside the folder and check where it came from; anyone can publish a skill, the same way anyone can publish a repo (How to Tell If a GitHub Repo Is Safe to Run covers that check).
How do I compare two AI models fairly?
Give them the same attachments, or give them both none. Change one thing at a time: same prompts, same files, same tools, different models. When you can't control the setup, as with published comparisons, read the result as a comparison of systems and look for the setup list before weighing the ranking.
Want to go further?
Three posts pair with this one: How to Check Which AI Model Your Tool Is Using covers the model a product calls, What Is MCP? Model Context Protocol Explained Simply covers the connection layer, and What Is AI Washing? How to Spot Features That Don't Need AI covers features that never needed a model. To run the checks on something you've watched, pick one demo from this week and note which parts you can attribute.