Writing · Jarvis Learning Assistant: AI Consulting

The template trap: why the template you downloaded isn't working

2026-07-15T09:00:00Z

I’ve been building AI workflows for over two years. In that time I’ve watched an entire industry spring up around selling prompt templates, and I think most of it is built on a shaky foundation.

Wharton’s Generative AI Labs put this to the test. They ran “You are a world-class expert” persona prompts across six models, tens of thousands of times. Expert personas had no significant impact on accuracy.¹ Mismatched experts sometimes made things worse. The tricks that template sellers charge for, the persona assignments and specific phrasing and structure, didn’t survive contact with rigorous testing.

That tracks with what I’ve seen firsthand.

In 2024, when I started working seriously with AI, I believed the prompt was the thing. Get the wording right, the structure tight, the instructions precise, and you’d get consistent results. Sometimes that worked. But just as often, the same prompt gave me something completely different on the next run. Inaccurate, unfocused, or disconnected from what I was trying to do. What I figured out over time was that the prompt only did heavy lifting when it had real context behind it. Reference documents, research, brand guidelines, project background. Without those, even a good prompt missed more than it hit.

What changed

By 2026, the prompt became the smallest part of how I work. The context around it does almost everything now. When I set up a workflow, the prompt is a plain explanation of what I’m trying to accomplish, what needs to be included, and what I don’t want. No personas. No magic phrasing. The real work lives in the workspace: project instructions, reference materials, accumulated context from everything I’ve already done in that project.

The broader field caught up to this too. Researchers started calling it “context engineering” instead of “prompt engineering,” because the data keeps showing that what you feed the model matters more than how you ask. Sclar et al. found that formatting changes alone can swing accuracy by up to 76 points.² Another team documented a “butterfly effect” where tiny edits that don’t change meaning at all can flip the output.³ If a prompt is that sensitive to surface-level changes, packaging one up and selling it as a finished product doesn’t make a lot of sense.

How I actually build things

My process has four parts, and the prompt barely registers as one of them.

I start by thinking through the idea. Not asking AI to produce anything, just going back and forth in conversation to get clear on what I’m actually making. This is where the real work happens, and it’s the part that every template skips.
Then I build a plan. I pull in research, brand guidelines, format requirements, whatever context is relevant to what I’m creating. If I’m building a social media post, the plan includes the data the post draws from, the voice it needs to match, and the direction it’s heading. A plan built on evidence will outperform a clever prompt running on nothing, every single time.
From there I create the output. With enough context loaded, the first pass is usually close. Not done, but close. The gap between starting from context and starting from a template is that one sounds like it came from your business and the other sounds like it could’ve come from anyone’s.
Then I humanize it. I go through and fix what doesn’t sound like me, adjust the design, catch the spots where the writing feels generated. AI handles most of the distance, but the last stretch is where your voice and your judgment finish the work. I don’t think AI should run a full workflow untouched. But getting to a point where you can build on what it produces, that’s worth the effort.

Why templates break down

When you buy a prompt template, you’re buying someone else’s context stripped out and replaced with fill-in-the-blank fields. Your brand, your audience, your constraints, your standards, none of that comes with it.

The data backs this up at scale too. McKinsey’s 2025 State of AI survey found that organizations actually succeeding with AI are 2.8 times more likely to have fundamentally redesigned their workflows, and only about 20% of other companies have done that at all.⁴ The gap isn’t about access to better prompts. It’s that most people and most organizations are bolting AI onto what they already do instead of rethinking how the work gets done.

There’s a judgment problem on top of that. A Harvard and BCG study tracked 758 consultants using AI and found that on tasks inside AI’s strengths, output quality jumped about 40%. But on tasks outside that range, people who used AI were 19% less likely to get the right answer than people who worked without it.⁵ A template can’t tell you which side of that line your task falls on. Only your own understanding of the work can.

And there’s a sameness problem. Doshi and Hauser found in a 2024 Science Advances study that AI boosts individual creativity but makes outputs 10.7% more similar to each other.⁶ The more people leaning on the same shared templates, the more everyone’s work starts converging. The thing that makes your output recognizable gets averaged away.

I’ll be fair though. Templates aren’t useless as a starting point. If you’ve never worked with AI before, a template can show you what a prompt looks like and get you moving. The problem is staying there. Treating the template as the destination instead of the on-ramp means you never build the context layer that makes AI actually useful for your specific work.

If you build your own process instead, going back and forth with AI, pulling in your own research, planning around your own context, the output carries your thinking in it, shaped by your constraints and finished by your judgment. A prompt pack can’t give you that, because the valuable part was never the prompt. It was the thinking behind it.

Sources

Basil, Shapiro, Shapiro, Mollick, Mollick & Meincke, Prompting Science Report 4: Playing Pretend: Expert Personas Don’t Improve Factual Accuracy, Wharton Generative AI Labs (December 2025). gail.wharton.upenn.edu ↩︎
Sclar, Choi, Tsvetkov & Suhr, Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design, ICLR 2024. The 76-point gap was measured on one model (LLaMA-2-13B). arxiv.org/abs/2310.11324 ↩︎
Salinas & Morstatter, The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance, Findings of ACL 2024. aclanthology.org ↩︎
McKinsey & Company, The State of AI in 2025: Agents, Innovation, and Transformation (November 2025). High performers: 55% had fundamentally redesigned workflows, against 20% of other respondents. mckinsey.com ↩︎
Dell’Acqua et al., Navigating the Jagged Technological Frontier, Organization Science (2026); working paper SSRN 4573321 (2023). The study reports the gap as 19 percentage points. Summary from Harvard Business School · Working paper ↩︎
Doshi & Hauser, Generative AI Enhances Individual Creativity but Reduces the Collective Diversity of Novel Content, Science Advances 10(28) (July 2024). doi.org/10.1126/sciadv.adn5290 ↩︎

Fishing rods and AI: understanding the tool

2026-07-15T10:00:00Z

A friend of mine shared a video the other day about SaaS prices going up across the board. The guy in the video made a point that stuck with me: these companies aren’t raising prices because their products got better. They’re raising prices because they bolted AI onto their platforms and called it an upgrade.

“Enhanced by AI.” That’s the pitch. And I keep asking the same question every time I see it: enhanced how?

What does it actually do for me? Does it save time on a specific task? Does it catch errors I’d miss? Does it change the way I work in some concrete way I can point to? Most of the time, the answer is vague at best. There’s a chatbot in the corner now, or a “smart suggestions” feature nobody asked for, and the subscription went up 20 percent.

The data says the same thing, just at a much larger scale.

McKinsey’s 2025 State of AI report found that only about 5 to 6 percent of enterprises attribute more than 5 percent of their earnings to AI.¹ Not vague sentiment. Actual bottom-line, show-it-on-the-books impact. MIT’s NANDA initiative reviewed more than 300 public AI deployments, interviewed 52 organizations and surveyed 153 senior leaders, and found that 95 percent of generative AI pilots produced no measurable financial return.² That stat has some caveats. MIT used a narrow definition of success, measured over a short window, and excluded productivity gains that hadn’t hit the P&L yet. But the direction is corroborated everywhere. RAND Corporation notes that, by some estimates, over 80 percent of AI projects fail, roughly double the failure rate of non-AI IT projects.³

So companies are spending billions on AI, packaging it into products, passing the cost to customers, and the vast majority can’t even prove it’s working inside their own organizations.

That’s a real problem.

The adoption problem nobody wants to talk about

The pattern I keep seeing is the same whether it’s a SaaS company bundling AI into a product or a company mandating AI use internally. The technology shows up, but the understanding doesn’t.

ManpowerGroup reported in January 2026 that AI usage among workers jumped 13 percent in 2025. In the same period, worker confidence in using AI dropped 18 percent. Only 64 percent of workers now feel confident using AI and the latest technology in their jobs, down from 78 percent a year earlier.⁴ Usage going up and confidence going down at the same time tells you something important about how these tools are being rolled out.

KPMG and the University of Melbourne ran a global study across 48,000 respondents in 47 countries.⁵ They found that 57 percent of employees hide their AI use or present the output as their own. Another 56 percent have used AI tools at work without knowing whether it’s even allowed. And 66 percent rely on AI output without checking whether it’s accurate.

People are using the tools. They’re just using them badly, secretly, and without support. None of that is adoption. It’s chaos dressed up as progress.

And it mirrors what’s happening on the product side. SaaS companies are integrating AI the same way employers are rolling it out internally: here’s the tool, figure it out, pay more for the privilege.

What people actually want

The guy in the video said something I agree with completely. People don’t want AI-enhanced products. They want better products — software that’s more intuitive, that takes their actual needs into account, that solves problems they care about. Whether AI is involved in making that happen is beside the point. The outcome is what matters, not the label.

This is the same thing I see in the organizations I work with. When someone tells me their team tried AI and it didn’t stick, the first thing I ask is whether anyone sat down and figured out which specific tasks AI could actually help with. The answer is almost always no. Someone bought a license, sent an email saying “we have AI now,” and expected results.

The nonprofit data makes this painfully clear. About 92 percent of nonprofits report using AI in some capacity, but only 7 percent report mission-level impact.⁶ Which makes it an implementation problem, not a technology one. And the Imagine Canada research is explicit about what’s holding organizations back: “uncertainty and limited hands-on experience, not resistance or lack of interest.”⁷

It’s the same story at the enterprise level. S&P Global found in early 2025 that 42 percent of companies had abandoned most of their AI initiatives, up from 17 percent a year earlier. Cost, data privacy and security risks topped the list of reasons.⁸ KPMG Canada found that 93 percent of Canadian businesses use AI, but only 2 percent see returns on generative AI investment.⁹ 93 versus 2. That gap should alarm anyone who’s being told AI is the answer to everything.

The part that doesn’t get said enough

AI works. It can save significant time, reduce repetitive work, and open up capacity that didn’t exist before. But only when someone takes the time to understand the tool and match it to specific tasks in a specific context. There’s no shortcut past that step.

The old saying about teaching someone to fish applies here more than anywhere. If you hand someone a fishing rod and walk away, they’re not going to eat. If you show them where the fish are, how to cast, and what bait to use for the conditions they’re dealing with, they’ll feed themselves every day after that.

AI is the same. When someone learns how to use it within their actual workflows, for the problems they actually face, the speed and quality of their work can change significantly. But that requires investment in people, not just in software licenses with “AI-powered” stamped on the label.

The SaaS price hikes and the enterprise failure rates are symptoms of the same problem. Organizations and product companies are treating AI as a feature to sell rather than a capability to build. And until that changes, most of the money being spent on AI is going to keep producing the same result: expensive tools that nobody really knows how to use.

Sources

McKinsey & Company, The State of AI in 2025: Agents, Innovation, and Transformation (November 2025). Self-reported by about 6% of 1,993 respondents. mckinsey.com ↩︎
Challapally et al., MIT Project NANDA, The GenAI Divide: State of AI in Business 2025 (2025). Coverage in Fortune ↩︎
RAND Corporation, The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed (August 2024). rand.org ↩︎
ManpowerGroup, 2026 Global Talent Barometer (January 2026), 13,918 workers in 19 countries. The survey’s confidence question changed between editions. Press release · Coverage in Fortune ↩︎
Gillespie et al., University of Melbourne & KPMG, Trust, Attitudes and Use of Artificial Intelligence: A Global Study 2025 (April 2025), 48,340 respondents. Figures on hidden use, unclear permission and unchecked output are from employees who use AI at work. doi.org/10.26188/28822919 ↩︎
Virtuous, 2026 Nonprofit AI Adoption Report (February 2026), survey of 346 nonprofits. virtuous.org ↩︎
Lasby & Baker, The State of Artificial Intelligence Adoption in Canadian Nonprofits, Imagine Canada and the Canadian Centre for Nonprofit Digital Resilience (January 2026), 963 responses. imaginecanada.ca ↩︎
S&P Global Market Intelligence, Voice of the Enterprise: AI & Machine Learning, Use Cases 2025. Coverage in CIO Dive ↩︎
KPMG in Canada, Generative AI Adoption Index (November 2025), 753 business leaders. kpmg.com ↩︎

Writing · Jarvis Learning Assistant: AI Consulting

The template trap: *why the template you downloaded isn't working*

What changed

How I actually build things

Why templates break down

Sources

Fishing rods and AI: *understanding the tool*

The adoption problem nobody wants to talk about

What people actually want

The part that doesn’t get said enough

Sources

The template trap: why the template you downloaded isn't working

Fishing rods and AI: understanding the tool