AI Trends 2025: What Karpathy’s Talk Didn’t Tell You (But You Need to Know) - Software 3.0
- saurabhsarkar
- 2 days ago
- 4 min read
In his keynote at YC AI Startup School 2025, Andrej Karpathy drew a clear arc from Software 1.0 (code) to Software 2.0 (learned weights) and now Software 3.0 (prompt-based systems). Prompts, he argued, are the new source code. English is the new programming language. And large language models are the new CPUs. It's a compelling narrative that is clear, inspiring, and widely shared.
But for those of us building in the trenches, Software 3.0 isn't just a new way, it's a minefield of brittle abstractions, missing tooling, and architectural confusion. Karpathy's talk captured the high-level shift. This post covers what he didn't say and what every AI builder needs to understand to thrive in 2025.
1. What Karpathy Didn’t Tell You (But You Need to Know)
A. Prompt Engineering Is Not Software EngineeringPrompts are not deterministic. They don’t compose cleanly. There’s no IDE, no debugger, no static typing. Every time you deploy a prompt, you're crossing your fingers.
B. Tooling Lags Behind the HypeYou wouldn't ship a Python app without tests, CI, and observability but most prompt chains go into production with zero coverage and no version control. Software 3.0 lacks the scaffolding that made Software 1.0 robust.
C. Hybrid Systems Are the Norm (and a Mess)Most real-world AI apps are a Frankenstein mix of:
Software 1.0 wrappers (Python or JS glue code)
Software 2.0 model calls (custom embeddings or classifiers)
Software 3.0 prompts (ChatGPT, Claude, open-source LLMs)
The seams show. And they break. Often.
2. Insight : Prompts Are Contracts, Not Code
Karpathy says prompts are the new programs. But that analogy breaks. Code is deterministic. Prompts are probabilistic. They don’t tell the model what to do, they negotiate with it.
That makes prompt engineers more like diplomats than devs. You’re not writing logic. You’re drafting intent. A prompt is more like a negotiation or soft agreement with a model.
The core skill in Software 3.0 is not programming. It’s product thinking.
You have to:
Anticipate failure paths
Write for robustness, not precision
Iterate through user feedback loops fast
3. Insight: Judgment Is the New Moat
If LLMs can execute most "how" instructions (code, queries, drafts), then the edge moves to knowing what to ask and why it matters. That’s where domain knowledge and judgment shine.
In this world:
Prompt libraries will be commoditized.
Access to models is no longer a moat.
Taste, insight, and context are king.
The best AI builders in 2025 are not just engineers. They’re curators, editors, and orchestrators.
4. Patterns That Work: Building Software 3.0 That Doesn’t Suck
A. Design Prompts Like APIs
Each prompt should:
Have clear input/output expectations
Be testable in isolation
Avoid brittle regex-style overfitting
B. Use Prompt Modules
Break complex workflows into prompt modules:
"Summarize"
"Decide"
"Rewrite"
"Validate"
Chaining them reduces hallucination and increases control.
C. Log Everything, Version Everything
Treat prompts like code:
Use prompt versioning tools (e.g., PromptLayer, LangSmith)
Log prompts, completions, user feedback
A/B test prompt variants
5. Beyond the Stack: Org Charts Are Breaking Too
Karpathy calls LLMs the new OS. Here’s a scarier thought: they’re also breaking your org chart.
With AI agents, a single prompt chain can:
File customer tickets
Route legal contracts
Draft code
Update CRM
So why do you still have 4 different departments handling these workflows manually?
Software 3.0 collapses silos. The orgs that win will be the ones that flatten workflows around AI.
What to Do Now
Software is changing again but success won’t come from being early. It’ll come from being intentional. The teams that thrive in Software 3.0 will:
Build with hybrid awareness
Design with prompt rigor
Operate with product taste
We’re not coding the future. We’re conversing with it. Time to learn how to speak clearly.
FAQ: Software 3.0, Prompt Engineering, and AI in 2025
What is Software 3.0?
Software 3.0 is a new paradigm where natural language prompts replace traditional code, powered by large language models. Instead of writing algorithms, developers write instructions in English that LLMs interpret and execute.
Why are prompts considered contracts and not code?
Because prompts are probabilistic and open to interpretation. Unlike deterministic code, they don’t guarantee consistent output. A prompt acts more like a soft agreement with an AI system, you’re negotiating behavior rather than commanding it.
What makes prompt engineering difficult?
Prompt engineering lacks many of the developer tools we’re used to debuggers, tests, IDEs. It's trial and error, heavily dependent on model quirks, with no clear standards for modularity or reliability.
How do you build reliable Software 3.0 systems?
Use modular prompt patterns, log all completions, version prompts, and test components independently. Treat each prompt like a product feature: design, test, iterate, and monitor.
What skills matter most in Software 3.0?
Judgment, domain expertise, and the ability to articulate clear intent. The value shifts from raw coding skill to understanding the business problem deeply and shaping high-quality interactions with LLMs.
What tools help manage prompts at scale?
PromptLayer, LangSmith, and LLMOps platforms are emerging to handle version control, logging, testing, and experimentation for LLM-based workflows.
Will LLMs replace developers?
No, but they will change what developers do. Coding will increasingly involve specifying intent, curating workflows, and evaluating AI outputs rather than hand-writing every line.
How should organizations prepare for this shift?
Start by integrating AI agents into cross-functional workflows. Break down silos. Invest in training product managers, engineers, and analysts to become fluent in prompt engineering and AI literacy.
Is this hype or real?
It’s both. LLMs are powerful but still unreliable. Building with them today requires vision and pragmatism. Software 3.0 is real, but it’s not mature. That’s your opportunity.