How to Add LLM Features to Your Next.js App Without Rewriting Everything: The /agents Pattern Explained
Adding LLM capabilities to an existing Next.js app doesn't require a complete rewrite. We'll show you the /agents pattern that keeps your codebase clean and lets you scale AI features incrementally.
The Real Problem: AI Integration Chaos in Production Next.js Apps
You've got a working Next.js application. It's handling users, payments, databases—the whole stack. Then the requirement lands: "We need AI features."
Your first instinct might be to panic. Do you restructure the entire codebase? Do you add API calls to every route handler? Do you scatter prompts across utility files and hope you remember which version is live?
This is exactly where most teams go wrong. I've watched projects get stuck for weeks because developers tried to retrofit AI into their existing architecture without a clear pattern. Prompt versions conflict with deployed code. LLM calls spread across five different files. Observability becomes impossible. And when you need to switch from Claude to Grok or add token counting, you're touching half the codebase.
The solution isn't a rewrite. It's a pattern: the /agents directory structure that lets you add AI capabilities as an isolated, testable layer on top of your existing Next.js application.
Why Architecture Matters for AI Integration
Before we get to code, let's be clear about why this matters. Unlike regular backend features, LLM integrations have unique characteristics that break typical folder structures:
If you scatter LLM calls across your route handlers and Server Actions, you'll end up maintaining the same prompt logic in three places. If you put all prompts in one utility file, it becomes a dumping ground that nobody wants to touch.
The /agents pattern solves this by treating AI capabilities as a first-class citizen in your codebase—not as a side effect of your API routes or business logic.
The /agents Directory Pattern: Structure First, Integration Second
Here's the folder structure you'll add to your existing Next.js project:
```
app/
├── api/
│ └── ai/
│ ├── stream/
│ │ └── route.ts
│ └── chat/
│ └── route.ts
└── agents/
├── prompts/
│ ├── customer-support.ts
│ ├── content-generator.ts
│ └── code-analyzer.ts
├── schemas/
│ ├── chat.ts
│ └── document.ts
├── tools/
│ ├── search-docs.ts
│ └── fetch-user-data.ts
└── index.ts
```
Notice: this doesn't touch your existing `/app` structure. No routes get moved. No components get reorganized. You're adding alongside, not rewriting.
Step 1: Define Your First Prompt in /agents/prompts
Start here. This is where your AI instructions live, versioned and organized.
```
// app/agents/prompts/customer-support.ts
export const customerSupportPrompt = `You are a helpful customer support agent for our SaaS platform.
Your job is to:
Be concise, friendly, and ask clarifying questions when needed.
Never make up features or pricing. If you don't know, say so.`;
export const systemPrompt = {
role: "system" as const,
content: customerSupportPrompt
};
```
This is intentionally simple. No complex abstractions yet. Just the prompt, versioned in your codebase, where you can track changes in git.
Step 2: Create Your First Route Handler
Add a streaming route at `/app/api/ai/stream/route.ts`. This is your entry point for LLM calls.
```
// app/api/ai/stream/route.ts
import { Anthropic } from "@anthropic-ai/sdk";
import { customerSupportPrompt } from "@/app/agents/prompts/customer-support";
const anthropic = new Anthropic({
apiKey: process.env.ANTHROPIC_API_KEY
});
export async function POST(request: Request) {
const { messages } = await request.json();
const stream = await anthropic.messages.stream({
model: "claude-3-5-sonnet-20241022",
max_tokens: 1024,
system: customerSupportPrompt,
messages: messages
});
const reader = stream.toReadableStream();
return new Response(reader, {
headers: {
"Content-Type": "text/event-stream",
"Cache-Control": "no-cache",
"Connection": "keep-alive"
}
});
}
```
This handler:
Test this endpoint in isolation. Make sure streaming works. Then move forward.
Step 3: Add Tools and Context in /agents/tools
Once your basic streaming works, add tools that your agent can call:
```
// app/agents/tools/fetch-user-data.ts
export async function fetchUserDataTool(userId: string) {
const response = await fetch(`/api/users/${userId}`, {
headers: {
Authorization: `Bearer ${process.env.INTERNAL_API_KEY}`
}
});
if (!response.ok) throw new Error("Failed to fetch user data");
return response.json();
}
```
Keep tools simple and single-purpose. Each tool should do one thing well.
Step 4: Wire AI Into Your Existing Flows (Carefully)
Only after your /agents layer is solid, connect it to your existing Server Actions or route handlers:
```
// app/actions/create-support-ticket.ts
"use server";
import { db } from "@/lib/db";
import { generateTicketSummary } from "@/app/agents/ticket-agent";
export async function createSupportTicket(
userId: string,
description: string
) {
// Your existing business logic
const user = await db.user.findUnique({ where: { id: userId } });
// NEW: Generate AI summary using the /agents layer
const summary = await generateTicketSummary(description);
// Save to database with AI-generated summary
const ticket = await db.ticket.create({
data: {
userId,
description,
summary,
status: "open"
}
});
return ticket;
}
```
Notice: your existing Server Action barely changes. The AI logic is isolated in /agents/ticket-agent.ts. When you need to modify the prompt or swap models, you only touch that one file.
Common Mistakes to Avoid
Scaling This Pattern
As your AI features grow, this structure scales naturally:
When you're ready to scale further, tools like LangChain or Vercel's ai package integrate cleanly on top of this foundation. You're not locked in—you've just got a clean baseline.
Why This Approach Wins
This pattern works because it respects two truths about production software:
ZipBuild handles this architectural problem at scale—it generates production-ready scaffolding with AI patterns already baked in, so you skip the setup decisions entirely. But whether you build it yourself using the /agents pattern or use a scaffold, the principle is the same: isolate your AI logic, version your prompts, and integrate carefully.
Start with /agents/prompts. Add one streaming route. Get it working. Then expand. Don't rewrite your entire application because you need AI features. Just add the layer, test it in isolation, and connect it carefully.
Try the free discovery chat at zipbuild.dev to explore how structured scaffolding can accelerate your AI integration from the start.
Written by ZipBuild Team
Ready to build with structure?
Try the free discovery chat and see how ZipBuild architects your idea.
Start Building