The AI Engineer Roadmap: From "What's a Token?" to Shipping Real AI Products

So you opened X, saw 47 new AI tools launched before breakfast, and now you feel like everyone got a memo you didn't. Good news: there is no memo. Just a mountain of hype, a few buzzwords used wrong on purpose, and a real set of skills you can learn in a sane order. I put this roadmap together for devs like me who already ship code and want to go from "I use ChatGPT for regex" to "I build AI features people actually use." Each phase builds on the last, and wherever real money is involved I added a cost estimate, because nothing ruins a side project like a surprise API bill at 3 AM. Grab your coffee (or a token-efficient energy drink) and let's dive in. TL;DR - AI Engineering is mostly integration and reliability, not training models from scratch. - The path: LLM basics → prompting → APIs → RAG → agents → evals and security → production. - You can do almost all of it for $0 to ~$15 USD with free tiers, local models, and cheap API models. - Build as you learn: there's a 5-tier project ladder at the end, from "simple API app" to "agents building agents." - Tools change every month, concepts don't. Learn the concepts first and treat tools as examples. What Does an AI Engineer Actually Do? Before we start learning things, let's answer the obvious question: what's the job? Spoiler: it's less "train a neural network in a dark room" and more "make a model behave inside a real product, reliably, without bankrupting the company." Day to day, an AI Engineer usually does some mix of these: - Work with AI APIs. Call models from code, handle streaming, errors, rate limits, and keep an eye on token costs. - Design effective prompts. Write instructions that get consistent results, not just one lucky answer. - Build RAG pipelines. Give a model access to your own data (docs, tickets, databases) so it stops making things up. - Create agents. Let a model use tools, make decisions, and take actions instead of just chatting. - Write evaluation frameworks. Build the test suites that tell you if a prompt, model, or pipeline is actually good. - Fine-tune models. Adapt a model to a specific task or style when prompting and RAG aren't enough. - Evaluate continuously. Measure quality in production, catch regressions, and compare models when a new one drops (which is every other Tuesday). Notice what's not on the list: inventing new model architectures. That's research. This roadmap is about the engineering side, and if you can already build backends and frontends, you're closer than you think. Phase 0 - The Modern AI Toolbox (Vocabulary You'll Keep Hearing) Before the fundamentals, a quick vocabulary tour. These are terms you'll see in every AI conversation, tweet, and job description. You don't need to master them yet, just know what they are so you don't nod along like you understand (we've all done it). - Skills: Reusable instructions that tell an AI how to do a specific kind of task better. - llms.txt : A file that lets AI models read what your website is about in a direct, machine-friendly way. - AGENTS.md : A context file an AI agent loads every time it starts, so it always knows the rules of your project. If you've seenCLAUDE.md , it's the same idea. - MCP (Model Context Protocol): A standard that lets an AI connect to the services you use every day (Google Drive, Gmail, Notion...) and take actions in them. - Commands: Reusable shortcuts you trigger on demand to run a predefined workflow with your AI tool. And a few buzzwords you'll bump into later. We'll come back to them, but for now, just recognize the names: - Harness Engineering - Specs-Driven Development - Forward Deployed Engineer Think of Phase 0 as the map legend: you don't need to memorize it, but the rest of the trip is easier when you know what the symbols mean. Phase 1 - LLM Fundamentals Goal: understand how modern AI systems actually work under the hood, so you stop treating them like magic (or like a very confident intern). - [ ] Generative AI: AI that creates new content (text, images, code, audio) instead of just classifying or predicting. - [ ] Large Language Models (LLMs): Models trained on huge amounts of text to predict what comes next, which turns out to be surprisingly powerful. - [ ] Fine-Tuning: Further training a model on your own data to adapt its behavior or style. - [ ] Tokens: The chunks of text a model reads and writes. They're also how you get billed. - [ ] Context: Everything the model can "see" when it generates a response. - [ ] Memory: How information is carried across conversations, since models don't remember anything by default. - [ ] Context Window: The maximum amount of tokens a model can handle at once. - [ ] Hallucinations: When a model confidently makes things up. Very confident. Zero shame. - [ ] Temperature: A setting that controls how predictable or creative the output is. - [ ] Embeddings: Numeric representations of text that capture meaning, so you can compare things by similarity. - [ ] System Prompts: Hidden instructions that define how the model should behave. - [ ] RAG: Giving a model external knowledge at query time. We'll get to it in Phase 4. Try it yourself: you don't need to pay anything to play with these ideas. Tools like Ollama let you run open models locally on your machine, and Hugging Face is the place to browse and discover thousands of open models. Don't try to memorize all of this in one sitting. Come back to this list as the later phases make each concept click. Phase 2 - Prompt Engineering (and Context Engineering) This is the foundation for everything that comes next, and most devs underestimate it. "It's just writing text to a chatbot," they say, right before spending three days debugging a prompt that works 70% of the time. - [ ] Core techniques: few-shot prompting (show examples), chain-of-thought (ask the model to reason step by step), and role prompting (tell it who it is). - [ ] Structured outputs: JSON mode and other ways to force a specific output format, so your code can actually parse what comes back. - [ ] System prompts vs. user prompts: what goes where, and why the order matters. - [ ] Measurable prompt evaluation: test your prompts with real criteria, not just "looks good to me." - [ ] Context Engineering: deciding what goes into the model's context (instructions, examples, retrieved data, tool results, history), not just how you word the prompt. A good mental shift: prompt engineering is about how you ask, while context engineering is about what the model gets to see. As your apps grow, the second one starts to matter more than the first. Quick rule of thumb: if you change a prompt and can't tell whether it got better or worse, you don't have a prompt problem yet, you have an evaluation problem. We'll fix that in Phase 6. Phase 3 - LLM APIs and SDKs Time to stop chatting in a UI and start calling models from code. This is the phase where AI stops being a toy and becomes just another service in your stack (one that occasionally answers in Shakespearean English when you asked for JSON). - [ ] Anthropic API and OpenAI API: authentication, streaming responses, error handling, and rate limits. - [ ] Token costs: how to estimate them before you ship, and how to keep them under control. - [ ] Function calling / Tool use: letting the model request actions in your code. This is the foundation of agents. - [ ] Structured outputs with schemas: define the shape of the response with Pydantic (Python) or Zod (TypeScript) and validate what comes back. 💰 Estimated cost: ~$5-15 USD in API credits if you use cheap models like Haiku or GPT-4o-mini. 🆓 Free option: run open models locally with Ollama, or use the free tiers listed in the "Free APIs and Resources" section at the end of this post. Suggested exercise: build a tiny script that takes a messy text (an email, a support ticket, a review) and returns a validated JSON object with the fields you care about. It covers auth, structured outputs, error handling, and cost tracking in one afternoon. Phase 4 - RAG (Retrieval-Augmented Generation) How to give an LLM "memory" or external knowledge without fine-tuning. Instead of retraining the model on your data, you fetch the relevant pieces at query time and hand them over along with the question. It's the difference between making someone memorize the whole library and letting them use the index. - [ ] Embeddings: what they are (no math required) and how to generate them through an API. - [ ] Vector databases: the core concepts, and how similarity search finds "things that mean something similar." - [ ] Chunking strategies: why the way you split your documents matters more than you'd expect. - [ ] The full pipeline: ingest → chunk → embed → store → retrieve → generate. Stack options: - [ ] pgvector: a Postgres extension. If you already use Postgres, it's $0 extra. - [ ] 100% free alternative: ChromaDB or Qdrant, self-hosted and running locally. - [ ] Embeddings: text-embedding-3-small (cheap) or Ollama (free, runs locally). 💰 Estimated cost: close to $0 with the local stack, and only cents with text-embedding-3-small for a small project. Suggested exercise: build a "chat with your docs" app over a folder of Markdown files. When it answers wrong (and it will), figure out whether the problem was retrieval or generation. That debugging habit is worth more than any tutorial. Phase 5 - Agents and Orchestration This is where an LLM stops answering and starts acting. It reads a goal, picks a tool, looks at the result, and decides what to do next. It's also where the hype is loudest, so let's keep our feet on the ground. - [ ] Real agent vs. marketing hype: a chatbot with a fancy name isn't an agent. Learn to tell the difference. - [ ] Agent Tools: the functions and services an agent can call (search, databases, APIs, your file system). This builds directly on the function calling you learned in Phase 3. - [ ] Patterns: ReAct, planning, multi-step tool use, and loops with guardrails. - [ ] Error handling: infinite loops, runa