Building AI Apps

Building Apps: Your Code and the LLM API (Part 4)

Recap: The AI Computer and Its Operating System

In Part 1: The AI Transistor, we opened the black box of the neural network computer, revealing how its core components are very different from a regular computer's main processor (CPU). They work by recognizing patterns at the same time, instead of following instructions one after another. Then, in Part 2: The Pattern Matching Computer, we looked closely at how this special way of working operates. We compared how LLMs find patterns (using statistics and probability) with how traditional software finds patterns (using clear, fixed rules). Finally, in Part 3: The LLM Operating System, we learned how a large language model acts like a new type of operating system, organizing digital information, tools, and tasks.

Now that we understand what the "AI computer" is, how its "brain" works, and how its "operating system" truly operates, the big question is: How do we, as developers, actually write code to work with this new kind of operating system? How do we link our usual code to its many features and truly "build apps" that use its strength?

Connecting to the LLM OS: The API Gateway

The main way to connect to the LLM operating system is surprisingly simple: through APIs. Just like your code sends requests over the internet to a regular web service or uses functions from another software library, it talks to an LLM. You send data (your natural language instructions, or prompts) over a network, and you receive data (the LLM-generated response).

But programming this new operating system isn't exactly like calling a standard web address (a REST endpoint). The "commands" we give are in everyday language. How we put these commands together—which is called prompt engineering—is very important. Simple methods include "zero-shot" prompting (giving a direct instruction) and "few-shot" prompting (giving examples of what you want to put in and get out). The AI's answer then needs to be correctly understood and broken down. This can be as simple as showing text or as complex as pulling out organized information.

New System Calls: Tool-Calling and the Model Context Protocol

To truly build strong applications, our code needs to do more than just send text back and forth. It needs to use the main features of the LLM operating system. This is where Tool-Calling (often called function calling) and the Model Context Protocol (MCP) come into play.

Think of Tool-Calling as the LLM operating system's version of a system call or a shell command. Instead of just creating text, the LLM can decide to run an outside function in your app based on what it understands the user wants to do. For example, if a user says "Find me the weather in Paris," the LLM operating system, using its design, knows this means it should call a getWeather(location: string) function that you've made available to it. The LLM doesn't find the weather itself; it smartly manages when and how that action happens inside your application.

The Model Context Protocol (MCP) builds on this by giving developers an organized way to control the LLM's memory (its context window) and guide its attention. It's more than just one command. It's about setting up the continuing conversation, giving important past information, and assigning roles (like system instructions, user messages, AI answers, and tool results). MCP lets you effectively control how information moves and how decisions are made during a continuous conversation with the LLM. This is similar to how a regular operating system manages tasks and memory.

Streamlining Integration: The Vercel AI SDK core primitives

Connecting our code to this new "LLM operating system" can involve a lot of repetitive setup work: like handling secret API keys, dealing with answers that come in parts, understanding outputs from different AI models, and arranging complicated lists of commands. Libraries like the Vercel AI SDK make this easier by offering simple, ready-to-use tools that hide most of the complex details of talking to the API. They are your main set of tools for programming the LLM operating system from a React/Next.js application.

Here's a breakdown of the Vercel AI SDK core tools and key methods that make working with the LLM operating system easier:

(Tech Hint: Example code snippets (Next.js, Vercel AI SDK), effective prompt patterns for tool calls would follow the table to illustrate the concepts.)

Conclusion

By learning how to use APIs, prompt engineering, and Tool-Calling, and by using tools like the Vercel AI SDK and understanding LM Middleware, developers can successfully "program" the LLM operating system. This lets us build strong, smart applications that smoothly combine the AI's special ability to find patterns with the clear, fixed rules of our regular code. In the next part of this series, "Managing Many AIs," we'll look at how to put these parts together and handle several LLM operating systems to create complicated, multi-step AI programs and work processes.