Parts 1 and 2 explained the stack. A model does math, the compute layer translates that math, and the hardware path decides whether your GPU or CPU does the work.
Now we test the path.
The goal is simple: install Lemonade Server, pull a model that fits normal desktop hardware, run one request, and confirm that the machine is using the acceleration path you expected.
This is not a benchmark contest. It is a sanity check. Before you tune prompts, swap models, or build tools on top of local AI, you need to know the foundation is actually working.
What You Need
| Requirement | Why it matters |
|---|---|
| Lemonade Server | The local server, model manager, and API bridge |
| 8 GB RAM minimum | Enough memory for compact models |
| 5 to 20 GB free storage | Models and runtime packages need local disk |
| A GPU, optional | Faster inference when the backend supports it |
| A terminal | The fastest way to verify the install |
If you do not have a supported GPU, the workflow still works. Lemonade falls back to CPU. The response will be slower, but the API and model flow are the same.
Step 1: Install Lemonade
On Windows, install the current Lemonade Server package from the project releases page:
After installation, open a new terminal and check that the command is available:
lemonade --help
That command should print the available Lemonade CLI actions. If your terminal cannot find lemonade, close the terminal and open a new one so your PATH refreshes.
Step 2: Pull a Starter Model
Pull a compact model first. The point is to prove the loop works before asking your machine to load something large.
lemonade pull Gemma-4-E2B-it-GGUF
This downloads the model once and keeps it in the local cache. After that, the model is available without another download.
To see what Lemonade knows about your local models:
lemonade list
If the model appears in the list, the model management layer is working.
Step 3: Start a Local Chat
Launch the browser chat interface:
lemonade launch claude
The local UI opens against your Lemonade Server instance. The important part is the address: your browser is talking to a server on your own machine, usually at http://localhost:13305.
Ask something small first:
Give me a three bullet summary of what local AI is.
If you get a response, the basic loop is complete:
- Your prompt reached the local server
- The server loaded a model
- The model generated tokens
- The response came back through the local API
Step 4: Check the API Directly
The browser UI is useful, but the API is what makes Lemonade interesting. Tools can point at the same local server.
Use the models endpoint as a quick health check:
curl http://localhost:13305/api/v1/models
You should see JSON describing available models. That tells you the server is reachable and responding through the OpenAI-compatible API shape.
Here is a minimal Python request using the OpenAI client:
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:13305/api/v1",
api_key="not-needed",
)
response = client.chat.completions.create(
model="Gemma-4-E2B-it-GGUF",
messages=[
{"role": "user", "content": "Write one sentence about local AI."}
],
)
print(response.choices[0].message.content)
The code looks like a cloud API call, but the base URL points to your machine.
Step 5: Verify the Hardware Path
This is the part most people skip.
If your machine has an AMD Radeon card in the supported RX 5000 through RX 9000 families, Lemonade should prefer the ROCm path when the backend and model support it. If that path is unavailable, it can step down to Vulkan or CPU.
For a practical check, watch three things:
| Signal | What it tells you |
|---|---|
| Install logs | Which backend packages were installed |
| Model load messages | Which recipe and backend the model used |
| Response speed | Whether the request feels GPU-class or CPU-class |
Do not use "it responded" as your only test. CPU fallback can still respond. The question is whether the fast path is active.
If a small model takes a long time to answer short prompts, you are probably not on the GPU path. That does not mean the system is broken. It means the fallback ladder is doing its job, and you should check your backend support, driver state, and model recipe.
A Clean Mental Model
After this walkthrough, the local stack should feel less mysterious:
Lemonade CLI
installs and manages models
Lemonade Server
exposes local APIs on localhost
Backend recipe
picks llama.cpp, stable-diffusion.cpp, whisper.cpp, or another engine
Compute path
uses ROCm, Vulkan, CUDA, Metal, NPU, or CPU depending on hardware support
The server hides most of that complexity from the tool using it. That is the point. The complexity still exists, but it is concentrated in one local service instead of being scattered across every app.
What Comes Next
Text chat is the first proof. The more interesting step is that Lemonade is not only a chat server. It exposes local endpoints for images, speech, transcription, embeddings, and multimodal workflows.
Part 4 looks at that bigger surface area: why a local AI server becomes much more valuable when text, image, and audio all share the same machine, the same privacy boundary, and the same API layer.