Hey it's Paul, welcome to my 80th newsletter!

Have you ever thought about creating your own AI agent?

It might sound complicated, but what if you could build and run one entirely on your own computer, without relying on external services?

That's now a reality, and it opens up a world of possibilities!

In this week's tutorial, I'll guide you through the steps to build, power, and run your very own AI agent locally. No need for expensive APIs or concerns about data privacy.

Let's dive in!

I've created a video version of this tutorial

If you prefer to follow it on YouTube, you can click here to watch it:

Watch the video on YouTube

1. Install LM Studio

First things first, we need a way to host and interact with an AI model on your local machine. That's where LM Studio comes in.

Go to LM Studio
Click the "Download" button.
Choose the correct version for your operating system (Windows, macOS, Linux).

2. Download an AI model

After launching LM Studio, you'll need to download an AI model. Think of this as the "brain" of your AI agent.

Click on the "Discover" tab within LM Studio.
You'll see a wide variety of models available. For this tutorial, I recommend using Gemma 3 12B from Google. It's a relatively new model that's quite powerful for its size.
In the search bar, type "Gemma 3 12B".
Look for the version by "Bartowski". You'll see different "quantizations" (sizes) of the model. For most users, the Q4 version is a good choice.

Click the download icon next to the Q4 version. The download time will depend on your internet connection.
Once downloaded, you'll be able to chat with the model directly within LM Studio to test it out!

3. Install necessary tools

To build our AI agent, we'll need a few more tools (keep in mind that you won't have to write a single line of code by yourself!):

Python:If you don't have it already, download the latest version from the official Python website.
Smolagents:This is a lightweight Python library that simplifies the process of building AI agents. We'll install this later using pip (Python's package installer).
VS Code:This is a popular and free code editor. You can download it from. While I'll be using VS Code, you can use any code editor you prefer.

4. Craft the prompt to build your AI Agent

Since we're not writing the code ourselves, we'll leverage the power of our local LLM to generate it for us!

Here's the prompt I used, broken down into parts:

Instructions:"I'm trying to run this Python app that uses Smolagents, a Python library for building Al agents. Explain step by step how I can run it on my Windows 11 laptop in a uv Python environment. I've already installed VS Code and Python 3.12 on my machine. I also want to use a local LLM using LM Studio to run this agent (adapt the script if necessary)."

Script Example (Description of the Task): I described the specific task I want the AI agent to perform:
- Go to the Hugging Face Models hub (huggingface.co/models).
- Filter the models by the "CC BY-SA" license.
- Sort the filtered models by the number of "likes".
- Extract the names of the top 5 most liked models with that license.
- Save these names into a .txt file.
LightLLM documentation:I included a brief mention of LightLLM. This is another library that "Smolagents" can use to connect to LM Studio, allowing our AI agent to use the local LLM without needing API keys from OpenAI or other providers.

5. Generate the AI Agent Code with LM Studio

Now it's time to put that prompt to work!

Open LM Studio and start a new chat.
At the top, select the Gemma 3 12B model you downloaded.
You can keep the default context length. Click "Load Model".
Paste the prompt you crafted in the chat window and click "Send".

The Gemma 3 12B model will now process your request and generate the Python code for your AI agent, along with step-by-step instructions on how to run it.

This process happens entirely on your local machine!

6. Run Your AI Agent

Once LM Studio has generated the code and instructions, follow the steps generated by Gemma above:

Copy the code
Create a project folder

Create the Python File: inside the project folder, create a new file named app.py
Set up the environment (if needed) and install the dependencies
Configure LM Studio API:
- In LM Studio, go to the "Developer" tab (usually the last icon on the left).
- You should see the Gemma 3 12B model listed as "Loaded".
- Click the toggle button to "Start Server". This will start a local API that your AI agent can communicate with.
- Note the API base URL provided (it will likely be something like http://localhost:1234).
- In your app.py file, you'll need to configure the LightLLM connection within the small_agent code.Make sure the api_base in your Python script matches the URL from LM Studio, and append /v1 to it (e.g., http://localhost:1234/v1). Also, ensure the model_id in your script matches the name of the model you downloaded in LM Studio (e.g., "google_gemma-3-12b-it").
Run the Script: Open your terminal in VS Code (Terminal > New Terminal) and navigate to your project folder. Then, run the Python script using the command: python app.py (if your file is named app.py).

You should now see output in your terminal as your AI agent interacts with the local LM Studio API, performs the web scraping task on Hugging Face, and saves the results to a .txt file in your project folder!

You can also monitor the API requests in the LM Studio "Developer" tab.

This demonstrates the power of running AI agents entirely locally, giving you control over your data and eliminating the need for external API dependencies.

Have a great weekend and see you next week!

Paul

Promote your business in this newsletter

0 comments

Build faster with Gemini and its new integration

Build anything with ChatGPT o3 and Canvas

Google Sheets now has a "=AI" formula!

Generate & Deploy a landing page with Gemini (for free!)

Build your Second AI Brain with NotebookLM

See all posts from Paul Couvert