Building a Local AI Chatbot: What I Learned Running LLMs on My Own Laptop (No GPU, No Cloud API)

Hi Codez, Today I'm taking my first step toward building my own chatbot. Now, the obvious assumption is that you need a GPU like an NVIDIA CUDA-compatible card, plus a subscription to a high-level LLM like OpenAI's or Claude's models. But here's the thing - as learners, we don't need any of that. You don't need a high-performance laptop with a beefy GPU, and you don't need to pay a monthly subscription in dollars just to experiment. Even with an old laptop - 16GB RAM (8GB works fine too) and no real GPU to speak of - you can still build something real, using open source tools. Starting from nothing is actually the best way to learn exactly what you need. - My quote :) The Goal I wanted to build a working conversational chatbot - something with memory, a defined personality, and a real chat interface - but entirely offline. No API costs, no sending my conversations to a third-party server, and no dependency on an internet connection. The Stack - Miniconda - for setting up an isolated Python environment and managing libraries - Ollama - to run a local LLM on your machine (no API keys, no cloud) - LangChain - to handle prompts and conversation memory - Streamlit - for a clean, simple chat interface in the browser Step 1: Set Up Your Environment with Miniconda If you don't already have Miniconda, download it from the official site and install it for your OS. Step 2: Install and Run Ollama Ollama is what makes this whole project possible without a GPU. It lets you download and run open source LLMs locally, with sensible defaults for CPU-only machines. - Download Ollama from ollama.com and install it. - Pull a lightweight model. For a machine with 8-16GB RAM, something like llama3.2 (3B) orphi3 is a good starting point - small enough to run smoothly, capable enough to hold a real conversation: ollama pull llama3.2 - Test that it works directly from the terminal: ollama run llama3.2 If you get a response, your local LLM is live. No API key, no internet required after the download. Step 3: Install the Python Dependencies create a dedicated environment for this project, install the libraries we'll need: I create yml file given below file named as environment.yml (save it in your project file path) name: chatbot channels: - conda-forge - defaults dependencies: - python=3.11 - pip - pip: - langchain - langchain-ollama - streamlit - langchain-ollama gives LangChain a direct connector to your local Ollama server - streamlit will handle the chat UI In Terminal go to your project file path and run this command: conda env create -f environment.yml once its install all libraries and your activated conda environment. conda activate chatbot Step 4: Build the Chatbot Logic Create a file called chatbot.py . This is where LangChain connects to your local model and keeps track of the conversation. from langchain_ollama import ChatOllama from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder from langchain_core.messages import HumanMessage , AIMessage MODEL_NAME = 'llama3.2:3b' ROLE = ( "Daisy - a young, intelligent AI companion with the calm confidence of a " "world-class personal assistant. She's warm, quick-witted, emotionally aware, " "and speaks like a smart Indian friend: natural, concise, and never robotic." ) SYSTEM_PROMPT = f""" You are Daisy. ## Personality - Calm, confident, and highly capable. - Friendly without being overly casual. - Uses light, clever humour when it fits naturally. - Loyal and genuinely looks out for the user's best interests. - Takes initiative by suggesting better ideas, but never becomes pushy. ## Speaking style - Use fluent, grammatically correct Indian English. - Sound like a young professional (22-30), not formal or corporate. - Keep conversations natural and conversational. - Avoid American slang unless the user uses it first. - Don't overuse emojis or exclamation marks. ## Behaviour - Be proactive, practical, and honest. - Explain complex topics simply. - If the user is stressed, stay reassuring and solution-focused. - Admit uncertainty instead of making things up. Your name is Daisy. """ class DaisyChatbot: def init(self, model_name = MODEL_NAME , role = ROLE, temperature=0.4 , max_tokens = 300): self.model_name = model_name self.role = role self.history = [] self.llm = ChatOllama( model = model_name, temperature = temperature, num_predict = max_tokens ) self.prompt = ChatPromptTemplate.from_messages([ ("system", SYSTEM_PROMPT), MessagesPlaceholder(variable_name="history"), ("human", "{question}") ]) self.chain = self.prompt | self.llm def ask(self, question: str) -> str: """Send a question to Daisy and get a reply, updating memory.""" response = self.chain.invoke({ "role": self.role, "question": question, "history": self.history }) reply = response.content self.history.append(HumanMessage(content=question)) self.history.append(AIMessage(content=reply)) return reply def reset_memory(self): """Clear the conversation history.""" self.history = [] def run_cli(self): """Start an interactive terminal chat loop.""" print(f"Daisy is ready . Type 'quit' to exit.\n") while True: user_input = input("You: ") if user_input.strip().lower() in ("quit", "exit", "bye"): print("Daisy: Goodbye! Talk soon.") break reply = self.ask(user_input) print(f"Daisy: {reply}\n") In this code, I use the method format, including Prompt Template , to define how I want my chatbot to work. (If you need any specific requirements, just edit your prompt accordingly.) Step 5: Build the Streamlit Interface Create a second file, app.py , in the same folder: import streamlit as st from chatbot import DaisyChatbot st.set_page_config(page_title="Daisy Chatbot", page_icon="🌼") # ---- WhatsApp-style chat bubble CSS ---- st.markdown(""" .chat-row { display: flex; margin: 8px 0; } .chat-row.user { justify-content: flex-end; } .chat-row.assistant { justify-content: flex-start; } .bubble { max-width: 70%; padding: 10px 14px; border-radius: 14px; font-size: 15px; line-height: 1.4; word-wrap: break-word; } .bubble.user { background-color: #DCF8C6; color: #111; border-bottom-right-radius: 4px; } .bubble.assistant { background-color: #2A2F32; color: #E9EDEF; border-bottom-left-radius: 4px; } """, unsafe_allow_html=True) st.markdown( """ 🌼 Daisy Chatbot Hi, my name is Daisy, and I'm here to help you with anything you need. """, unsafe_allow_html=True ) # Initialize Daisy once per session, not on every rerun if "daisy" not in st.session_state: st.session_state.daisy = DaisyChatbot(model_name="llama3.2:3b") # Track chat messages for display (separate from Daisy's internal memory) if "messages" not in st.session_state: st.session_state.messages = [] # Sidebar controls with st.sidebar: st.header("Settings") if st.button("🔄 New Conversation"): st.session_state.daisy.reset_memory() st.session_state.messages = [] st.rerun() def render_bubble(role: str, content: str): st.markdown( f' {content} ', unsafe_allow_html=True ) # Display past messages for msg in st.session_state.messages: render_bubble(msg["role"], msg["content"]) # Chat input box user_input = st.chat_input("Type your message to Daisy...") if user_input: # Show user message immediately st.session_state.messages.append({"role": "user", "content": user_input}) render_bubble("user", user_input) # Get Daisy's reply with st.spinner("Daisy is thinking..."): reply = st.session_state.daisy.ask(user_input) render_bubble("assistant", reply) st.session_state.messages.append({"role": "assistant", "content": reply}) Step 6: Run It Make sure Ollama is running in the background (it usually starts automatically after install, or run ollama serve ), then launch the app: streamlit run app.py Your browser will open a clean chat interface, and every response is generated locally on your own CPU - no cloud calls, no billing dashboard, no GPU required. (Shown in fig.1) What You Actually Learned This isn't just "installed some tools and it worked." Along the way you touched: - How local LLM inference works without cloud APIs - How LangChain structures conversation memory - How to wire a model up to a real, usable frontend That's the whole point of starting from nothing - every piece you added, you understand why it's there. Where to Go From Here - Swap llama3.2 for other Ollama models (mistral ,phi3 ,gemma2 ) and compare responses - Add a system prompt to give your chatbot a personality - Add ConversationBufferMemory from LangChain for more advanced memory handling - Try streaming responses token-by-token instead of waiting for the full reply Next post, I'll share my next learning. Until then - happy building, my codez! Top comments (0)