Building an HTTP Server From Scratch using TCP.

HTTP is something I interact with every single day, whether I am fetching data on the frontend or building backend APIs. But for a long time, if you had asked me how it actually worked under the hood, I wouldn't have had a good answer. I always treated HTTP like this intimidating, hyper-complex black box that would be miserable to understand, let alone rebuild from scratch. Once I actually dug into the internals, though, I realized the core mechanics are surprisingly straightforward. I am using Go for this project, but the concepts are entirely language-agnostic. You can follow along in C, Rust, Python, Node, or whatever language you like; as long as it can open a network socket (sorry, HTML and CSS won't cut it here). To understand HTTP, we first have to talk about TCP/IP, because HTTP is just a text-protocol riding on top of TCP/IP stack. The Internet Protocol (IP) has one basic job: move raw data from Machine A to Machine B across the world. But it doesn't dump a massive file across the wire all at once. If you download a 10 GB file, IP chops that data into tiny pieces called packets, usually around 1,500 bytes each. It does this because network hardware has physical transmission limits, routers need to share bandwidth fairly among thousands of users, and resending one dropped 1.5 KB packet over flaky Wi-Fi is painless compared to re-downloading an entire 10 GB stream. Every device on the internet gets an IP address so these packets know where to go. When you want to visit google.com , your computer doesn't magically know Google's physical server address right away. It first asks a DNS server to translate google.com into an actual IP address, like 142.250.190.46 . Once your machine has that destination IP, it stamps its own IP address into the packet header so Google knows where to send the reply, and fires the packets into the wild. The catch is that IP is completely "best-effort." It launches packets into the network and immediately stops caring. If a router gets overloaded and drops your packet, or if packets take different routes and arrive completely out of order, IP won't fix it. That is why we need TCP (Transmission Control Protocol). TCP sits right on top of IP to turn that chaotic packet delivery into a reliable stream. It tracks every byte with sequence numbers so out-of-order data gets assembled correctly. If a packet goes missing, TCP notices and asks the sender to retransmit it. It also introduces the concept of ports; so when data finally arrives at a computer's IP address, the operating system knows whether to hand those bytes to your web server on port 80 or your database on port 5432. So now TCP has solved our biggest headache. We have a solid, reliable pipe open between our client and our server. You throw bytes into one end, and they pop out the other end in the exact right order without getting lost. Problem solved, right? Not even close. Because while TCP is great at moving bytes from one place to another reliably, it is completely blind to what those bytes actually mean. It treats everything as one never-ending, continuous river of data (a raw byte stream). If your browser sends a request to load a profile picture, and immediately sends another request for a stylesheet, TCP just mashes all those bytes together in a single stream. Your server is now sitting there staring at a raw chunk of bytes, completely clueless: - Where does the first request end and the second one begin? - Is the client trying to download a file, submit a form, or delete a record? - Is this data plain text, an image, or JSON? - If something goes wrong on the server, how do we tell the client? If we didn't have a standard rulebook, every single developer would invent their own chaotic format. You'd write your own custom protocol where maybe you put an exclamation mark at the end of a message, while someone else uses a random binary flag. Your backend wouldn't be able to talk to any standard browser because neither speaks the same language. This is where HTTP (Hypertext Transfer Protocol) steps in. HTTP is nothing more than an agreed-upon rulebook. If TCP is a telephone line connecting two people, HTTP is the grammar they agree to speak so they understand each other. At its core, HTTP turns that blind stream of TCP bytes into predictable, structured messages. In HTTP/1.1, it does this entirely using plain text: First, it forces the client to state its intent right on the very first line: like GET /index.html HTTP/1.1 . Now the server instantly knows the action (GET ), the target (/index.html ), and the protocol version. Next, it uses standard key-value headers separated by clean line breaks; specifically \r\n (CRLF: Carriage Return + Line Feed). Why two characters instead of just \n ? Because early internet protocols inherited typewriter conventions from telegraph and terminal days, and now we are stuck with it forever. Then, it solves the boundary problem with an empty line (\r\n\r\n ), which screams to the parser: "Hey, the headers are done! Whatever comes next is the actual body payload." Finally, the server replies with a standardized response that includes a status code like 200 OK if everything went well, or 404 Not Found if you asked for something that doesn't exist. That's all HTTP really is. It's not magic, and it's not an intimidating engine. It's just a structured text format running over a raw TCP socket. Once you realize it's just plain text over a byte stream, building one yourself becomes a whole lot less scary. What Does Raw HTTP Actually Look Like? Before we write the code to parse requests and generate responses, let's look at the exact text format traveling across the wire. 1. The HTTP Request Format When a client wants something from our server, it sends a plain-text payload formatted like this: POST /users HTTP/1.1 Host: localhost:8080 User-Agent: curl/8.0.0 Accept: / Content-Type: application/json Content-Length: 26 {"name": "Dev", "age": 22} - Line 1 (Request Line): Action ( POST ), path (/users ), and version (HTTP/1.1 ), terminated by\r\n . - Lines 2-6 (Headers): Key-value metadata lines, each terminated by \r\n . - Line 7 (Empty Line): A single blank \r\n with no characters. This tells our server: "The headers are done." - Line 8 (Body Payload): Exactly 26 raw bytes of data matching the Content-Length header. 2. The HTTP Response Format Once our server finishes processing, it writes back an answer formatted like this: HTTP/1.1 200 OK Content-Type: application/json Content-Length: 35 {"message": "user list endpoint"} - Line 1 (Status Line): Version ( HTTP/1.1 ), status number (200 ), and status message (OK ), followed by\r\n . - Lines 2-3 (Headers): Key-value details about what we are sending back, terminated by \r\n . - Line 4 (Empty Line): A single blank \r\n to mark the end of response headers. - Line 5 (Body Payload): Exactly 35 raw bytes of data sent down the wire. Now that we know the format for both directions, let's build the server to handle it. Building an HTTP Server from Raw TCP I like to break the implementation down into four distinct steps: - Initialize a TCP socket and bind it to a local port. - Accept incoming client connections in a loop. - Read raw bytes from the socket and parse the HTTP request. - Construct an HTTP response and write those bytes back over the wire. 1. Setting Up the TCP Listener In any programming language, listening for network traffic comes down to a few basic steps. You ask the operating system to reserve a port (like 8080), and then you wait for someone to connect. package main import ( "fmt" "log" "net" ) func main() { // Ask the OS to open port 8080 and listen for incoming traffic listener, err := net.Listen("tcp", ":8080") if err != nil { log.Fatal("[SERVER] Failed to bind to port: ", err) } // Make sure we release the port when the server stops defer listener.Close() fmt.Println("[SERVER] Listening on port :8080...") // Keep the server running in an infinite loop to accept other connections for { // Our program pauses right here until a client connects conn, err := listener.Accept() if err != nil { fmt.Printf("[SERVER] Failed to accept connection: %v\n", err) continue } // Pass the connection to a background worker so the loop can keep spinning go handleConnection(conn) } } What is happening here? - Binding: The operating system locks port 8080 for us. From now on, any data sent to this port comes straight to our app. - Accepting: Our program pauses and waits. When a client finally connects, the operating system wakes us up and hands us a connection object ( conn ). - Concurrency: If we try to process this connection right here in the main loop, our server will freeze for everyone else. So, we hand the connection off to run in the background. Go uses goroutines , Python might use threads, and Node uses its event loop. The idea is exactly the same: move the work out of the way so we can instantly wait for the next person. 2. The Request Lifecycle (handleConnection ) Before we dive into the details of parsing, let's look at the whole journey of a single connection. Our handleConnection function does four things in order: it wraps the raw socket to read from it easily, parses the incoming text into a request we can understand, checks the URL path to see what the user wants, and sends back a text response. func handleConnection(conn net.Conn) { // Always close the connection when we are completely done defer conn.Close() clientAddr := conn.RemoteAddr().String() // Wrap the raw connection in our custom stream reader reader := stream.NewReader(conn) // Try to make sense of the incoming bytes req, err := request.Parse(reader) if err != nil { // If they sent garbage data, reply with a 400 Bad Request res := response.New() res.SetStatus(400) res.SetBody([]byte("400 Bad Request"), "text/plain") _ = res.Send(conn) return } // Prepare a blank response to fill out res := response.New() // Basic routing: look at the path and decide what to send back switch r