DEV Community

Build a Simple Keyword Ranking Monitor with Python and a SERP API

What We Are Building

At some point, every SEO script starts with a small question: Where does my site rank for this keyword? Then the small question grows legs. You want to track 10 keywords. Then 100. Then multiple competitors. Then different countries. Then daily changes. Then someone asks for a CSV report every Monday. Now the tiny script has become a tiny SEO robot with a clipboard.

In this article, we will build a simple keyword ranking monitor with Python and a SERP API. The goal is not to build a full SEO platform. The goal is to build a clean, practical script that can:

keyword list → SERP API → organic results → target domain match → ranking snapshot → CSV report

You can use this as a starting point for SEO monitoring, competitor tracking, content performance checks, or internal reporting.

We will build a Python script that:

  • Reads keywords from a text file
  • Calls a SERP API for each keyword
  • Extracts organic results
  • Checks whether a target domain appears
  • Records ranking position
  • Saves a daily CSV snapshot
  • Compares today's ranking with a previous snapshot

The final output will look like this:

date,keyword,target_domain,found,position,url,title
2026-01-01,best serp api,example.com,true,3,https://example.com/serp-api,Example SERP API Page

That is enough to answer the basic SEO question: Did my ranking go up, down, disappear, or stay the same?

Why Use a SERP API?

You can scrape search result pages yourself. For a demo, that may work. But for ranking monitoring, scraping gets annoying quickly. You need to deal with:

  • HTML changes
  • Blocked requests
  • CAPTCHA
  • Location differences
  • Device differences
  • Missing snippets
  • Ads shifting layout
  • Unexpected SERP features

A SERP API gives you structured search result data. Instead of parsing a search page manually, you call an API and get JSON. The workflow becomes:

keyword → SERP API → organic_results → ranking check

Much cleaner. Less duct tape. Fewer midnight gremlins.

Install Dependencies

Create a new project folder and install the packages:

pip install requests python-dotenv pandas

We will use:

  • requests → call the SERP API
  • python-dotenv → load API keys
  • pandas → save and compare CSV files

Create a .env File

Create a .env file:

SERP_API_KEY=your_api_key
SERP_API_URL=https://your-serp-api-endpoint.example.com/search
TARGET_DOMAIN=example.com

This tutorial uses a generic SERP API style. Your provider may use different parameter names, such as:

  • q / query
  • engine
  • location
  • gl
  • hl
  • device

Adjust the request function to match your provider's docs. The rest of the ranking logic stays the same.

Create a Keyword File

Create keywords.txt:

best serp api
google search api
serp api for ai agents
rank tracking api
local seo rank tracker

Use real keywords from your workflow. Do not only test easy keywords. Real search results are messy, and messy is where your monitor proves whether it is useful.

Step 1: Load Settings

Create a file called rank_monitor.py.

import os
import time
import requests
import pandas as pd
from datetime import date
from urllib.parse import urlparse
from dotenv import load_dotenv

load_dotenv()

SERP_API_KEY = os.getenv("SERP_API_KEY")
SERP_API_URL = os.getenv("SERP_API_URL")
TARGET_DOMAIN = os.getenv("TARGET_DOMAIN")

Now add a quick validation function.

def validate_settings():
    if not SERP_API_KEY:
        raise ValueError("Missing SERP_API_KEY")
    if not SERP_API_URL:
        raise ValueError("Missing SERP_API_URL")
    if not TARGET_DOMAIN:
        raise ValueError("Missing TARGET_DOMAIN")

It is better to fail early than to discover 200 empty rows later. Empty CSV files have a special talent for wasting afternoons.

Step 2: Load Keywords

Add a helper to read keywords from keywords.txt.

def load_keywords(filename="keywords.txt"):
    with open(filename, "r", encoding="utf-8") as file:
        return [line.strip() for line in file if line.strip()]

Keeping keywords outside the code makes the script easier to reuse. Tomorrow you can test a different keyword set without touching Python.

Step 3: Call the SERP API

Now write the API function.

def fetch_serp(query, location="United States", language="en"):
    params = {
        "api_key": SERP_API_KEY,
        "engine": "google",
        "q": query,
        "location": location,
        "language": language,
        "output": "json",
    }
    response = requests.get(SERP_API_URL, params=params, timeout=30)
    response.raise_for_status()
    return response.json()

This function returns raw JSON. Keep it separate from the ranking logic. When something fails, you want to know whether the API call failed or your parser failed.

Step 4: Extract Organic Results

Different SERP APIs may use slightly different field names. Common examples include:

  • organic_results
  • organic
  • results

Let's support a few common shapes.

def get_organic_items(data):
    possible_keys = ["organic_results", "organic", "results"]
    for key in possible_keys:
        value = data.get(key)
        if isinstance(value, list):
            return value
    return []

This small defensive parser makes the script easier to adapt across providers.

Step 5: Normalize Organic Results

Now normalize each result into one internal format.

def normalize_result(item):
    return {
        "position": item.get("position") or item.get("rank") or "",
        "title": item.get("title") or "",
        "url": item.get("link") or item.get("url") or "",
        "snippet": item.get("snippet") or item.get("description") or "",
    }

Your ranking code should not care whether the API calls the URL field link or url. Normalize once. Use clean data everywhere else. That tiny layer is the broom closet where future bugs go to behave.

Step 6: Extract Domains

To check ranking, we need to compare domains. A search result URL like this:

https://www.example.com/blog/best-serp-api

should match:

example.com

Add a domain helper.

def extract_domain(url):
    if not url:
        return ""
    parsed = urlparse(url)
    domain = parsed.netloc.lower()
    if domain.startswith("www."):
        domain = domain[4:]
    return domain

Now add a domain matching function.

def domain_matches(result_url, target_domain):
    result_domain = extract_domain(result_url)
    target_domain = target_domain.lower().strip()
    if target_domain.startswith("www."):
        target_domain = target_domain[4:]
    return result_domain == target_domain or result_domain.endswith("." + target_domain)

This allows subdomains to match too. For example: blog.example.com can match example.com. If you only want exact domain matching, replace the return line with:

return result_domain == target_domain

Step 7: Find the Ranking Position

Now we can search through organic results and find the first matching URL.

def find_domain_ranking(results, target_domain):
    for result in results:
        url = result["url"]
        if domain_matches(url, target_domain):
            return {
                "found": True,
                "position": result["position"],
                "url": result["url"],
                "title": result["title"],
                "snippet": result["snippet"],
            }
    return {
        "found": False,
        "position": "",
        "url": "",
        "title": "",
        "snippet": "",
    }

This checks whether the target domain appears in the organic results. If it appears, we save the position. If not, we mark it as not found.

Step 8: Track One Keyword

Now combine the API call, parsing, normalization, and ranking check.

def track_keyword(keyword, target_domain, location="United States", language="en"):
    data = fetch_serp(query=keyword, location=location, language=language)
    organic_items = get_organic_items(data)
    normalized_results = [normalize_result(item) for item in organic_items]
    ranking = find_domain_ranking(results=normalized_results, target_domain=target_domain)
    return {
        "date": date.today().isoformat(),
        "keyword": keyword,
        "target_domain": target_domain,
        "location": location,
        "language": language,
        "found": ranking["found"],
        "position": ranking["position"],
        "url": ranking["url"],
        "title": ranking["title"],
        "snippet": ranking["snippet"],
        "result_count": len(normalized_results),
    }

The result_count field is useful. If the API returns zero organic results, you want to know. A "not found" result means something different when there were 10 results versus zero results.

Step 9: Track All Keywords

Add a loop for all keywords.

def track_keywords(keywords, target_domain, location="United States", language="en", delay=1):
    rows = []
    for keyword in keywords:
        print(f"Tracking keyword: {keyword}")
        try:
            row = track_keyword(
                keyword=keyword,
                target_domain=target_domain,
                location=location,
                language=language,
            )
            rows.append(row)
        except Exception as exc:
            print(f"Failed keyword: {keyword}")
            print(f"Error: {exc}")
            rows.append({
                "date": date.today().isoformat(),
                "keyword": keyword,
                "target_domain": target_domain,
                "location": location,
                "language": language,
                "found": False,
                "position": "",
                "url": "",
                "title": "",
                "snippet": "",
                "result_count": 0,
                "error": str(exc),
            })
        time.sleep(delay)
    return rows

The delay keeps the script from firing requests too aggressively. Respect rate limits. A ranking monitor should be boring and polite, not a request cannon in a trench coat.

Step 10: Save a Snapshot

Now save the results as a daily CSV.

def save_snapshot(rows):
    today = date.today().isoformat()
    filename = f"ranking_snapshot_{today}.csv"
    df = pd.DataFrame(rows)
    df.to_csv(filename, index=False)
    print(f"Saved snapshot: {filename}")
    return filename

You will get files like:

  • ranking_snapshot_2026-01-01.csv
  • ranking_snapshot_2026-01-02.csv
  • ranking_snapshot_2026-01-03.csv

That gives you a ranking history.

Full Script

Here is the complete first version.

import os
import time
import requests
import pandas as pd
from datetime import date
from urllib.parse import urlparse
from dotenv import load_dotenv

load_dotenv()

SERP_API_KEY = os.getenv("SERP_API_KEY")
SERP_API_URL = os.getenv("SERP_API_URL")
TARGET_DOMAIN = os.getenv("TARGET_DOMAIN")

def validate_settings():
    if not SERP_API_KEY:
        raise ValueError("Missing SERP_API_KEY")
    if not SERP_API_URL:
        raise ValueError("Missing SERP_API_URL")
    if not TARGET_DOMAIN:
        raise ValueError("Missing TARGET_DOMAIN")

def load_keywords(filename="keywords.txt"):
    with open(filename, "r", encoding="utf-8") as file:
        return [line.strip() for line in file if line.strip()]

def fetch_serp(query, location="United States", language="en"):
    params = {
        "api_key": SERP_API_KEY,
        "engine": "google",
        "q": query,
        "location": location,
        "language": language,
        "output": "json",
    }
    response = requests.get(SERP_API_URL, params=params, timeout=30)
    response.raise_for_status()
    return response.json()

def get_organic_items(data):
    possible_keys = ["organic_results", "organic", "results"]
    for key in possible_keys:
        value = data.get(key)
        if isinstance(value, list):
            return value
    return []

def normalize_result(item):
    return {
        "position": item.get("position") or item.get("rank") or "",
        "title": item.get("title") or "",
        "url": item.get("link") or item.get("url") or "",
        "snippet": item.get("snippet") or item.get("description") or "",
    }

def extract_domain(url):
    if not url:
        return ""
    parsed = urlparse(url)
    domain = parsed.netloc.lower()
    if domain.startswith("www."):
        domain = domain[4:]
    return domain

def domain_matches(result_url, target_domain):
    result_domain = extract_domain(result_url)
    target_domain = target_domain.lower().strip()
    if target_domain.startswith("www."):
        target_domain = target_domain[4:]
    return result_domain == target_domain or result_domain.endswith("." + target_domain)

def find_domain_ranking(results, target_domain):
    for result in results:
        url = result["url"]
        if domain_matches(url, target_domain):
            return {
                "found": True,
                "position": result["position"],
                "url": result["url"],
                "title": result["title"],
                "snippet": result["snippet"],
            }
    return {
        "found": False,
        "position": "",
        "url": "",
        "title": "",
        "snippet": "",
    }

def track_keyword(keyword, target_domain, location="United States", language="en"):
    data = fetch_serp(query=keyword, location=location, language=language)
    organic_items = get_organic_items(data)
    normalized_results = [normalize_result(item) for item in organic_items]
    ranking = find_domain_ranking(results=normalized_results, target_domain=target_domain)
    return {
        "date": date.today().isoformat(),
        "keyword": keyword,
        "target_domain": target_domain,
        "location": location,
        "language": language,
        "found": ranking["found"],
        "position": ranking["position"],
        "url": ranking["url"],
        "title": ranking["title"],
        "snippet": ranking["snippet"],
        "result_count": len(normalized_results),
    }

def track_keywords(keywords, target_domain, location="United States", language="en", delay=1):
    rows = []
    for keyword in keywords:
        print(f"Tracking keyword: {keyword}")
        try:
            row = track_keyword(
                keyword=keyword,
                target_domain=target_domain,
                location=location,
                language=language,
            )
            rows.append(row)
        except Exception as exc:
            print(f"Failed keyword: {keyword}")
            print(f"Error: {exc}")
            rows.append({
                "date": date.today().isoformat(),
                "keyword": keyword,
                "target_domain": target_domain,
                "location": location,
                "language": language,
                "found": False,
                "position": "",
                "url": "",
                "title": "",
                "snippet": "",
                "result_count": 0,
                "error": str(exc),
            })
        time.sleep(delay)
    return rows

def save_snapshot(rows):
    today = date.today().isoformat()
    filename = f"ranking_snapshot_{today}.csv"
    df = pd.DataFrame(rows)
    df.to_csv(filename, index=False)
    print(f"Saved snapshot: {filename}")
    return filename
Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.