How to Get Google Search Results in Python?

To get Google search results in Python, the safest and most reliable method is the official Google Custom Search JSON API (using requests or google-api-python-client), which returns structured JSON for any query. For quick prototypes you can use the unofficial googlesearch-python library; for production-scale SERP data use a third-party API such as SerpApi; and for rendered, JavaScript-dependent results use Selenium. Avoid scraping google.com/search directly in production, as it violates Google's Terms of Service and gets IP-blocked.

The five legitimate methods covered in this guide are:

• Method 1 - Google Custom Search JSON API (official, recommended)
• Method 2 - the googlesearch-python library (quick, unofficial)
• Method 3 - third-party SERP APIs such as SerpApi (for scale)
• Method 4 - scraping with requests and BeautifulSoup (educational only)
• Method 5 - Selenium for JavaScript-rendered results (the testing angle)

Method 1 - Google Custom Search JSON API (Official, Recommended)

The Custom Search JSON API is Google's official, Terms-of-Service-safe way to retrieve search results programmatically. It requires a Google Cloud API key plus a Programmable Search Engine ID, and it returns clean structured JSON. The free tier covers 100 queries per day, after which paid pricing applies. You can scope the engine to specific sites or configure it to search the entire web.

Note: Google has begun winding this API down, closing new sign-ups and announcing full deprecation. Existing keys still work, but if you cannot create a new Programmable Search Engine and key, use a third-party SERP API (Method 3) instead.

Setup steps:

• Create a Programmable Search Engine at programmablesearchengine.google.com and enable "Search the entire web".
• Copy the Search engine ID (cx) from the engine's control panel.
• In the Google Cloud Console, enable the Custom Search API for your project.
• Create an API key under Credentials.

With those two values you can query the API directly with requests, no SDK required:

import requests

API_KEY = "YOUR_API_KEY"
CSE_ID = "YOUR_SEARCH_ENGINE_ID"

def google_search(query, num=10):
    url = "https://www.googleapis.com/customsearch/v1"
    params = {"key": API_KEY, "cx": CSE_ID, "q": query, "num": num}
    response = requests.get(url, params=params, timeout=10)
    response.raise_for_status()
    data = response.json()
    return data.get("items", [])

for item in google_search("selenium python automation"):
    print(item["title"])
    print(item["link"])
    print(item.get("snippet", ""), "\n")

Alternatively, use the official Google client library. Install it first (this corrects the older typo !pip instal):

pip install google-api-python-client

Then build the service and fetch results:

from googleapiclient.discovery import build

API_KEY = "YOUR_API_KEY"
CSE_ID = "YOUR_SEARCH_ENGINE_ID"

service = build("customsearch", "v1", developerKey=API_KEY)
result = service.cse().list(q="selenium python automation", cx=CSE_ID, num=10).execute()

for item in result.get("items", []):
    print(item["title"], "->", item["link"])

A single call returns up to 10 results. To page further, pass the start parameter (1, 11, 21, and so on), keeping in mind the API caps at 100 results per query. For production, never hardcode credentials, load the API key and engine ID from environment variables instead.

Method 2 - The googlesearch-python Library (Quick, Unofficial)

The googlesearch-python package scrapes Google under the hood (using requests and BeautifulSoup) and needs no API key. It is ideal for quick prototypes and small scripts, but it is unofficial and not suited to production because it can be rate-limited or blocked.

pip install googlesearch-python

The basic call yields result URLs, while advanced mode returns the title, URL, and description for each result:

from googlesearch import search

# Simple: yields result URLs
for url in search("selenium python automation", num_results=10):
    print(url)

# Advanced: title, url, and description for each result
for result in search("selenium python automation", num_results=10, advanced=True, lang="en", region="us"):
    print(result.title)
    print(result.url)
    print(result.description, "\n")

Caveat: add a sleep_interval between requests to reduce the chance of temporary blocks, and respect Google's Terms of Service. If you start seeing empty results, you are most likely being rate-limited.

Method 3 - Third-Party SERP APIs for Scale

When the official API's limits or scope aren't enough, third-party SERP APIs handle proxies, CAPTCHAs, parsing, and the legal liability on their side. Options include SerpApi, Serper, ScraperAPI, and Zenserp; all are paid services with free tiers for testing. They return rich structured SERP data (organic results, ads, People Also Ask, knowledge graph) so you don't parse fragile HTML yourself.

pip install google-search-results

from serpapi import GoogleSearch

params = {
    "engine": "google",
    "q": "selenium python automation",
    "api_key": "YOUR_SERPAPI_KEY",
}
search = GoogleSearch(params)
results = search.get_dict()

for item in results.get("organic_results", []):
    print(item.get("position"), item.get("title"), item.get("link"))

Because the vendor manages compliance and blocking, this is the most practical choice for high-volume or richly structured SERP collection.

Method 4 - Scraping With requests and BeautifulSoup (Educational)

This approach is shown for learning only. Scraping google.com/search directly violates Google's Terms of Service, breaks whenever Google changes its markup, and quickly triggers CAPTCHAs and IP bans. Use it for experiments, and prefer the official or third-party APIs in production.

pip install requests beautifulsoup4

import requests
from bs4 import BeautifulSoup

headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"}
params = {"q": "selenium python automation", "hl": "en"}
resp = requests.get("https://www.google.com/search", headers=headers, params=params, timeout=10)

soup = BeautifulSoup(resp.text, "html.parser")
for h3 in soup.select("h3"):
    print(h3.get_text())

Caveat: the CSS selectors are fragile because Google rotates and obfuscates its class names, and you should expect to be blocked at any meaningful volume.

Method 5 - Selenium for JavaScript-Rendered Results

Some result data is rendered or lazy-loaded by JavaScript, so a plain HTTP request never sees it. Selenium drives a real browser and gives you the fully rendered DOM. This is also exactly how QA teams test search and result UIs across browsers. For a deeper walkthrough, see the TestMu AI Selenium Python resources.

pip install selenium

from selenium import webdriver
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()
driver.get("https://www.google.com/search?q=selenium+python+automation&hl=en")

for el in driver.find_elements(By.CSS_SELECTOR, "h3"):
    print(el.text)

driver.quit()

To verify that rendered results and result UIs look and behave correctly across many browser and OS combinations, you can run the same Selenium script on a cloud grid by pointing the driver at a remote WebDriver endpoint instead of a local browser. TestMu AI offers Selenium Automation across 3000+ browser and OS combinations, so you confirm consistent rendering everywhere from a single script.

from selenium import webdriver

options = webdriver.ChromeOptions()
options.set_capability("browserName", "Chrome")
options.set_capability("browserVersion", "latest")
options.set_capability("LT:Options", {
    "platformName": "Windows 11",
    "build": "Google SERP Test",
})

driver = webdriver.Remote(
    command_executor="https://USERNAME:[email protected]/wd/hub",
    options=options,
)
driver.get("https://www.google.com/search?q=selenium+python+automation")
# ... extract and assert on rendered results ...
driver.quit()

Comparison - Which Method Should You Use?

Legal and Rate-Limit Notes

Google's Terms of Service prohibit automated scraping of its search pages, so prefer the official Custom Search JSON API or a licensed SERP API for anything beyond a quick experiment. Respect robots.txt, stay within published query quotas, and add exponential backoff between requests to avoid temporary blocks. The legality of scraping varies by jurisdiction, so when in doubt, use a sanctioned API that puts the compliance burden on the provider.

Frequently Asked Questions

Is it legal to scrape Google search results in Python?

Google's Terms of Service prohibit automated scraping of its search result pages, and doing so at scale typically results in CAPTCHAs and IP blocks. Legality also varies by jurisdiction. The safe, compliant path is the official Custom Search JSON API or a licensed third-party SERP API rather than scraping google.com/search directly.

What is the official way to get Google search results in Python?

The official way is the Google Custom Search JSON API. You create a Programmable Search Engine to get a search engine ID (cx), enable the Custom Search API in Google Cloud and create an API key, then query the endpoint with requests or the google-api-python-client SDK to receive structured JSON results.

How do I get more than 10 or 100 results from Google in Python?

The Custom Search JSON API returns up to 10 results per call and you paginate with the start parameter, but it caps at 100 results per query. For larger volumes of SERP data, use a third-party SERP API such as SerpApi that handles pagination and proxies for you.

Why is googlesearch-python getting blocked or returning no results?

googlesearch-python scrapes Google under the hood, so rapid requests trigger rate limiting and temporary IP blocks, which return empty results. Add a sleep_interval between requests, reduce volume, or switch to the official Custom Search JSON API or a third-party SERP API for reliable access.

Can I get Google search results for free in Python?

Yes, within limits. The Custom Search JSON API includes 100 free queries per day before paid pricing applies. The googlesearch-python library is free but unofficial and can be blocked. Most third-party SERP APIs also offer a small free quota for testing.

How do I test that search results render correctly across browsers?

Use Selenium to drive a real browser so JavaScript-rendered results load, then run the same script across multiple browser and OS combinations on a cloud Selenium grid. This validates that your result UI renders and behaves consistently everywhere rather than only on your local machine.

• Fix Errors Python Automation Testing
• How Many Python Web Frameworks Are There?
• How to Comment Multiple Lines in Python?
• How to Use and and or in Python?