Easy Google Search Result API with Zenserp

By (Sponsor) on September 24, 2019

No matter how much experience I gain in this industry, one task I continue to fail at is building an accurate, flexible, and maintainable scraper for site search. As soon as sites change their HTML structure, my scrape is borked. When looking to build my own search results page, I looked around and the best solution I found was Zenserp. Zenserp, created by Saas Industries, is an easy to use, logical SERP API for getting customizable Google search results!

Quick Hits

Rolling your own SERP is verify difficult and impossible to maintain (think proxy networks and CAPTCHAs) -- trust Zenserp to do it for you
API payloads are logical and easy to use
Get multiple data types: organic, paid, local, maps, and more
Zenserp does not store or sell your query requests
Very strong infrastructure - no QPS limit!
Provides a method for bulk data retrieval to consolidate API requests
Free to start!

Let's have a look at using Zenserp's API!

Using zenserp

After signing up for a free Zenserp account, you can start making requests. zenserp even has a nice question builder that allows you to export search code in cURL, Python, PHP, or Node.js:

Note: the API key used here was rotated and will no longer work

Note the available options: the search phrase, location requesting from, which Google domain you prefer, and language. Additional options include whether you want a text or image search, the number of results, and which page of results (page 1, page 2, etc.) you'd like. Super customizable!

Let's have a look at the resulting payload:

{
  "query": {
    "q": "david walsh blog",
    "hl": "en",
    "gl": "US",
    "location": "United States",
    "search_engine": "google.com",
    "apikey": "MY_API_KEY",
    "url": "https://google.com/search?q=david+walsh+blog&uule=w+CAIQICINVW5pdGVkIFN0YXRlcw&hl=en&gl=US&sourceid=chrome&ie=UTF-8"
  },
  "organic": [
    {
      "position": 1,
      "title": "David Walsh Blog - JavaScript Consultant",
      "url": "https://davidwalsh.name/",
      "destination": "https://davidwalsh.name",
      "description": "A blog featuring tutorials about JavaScript, HTML5, AJAX, PHP, CSS, WordPress, and everything else development.",
      "isAmp": false,
      "sitelinks": [
        {
          "title": "About David Walsh",
          "description": "David Walsh. My name is David Walsh. I'm a 33-year old web ...",
          "url": "https://davidwalsh.name/about-david-walsh"
        },
        {
          "title": "7 Essential JavaScript Functions",
          "description": "Seven JavaScript functions that every developer should keep ...",
          "url": "https://davidwalsh.name/essential-javascript-functions"
        },
        {
          "title": "JavaScript",
          "description": "The webpack JavaScript utility has taken over the modern ...",
          "url": "https://davidwalsh.name/tutorials/javascript"
        },
        {
          "title": "Incredible Demos",
          "description": "Chris Coyier's Favorite CodePen Demos IV. Did you know you ...",
          "url": "https://davidwalsh.name/tutorials/demos"
        },
        {
          "title": "Recent Tutorials",
          "description": "A blog featuring tutorials about JavaScript, HTML5, AJAX, PHP ...",
          "url": "https://davidwalsh.name/page/2"
        },
        {
          "title": "I Don't Hate Arrow Functions",
          "description": "While arrow functions clearly have a ubiquitous community ...",
          "url": "https://davidwalsh.name/i-dont-hate-arrow-functions"
        }
      ]
    },
    {
      "position": 2,
      "title": "David Walsh Blog - Home | Facebook",
      "url": "https://www.facebook.com/davidwalshblog/",
      "destination": "https://www.facebook.com › davidwalshblog",
      "description": "David Walsh Blog, Madison. 7996 likes · 23 talking about this. David Walsh's blog about HTML5, JavaScript (MooTools, jQuery, Dojo), CSS, PHP, AJAX,...",
      "isAmp": false
    },
    {
      "position": 3,
      "title": "David Walsh Blog - Facebook",
      "url": "https://d.facebook.com/davidwalshblog/?__tn__=%2As-R",
      "destination": "https://d.facebook.com › davidwalshblog",
      "description": "David Walsh Blog, Madison, WI. 8005 likes · 7 talking about this. David Walsh's blog about HTML5, JavaScript (MooTools, jQuery, Dojo), CSS, PHP, AJAX,...",
      "isAmp": false
    },
    // ...
  ],
  "number_of_results": 32600000
}

Since I wanted organic results, the organic property provides an interesting array of results which match when I do a manual query of david walsh blog. Notice the amount of detail in the first result, especially sitelinks, which represents the sub-links you see within a manual search on Google -- wow!

If there are paid search results, those will be in the payload as well. Let's look up bitcoin:

{
  "query": {
    "q": "bitcoin",
    "hl": "en",
    "gl": "US",
    "location": "United States",
    "search_engine": "google.com",
    "apikey": "API_KEY",
    "url": "https://google.com/search?q=bitcoin&uule=w+CAIQICINVW5pdGVkIFN0YXRlcw&hl=en&gl=US&sourceid=chrome&ie=UTF-8"
  },
  "paid": [
    {
      "paidPosition": "top",
      "title": "Buy & Sell Bitcoin | #1 in Crypto Security & Safety | coinbase.com‎",
      "trackingUrl": "https://google.com/aclk?sa=l&ai=DChcSEwjDssyHo9HkAhUahtUKHXNyAlUYABAAGgJ3cw&sig=AOD64_1Gml8jrX5PSNhnHex8ZD2ErH6vIg&q&ved=2ahUKEwjXq8aHo9HkAhXSZVAKHXrWAUoQ0Qx6BAgVEAE&adurl",
      "destinationUrl": "https://www.coinbase.com/signup",
      "breadcrumbs": [],
      "visurl": "www.coinbase.com/",
      "description": "Coinbase is the Safest, Most Secure Place to Buy and Sell Bitcoin, Ethereum, and More. We’re Obsessed with Security So You Don’t Have to Be. Buy, Sell, and Trade Crypto Safely. iOS & Android App. Secure Storage. Over 20M+ Users. Most Trusted."
    },
    {
      "paidPosition": "top",
      "title": "Buy Bitcoin In 90 Seconds | Get Started For Free‎",
      "trackingUrl": "https://google.com/aclk?sa=l&ai=DChcSEwjDssyHo9HkAhUahtUKHXNyAlUYABABGgJ3cw&sig=AOD64_0a5ShbhklcaLMdUNRMqxNkiDtDSw&q&ved=2ahUKEwjXq8aHo9HkAhXSZVAKHXrWAUoQ0Qx6BAgWEAE&adurl",
      "destinationUrl": "https://gemini.com/",
      "breadcrumbs": [
        {
          "url": "https://exchange.gemini.com/register",
          "text": "Buy Bitcoin Instantly"
        },
        {
          "url": "https://gemini.com/",
          "text": "\"Best Crypto Exchange\""
        },
        {
          "url": "https://gemini.com/security/",
          "text": "Industry-Best Security"
        }
      ],
      "visurl": "www.gemini.com/",
      "description": "Gemini Makes Buying Bitcoin Simple, Safe & Secure. Open A Free Account Today & Start Experiencing The Future of Money! Trade In Minutes. Superior Security. Most Trusted. Free Sign Up. Services: Buy & Sell Bitcoin, Price Charts, Price Alerts, Secure Custody."
    }
  ],
  "organic": [
    {
      "position": 1,
      "isCarousel": true,
      "isVertical": false,
      "news": [
        {
          "title": "China's PBC Is Warming Up To Digital Currencies -- Good News For ...Forbes9 hours ago",
          "url": "https://www.forbes.com/sites/panosmourdoukoutas/2019/09/14/chinas-pbc-is-warming-up-to-digital-currencies-good-news-for-bitcoin-eth-xrp-and-ltc/"
        },
        {
          "title": "Bitcoin Price Indicator May Signal Next Leg HigherCoinDesk12 hours ago",
          "url": "https://www.coindesk.com/bitcoin-price-indicator-may-signal-next-leg-higher"
        },
        {
          "title": "Ethereum Network Demand Surges; Will ETH’s Price Follow?NewsBTC3 mins ago",
          "url": "https://www.newsbtc.com/2019/09/14/ethereum-network-demand-surges-will-eths-price-follow/"
        }
      ]
    },
    // ...
  ],
  "number_of_results": 513000000
}

Notice the paid key which provides results usually seen at the top of searches for which were paid for by each vendor.

The above queries are the simplest but only the tip of the iceberg; here are a few other valuable search options:

device - mobile or desktop results
tbm - image search (isch) or maps (lcl)
timeframe - h, d, w, etc. for search results based on time units

There are loads of other customizations available to see in the Zenserp documentation.

Google Trends

Zenserp also provides a /trends API endpoint to query Google Trends:

curl https://app.zenserp.com/api/v1/trends \
    -F "keyword[]=bitcoin" \
    -F "keyword[]=ethereum" \
    -F "apikey=YOUR-API-KEY"

Notice that you can provide multiple keywords using the array syntax!

Use Cases

The obvious use case for Zenserp is creating my own custom search page but SERPs are so much more useful than that -- you could brainstorm any variety of use cases:

Monitoring your (or the competition's) search placement for an important keyword
Tracking popular trends
Analyzing searches your users are making via your own search forms using SERP in the background
Downloading images that match a given term
Tracking a given business type's density in a given area using Maps search
....too many to list!

As much as I love trusting my own scraping talents, it's so much easier to trust Zenserp. I save time in maintenance and get the advantage of their easy to use API and its wealth of options. The more I learn about topics in this industry, the more I've learned to delegate difficult tasks to experts. Zenserp is the my expert in Google search results APIs.