Suppose you plan to go on a coffee date but can’t think of any places. You navigate to Google and search for ‘coffee places near me,’ and Google presents you with these results:
To explore more options, we click on the ‘More places’ button, and Google provides us with a list of local businesses in the nearby area.
Google Local Results can be useful for collecting and generating local leads in a particular area of interest, while also providing various information about these businesses, including customer ratings, reviews, pricing, websites, and much more. This information can later be used for competitor and sentiment analysis, price tracking, improving rankings, and more.
Scraping Google Local Results With Scrapingdog
Bulk data collection, including phone numbers, popular times, customer ratings, and reviews, is only possible using a scraper that has already built a large infrastructure of proxies and other technical resources to bypass any kind of protection.
In this tutorial, we will use Scrapingdog’s Google Local API to extract local business listings from Google.
Setting up a Scrapingdog Account
Navigate to the Scrapingdog registration page and get your API Key to access the API.
We will be scraping the title, rating, reviews, price description, and other data from the webpage.
First, we will install the Requests library.
pip install requests
After that, we will use the Scrapingdog Google Local API to extract the local businesses in a particular area.
import requests
payload = {'api_key': 'APIKEY', 'query':'coffee+in+manhattan'}
resp = requests.get('https://api.scrapingdog.com/google_local', params=payload)
data = resp.json()
print(data["local_results"])
Running this code will give you the following results:
[
{
"title": "Bird & Branch Coffee Roasters",
"rating": "4.7",
"reviews": "762",
"price": "$1–10",
"description": "Family-run specialty coffee shop",
"address": "359 W 45th St",
"type": " Coffee shop",
"thumbnail": "data:image/jpeg;base64,/9j....."
"place_id": "197653116721562462",
"place_id_search": "https://api.scrapingdog.com/google_local?api_key=67447bf02cb60bc491d62040&ludocid=197653116721562462",
"gps_coordinates": {
"lat": "40.7602998",
"lng": "-73.9907758"
}
},
{
"title": "Copper Mug Coffee",
"rating": "4.8",
"reviews": "538",
"price": "$1–10",
"description": "Dine-in·Curbside pickup·No-contact delivery",
"address": "38 W 30th St",
"type": " Coffee shop",
"thumbnail": "data:image/jpeg;base64,/9j/...",
"services": [
"Dine-in",
"Curbside pickup",
"No-contact delivery"
],
"place_id": "14733505741487564719",
"place_id_search": "https://api.scrapingdog.com/google_local?api_key=67447bf02cb60bc491d62040&ludocid=14733505741487564719",
"gps_coordinates": {
"lat": "40.7465799",
"lng": "-73.98822419999999"
}
},
....
]
You can also save this data to a CSV file using the following code:
local_results = data.get("local_results", [])
# Define the CSV file name
csv_file = 'coffee_shops_manhattan.csv'
# Define the CSV column headers
headers = [
"title", "rating", "reviews", "price", "description", "address", "type", "thumbnail",
"place_id", "place_id_search", "latitude", "longitude", "services"
]
# Open the CSV file for writing
with open(csv_file, mode='w', newline='', encoding='utf-8') as file:
writer = csv.DictWriter(file, fieldnames=headers)
writer.writeheader()
# Write each result as a row in the CSV
for result in local_results:
row = {
"title": result.get("title", ""),
"rating": result.get("rating", ""),
"reviews": result.get("reviews", ""),
"price": result.get("price", ""),
"description": result.get("description", ""),
"address": result.get("address", ""),
"type": result.get("type", ""),
"thumbnail": result.get("thumbnail", ""),
"place_id": result.get("place_id", ""),
"place_id_search": result.get("place_id_search", ""),
"latitude": result.get("gps_coordinates", {}).get("lat", ""),
"longitude": result.get("gps_coordinates", {}).get("lng", ""),
"services": ', '.join(result.get("services", [])) # Join services as a comma-separated string
}
writer.writerow(row)
print(f"Data saved to {csv_file}")
This is how you can scrape Google Local Results using Python without creating a large, dedicated scraper. You can also perform this task in other languages, such as Node.js, Java, Go, and more.
Conclusion
In this tutorial, we learned to scrape Google Local Results using Python with Scrapingdog’s API.
I hope you liked this article and if you do, please share this article on Social Media.
Thanks for reading it!👋
