How to Scrape Google Finance with Python

TL;DR

Python guide to scrape Google Finance.
Use requests + BeautifulSoup; build the quote URL from symbol + index; parse key fields like price ranges, market cap, P/E, CEO; output as a dict.
Complete script provided.
For scale (to avoid blocks), switch the fetch layer to a scraping API like Scrapingdog.

If you are an investor, a trader, an analyst, or just curious about the overall stock market. You’ve probably already stumbled on Google Finance. It provides up-to-date stock quotes from indexes, historical financial data, news, and currency conversion rates.

Knowing how to scrape the Google Finance website, can be advantageous when it comes to:

Data Aggregation: Google Finance hosts data from different sources, minimizing the need to look for data elsewhere.
Sentiment Analysis: The website displays news from several sources. These can be scraped to gather insights about the market’s sentiment.
Market Predictions: It provides historical data and real-time information from several stock market indexes. Resulting in a very effective source for price predictions.
Risk Management: Google Finance minimizes arbitrage, thanks to its accurate and up-to-date data, which is crucial for assessing the risk associated with specific investment strategies.

Web scraping Google Finance can be achieved using Beautiful Soup and Requests Python libraries.

Why Beautiful Soup as the scraping tool?

Beautiful Soup is one of the most used web scraping libraries in Python. It comprises extensive documentation, it’s easy to implement and to integrate with other libraries. To use it, you first need to set your Python’s virtual environment, then you can easily install it using the following command.

1pip install beautifulsoup4

It is usually used side-by-side with the Requests Python library, which serves as the standard package for making HTTP requests. It generates the HTML instance, from which Beautiful Soup will interact to grab the required information. This library can also be installed via pip.

1pip install requests

How to extract information from stocks?

To extract stock information from Google Finance, we first need to understand how to play with the website’s URL to
crawl the desired stock. Let’s take, for instance, the NASDAQ index, which hosts several stocks from which we can grab
information. To have access to the symbols of each stock, we can use NASDAQ’s stock screener at this link. Now, let’s take META as our target stock. With both the index and stock, we can build the first code snippet of our script.

1import requests
2from bs4 import BeautifulSoup
3 
4 
5BASE_URL = "https://www.google.com/finance"
6INDEX = "NASDAQ"
7SYMBOL = "META"
8LANGUAGE = "en"
9TARGET_URL = f"{BASE_URL}/quote/{SYMBOL}:{INDEX}?hl={LANGUAGE}"

Now we can use the Requests library to make an HTTP request on the TARGET_URL and create a Beautiful Soup instance to crawl the HTML content.

1# make an HTTP request
2page = requests.get(TARGET_URL)
3 
4 
5# use an HTML parser to grab the content from "page"
6soup = BeautifulSoup(page.content, "html.parser")

Before getting into scraping, we first need to tackle the HTML elements by inspecting the web page (TARGET_URL).

How to Scrape Google Finance with Python

The items that describe the stock are represented by the class gyFHrc. Inside each one of these elements, there’s a class that represents the title of the item (Previous close for instance) and the value ($295.89). The first can be grabbed from the mfs7Fc class, and the second from the P6K39c respectively. The complete list of items to be scraped is the following:

Previous Close
Day Range
Year Range
Market Cap
AVG Volume
P/E Ratio
Dividend Yield
Primary Exchange
CEO
Founded
Website
Employees

Let’s now see how we can crawl these items with Python code.

1# get the items that describe the stock
2items = soup.find_all("div", {"class": "gyFHrc"})
3 
4 
5# create a dictionary to store the stock description
6stock_description = {}
7 
8# iterate over the items and append them to the dictionary
9for item in items:
10    item_description = item.find("div", {"class": "mfs7Fc"}).text
11    item_value = item.find("div", {"class": "P6K39c"}).text
12    stock_description[item_description] = item_value
13 
14 
15print(stock_description)

The function .find_all() was used to target all the elements containing the class gyFHrc. Unlike .find_all(), the function .find() only retrieves one element. That’s why it is used inside the for loop because in this case, we know that there’s only one mfs7Fc and P6K39c for each iterable item. The .text() attribute, concatenates all the pieces of text that are inside each element which is the information displayed on the webpage.

The loop in the code snippet above serves to build a dictionary of items that represent the stock. This is a good practice because the dictionary structure can easily be converted to other file formats such as a .json file or a .csv file, depending on the use case.

The output:

1{'Previous close': '$295.89', 'Day range': '$294.47 - $301.74', 'Year range': '$88.09 - $326.20', 'Market cap': '762.63B USD', 'Avg Volume': '22.93M', 'P/E ratio': '35.49', 'Dividend yield': '-', 'Primary exchange': 'NASDAQ', 'CEO': 'Mark Zuckerberg', 'Founded': 'Feb 2004', 'Website': 'investor.fb.com', 'Employees': '71,469'}

This is just an example of a simple script, that can be integrated into a trading bot, an application, or a simple dashboard to keep track of your favorite stocks.

Complete Code

You can scrape many more data attributes from the page but for now, the complete code will look somewhat like this.

1import requests
2from bs4 import BeautifulSoup
3 
4 
5BASE_URL = "https://www.google.com/finance"
6INDEX = "NASDAQ"
7SYMBOL = "META"
8LANGUAGE = "en"
9TARGET_URL = f"{BASE_URL}/quote/{SYMBOL}:{INDEX}?hl={LANGUAGE}"
10 
11# make an HTTP request
12page = requests.get(TARGET_URL)
13 
14 
15# use an HTML parser to grab the content from "page"
16soup = BeautifulSoup(page.content, "html.parser")
17 
18 
19# get the items that describe the stock
20items = soup.find_all("div", {"class": "gyFHrc"})
21 
22 
23# create a dictionary to store the stock description
24stock_description = {}
25 
26 
27# iterate over the items and append them to the dictionary
28for item in items:
29    item_description = item.find("div", {"class": "mfs7Fc"}).text
30    item_value = item.find("div", {"class": "P6K39c"}).text
31    stock_description[item_description] = item_value
32 
33 
34print(stock_description)

Limitations while scraping Google Finance

Using the above method you can create a small scraper but this scraper will not continue to supply you with data if you are going to do mass scraping. Google is very sensitive to data crawling and it will ultimately block your IP.

Once your IP is blocked you will not be able to scrape anything and your data pipeline will finally break. Now, how to overcome this issue? Well, there is a very easy solution for this and that is to use a Google Scraping API.

Let’s see how we can use this API to crawl limitless data from Google Finance.

Using Scrapingdog for scraping Google Finance

Once you sign up for Scrapindog you will get your API key(available on the dashboard). Now, just copy that API key to the below-provided code.

1import requests
2from bs4 import BeautifulSoup
3 
4BASE_URL = "http://api.scrapingdog.com/google/?api_key=YOUR-API-KEY&query=https://www.google.com/finance"
5INDEX = "NASDAQ"
6SYMBOL = "META"
7LANGUAGE = "en"
8TARGET_URL = f"{BASE_URL}/quote/{SYMBOL}:{INDEX}?hl={LANGUAGE}"
9 
10# make an HTTP request
11page = requests.get(TARGET_URL)
12 
13# use an HTML parser to grab the content from "page"
14soup = BeautifulSoup(page.content, "html.parser")
15 
16# get the items that describe the stock
17items = soup.find_all("div", {"class": "gyFHrc"})
18 
19# create a dictionary to store the stock description
20stock_description = {}
21 
22# iterate over the items and append them to the dictionary
23for item in items:
24    item_description = item.find("div", {"class": "mfs7Fc"}).text
25    item_value = item.find("div", {"class": "P6K39c"}).text
26    stock_description[item_description] = item_value
27 
28print(stock_description)

In place of YOUR-API-KEY you have to paste your API key. One thing you might have noticed is that apart from the
BASE_URL nothing has changed in the code. This is the beauty of using the web scraping APIs.

Using this code you can scrape endless Google Finance pages. If you want to crawl this then I would advise you to read web crawling with Python.

Key Takeaways

The blog explains how Google Finance aggregates real-time financial data such as stock prices, market trends, and company information directly in Google Search.
It shows that Google Finance pages are dynamically rendered, which makes traditional request-based scraping unreliable.
The article demonstrates how using a dedicated scraping API helps bypass rendering issues, bot detection, and request blocking on Google Finance.
It highlights how structured financial data like price, market cap, and related metrics can be extracted programmatically for analysis.
The post emphasizes practical use cases such as tracking stock performance, building financial dashboards, and automating market research workflows.

Conclusion

With the combination of requests and bs4, we were able to scrape Google Finance. Of course, if the scraper needs to survive then you have to use a proxy scraping APIs.

We have explored the fascinating world of web scraping Google Finance using Python. Throughout this article, we have learned how to harness the power of various Python libraries, such as BeautifulSoup and Requests, to extract valuable financial data from one of the most trusted sources on the internet.

Scraping financial data from Google Finance can be a valuable skill for investors, data analysts, and financial professionals alike. It allows us to access real-time and historical information about stocks, indices, currencies, and more, enabling us to make informed decisions in the world of finance.

I hope you like this tutorial and if you do then please do not forget to share it with your friends and on your social media.

Additional Resources

Web Scraping with Scrapingdog

Scrape the web without the hassle of getting blocked Try for Free Contact sales