YouTube is the largest online video-sharing platform in the world and currently ranks as the second most popular social media platform in terms of active users.
The recent new feature by YouTube — “YouTube Shorts”- has garnered over 5 trillion views in just a short period.
The vast amount of data including videos, ads, comments, channel information, likes and dislikes adds to the massive opportunity for businesses and can be utilized in training AI models for generating videos and understanding human language for sentimental analysis, creating LLMs, and much more. To leverage this insane size of data, you undoubtedly need a scalable scraping API that can make this possible. In this article, we will use Scrapingdog’s YouTube Search API with Python to extract search results from YouTube and harness the potential of its data.
Why Scrape YouTube?
- Identify Potential Influencers—Scraping YouTube can help you identify upcoming influencers in a specific industry. These influencers can then be used to promote products, increase sales and conversion rates, drive engagement, and enhance the product’s trustworthiness.
- Train AI Models — The recent introduction of the capabilities of AI to the general public also came with AI video generator models that are of course trained on data collected using web scraping. The insane videos generated by these models give you an indication of how this data can be used to create such advanced models.
- SEO Optimization — Extracting titles, descriptions, and tags from Youtube can help you find effective keyword strategies to rank higher on Youtube Search results and gain a higher traction from the content consumers.
Scraping Youtube Search Results Using Scrapingdog’s Youtube API
Setting Up a Scrapingdog Account
Access to Scrapingdog API requires an API Key which you will get by registering on its platform.
You will be redirected to the Scrapingdog dashboard after successful registration, where you will get your API KEY.
Store this API Key in a safer place, as we will use it in a bit.
Also, you can easily test the API on the dashboard itself. Here is a video, showing how you can get the people also ask data in our platform only ⬇️
Let’s start Scraping
Let’s first start by setting up the environment by installing the library we will need to create an API request.
pip install requests
After installing, we will import this library into our project file.
import requests
Let’s say we want to find every video related to the term, “home workout videos” and check their rankings, title, and description to later use it for SEO.
You can refer to this documentation page, for adding more parameters to API requests.
payload = {'api_key': 'APIKEY', 'query':'home+workout+videos'}
resp = requests.get('https://api.scrapingdog.com/youtube/search', params=payload)
data = resp.json()
print(data["video_results"])
And that’s it!
Save this program and run it in your project terminal to get the list of videos ranking on this keyword.
[
{
"title": "FAST Walking in 30 minutes | Fitness Videos",
"link": "https://www.youtube.com/watch?v=enYITYwvPAQ",
"channel": {
"name": "Walk at Home",
"link": "https://www.youtube.com/channel/@LeslieSansonesWalkatHome",
"thumbmail": "https://yt3.ggpht.com/vDtT2JImQafKsR1vlxp-pJkobu6atjbCEHsezfpSbXTVMsksqKqLPVW3P6E2ncskWwFLSWHyBw=s68-c-k-c0x00ffffff-no-rj",
"verified": false
},
"published_date": "5 years ago",
"views": "151,310,059 views",
"length": "31:31",
"description": "30 minutes of fast walking MOST days of the week is super HEALTHY! A very special EDIT from the hit series Miracle Miles!",
"thumbnail": {
"static": "https://i.ytimg.com/vi/enYITYwvPAQ/hq720.jpg?sqp=-oaymwEcCOgCEMoBSFXyq4qpAw4IARUAAIhCGAFwAcABBg==&rs=AOn4CLAf1E-Vun2KI6qFx9BmAlAWf-wgGA",
"rich": "https://i.ytimg.com/an_webp/enYITYwvPAQ/mqdefault_6s.webp?du=3000&sqp=CPSqzrwG&rs=AOn4CLCYV_Cy820bpTiUYZZkVj2DAjHdBw"
},
"position": 0
},
{
"title": "25 MIN FULL BODY HIIT for Beginners - No Equipment - No Repeat Home Workout",
"link": "https://www.youtube.com/watch?v=cbKkB3POqaY",
"channel": {
"name": "growingannanas",
"link": "https://www.youtube.com/channel/@growingannanas",
"thumbmail": "https://yt3.ggpht.com/ytc/AIdro_m59N6Yys5ZGin7QpYC047fGfJY86bhbDJRhDjvrKlAVec=s68-c-k-c0x00ffffff-no-rj",
"verified": true
},
"published_date": "1 year ago",
"views": "15,391,227 views",
"length": "28:19",
"description": "Team, get ready for 25 minutes of full body HIIT! You can expect no repeat cardio & strength exercises and all you need is a mat!",
"thumbnail": {
"static": "https://i.ytimg.com/vi/cbKkB3POqaY/hq720.jpg?sqp=-oaymwEcCOgCEMoBSFXyq4qpAw4IARUAAIhCGAFwAcABBg==&rs=AOn4CLAPCs8hlUAq6ARM5digf5vuYtWiNg",
"rich": "https://i.ytimg.com/an_webp/cbKkB3POqaY/mqdefault_6s.webp?du=3000&sqp=CIStzrwG&rs=AOn4CLAc9hiwz2iz5iEi4ewnQCO93Q3DvQ"
},
"position": 1
},
......
]
We got a lot of information including title, link, channel, views and other entities by just running a small program which didn’t even require any hassle.
You may be thinking, why did we used the API instead of building our own scraper?
Obviously, you can create your own scraper and can extract data from YouTube. However, there is a limit, and your IP will soon be get blocked by YouTube for scraping their web content at scale.
That’s why we have used an API which has already setup a huge infrastructure to allow you scrape data from YouTube seamlessly without any blockages.
Conclusion
In this tutorial, we learned the to scrape Youtube Search Reuslts using Python with Scrapingdog’s API.
I hope you liked this article and if you do, please share this article on Social Media.
Thanks for reading it!👋
Additional Resources
