In this tutorial, I will explain how you can use Scrapingdog’s LinkedIn Scraping API to scrape any Person’s Public Profile and Company’s Public Profile.
How To Scrape LinkedIn Public Profile via Scrapingdog
There are two kinds of public profiles on Linkedin when it comes to scraping them.
- Profile with a captcha
- A profile without a captcha
Let’s first discuss how you can scrape a profile without a captcha on it.
Let’s say I want to scrape this profile. Then simply you will try to scrape it like this.
o["rating"]=soup.find("span",{"class":"ebay-review-start-rating"}).text.strip(
This is the most basic way in which you can scrape any Linkedin Profile with our API. But sometimes some profiles throw a captcha like this.
In this case, our API will respond with either a 404 or 400. In this case, you can take advantage of the private parameter.
Try to open this profile in incognito mode. You will face a captcha similar to that shown in the above image. Scrapingdog’s API can scrape these profiles as well. Just add a private parameter and you are done.
https://api.scrapingdog.com/linkedin/?api_key=Your-API-Key&type=profile&linkId=bradshapcott&private=true
When you make the above API call then you will get one of the two status codes.
200202
When it is 200 then it means the data has been successfully scraped and returned to you.
But if it is 202 then it means your request is accepted but the profile is still being scraped. Once the profile is scraped the copy of that profile will be saved in our DB. Of course, you will be not charged when you get a 202. Now anytime after 2 to 3 minutes, you can scrape this profile and you will get a 200 in that case.
When you get a 202 you are advised to scrape other profiles in your list rather than waiting for 2 to 3 minutes. This will increase your work productivity.
Let us know if you have any questions regarding the API. You can contact us here. You can watch this quick tutorial in the video to further understand how this API works.
Web Scraping with Scrapingdog
