Instagram has over 2 Billion monthly active users (source).
Clearly, the number is huge and no wonders why would marketers would need this data.
The platform is great to generate potential leads, get competitive insights, see what is trending.
In this blog, we are going to scrape data from Instagram using Python. We are going to scrape details like bio, number of followers, top posts, top comments, etc., without even logging into your Instagram account.
Setting Up Environment for Scraping Instagram
Before you start make sure you have Python 3.x installed on your machine. If it is not then you can download it from here. Now, create a folder with any name you like. I am naming the folder scraper.
mkdir scraper
Now, install the requests library using pip inside scraper folder. This library will be used for making HTTP connections with the host website.
pip install requests
Let's Start Scraping
In this section, we will learn to scrape basic Instagram page details.
- Number of posts
- Number of followers
- How many pages are followed?
- Bio details
- Profile picture URL.
and many other data points.
If you inspect the network tab while loading any Instagram page then you will find a GET API.
This endpoint will generate a JSON response that contains a lot of data. We will write a Python code to extract details from this endpoint and then we will pull out all the necessary data from that JSON,
import requests
# URL for the request
url = "https://www.instagram.com/api/v1/users/web_profile_info/?username=instagram"
# Headers for the request
headers = {
"accept": "*/*",
"accept-language": "en-US,en;q=0.9",
"x-ig-app-id": "936619743392459",
}
# Make the GET request
response = requests.get(url, headers=headers)
# Print the response
if response.status_code == 200:
print("Response:", response.json())
else:
print(f"Error: {response.status_code} - {response.text}")
We are making a GET request to https://www.instagram.com/api/v1/users/web_profile_info/?username=instagram. We are passing three headers out of which one is x-ig-app-id which is an internal ID of Instagram APIs. This remains constant all the time. Once you run this code you get this.
Let’s parse the data we need from this JSON.
import requests
# URL for the request
url = "https://www.instagram.com/api/v1/users/web_profile_info/?username=instagram"
o={}
post={}
allpost=[]
insta_arr=[]
# Headers for the request
headers = {
"accept": "*/*",
"accept-language": "en-US,en;q=0.9",
"x-ig-app-id": "936619743392459",
}
# Make the GET request
response = requests.get(url, headers=headers)
# Print the response
if response.status_code == 200:
allData = response.json()['data']['user']
o['biography']=allData['biography']
o['link_in_bio']=allData['bio_links'][0]['url']
o['followers']=allData['edge_followed_by']['count']
o['following']=allData['edge_follow']['count']
o['num_posts']=allData['edge_owner_to_timeline_media']['count']
o['profile_pic_url']=allData['profile_pic_url_hd']
o['verified']=allData['is_verified']
allPosts=allData['edge_owner_to_timeline_media']['edges']
for i in range(0,len(allPosts)):
if(allPosts[i]['node']['is_video']==True):
post['display_url']=allPosts[i]['node']['display_url']
post['video_view_count']=allPosts[i]['node']['video_view_count']
post['video_url']=allPosts[i]['node']['video_url']
post['num_comments']=allPosts[i]['node']['edge_media_to_comment']['count']
post['num_likes']=allPosts[i]['node']['edge_liked_by']['count']
else:
post['display_url']=allPosts[i]['node']['display_url']
post['num_comments']=allPosts[i]['node']['edge_media_to_comment']['count']
post['num_likes']=allPosts[i]['node']['edge_liked_by']['count']
allpost.append(post)
post={}
insta_arr.append(o)
insta_arr.append(allpost)
print(insta_arr)
else:
print(f"Error: {response.status_code} - {response.text}")
The response is quite big but with the help of parsing, we have extracted other details like the display URL of the first 12 posts, the number of likes on them, the number of comments on them, their display URL, reel URL, etc.
Once you run this code you get data like this.
[{'biography': "Discover what's new on Instagram 🔎✨", 'link_in_bio': 'https://about.fb.com/news/2024/09/instagram-teen-accounts/', 'followers': 681289082, 'following': 130, 'num_posts': 7885, 'profile_pic_url': 'https://instagram.fjai2-5.fna.fbcdn.net/v/t51.2885-19/281440578_1088265838702675_6233856337905829714_n.jpg?stp=dst-jpg_s320x320_tt6&_nc_ht=instagram.fjai2-5.fna.fbcdn.net&_nc_cat=1&_nc_ohc=0pfpBOyFhoEQ7kNvgHl7i1V&_nc_gid=78c7d65af5ee48de8a589402203d493e&edm=AOQ1c0wBAAAA&ccb=7-5&oh=00_AYANzVn2JCuvHIMJv2LZtZDlZyl6Ifjl4brNix2SKkqW-A&oe=674DF958&_nc_sid=8b3546', 'verified': True}, [{'display_url': 'https://instagram.fjai2-5.fna.fbcdn.net/v/t51.2885-15/468701809_18610788898001321_3539461778809962519_n.jpg?stp=dst-jpg_e35_p1080x1080_sh0.08&_nc_ht=instagram.fjai2-5.fna.fbcdn.net&_nc_cat=1&_nc_ohc=1dnLfpYS4iQQ7kNvgGo8Ik-&_nc_gid=78c7d65af5ee48de8a589402203d493e&edm=AOQ1c0wBAAAA&ccb=7-5&oh=00_AYDxP5KFkFgnLdRWhRrAPw05f8OaOFQypmjtoSiEWsqfYg&oe=674DF7DF&_nc_sid=8b3546', 'video_view_count': 1602346, 'video_url': 'https://instagram.fjai2-5.fna.fbcdn.net/o1/v/t16/f1/m86/3D4B7AB942A0ECE4FE6A6536276D87A0_video_dashinit.mp4?stp=dst-mp4&efg=eyJxZV9ncm91cHMiOiJbXCJpZ193ZWJfZGVsaXZlcnlfdnRzX290ZlwiXSIsInZlbmNvZGVfdGFnIjoidnRzX3ZvZF91cmxnZW4uY2xpcHMuYzIuNzIwLmJhc2VsaW5lIn0&_nc_cat=1&vs=1293056231601324_1475431623&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC8zRDRCN0FCOTQyQTBFQ0U0RkU2QTY1MzYyNzZEODdBMF92aWRlb19kYXNoaW5pdC5tcDQVAALIAQAVAhg6cGFzc3Rocm91Z2hfZXZlcnN0b3JlL0dQWlg2eHVyYzhIdjFLd0JBUGdkN2tBSGpiWlJicV9FQUFBRhUCAsgBACgAGAAbABUAACaMmNuArKaaQRUCKAJDMywXQFaAYk3S8aoYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HAA%3D%3D&ccb=9-4&oh=00_AYCr9iWgiKMhImniOqOqeHm84CMQAReYxuLWhT7ZU71y2g&oe=674A264E&_nc_sid=8b3546', 'num_comments': 2196, 'num_likes': 59036}, {'display_url': 'https://instagram.fjai2-5.fna.fbcdn.net/v/t51.2885-15/468522558_18610740931001321_7321842896945779589_n.jpg?stp=dst-jpg_e15_tt6&_nc_ht=instagram.fjai2-5.fna.fbcdn.net&_nc_cat=1&_nc_ohc=vHy6zbSDUz8Q7kNvgGVeY3W&_nc_gid=78c7d65af5ee48de8a589402203d493e&edm=AOQ1c0wBAAAA&ccb=7-5&oh=00_AYAaQyYEMZGIXNWY3OiL5bgUyQKdDBlNnfq0MIiq0WrbKA&oe=674DF72F&_nc_sid=8b3546', 'video_view_count': 330050, 'video_url': 'https://instagram.fjai2-6.fna.fbcdn.net/o1/v/t16/f1/m86/DB4DC796F36F4BD0A513826E2D00C39D_video_dashinit.mp4?stp=dst-mp4&efg=eyJxZV9ncm91cHMiOiJbXCJpZ193ZWJfZGVsaXZlcnlfdnRzX290ZlwiXSIsInZlbmNvZGVfdGFnIjoidnRzX3ZvZF91cmxnZW4uY2xpcHMuYzIuNzIwLmJhc2VsaW5lIn0&_nc_cat=103&vs=552574571001318_670040905&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC9EQjREQzc5NkYzNkY0QkQwQTUxMzgyNkUyRDAwQzM5RF92aWRlb19kYXNoaW5pdC5tcDQVAALIAQAVAhg6cGFzc3Rocm91Z2hfZXZlcnN0b3JlL0dHckg3QnN2TVpac0kyOEVBSjNBeW5LQmxTZ2dicV9FQUFBRhUCAsgBACgAGAAbABUAACaEgpKm5siMQBUCKAJDMywXQEPAAAAAAAAYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HAA%3D%3D&ccb=9-4&oh=00_AYBluOpMpNi6vTjFvGltMha8V8Ua1K8HxMWXdTaih2YgpA&oe=674A1CFB&_nc_sid=8b3546', 'num_comments': 806, 'num_likes': 24026}, {'display_url': 'https://instagram.fjai2-5.fna.fbcdn.net/v/t51.2885-15/468616695_18610611394001321_1247804736818079431_n.jpg?stp=dst-jpg_e15_fr_p1080x1080&_nc_ht=instagram.fjai2-5.fna.fbcdn.net&_nc_cat=1&_nc_ohc=kZYODaYR4dAQ7kNvgGesAsJ&_nc_gid=78c7d65af5ee48de8a589402203d493e&edm=AOQ1c0wBAAAA&ccb=7-5&oh=00_AYCGiu9Tro1slT_oenPtZPu09pQaNM_Vg_CHVvlEs8jMNQ&oe=674E05C0&_nc_sid=8b3546', 'video_view_count': 2618077, 'video_url': 'https://instagram.fjai2-5.fna.fbcdn.net/o1/v/t16/f1/m86/5A4C97D3073E436536B77A653FB43293_video_dashinit.mp4?stp=dst-mp4&efg=eyJxZV9ncm91cHMiOiJbXCJpZ193ZWJfZGVsaXZlcnlfdnRzX290ZlwiXSIsInZlbmNvZGVfdGFnIjoidnRzX3ZvZF91cmxnZW4uY2xpcHMuYzIuNzIwLmJhc2VsaW5lIn0&_nc_cat=1&vs=1079020997239614_2839075097&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC81QTRDOTdEMzA3M0U0MzY1MzZCNzdBNjUzRkI0MzI5M192aWRlb19kYXNoaW5pdC5tcDQVAALIAQAVAhg6cGFzc3Rocm91Z2hfZXZlcnN0b3JlL0dFY2E3UnM0MS1Rd2g4NEZBSWNrNTNYWVhPUXZicV9FQUFBRhUCAsgBACgAGAAbABUAACbS%2Fqf7nNDTPxUCKAJDMywXQC9crAgxJukYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HAA%3D%3D&ccb=9-4&oh=00_AYBMiKWAZQic-HpLq3svXC2QSPIhzaVuzp_tmchMwxpLVw&oe=674A25D8&_nc_sid=8b3546', 'num_comments': 1827, 'num_likes': 60986}, {'display_url': 'https://instagram.fjai2-5.fna.fbcdn.net/v/t51.2885-15/468274355_18610447756001321_3305237204360897748_n.jpg?stp=dst-jpg_e15&_nc_ht=instagram.fjai2-5.fna.fbcdn.net&_nc_cat=1&_nc_ohc=IeIx9cSrOCoQ7kNvgGPzZcZ&_nc_gid=78c7d65af5ee48de8a589402203d493e&edm=AOQ1c0wBAAAA&ccb=7-5&oh=00_AYDIjFTHXEa4quvF_Cod5A007RkY5XSkV9SgpnlsGiUP-Q&oe=674DFEEE&_nc_sid=8b3546', 'video_view_count': 2239778, 'video_url': 'https://instagram.fjai2-5.fna.fbcdn.net/o1/v/t16/f1/m86/5C476954A7880F8C8EA925B78D52C38D_video_dashinit.mp4?stp=dst-mp4&efg=eyJxZV9ncm91cHMiOiJbXCJpZ193ZWJfZGVsaXZlcnlfdnRzX290ZlwiXSIsInZlbmNvZGVfdGFnIjoidnRzX3ZvZF91cmxnZW4uY2xpcHMuYzIuNzIwLmJhc2VsaW5lIn0&_nc_cat=105&vs=495088089537103_1085294210&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC81QzQ3Njk1NEE3ODgwRjhDOEVBOTI1Qjc4RDUyQzM4RF92aWRlb19kYXNoaW5pdC5tcDQVAALIAQAVAhg6cGFzc3Rocm91Z2hfZXZlcnN0b3JlL0dKTTE2QnUwSUVLZ2Rfa0JBSFhFYjBTR3RBSmdicV9FQUFBRhUCAsgBACgAGAAbABUAACbOvKbhlsiGQBUCKAJDMywXQA2ZmZmZmZoYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HAA%3D%3D&ccb=9-4&oh=00_AYBYJo1A0xlBz0XTfGLG5IH9uwFiKYRb77rOI0i5iUZ4og&oe=674A16ED&_nc_sid=8b3546', 'num_comments': 2176, 'num_likes': 101902}, {'display_url': 'https://instagram.fjai2-5.fna.fbcdn.net/v/t51.2885-15/468071289_18609987304001321_3331816174249952448_n.jpg?stp=dst-jpg_e35_p1080x1080_sh0.08_tt6&_nc_ht=instagram.fjai2-5.fna.fbcdn.net&_nc_cat=1&_nc_ohc=vxrRJlPrByQQ7kNvgFd2PKf&_nc_gid=78c7d65af5ee48de8a589402203d493e&edm=AOQ1c0wBAAAA&ccb=7-5&oh=00_AYDEaqm0zkC4GR0weOpLiP6cRaIUWWEnJMjZDJvbqwwPPA&oe=674E25EA&_nc_sid=8b3546', 'video_view_count': 566877, 'video_url': 'https://instagram.fjai2-6.fna.fbcdn.net/o1/v/t16/f1/m86/1E4FB22C03B655CA1455FBBB86B1BBAE_video_dashinit.mp4?stp=dst-mp4&efg=eyJxZV9ncm91cHMiOiJbXCJpZ193ZWJfZGVsaXZlcnlfdnRzX290ZlwiXSIsInZlbmNvZGVfdGFnIjoidnRzX3ZvZF91cmxnZW4uY2xpcHMuYzIuNzIwLmJhc2VsaW5lIn0&_nc_cat=107&vs=1055017179735209_3595928036&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC8xRTRGQjIyQzAzQjY1NUNBMTQ1NUZCQkI4NkIxQkJBRV92aWRlb19kYXNoaW5pdC5tcDQVAALIAQAVAhg6cGFzc3Rocm91Z2hfZXZlcnN0b3JlL0dJVlE0QnU2Vjg2WXZna0NBQ1hhVmFwRWZJUjNicV9FQUFBRhUCAsgBACgAGAAbABUAACaU6J%2BHi9ipQBUCKAJDMywXQDeqfvnbItEYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HAA%3D%3D&ccb=9-4&oh=00_AYBmcRpc1_tTc1C3q4vkE2F4myGKHCri_BmCUl7zqHWu8A&oe=674A22F3&_nc_sid=8b3546', 'num_comments': 977, 'num_likes': 25487}, {'display_url': 'https://instagram.fjai2-5.fna.fbcdn.net/v/t51.2885-15/468019396_18609846838001321_5343207079704224996_n.jpg?stp=dst-jpg_e15_tt6&_nc_ht=instagram.fjai2-5.fna.fbcdn.net&_nc_cat=1&_nc_ohc=WYH69lFEL2cQ7kNvgE9QynE&_nc_gid=78c7d65af5ee48de8a589402203d493e&edm=AOQ1c0wBAAAA&ccb=7-5&oh=00_AYB1YUDypKM8eJ3nDekT6Al7eYHJx60eDrXRvtlnPENrKA&oe=674E2B69&_nc_sid=8b3546', 'video_view_count': 4626383, 'video_url': 'https://instagram.fjai2-2.fna.fbcdn.net/o1/v/t16/f1/m86/2749B6F54BB12CE42C28C3C93B3C0EBE_video_dashinit.mp4?stp=dst-mp4&efg=eyJxZV9ncm91cHMiOiJbXCJpZ193ZWJfZGVsaXZlcnlfdnRzX290ZlwiXSIsInZlbmNvZGVfdGFnIjoidnRzX3ZvZF91cmxnZW4uY2xpcHMuYzIuNzIwLmJhc2VsaW5lIn0&_nc_cat=108&vs=495082396885160_3050337303&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC8yNzQ5QjZGNTRCQjEyQ0U0MkMyOEMzQzkzQjNDMEVCRV92aWRlb19kYXNoaW5pdC5tcDQVAALIAQAVAhg6cGFzc3Rocm91Z2hfZXZlcnN0b3JlL0dKV3Y0QnU2djhqMlNtSUdBR1lzc25WaDhZQWticV9FQUFBRhUCAsgBACgAGAAbABUAACa447mqgY2EQBUCKAJDMywXQC%2B7ZFocrAgYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HAA%3D%3D&ccb=9-4&oh=00_AYDKEAHVNZX7SyL92gOg_FiRPtbzECKv2wgnLGuWGp6XvA&oe=674A013A&_nc_sid=8b3546', 'num_comments': 4520, 'num_likes': 351955}, {'display_url': 'https://instagram.fjai2-5.fna.fbcdn.net/v/t51.2885-15/467749118_18609530869001321_9163764497276818023_n.jpg?stp=dst-jpg_e15&_nc_ht=instagram.fjai2-5.fna.fbcdn.net&_nc_cat=1&_nc_ohc=RLUpCtzw6SMQ7kNvgEJXNFP&_nc_gid=78c7d65af5ee48de8a589402203d493e&edm=AOQ1c0wBAAAA&ccb=7-5&oh=00_AYBbkovHxmZ2pH9sYx5TC37pHIIu28DeCnfh4KEkC7tgoA&oe=674E1817&_nc_sid=8b3546', 'num_comments': 3250, 'num_likes': 135852}, {'display_url': 'https://instagram.fjai2-5.fna.fbcdn.net/v/t51.2885-15/467768464_18609478921001321_7892186948542213391_n.jpg?stp=dst-jpg_e35_p1080x1080_sh0.08_tt6&_nc_ht=instagram.fjai2-5.fna.fbcdn.net&_nc_cat=1&_nc_ohc=vANgbpgtSEoQ7kNvgGDs-XM&_nc_gid=78c7d65af5ee48de8a589402203d493e&edm=AOQ1c0wBAAAA&ccb=7-5&oh=00_AYAY6DzCSw__I3UMc0UFt0pHL_5NLeuAG0N3hreKrneClA&oe=674E1F64&_nc_sid=8b3546', 'video_view_count': 1139647, 'video_url': 'https://instagram.fjai2-2.fna.fbcdn.net/o1/v/t16/f1/m86/904F6DED25829D582C407E0DD5CB4D9E_video_dashinit.mp4?stp=dst-mp4&efg=eyJxZV9ncm91cHMiOiJbXCJpZ193ZWJfZGVsaXZlcnlfdnRzX290ZlwiXSIsInZlbmNvZGVfdGFnIjoidnRzX3ZvZF91cmxnZW4uY2xpcHMuYzIuNzIwLmJhc2VsaW5lIn0&_nc_cat=108&vs=785859783664607_2002347264&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC85MDRGNkRFRDI1ODI5RDU4MkM0MDdFMERENUNCNEQ5RV92aWRlb19kYXNoaW5pdC5tcDQVAALIAQAVAhg6cGFzc3Rocm91Z2hfZXZlcnN0b3JlL0dQV2wyUnZXdHJoR2JKQUZBQXRIOThhYkZndDZicV9FQUFBRhUCAsgBACgAGAAbABUAACaWk%2ByqnpSZQBUCKAJDMywXQEFMzMzMzM0YEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HAA%3D%3D&ccb=9-4&oh=00_AYDW6E8VQXIhUMilKWbVlL2eQqV0yLA6-iIC93AY7udAAg&oe=674A290A&_nc_sid=8b3546', 'num_comments': 2496, 'num_likes': 49486}, {'display_url': 'https://instagram.fjai2-5.fna.fbcdn.net/v/t51.2885-15/467243126_18609418324001321_935926265623073471_n.jpg?stp=dst-jpg_e15&_nc_ht=instagram.fjai2-5.fna.fbcdn.net&_nc_cat=1&_nc_ohc=dnr_oc8AUw0Q7kNvgG6X2JC&_nc_gid=78c7d65af5ee48de8a589402203d493e&edm=AOQ1c0wBAAAA&ccb=7-5&oh=00_AYDEF-EaRIIP1zJWOwtBwihOLNy4Gcag6qv7oDVGg_joYA&oe=674E07AE&_nc_sid=8b3546', 'video_view_count': 2464476, 'video_url': 'https://instagram.fjai2-6.fna.fbcdn.net/o1/v/t16/f1/m86/9B49187DAD6EA29F3C846F07126FB4BD_video_dashinit.mp4?stp=dst-mp4&efg=eyJxZV9ncm91cHMiOiJbXCJpZ193ZWJfZGVsaXZlcnlfdnRzX290ZlwiXSIsInZlbmNvZGVfdGFnIjoidnRzX3ZvZF91cmxnZW4uY2xpcHMuYzIuNzIwLmJhc2VsaW5lIn0&_nc_cat=107&vs=1708718609978738_2571349805&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC85QjQ5MTg3REFENkVBMjlGM0M4NDZGMDcxMjZGQjRCRF92aWRlb19kYXNoaW5pdC5tcDQVAALIAQAVAhg6cGFzc3Rocm91Z2hfZXZlcnN0b3JlL0dPUjYyeHUwSlNRTUhNd0RBS19lUTlKZkxZbElicV9FQUFBRhUCAsgBACgAGAAbABUAACagsPGbue%2FRQRUCKAJDMywXQCwhysCDEm8YEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HAA%3D%3D&ccb=9-4&oh=00_AYC3iemJQdhKdvNt-FronpXYfRs-McR8Csv0d_0AmZJeCA&oe=674A2CF3&_nc_sid=8b3546', 'num_comments': 2178, 'num_likes': 166627}, {'display_url': 'https://instagram.fjai2-5.fna.fbcdn.net/v/t51.2885-15/467554495_18609346108001321_8446496511191069471_n.jpg?stp=dst-jpg_e15_tt6&_nc_ht=instagram.fjai2-5.fna.fbcdn.net&_nc_cat=1&_nc_ohc=njKYIMHWW6UQ7kNvgHX51tk&_nc_gid=78c7d65af5ee48de8a589402203d493e&edm=AOQ1c0wBAAAA&ccb=7-5&oh=00_AYBkM79h1LU9w6T_9ZluK3QkahmZJ6WDMvfWGDZ1hSkcig&oe=674E11EB&_nc_sid=8b3546', 'video_view_count': 1636652, 'video_url': 'https://instagram.fjai2-4.fna.fbcdn.net/o1/v/t16/f1/m86/FA48EBB91527B32B8F01501509F7EC91_video_dashinit.mp4?stp=dst-mp4&efg=eyJxZV9ncm91cHMiOiJbXCJpZ193ZWJfZGVsaXZlcnlfdnRzX290ZlwiXSIsInZlbmNvZGVfdGFnIjoidnRzX3ZvZF91cmxnZW4uY2xpcHMuYzIuNzIwLmJhc2VsaW5lIn0&_nc_cat=109&vs=1295481824781381_637289147&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC9GQTQ4RUJCOTE1MjdCMzJCOEYwMTUwMTUwOUY3RUM5MV92aWRlb19kYXNoaW5pdC5tcDQVAALIAQAVAhg6cGFzc3Rocm91Z2hfZXZlcnN0b3JlL0dBUEQ0eHRiTGR5MUVQSUJBQUcyY0FyX3BsNXJicV9FQUFBRhUCAsgBACgAGAAbABUAACaOv%2BXvv%2B%2FkPxUCKAJDMywXQE6RBiTdLxsYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HAA%3D%3D&ccb=9-4&oh=00_AYAplchbXyZ3bjRehBMWRn6QyZer3K2wGvooFVAOjLqFqQ&oe=674A2709&_nc_sid=8b3546', 'num_comments': 2566, 'num_likes': 61865}, {'display_url': 'https://instagram.fjai2-5.fna.fbcdn.net/v/t51.2885-15/467763988_18609277435001321_8167609778074325034_n.jpg?stp=dst-jpg_e15_tt6&_nc_ht=instagram.fjai2-5.fna.fbcdn.net&_nc_cat=1&_nc_ohc=nEWz9VpTmJEQ7kNvgEBZ5g-&_nc_gid=78c7d65af5ee48de8a589402203d493e&edm=AOQ1c0wBAAAA&ccb=7-5&oh=00_AYA1Zf1KvP8OwkSpitNtN7A831oacNkAVQUFh9j2lH7Zig&oe=674E066D&_nc_sid=8b3546', 'video_view_count': 3693031, 'video_url': 'https://instagram.fjai2-1.fna.fbcdn.net/o1/v/t16/f1/m86/34409115CA0FD8B3F82A7141F7896590_video_dashinit.mp4?stp=dst-mp4&efg=eyJxZV9ncm91cHMiOiJbXCJpZ193ZWJfZGVsaXZlcnlfdnRzX290ZlwiXSIsInZlbmNvZGVfdGFnIjoidnRzX3ZvZF91cmxnZW4uY2xpcHMuYzIuNzIwLmJhc2VsaW5lIn0&_nc_cat=102&vs=1212613399814193_3303072675&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC8zNDQwOTExNUNBMEZEOEIzRjgyQTcxNDFGNzg5NjU5MF92aWRlb19kYXNoaW5pdC5tcDQVAALIAQAVAhg6cGFzc3Rocm91Z2hfZXZlcnN0b3JlL0dDWngyUnRRdjJsaG9lUURBTG9MWGE2anNya3picV9FQUFBRhUCAsgBACgAGAAbABUAACaezfX%2F6e3FPxUCKAJDMywXQDAIcrAgxJwYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HAA%3D%3D&ccb=9-4&oh=00_AYDX3I7X5zCJ4Y3ehmr4LgnbV2ao48mOs9GfAMg63DjehA&oe=674A1C04&_nc_sid=8b3546', 'num_comments': 2680, 'num_likes': 159823}, {'display_url': 'https://instagram.fjai2-5.fna.fbcdn.net/v/t51.2885-15/467532948_18609103744001321_4066765072496668197_n.jpg?stp=dst-jpg_e15_fr_p1080x1080_tt6&_nc_ht=instagram.fjai2-5.fna.fbcdn.net&_nc_cat=1&_nc_ohc=2ODZVySFvMgQ7kNvgFMpgDh&_nc_gid=78c7d65af5ee48de8a589402203d493e&edm=AOQ1c0wBAAAA&ccb=7-5&oh=00_AYAAzJYJOqrKNL-DzihkZwW8oQiNVbP7jGiYfa9mNs_FkA&oe=674E224D&_nc_sid=8b3546', 'video_view_count': 3097169, 'video_url': 'https://instagram.fjai2-4.fna.fbcdn.net/o1/v/t16/f1/m86/EE42BC08583F1BE1789B48947915C384_video_dashinit.mp4?stp=dst-mp4&efg=eyJxZV9ncm91cHMiOiJbXCJpZ193ZWJfZGVsaXZlcnlfdnRzX290ZlwiXSIsInZlbmNvZGVfdGFnIjoidnRzX3ZvZF91cmxnZW4uY2xpcHMuYzIuNzIwLmJhc2VsaW5lIn0&_nc_cat=106&vs=2832430600262052_2952551441&_nc_vs=HBksFQIYUmlnX3hwdl9yZWVsc19wZXJtYW5lbnRfc3JfcHJvZC9FRTQyQkMwODU4M0YxQkUxNzg5QjQ4OTQ3OTE1QzM4NF92aWRlb19kYXNoaW5pdC5tcDQVAALIAQAVAhg6cGFzc3Rocm91Z2hfZXZlcnN0b3JlL0dEb0o0QnRHbDhlVHllNEJBR0FJWkk0WEczOGlicV9FQUFBRhUCAsgBACgAGAAbABUAACaEjvDwwuGEQBUCKAJDMywXQCsiTdLxqfwYEmRhc2hfYmFzZWxpbmVfMV92MREAdf4HAA%3D%3D&ccb=9-4&oh=00_AYAokQmktQ4Why8UFbUUP6pdo5QKp_wma4GOQKexY7kmfw&oe=674A0A58&_nc_sid=8b3546', 'num_comments': 4011, 'num_likes': 141269}]]
The response is quite big but with the help of parsing, we have extracted other details like the display URL of the first 12 posts, the number of likes on them, the number of comments on them, their display URL, reel URL, etc.
Once you run this code you get data like this.
To avoid such blockage you should prefer a web scraping API like Scrapingdog which can help you scrape Instagram at scale without getting blocked.
Scraping Instagram with Scrapingdog
To start with Scrapingdog, you must first sign up for the free trial. The free trial will provide you with 1000 free credits which are enough for testing the API.
Once you sign up you will be redirected to your personal dashboard where you will find an API key.
Here is how you can scrape Instagram using Scrapingdog with simple Python code. Use your own API key while using the below code.
import requests
api_key = "Paste-your-API-key"
url = "https://api.scrapingdog.com/instagram"
params = {
"api_key": api_key,
"username": "instagram"
}
response = requests.get(url, params=params)
if response.status_code == 200:
data = response.json()
print(data)
else:
print(f"Request failed with status code: {response.status_code}")
Once you run this code you will get already parsed JSON data from Instagram.
You can leave the complexities of proxies and headers to Scrapingdog. Focus on collecting valuable data while Scrapingdog handles all the details of data extraction.
Conclusion
In this article we have looked how you can easily scrape Instagram and to do it in scale you can use Scrapingdog’s API.
Scraping social media has become an essential today to get insights on the businesses today. And Instagram is one such valuable source. If you liked the article do share it!!
