Login Register


Help! an extension that automatically switch the proxy when I scrape a website filter_list
Author
Message
Help! an extension that automatically switch the proxy when I scrape a website #1
Hi dear Sinsiters,

I am newbie in the domain and I need the expert members' help

I'm scraping data from a website, that later hide its data from non US members.

Since I'm not yet a professional, I use only an extension on my browser to have a US Ip.

The issue is that though I use the US-ip, the website hide the content from many links I visit while scraping.

The solution I can think of it is that I have to install an extension on my browser so that any time my scraping tool visit an URL from the website, the extension change the Ip from an US proxy list.

My question is do you have any suggestion for such extension, or do you have better ideas ?

Thank you in advance for your consideration and your help, I appreciate a lot

Reply

RE: Help! an extension that automatically switch the proxy when I scrape a website #2
Quote:I use only an extension on my browser to have a US Ip.
Have you tried using a VPN, and changing US servers In different localities?
[Image: AD83g1A.png]

Reply

RE: Help! an extension that automatically switch the proxy when I scrape a website #3
The Vpn that I a have already chose allow me to visit all the URL list with one US-ip, and the website is able to hide the data when it detect the scraper bot...

I cannot find an Vpn extension that can change the ip which each URL of the website
(This post was last modified: 03-19-2022, 10:34 AM by TropMuch.)

Reply







Users browsing this thread: 5 Guest(s)