Many of the websites are limited to particular countries as per their business. So their parent company decides to disable the worldwide IP on their server to save bandwidth and cost.
So, if you happen to scrape a website which works in a particular country. For example example.com works in the UK only.
The Agenty’s geo-based web scraping tool is designed to allow you to scrape data based on a given geographical location, country, city which routes the requests from selected locations and set the latitude and longitude, GPS values automatically in the browser.
For example, we can specify the UK location in web scraping agent to use the UK origin IP addresses only for scraping data from that website to avoid server errors, as by default Agenty rotate the IP address from thousands of proxies across the world to avoid blocking.
- Go to your scraping agent page
- Click on Configuration tab
- Click on proxy tab
- Select country and save
This will change the scraper IP address dynamically from thousands of IP ranges available for each country to make your scraper looks like coming from selected country.
Sometimes it’s also important to specify the
language headers to scrape a website hosted on another country server to extract data on the dynamic language served based on the browser headers.
For example, if you open a french website on your browser it may show content in English language based on Accept-Language header set in your browser.
But, when you specify the country in the scraping agent as
France, it may scrape the content in french language assuming you are visiting from France.
For such websites which use the
Accept-Language header to dynamically display their webpages content in different languages, we can pass a custom header in the scraping agent to specify our choice of language to extract data as per our geo-location and header combined to make it more advance scraper for location based web scraping .