The scraping agent allows you to capture the webpage screenshot automatically while crawling a website, as it is very common in web scraping to capture a screenshot of a website for testing or as a proof of data being scraped is exactly same as it’s shown on the website.
There are 3 types of screenshot option available in Agenty
- Small screenshot : Visible portion on screen or viewport set
- Full screenshot: Full web page
- HD screenshot: High definition screenshot of full web page
Step by step guide: Let’s make them easy to understand how to capture website screenshot
To capture the Visible page screenshot only, follow these steps:
- Go to your agent page
- Click on the Configuration tab and scroll down to Fields section
- Add a new field with configuration like this
- Field Name : Any name of your choice
- Field Type
- From :
- Save and re-run your agent
- The scraping agent will capture the visible page screen.
This option allow you to capture the above the fold or visible part of website on computer screen without scrolling. The webpage will be captured as per the viewport width and height given in the web scraping engine. So, if you are looking to capture the visible page screenshot with web scraping, use this option. Here is an example :-
Full page screenshot
The full page screenshot option allow us to capture the full page screen, which make the task easier to diagnose to view full content of website.
To capture the full portion of a webpage - We need to follow the same steps as followed above, except change the from type to
- Agent page > Configuration Tab
- Add new Field
- Select Default as Type
- Select full screenshot option in system fields
- Save this and Click on Run button
Now, when the scraping job is executed, Agenty will capture the full page screenshot.
Here is an example of full page screenshot captured in this agent with viewport : full page screenshot for web scraping
The High Definition screenshot produce larger, wider and clearer picture of a website with increased pixels and wider scaled viewport. So, the HD screenshot option allows you to get the better experience to diagnose the web page during web scraping.
To capture the HD screenshot of web page, follow these steps:
- Agent Page > Configuration tab
- Add a default field
hd_screenshotoption from system field
- Save the configuration and Run it
Now you can see the high definition quality screenshot will be captured by Agenty automated scraping agent.