How to scrape all product color variants?

I am scraping a Shopify product website for a brand. That brand has multiple product color variants and when I scrape the product details page, only the default color details is scraped.

How can I scrape all the color variants for each product in the next row? Also the other product details should be repeated in the next rows in CSV.

Yes, you can scrape the color variants and then repeat the other values in next rows following these steps -

For example, I will use this example HTML page to showcase which has 3 colors variants as an option to select while making a purchase - product card shop -

Step 1 : Find the selector for colors, and add a field to scrape all colors first. In this case the selector is .swatches .swatch:nth-of-type(2) input and then I used the attribute option to extract the value attribute from this input html element.

Step 2 : : Add other fields and save your scraping agent and it will looks like this, by default -

Because 2 fields “product name” and “price” have 1 match, while the colors have 3 matches. So it’s taking 3 rows by default. You can always use the string join function to join more than one result together into one cell with comma separated(or some other delimiter).

But since you want to have it in the same format and name, price should be repeated to make your file importable in Shopify to create color variants.

So, we are going to use the AutoFill Blank Cells post-processing function on fields which should be repeated.

Step 3 : Add the auto fill blank cells function on fields which should be repeated.

Run it again after the changes in configuration and you’ll get the format you are looking for -

14 days free trial

Automate your business with advanced, fully-featured agents on Agenty. Fast, scalable and no-code web automation tool for scraping, change monitoring and more...

Sign up for free →

No credit card required