Octoparse is a tool for web scraping that also offers proxy service and helps the user carry out their activity without having many problems. Since the tool offers premium packages and services to those that can afford it, it can be said to be excellent in carrying out its job.

Unlike most scraping software that gives you limited scraping features for free, octoparse is generous to its free users and gives you more power as you scrape amazon reviews and other data for free.

The free features of octoparse are as follows:

Unlimited pages per crawl, 10 crawlers per time, and 10,000 records for every export. The number of records is the limit of the free plan and determines if there is a need to advance from the free plan.

So depending on your project, 10,000 record entries could be enough or not even close to being enough.

Interesting Read : How to Scrape URLs with Scrapebox?

In case you were wondering, octoparse is as effective as python packages for web scraping, and could even be preferable to some. Everything in their product overview is at it is but don’t be carried away as you need to be aware of its limitations too.

Octoparse is a software program and it has a lot of task templates already set for you to begin scraping. The templates available include those to scrape amazon reviews, eBay, Rakuten, Taobao, BestBuy, JD, and a lot more.

Simply put, octoparse is an interactive GUI and also a software tool that was made to ease the process of web scraping. It doesn’t run proxies by default as proxies are not necessary when small scale scraping is to be done. So if larger tasks are needed, proxies would be required for better performance and speed.

Interesting Read : How to scrape leads through proxies?

Proxies also have to be used alongside octoparse’s workflow for the best results. Note that octoparse doesn’t replace the need for a proxy when necessary.

DEDICATED PROXIES FOR OCTOPARSE

If your proxy service provider gives you dedicated proxies but doesn’t rotate them automatically, octoparse would assist you in doing so. This is by detecting when an IP address has been exhausted and moving on to the next one.

For built-in IP rotation when using octoparse, here are some proxy services you can trust:

1. Limeproxies – they provide you with dedicated proxies and fresh IP addresses that will be undetected, with good speed for scraping.

2. Myprivateproxy – this is budget-friendly and they offer you private proxies

3. Instantproxies - this is the best choice for those on a budget as they provide cheap private IPs

4. Squidproxies – you get a money-back guarantee with this if you are not pleased by their service.

ROTATING PROXIES FOR OCTOPARSE

For web scraping or crawling purposes, the best proxies to use with octoparse are rotating backconnect proxies. Usually, you would get two types of IP rotation from the backconnect proxy provider: one that would be rotated per session, and another that would be rotated by time.

So if you want to choose a proxy provider to use with octoparse, go for one that offers rotation per request. Some of the best proxy providers in this category are as follows:

1. Smartproxy – top choice and offers rotating residential and datacenter IP proxies.

2. Storm Proxies – they are the budget-friendly choice of proxies and offer cheap rotating reverse proxies for your use with octoparse.

3. GeoSurf – these are user friendly and the best choice or new proxy users. They provide a high rotation gateway and give you good residential IP proxies that are less likely to be blocked.

Note that octoparse supports the IP as the proxy setting, and not “host:port” so check that your proxy provider uses IP setting or you would have to change it to IP:Port if Host:Port is used.

This guide is based on the tutorial from octoparse’s website

For a new task

Create a pagination loop and click on “see all reviews”

Create a loop item to enable data extraction from the selected elements

If there is a need to have a list of items, octoparse would guide you on how to do that.

Create a list of items

Select the items from the list you want and save them. While you are still on this screen, you should rename the fields according to your preference. Double click on the field name to edit.

if you followed these instructions or the one on octoparse’s homepage you would have at this point completed your web crawling. The pictures are to guide and let you know you are doing it right. The final result if all went well should look like this:

with the file export option in the octoparse menu, you can save the data in a format of your choice. So you can say that octoparse made it easy to scrape amazon reviews.