What is a scraper

Scrapex scraper is a module that enables you view the extracted data and also view errors in extraction if any.

Sections of the configurator:

The primary sections of the scraper module are illustrated below:

  • URL: Configurator
  • Data Section
  • Toolbar
  • Options
    • Redownload All
    • Extract All
    • Export CSV
    • Export Config

Scrapex.ai Scraper Sections

URL Section

You can click on the URL to re-open up the configurator

Data Section

Data is extracted as per your configuration and displayed in the table. Column names correspond to the configured tag names and every row contains a unique URL.

Toolbar Section

Toolbar has the following options

Filter

Filter option enables to filter by scrape status (Success, Pending Error). It also helps you find out if there are any extraction error.

Options

Redownload all

Redownloads all URLs and updates the stored website state

Extract all

Runs extraction alone without redownloading on the stored website state

Export CSV

Exports the data into a CSV

Export Config

Exports the scraper config

Add URLs

Clicking on this button opens up a popup, please drop a CSV file or enter urls seperated by newline to add more urls to the dashboard.

API

Clicking on this button, shows you the endpoint to run an on demand scrape.