Code

Scavenger - Laravel Web Scraper Package

Scavenger - Laravel Web Scraper Package

A highly flexible and efficient Laravel 5.x scraper package. Easily scrape data from anywhere on the web and build content.

Top Features

Scavenger provides the following features and more out of the box.

  • Ease of use
    • Scavenger is super-easy to configure. Simple publish the config file and set your targets.
  • Scrape data from multiple sources at once.
  • Convert scraped data into useable Laravel model objects.
    • eg. You may scrape an article and have it converted into an object of your choice and saved in your database. Immediately available to your viewers.
  • You can easily perform one or more operations to each property of any scraped entity.
    • eg. You may call a paraphrase service from a model or package of your choice on data attributes before saving them to your database.
  • Data integrity constraints
    • Scavanger uses a hashing algorithm of your choice to maintain data integrity. This hash is used to ensure that one scrap (source article) is not converted to multiple output objects (model duplicates).
  • Console Command
    • Once scavenger is configured, a simple artisan command launches the seeker. Since this is a console command it is more efficient and timeouts are less likely to occur.
    • Artisan command: php artisan scavenger:seek
  • Schedule ready
    • Scavenger can easily be set to scrape on a schedule. Hence, creating a someone autonomous website is super easy!
  • SERP
    • Scavenger can be used to flexibly scrape Search Engine Result Pages.

Documentation

Full documentation can be found at scavenger.reliqarts.com.

Version Log

v2.0.0 (Feb. 21, 2018)
    - Introduces SERP support.
v1.0.6 (Jan. 13, 2018)
    - Stability update.
v1.0.5 (Sept. 2, 2017)
    - Doc update.
v1.0.4 (Aug. 23, 2017)
    - Stability update.
v1.0.3 (Aug. 22, 2017)
    - Initial release.
by
by
by
by
by
by