Code

Website crawler

Website crawler

Our Website crawler can import websites to crawl. It will get page url, page texts, page size, page domain, initial page url, page title. There is no need to download html files and images. Our crawler stored by text.

All crawled data will be stored to output folder and saved sql file. Example: yahoo.com.sql

First time launching crawler then it is required password. But don’t worry it is already written in crypt.txt

The crawler is - Windows based crawler/scraper - Multithreaded - Optimized for performance. - Depth and scope of the crawl. - It is fast. Because written by C# - Including source file

by
by
by
by
by
by