This tutorial will show you how to extend Norconex HTTP Collector using Java to create a link checker to ensure all URLs in your web pages are valid. The link checker will crawl…
Blog
Notes from the team on crawling, search, and building data pipelines that hold up in production.
HP Autonomy users, take control over your web crawling. Norconex recently released an HP Autonomy IDOL Committer module for its open-source web crawler, Norconex HTTP Collector.…
Norconex Committer and all its current concrete implementations (Solr, Elasticsearch, IDOL) have been upgraded and have seen a redesign of their web sites. Committers are…
GATINEAU, QC, CANADA – Thursday, August 25, 2014 – Norconex is announcing the launch of Norconex Filesystem Collector, providing organizations with a free “universal” filesystem…
Release 1.3 of Norconex HTTP Collector is now available. Among new features added to our open-source web crawler, you can expect the following: Now supports NTLM authentication.…
Norconex Importer 1.2.0 was just released along with a new website for it. New features: Now support text extraction from WordPerfect documents. New transformer to reduce…
