Apache Nutch™

Nutch is a highly extensible, highly scalable, matured, production-ready Web crawler which enables fine grained configuration and accomodates a wide variety of data acquisition tasks.

Download View on Github Get Started

Scalable

Relying on Apache Hadoop™ data structures, Nutch is great for batch processing large data volumes but can also be tailored to smaller jobs.

Pluggable

Out of the box Nutch offer powerful plugins i.e., parsing with Apache Tika™, indexing with Apache Solr™, Elasticsearch and more!

Extensible

Provides intuitive and stable interfaces for popular functions i.e., Parsers, HTML Filtering, Indexing and Scoring for custom implementations.