A crawler framework. It covers the whole lifecycle of crawler: downloading, url management, content extraction and persistent. It can simply the development of a specific crawler.

Compile Dependencies (0)

Category/LicenseGroup / ArtifactVersionUpdates

Managed Dependencies (26)

Category/LicenseGroup / ArtifactVersionUpdates
JSON Lib Apache 2.0
com.alibaba » fastjson2 vulnerabilities
Fastjson is a JSON processor (JSON parser + JSON generator) written in Java
1.2.682.0.64
Browser Automation BSD 2-clause
com.github.detro » phantomjsdriver
PhantomJSDriver is a Java binding for the PhantomJS WebDriver, GhostDriver.
1.2.01.5.0
MIT
com.github.dreamhead » moco-core
Moco is an easy setup stub framework, mainly focusing on testing and integration.
1.1.01.6.1
Core Utils Apache 2.0
com.google.guava » guava2 vulnerabilities
Guava is a suite of core and expanded libraries that include
29.0-jre33.7.1-jre
JSON Lib Apache 2.0
com.jayway.jsonpath » json-path1 vulnerability
A library to query and verify JSON
2.4.03.0.0
CLI Parser Apache 2.0
commons-cli » commons-cli
Apache Commons CLI provides a simple API for presenting, processing, and validating a Command Line Interface.
1.41.11.0
Collections Apache 2.0
commons-collections » commons-collections
The Apache Commons Collections package contains types that extend and augment the Java Collections Framework.
3.2.24.6.0
Testing EPL 2.0
junit » junit1 vulnerability
JUnit Jupiter is the API for writing tests using JUnit 5.
4.136.1.3
Logging Apache 2.0
log4j » log4j6 vulnerabilities
Implementation for Apache Log4J, a highly configurable logging tool that focuses on performance and low garbage ...
1.2.172.26.1
MPL 2.0
net.sf.saxon » Saxon-HE
The XSLT and XQuery Processor
10.113.0
net.sourceforge.htmlcleaner » htmlcleaner1 vulnerability
HtmlCleaner is an HTML parser written in Java. It transforms dirty HTML to well-formed XML following
2.52.29
Core Utils Apache 2.0
org.apache.commons » commons-lang31 vulnerability
Apache Commons Lang, a package of Java utility classes for the classes that are in java.lang's hierarchy, or are ...
3.103.20.0
I/O Apache 2.0
org.apache.commons » commons-io1 vulnerability
The Apache Commons IO library contains utility classes, stream implementations, file filters,
1.3.22.22.0
HTTP Clients Apache 2.0
org.apache.httpcomponents » httpclient1 vulnerability
Apache HttpComponents Client
4.5.125.6.4
HTTP Clients Apache 2.0
org.apache.httpcomponents » httpcore
Apache HttpComponents HTTP/1.1 core components
4.4.135.4.3
Assertion Apache 2.0
org.assertj » assertj-core1 vulnerability
Rich and fluent assertions for testing in Java
3.16.13.27.7
Lang Runtime Apache 2.0
org.codehaus.groovy » groovy-all1 vulnerability
Groovy: A powerful multi-faceted language for the JVM
2.4.195.1.0
Lang Runtime EPL 2.0 +2
org.jruby » jruby9.2.11.110.1.1.0
org.jsoup » jsoup2 vulnerabilities
jsoup is a Java library that simplifies working with real-world HTML and XML. It offers an easy-to-use API for URL ...
1.10.31.23.1
Mocking MIT
org.mockito » mockito-all
Mockito mock objects library core API and implementation
1.10.195.23.0
Lang Runtime JYTHON
org.python » jython
Jython is an implementation of the high-level, dynamic, object-oriented
2.7.22.7.4
Web Testing Apache 2.0
org.seleniumhq.selenium » selenium-java
Selenium provides support for the automation of web browsers. It provides extensions to emulate user interaction with ...
3.141.594.47.0
Logging MIT
org.slf4j » slf4j-api
API for SLF4J (The Simple Logging Facade for Java) which serves as a simple facade or abstraction for various logging ...
1.7.302.0.18
org.slf4j » slf4j-log4j12
SLF4J Reload4j Provider
1.7.302.0.18
redis.clients » jedis
Jedis is a blazingly small and sane Redis java client.
2.9.38.0.0
XPath MIT
us.codecraft » xsoup
HTML XPath selector based on Jsoup.
0.3.10.3.7