A crawler framework. It covers the whole lifecycle of crawler: downloading, url management, content
extraction and persistent. It can simply the development of a specific crawler.
Compile Dependencies (0)
| Category/License | Group / Artifact | Version | Updates |
|---|
Managed Dependencies (26)
| Category/License | Group / Artifact | Version | Updates | |
|---|---|---|---|---|
JSON Lib
Apache 2.0
| com.alibaba
»
fastjson2 vulnerabilities Fastjson is a JSON processor (JSON parser + JSON generator) written in Java | 1.2.68 | 2.0.64 | |
Browser Automation
BSD 2-clause
| com.github.detro
»
phantomjsdriver PhantomJSDriver is a Java binding for the PhantomJS WebDriver, GhostDriver. | 1.2.0 | 1.5.0 | |
MIT
| com.github.dreamhead
»
moco-core Moco is an easy setup stub framework, mainly focusing on testing and integration. | 1.1.0 | 1.6.1 | |
Core Utils
Apache 2.0
| com.google.guava
»
guava2 vulnerabilities Guava is a suite of core and expanded libraries that include | 29.0-jre | 33.7.1-jre | |
JSON Lib
Apache 2.0
| com.jayway.jsonpath
»
json-path1 vulnerability A library to query and verify JSON | 2.4.0 | 3.0.0 | |
CLI Parser
Apache 2.0
| commons-cli
»
commons-cli Apache Commons CLI provides a simple API for presenting, processing, and validating a Command Line Interface. | 1.4 | 1.11.0 | |
Collections
Apache 2.0
| commons-collections
»
commons-collections The Apache Commons Collections package contains types that extend and augment the Java Collections Framework. | 3.2.2 | 4.6.0 | |
Testing
EPL 2.0
| junit
»
junit1 vulnerability JUnit Jupiter is the API for writing tests using JUnit 5. | 4.13 | 6.1.3 | |
Logging
Apache 2.0
| log4j
»
log4j6 vulnerabilities Implementation for Apache Log4J, a highly configurable logging tool that focuses on performance and low garbage ... | 1.2.17 | 2.26.1 | |
MPL 2.0
| net.sf.saxon
»
Saxon-HE The XSLT and XQuery Processor | 10.1 | 13.0 | |
HTML Parser
BSD
| net.sourceforge.htmlcleaner
»
htmlcleaner1 vulnerability HtmlCleaner is an HTML parser written in Java. It transforms dirty HTML to well-formed XML following | 2.5 | 2.29 | |
Core Utils
Apache 2.0
| org.apache.commons
»
commons-lang31 vulnerability Apache Commons Lang, a package of Java utility classes for the classes that are in java.lang's hierarchy, or are ... | 3.10 | 3.20.0 | |
I/O
Apache 2.0
| org.apache.commons
»
commons-io1 vulnerability The Apache Commons IO library contains utility classes, stream implementations, file filters, | 1.3.2 | 2.22.0 | |
HTTP Clients
Apache 2.0
| org.apache.httpcomponents
»
httpclient1 vulnerability Apache HttpComponents Client | 4.5.12 | 5.6.4 | |
HTTP Clients
Apache 2.0
| org.apache.httpcomponents
»
httpcore Apache HttpComponents HTTP/1.1 core components | 4.4.13 | 5.4.3 | |
Assertion
Apache 2.0
| org.assertj
»
assertj-core1 vulnerability Rich and fluent assertions for testing in Java | 3.16.1 | 3.27.7 | |
Lang Runtime
Apache 2.0
| org.codehaus.groovy
»
groovy-all1 vulnerability Groovy: A powerful multi-faceted language for the JVM | 2.4.19 | 5.1.0 | |
Lang Runtime
EPL 2.0 +2
| org.jruby » jruby | 9.2.11.1 | 10.1.1.0 | |
HTML Parser
MIT
| org.jsoup
»
jsoup2 vulnerabilities jsoup is a Java library that simplifies working with real-world HTML and XML. It offers an easy-to-use API for URL ... | 1.10.3 | 1.23.1 | |
Mocking
MIT
| org.mockito
»
mockito-all Mockito mock objects library core API and implementation | 1.10.19 | 5.23.0 | |
Lang Runtime
JYTHON
| org.python
»
jython Jython is an implementation of the high-level, dynamic, object-oriented | 2.7.2 | 2.7.4 | |
Web Testing
Apache 2.0
| org.seleniumhq.selenium
»
selenium-java Selenium provides support for the automation of web browsers. It provides extensions to emulate user interaction with ... | 3.141.59 | 4.47.0 | |
Logging
MIT
| org.slf4j
»
slf4j-api API for SLF4J (The Simple Logging Facade for Java) which serves as a simple facade or abstraction for various logging ... | 1.7.30 | 2.0.18 | |
Logging Bridge
MIT
| org.slf4j
»
slf4j-log4j12 SLF4J Reload4j Provider | 1.7.30 | 2.0.18 | |
Redis Client
MIT
| redis.clients
»
jedis Jedis is a blazingly small and sane Redis java client. | 2.9.3 | 8.0.0 | |
XPath
MIT
| us.codecraft
»
xsoup HTML XPath selector based on Jsoup. | 0.3.1 | 0.3.7 |