Group: JWAT

Used for reading, writing and validating WARC files. Implemented to follow the WARC/1.0 ISO specification as closely as possible.

Various classes to help with streams, character encoding, HTTP and other Internet related protocols.

Used for reading, writing and validating ARC files. Trying to follow the adhoc standard as much as possible.

Used for reading, writing and validating GZip files. Internal header parser, constructor and validation. Implemented in accordance with the RFC.

JWAT-Tools uses the available JWAT libraries to make high level tasks available either from command-line or programmatically. Common tasks include: Test, Compress, Decompress, CDX, Arc2Warc. More specialised tasks include: Changed, ContainerMD, Delete, Extract, Interval, PathIndex, Unpack, Headers2CDX.

Work in progress on a unified reader API. Nothing useful yet.

Work on a distribution module that was never finalized. Currently only groups a bunch of jar files into two different archives(zip and tar.gz).

Common code used with both ARC and WARC archives. Also include some useful utilities to use when working with ARC/WARC/GZip files.

A collection of libraries to use for reading, writing and validating ARC, WARC and GZip files. Also includes various helper classes to help with different types of input streams. Finally there are also classes to help with HTTP, character encoding and other Internet related protocols.