Home » Categories » Tokenizers

Tokenizers
Sort by:Popular▼

1.DJL NLP Utilities For Huggingface Tokenizers86 usages

ai.djl.huggingface » tokenizers Apache

Deep Java Library (DJL) NLP utilities for Huggingface tokenizers
Last Release on Dec 16, 2025

2.JTokkit143 usages

com.knuddels » jtokkit MIT

JTokkit is a Java tokenizer library designed for use with OpenAI models.
Last Release on Jul 19, 2024

3.NLP Tokenizer JVM50 usages

ai.grazie.nlp » nlp-tokenizer-jvm

NLP Tokenizer JVM
Last Release on Aug 13, 2026

4.NLP Tokenizer31 usages

ai.grazie.nlp » nlp-tokenizer

NLP Tokenizer
Last Release on Aug 13, 2026

5.ML LLM Tokenizer6 usages

com.jetbrains.intellij.ml » ml-llm-tokenizer

ML LLM Tokenizer
Last Release on Aug 12, 2026

6.DJL NLP Utilities For SentencePiece5 usages

ai.djl.sentencepiece » sentencepiece Apache

Deep Java Library (DJL) NLP utilities for SentencePiece
Last Release on Dec 16, 2025

7.Xef5 usages

com.xebia » xef-tokenizer-jvm Apache

Building applications with LLMs through composability in Kotlin
Last Release on Nov 27, 2023