Group: ML

Sort by:Popular▼

✓Popular✓Newest

Client-facing OpenAPI 3.0 REST spec + generated Java code. Internal node-to-node gRPC definitions (inference.proto).

Inference Node — runs on each GPU machine, cuBLAS SGEMM via Panama FFI, pipeline forward pass, gRPC activation transfer

KV Cache Manager — three-tier GPU/CPU/Disk storage, prefix Trie cache, configurable eviction, Hazelcast metadata index

Model Registry and Shard Loader — GGUF weight loading, IMQ-inspired seed election, dynamic VRAM-aware sharding via Hazelcast

Health Monitor — Hazelcast membership events, GPU probes (via node), Resilience4j circuit breakers, Micrometer metrics

LoRA adapter tensors, checkpoint I/O, and Adam — no CUDA or GGUF dependency

Sampler — pure Java, zero external deps, temperature/top-k/top-p/repetition-penalty pipeline, configurable presets

Tokenizer — DJL SentencePiece wrapper, chat template formatter, BPE support for LLaMA 2/3, Mistral, Gemma

Bill of Materials for Juno cab.ml artifacts — import this POM in dependencyManagement to pin aligned versions.

Coordinator and Scheduler — continuous batching, leader election, gRPC streaming, Java 25 virtual threads

Java Unified Neural Orchestration (Juno) -- distributed LLM inference and fine-tuning engine

Juno-master standalone coordinator entry point. Full-cluster tests.

Juno node fat jar — NodeMain entry point for inference nodes.

Console player and cluster harness utilities used for manual runs.

JFR productivity metrics and model manifest