Adds a local chat model backed by jitLLM (io.github.beehive-lab:jitllm), the successor of GPULlama3.java. It runs GGUF models in-process on the CPU or, through TornadoVM, on a GPU (CUDA, OpenCL or ...

podcast with adam bien

GPU-accelerated Llama3.java inference in pure Java using TornadoVM. - GitHub - beehive-lab/GPULlama3.java: GPU-accelerated Llama3.java inference in pure Java using TornadoVM.

TornadoVM is an open-source framework that extends the Java Virtual Machine (JVM) to support hardware accelerators such as Graphics Processing Units (GPUs), Field-Programmable Gate Arrays (FPGAs), …