LLM speculative inference server for consumer hardware & heterogeneous computing
Categories & Topics
Repository Stats
Related Tools
bw24
memra — from-scratch LLM inference engine for NVIDIA RTX 50-series (Rust + CUDA)
cuda-oxide
cuda-oxide is an experimental Rust-to-CUDA compiler that lets you write (SIMT) GPU kernels in safe(ish), idiomatic Rust. It compiles standard Rust code directly to PTX — no DSLs, no foreign language bindings, just Rust.
ds4
DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm
InferenceX
Open Source Continuous Inference Benchmark Research Platform — Kimi K3 2.8T, MiniMax M3, DeepSeekv4, GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72 & soon™ TPUv6e/v7/Trainium2/3 | 开源持续推理基准研究平台 — Kimi K2.7-Code、MiniMax M3、DeepSeekv4、GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72,即将推出™ TPUv6e/v7/Trainium2/3
linux-insides
A comprehensive guide that explores the Linux kernel, detailing its inner workings and functionalities. It serves as an educational resource for those interested in understanding how Linux operates, making complex concepts accessible to a wider audience.
llm-algo-leetcode
LLM algorithm practice lab with theory, solutions, and test cases.《大模型算法与系统教程》面向大模型入门到进阶的算法实战教程,覆盖原理讲解、答案解析、测试用例与 CUDA/Triton 实战。