Skip to content
@kvcache-ai

kvcache.ai

KVCache.AI is a joint research project between MADSys and top industry collaborators, focusing on efficient LLM serving.

Pinned Loading

  1. Mooncake Mooncake Public

    Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

    C++ 4.9k 575

  2. ktransformers ktransformers Public

    A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

    Python 16.7k 1.2k

  3. TrEnv-X TrEnv-X Public

    Go 75 4

Repositories

Showing 10 of 11 repositories
  • sglang_awq Public Forked from sgl-project/sglang

    SGLang is a fast serving framework for large language models and vision language models.

    kvcache-ai/sglang_awq’s past year of commit activity
    Python 2 Apache-2.0 4,650 0 0 Updated Mar 3, 2026
  • sglang Public Forked from sgl-project/sglang

    SGLang is a fast serving framework for large language models and vision language models.

    kvcache-ai/sglang’s past year of commit activity
    Python 8 Apache-2.0 4,650 0 3 Updated Mar 3, 2026
  • Mooncake Public

    Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

    kvcache-ai/Mooncake’s past year of commit activity
    C++ 4,860 Apache-2.0 575 250 (1 issue needs help) 83 Updated Mar 3, 2026
  • ktransformers Public

    A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

    kvcache-ai/ktransformers’s past year of commit activity
    Python 16,677 Apache-2.0 1,222 408 (1 issue needs help) 3 Updated Mar 2, 2026
  • kvcache-blog Public
    kvcache-ai/kvcache-blog’s past year of commit activity
    HTML 2 MIT 1 0 0 Updated Feb 25, 2026
  • vllm Public Forked from vllm-project/vllm

    A high-throughput and memory-efficient inference and serving engine for LLMs

    kvcache-ai/vllm’s past year of commit activity
    Python 16 Apache-2.0 14,039 0 0 Updated Feb 18, 2026
  • DeepEP_fault_tolerance Public Forked from deepseek-ai/DeepEP

    DeepEP: an efficient expert-parallel communication library that supports fault tolerance

    kvcache-ai/DeepEP_fault_tolerance’s past year of commit activity
    Cuda 4 MIT 1,116 0 0 Updated Jan 5, 2026
  • gpustack Public Forked from gpustack/gpustack

    GPU cluster manager for optimized AI model deployment

    kvcache-ai/gpustack’s past year of commit activity
    Python 0 Apache-2.0 466 0 0 Updated Dec 7, 2025
  • TrEnv-X Public
    kvcache-ai/TrEnv-X’s past year of commit activity
    Go 75 Apache-2.0 4 0 0 Updated Sep 15, 2025
  • sglang-npu Public Forked from sgl-project/sglang

    SGLang is a fast serving framework for large language models and vision language models.

    kvcache-ai/sglang-npu’s past year of commit activity
    Python 0 Apache-2.0 4,650 0 0 Updated Aug 12, 2025
View all repositories

Most used topics

Loading…