GitHub - liventruth/UL-SMF-Cache-Compression: The Unified Latent-State Memory Fabric (UL-SMF) is a hardware-software co-designed memory compression fabric that solves the memory bottleneck in long-context Transformer inference. By combining FSQ with dynamic 16-dimensional latent mapping, UL-SMF compresses Key-Value (KV) cache tensors by up to 384x while maintaining >94% semantic retention.

Linear-Complexity KV Cache Compression via GLRP v2.0 & Aegis-KV

The Unified Latent-State Memory Fabric (UL-SMF) is a hardware-software co-designed memory compression fabric that solves the memory bottleneck in long-context Transformer inference. By combining Finite Scalar Quantization (FSQ) with dynamic 16-dimensional latent mapping, UL-SMF compresses Key-Value (KV) cache tensors by up to 384x while maintaining >94% semantic retention.

⚠️ Enterprise & Commercial Licensing Notice

UL-SMF is dual-licensed:

1. Open Source (AGPLv3): Free for non-commercial research, academic use, and open-source projects. Note: The AGPLv3 license requires any network-accessible service using this software to open-source its entire backend application code.
2. Commercial Enterprise License: Required for proprietary commercial deployments, closed-source SaaS platforms, and enterprise data center infrastructure. Commercial licenses grant full rights without AGPLv3 copyleft restrictions, plus integration support.

📩 For Enterprise Licensing Inquiries: inquiries@lawrencearchitectures.com

Key Benchmarks (Measured on CUDA Hardware)

Quickstart (Universal Integration)

UL-SMF dynamically maps any model hidden dimension (Mistral, Llama, Qwen, etc.) on-the-fly using orthogonal projection:

import torch
from ul_smf import UniversalLatentBridge

# 1. Load your compiled Aegis-KV oracle core binary
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
oracle_core = torch.jit.load("aegis_kv_oracle_core.pt", map_location=device)
oracle_core.eval()

# 2. Wrap it with the Universal Dynamic Bridge (auto-adapts to any model size)
ul_smf_bridge = UniversalLatentBridge(core_module=oracle_core, core_dim=3072).to(device)

# 3. Seamlessly compress any model hidden dimension (e.g., 4096 for Llama/Qwen)
kv_cache_tensor = torch.randn(1, 32, 4096, device=device)
reconstructed_cache, compressed_latents = ul_smf_bridge(kv_cache_tensor)

print(f"Compressed down to latent space: {compressed_latents.shape}")