In 2024, AI companies charged per request. By 2026, they may charge per context. Kimi's new research paper reframes KVCache—the technique that stores intermediate attention states—from a performance optimization into an entirely new economic model for AI services.
The thesis emerges clearly from the paper: context is not a feature to be optimized, but the actual product being sold. Traditional AI billing charges for each API call regardless of whether the model is processing five words or fifty thousand. Kimi's research proposes something different. What if billing scaled with the contextual relevance delivered—the specific knowledge, history, and relationships that make an AI response actually useful?
The technical foundation comes from ultra-long context capabilities and the KVCache mechanism that makes them practical. When a model processes a conversation, it generates key-value pairs for every token. Standard systems recompute these on every request. Kimi's architecture caches these states persistently, allowing the model to reference and build upon previous context without redundant computation. The paper demonstrates that this isn't just an efficiency trick—it changes what AI providers are actually selling.
Context becomes infrastructure. When cached states persist across sessions, they transform from ephemeral computation into durable assets. A user returning to a project after two weeks finds their AI assistant already loaded with relevant context—not because it was told to retrieve it, but because the system cached and maintained it. This shifts the economic equation fundamentally. The value isn't in the inference call; it's in what the model knows because someone paid for that context to be preserved and reused.
The paper explores monetization angles that follow from this reframing. Providers could charge per context-unit stored, creating a direct relationship between storage duration and billing. Enterprise customers might subscribe to context tiers based on how much institutional memory they need the AI to maintain. A law firm requiring perfect recall of every document reviewed deserves a different pricing tier than a casual user asking random questions.
Benchmarks within the paper show the approach maintains performance while enabling these new economic structures. Specific computational savings demonstrate that the model can sustain ultra-long contexts (up to millions of tokens) without proportional cost increases—exactly the condition needed for context-based pricing to work.
The implications extend beyond individual companies. If context becomes the unit of value, entire ecosystems could form around context marketplaces. Third parties might curate and sell specialized context sets—legal precedent databases, medical literature compilations, code repository histories—that users license and feed into AI systems. The paper doesn't fully explore this dimension, but the economic logic points there inevitably.
Critics will note the significant engineering challenges. Persistent KVCache requires sophisticated infrastructure and creates new reliability requirements. Some will argue this model benefits large providers at the expense of smaller ones, concentrating market power in companies that can afford the caching infrastructure. Security implications around cached context also warrant careful examination—sensitive information persisting longer creates new attack surfaces.
Yet these concerns describe implementation hurdles, not fundamental flaws. Kimi's research demonstrates that the next evolution of AI economics may involve charging for context consumed rather than queries processed. That changes everything about how AI services get built, priced, and delivered.