---
title: "Kimi K3 Is Open Weight. That Does Not Mean You Can Run It."
description: "Moonshot released the weights for a 2.8 trillion parameter model under a Modified MIT license. Open weights make it legally yours, but a sparse MoE decouples compute from memory, and memory is what decides whether self-hosting is a real option for your team."
image: "https://foundrysoft.co/api/og?type=article&title=Kimi+K3+Is+Open+Weight.+That+Does+Not+Mean+You+Can+Run+It.&cat=Insights+%2F%2F+Infrastructure&rt=12+min+read&au=Varun+Raj+Manoharan&dt=2026-08-12"
url: "https://foundrysoft.co/blog/kimi-k3-self-hosting-memory-math"
---

Insights // Infrastructure 2026-08-12 12 min read

# Kimi K3 Is Open Weight. That Does Not Mean You Can Run It.

Moonshot released the weights for a 2.8 trillion parameter model under a Modified MIT license. Open weights make it legally yours, but a sparse MoE decouples compute from memory, and memory is what decides whether self-hosting is a real option for your team.

![Varun Raj Manoharan](https://foundrysoft.co/images/about/founder.webp)

Varun Raj Manoharan Founder & Principal Engineer

Kimi K3 Open Weight LLMs Self-Hosted LLM MoE GPU Infrastructure

## Key takeaways

-   A sparse MoE separates the compute you pay per token from the memory you must hold resident. K3 activates 16 of 896 experts, so it is cheap to run per token and expensive to have available at all.
-   Memory is the binding constraint, and it is set by total parameters rather than active ones. All 2.8 trillion parameters have to be resident because routing can select any expert on any token.
-   The million token context window has its own memory cost that lands on top of the weights, and unlike weights it scales with how many concurrent sessions you are serving.
-   Open weights are worth having for reasons that survive the hardware math: licence certainty, no deprecation risk, and the option to serve it later on hardware that does not exist yet at a price that does not exist yet.

Moonshot AI released Kimi K3 on 16 July 2026 and published the full weights on 27 July under a Modified MIT licence. It is a 2.8 trillion parameter mixture-of-experts model with a one million token context window and native multimodal input, and by most accounts it is the largest open weight model anyone has shipped. In Moonshot's own evaluation suite it sits behind Claude Fable 5 and GPT-5.6 Sol on overall performance while beating Claude Opus 4.8 and GPT-5.5 across coding and agentic benchmarks. Arena ranked it first on Frontend Code at 1,679 points in blind developer testing, ahead of Fable 5.

Those numbers got the attention they deserved. What got less attention is the gap between "the weights are downloadable" and "we can run this," which for a model of this shape is wide enough that most teams asking the question should answer no.

I want to be precise about why, because the reason is interesting and it is not the one people assume.

## Sparse MoE separates two costs that used to move together

K3 contains 896 experts and activates 16 of them for any given computation. That is the architectural fact everything else follows from.

For a dense model, total parameters and per-token compute are the same number wearing two hats. Every parameter participates in every token, so a bigger model is proportionally more expensive to hold and proportionally more expensive to run.

A sparse MoE breaks that coupling. Per token, K3 does the compute of a model a fraction of its nominal size, because only 16 of 896 experts fire. That is what makes a 2.8 trillion parameter model practical to serve at all, and it is why the headline parameter count is not a throughput estimate.

The part that does not shrink is memory. Routing is per token and can select any expert, so every expert has to be resident and ready. You do not get to page in the 16 you need, because you do not know which 16 you need until the token arrives, and by then you have a latency budget measured in milliseconds. All 2.8 trillion parameters live in accelerator memory or the model does not serve.

So the model is cheap to run and expensive to have. That inversion is what catches teams out, because our intuitions about model cost were built on dense models where the two moved together.

## The memory arithmetic, with the caveats it needs

Here is the illustrative arithmetic. These are rough figures meant to establish an order of magnitude, not a deployment plan, and real numbers depend on quantisation scheme, kernel implementation, tensor parallel layout, and how much headroom your serving stack wants.

At eight-bit precision, one byte per parameter, 2.8 trillion parameters is roughly 2.8 terabytes of weights. On 80GB accelerators that is something like 35 cards before you have stored a single token of context or left any room for activations and fragmentation. Round up for reality and you are looking at five to eight nodes of eight cards each.

Quantise to four bits and the weights come down to roughly 1.4 terabytes, so perhaps 18 to 24 cards, call it three nodes. That is a meaningful improvement and it comes with a quality question you have to answer empirically on your own workload, because aggressive quantisation does not degrade all capabilities evenly, and the agentic and long-context behaviours are not the ones most quantisation evaluations measure.

Then add context. A one million token window is the headline feature, and the KV cache for it is charged separately from the weights. Unlike weights, that cost scales with concurrency: one user at full context is one allocation, ten concurrent users at full context is ten. A serving configuration that fits comfortably with a single session can fall over at modest concurrency, a failure mode that shows up in production and not in your evaluation.

Moonshot shipped production-ready support for Kimi Delta Attention in vLLM alongside the weights, and that detail matters more than it sounds. Attention efficiency work is exactly what moves the long-context memory profile from theoretical to practical, and having it land in a mainstream serving stack on day one is the difference between a research artifact and something you could operate.

## What that means in practice

Three nodes of current-generation accelerators, running continuously, staffed by someone who understands MoE serving, with a quantisation strategy you have validated yourself. That is the realistic floor for serving K3 in-house.

For most teams, that is not a hosting decision. It is a capital decision, and it competes with everything else the same money could do.

The honest framing is the one I would apply to any self-hosting question, only with larger numbers. Self-hosting does not remove operational burden, it relocates it. You are now operating inference: GPU provisioning, model updates, monitoring for degraded output, capacity planning against concurrency, and an on-call rotation for when the serving stack falls over at two in the morning. An API failure is a status page and a retry. A self-hosted failure is your afternoon, and at this scale it is your afternoon plus a hardware vendor.

## When the answer is still yes

None of that makes self-hosting wrong. It narrows the set of teams for whom it is right, and the qualifying conditions are specific.

**Data residency that is a hard requirement.** If the records your agent processes legally or contractually cannot leave your network, the hardware math stops being a comparison and becomes a cost of doing business. You are not choosing self-hosting over an API, you are choosing between self-hosting and not shipping the product. In that case the question is only which open weight model clears your quality bar at the smallest footprint, and K3 is one candidate among several rather than an obvious pick.

**Utilisation high enough to amortise the cluster.** The economics of owned hardware improve with the fraction of the day it is busy. A cluster serving a steady high-volume workload sixteen hours a day is a different proposition from one that idles between bursts. Work out your actual duty cycle before you model the crossover, because the crossover is extremely sensitive to it and most estimates are made against a peak that occurs twice a week.

**A workload that matches where K3 is strongest.** The benchmark profile is not uniform. Its standout results are frontend code in blind developer testing, 90.0 on Video-MME with subtitles against Sol's 89.5, and 91.1 on OmniDocBench against Fable's 89.8. Those are document and video understanding results sitting alongside coding, a specific shape. If your workload is heavy on document and video comprehension, the case is stronger than the general-purpose framing suggests.

## Rented weights are still open weights

There is a middle path that gets skipped in the self-host versus API framing, and it is where I would expect most teams who want K3 to actually land.

Open weights do not have to mean weights you host. Providers serve open weight models, and renting inference on a model whose weights are published is a materially different position from renting a proprietary model. You keep the ability to move: if the provider raises prices, degrades service, or exits, the weights are still public and another provider can serve the same model. You are not tied to a vendor's roadmap the way you are with a closed model that can be deprecated, repriced, or changed underneath you without a code change on your end.

What you do not get is data residency, because the tokens still leave your network. If residency was the reason you wanted open weights, this path does not solve your problem. If the reason was independence from a vendor's roadmap, it solves it almost completely, at a fraction of the cost and none of the operational burden.

Be precise about that distinction internally, because "we want open weights" is usually shorthand for one of those two motivations, and they have completely different answers.

## The option value argument

There is one more reason to care about a release like this even if you are certain you will never run it, and it is easy to dismiss as abstract until it is not.

A published set of weights under a permissive licence is an option that does not expire. The hardware that makes 2.8 trillion parameters awkward today gets cheaper on a schedule that has been reliable for a long time. Quantisation and serving techniques improve, and they improve fastest on models a lot of people want to run. Kimi Delta Attention landing in vLLM at release is an example of exactly that dynamic, and it will not be the last optimisation this model gets.

A proprietary model you depend on today can be deprecated on a vendor's timetable, and when it is, your options are migration or an unsupported dependency. A model whose weights you hold cannot be taken away. That does not mean you should download it and feel prepared, because a model you have never served is not a fallback plan. It does mean the calculus for a critical workload is different, and it is reasonable to weight that when choosing what to build on.

The right conclusion from the K3 release is not that everyone should self-host a frontier model. It is that the gap between the best open weight model and the best proprietary one is now small enough, and the licensing permissive enough, that "what happens if our provider changes the deal" has a real answer for the first time in a while. Most teams should still rent. They are just renting from a better position than they were a year ago.

#### Related reading

[Native Multimodal Models Beat Your OCR Pipeline, Then Take Away the Thing You Needed Most

Frontier models now post better than 90% on document understanding benchmarks and read a PDF page directly. That collapses a four-stage pipeline into one call, and it removes provenance, confidence, and loud failure. Here is how to get the accuracy without giving up the audit trail.

Document AI Multimodal OCR

](https://foundrysoft.co/blog/native-multimodal-document-extraction-provenance)[Best Open Weight LLMs for Agents in 2026

A practical look at the best open weight LLMs for agents in 2026, organized by which constraint, cost, latency, or data residency, should actually decide the pick.

Open Weight LLMs AI Agents LLM Comparison

](https://foundrysoft.co/blog/best-open-weight-llms-agents-2026)[The Ultimate Research Assistant: Processing 100-Page Contracts with Kimi K3

Explore how legal and financial professionals are using Moonshot AI's Kimi K3 model to analyze massive documents, identify loopholes, and extract key insights in seconds.

Kimi K3 Moonshot AI Legal AI

](https://foundrysoft.co/blog/kimi-k3-document-analysis)

#### Next Article

[

Your Agent's Real Unit Cost Is Per Completed Task, and You Are Probably Not Measuring It

](https://foundrysoft.co/blog/cost-per-completed-task-agent-economics)

Available for new projects

## Let's build something great.

Have a project in mind? We are an elite software and AI development studio ready to bring your ideas to production. Let's talk about your roadmap.

Start a Project [See our work](https://foundrysoft.co/work)

```json
{
  "@context": "https://schema.org",
  "@type": "Organization",
  "name": "FoundrySoft",
  "url": "https://foundrysoft.co",
  "logo": "https://foundrysoft.co/logo.svg",
  "description": "FoundrySoft builds production-grade software and AI systems for US companies, from an India-based team of senior engineers.",
  "sameAs": [
    "https://github.com/foundrysofthq",
    "https://www.linkedin.com/company/foundrysoft",
    "https://www.instagram.com/foundrysoft/"
  ]
}
```

```json
{
  "@context": "https://schema.org",
  "@type": "WebSite",
  "name": "FoundrySoft",
  "url": "https://foundrysoft.co"
}
```

```json
{
  "@context": "https://schema.org",
  "@type": "TechArticle",
  "headline": "Kimi K3 Is Open Weight. That Does Not Mean You Can Run It.",
  "description": "Moonshot released the weights for a 2.8 trillion parameter model under a Modified MIT license. Open weights make it legally yours, but a sparse MoE decouples compute from memory, and memory is what decides whether self-hosting is a real option for your team.",
  "url": "https://foundrysoft.co/blog/kimi-k3-self-hosting-memory-math",
  "mainEntityOfPage": "https://foundrysoft.co/blog/kimi-k3-self-hosting-memory-math",
  "image": [
    "https://foundrysoft.co/images/blog/kimi-k3-self-hosting-memory-math.webp"
  ],
  "datePublished": "2026-08-12",
  "dateModified": "2026-08-12",
  "keywords": "Kimi K3, Open Weight LLMs, Self-Hosted LLM, MoE, GPU Infrastructure",
  "author": {
    "@type": "Person",
    "name": "Varun Raj Manoharan"
  },
  "publisher": {
    "@type": "Organization",
    "name": "FoundrySoft",
    "logo": {
      "@type": "ImageObject",
      "url": "https://foundrysoft.co/logo.svg"
    }
  }
}
```

```json
{
  "@context": "https://schema.org",
  "@type": "BreadcrumbList",
  "itemListElement": [
    {
      "@type": "ListItem",
      "position": 1,
      "name": "Home",
      "item": "https://foundrysoft.co/"
    },
    {
      "@type": "ListItem",
      "position": 2,
      "name": "Blog",
      "item": "https://foundrysoft.co/blog"
    },
    {
      "@type": "ListItem",
      "position": 3,
      "name": "Kimi K3 Is Open Weight. That Does Not Mean You Can Run It.",
      "item": "https://foundrysoft.co/blog/kimi-k3-self-hosting-memory-math"
    }
  ]
}
```