Open-Weight vs Closed-Weight vs Open-Source LLMs Explained
Introduction
GPT, Claude and Gemini are used through an API. Llama, DeepSeek, Qwen and Mistral can be downloaded and run on your own machine. And a few models, such as OLMo, share even their training data. People call all of these "open", and the word hides differences which decide what you may build, where your data goes, and what the bill looks like. This post explains open-weight, closed-weight and open-source LLMs with diagrams: the three categories and what each really releases, what you can and cannot do with each, licences and privacy, what it takes to run a model yourself, a Java calculator for memory and for the break-even between an API and your own GPU, and a decision flow for choosing.(Note - This field changes quickly. The models and terms named here are examples as of late 2026; check each model's current licence before you rely on it. Nothing here is legal advice.)
Three meanings of "open"
A model has three parts which can be shared or kept: its weights (the learned numbers, a file of tens or hundreds of gigabytes), the code used to train it, and the data it was trained on. The labels differ in how much of this is released -What each label releases.
- Closed-weight models are used only through the provider's servers: you send text to an API and receive text back. The weights never leave the provider. This is how GPT, Claude and Gemini models are offered.
- Open-weight models publish the weights, so anyone can download them, run them on their own hardware, and fine-tune them. The terms are set by a licence - which may be permissive, like MIT for DeepSeek-R1Click to read tutorial: DeepSeek-R1 Paper Explained: Reasoning via RL and Apache 2.0 for many Qwen, Mistral and OpenAI gpt-oss models, or a custom one, like the Llama community licence and Google's terms for Gemma. The training code and the training data are usually not released.
- Open-source, in the strict sense, means more: the Open Source Initiative's Open Source AI Definition (version 1.0, 2024) asks for the weights, the code for training and running the model, and enough information about the training data, all under open-source terms. Models such as OLMo from the Allen Institute for AI aim to meet it. Very few of the models called "open-source" in the news do.
What you can and cannot do
| Closed weights (API) | Open weights | Open source | |
|---|---|---|---|
| Run it on your own hardware | No | Yes | Yes |
| Keep every request inside your network | No - data goes to the provider; check its terms | Yes, if you host it yourself | Yes, if you host it yourself |
| Fine-tune it | Only through the provider's service, for some models | Yes, for example with LoRAClick to read tutorial: LoRA Paper Explained: Low-Rank Adaptation of LLMs | Yes |
| Inspect weights and internal activations | No | Yes | Yes |
| Keep one version forever | No - versions are retired on the provider's schedule | Yes - keep the file | Yes |
| Modify it and share the result | No | Depends on the licence | Yes |
| Reproduce its training | No | No - code and data are usually missing | In principle, with huge compute |
| Quality at the frontier | The leading models have usually been here | Often close behind | Varies |
| Operations work for you | None: the provider runs it | Yours: hardware, serving, updates | Yours |
Licences, privacy and safety
- Read the licence. Questions to ask of every model: may it be used commercially? May modified versions be shared? Is there a limit on the size of the service you run with it, an acceptable-use policy, or an attribution rule? May its outputs be used to train other models - a question which also applies to closed APIs, as noted in the knowledge distillationClick to read tutorial: Knowledge Distillation Explained: Teacher and Student Models post? Some licences are permissive (MIT, Apache 2.0); others are custom and restrictive.
- Open weights do not by themselves mean private. Privacy comes from where the model runs. If you call an open-weight model through a third party's hosted API, your data goes to that third party just as it would with a closed model. Only a model which you host yourself keeps data inside your network.
- Know what you download. Prefer the safetensors file format to older pickle-based formats, which can run code when they are loaded, and download from a source you trust.
- Safety tuning is not permanent. The refusals which a lab trained in, as in RLHFClick to read tutorial: InstructGPT Paper Explained: Training LLMs with RLHF, can be fine-tuned away from open weights. Applications which need guardrails should add their own, around the model.
- Closed APIs have their own terms: data retention, whether your data may be used for training, where it is processed, and what happens when a model version is retired. Enterprise plans usually offer stronger commitments than free ones.
Running an open-weight model: memory and cost
Downloading the weights is the easy part. Running them takes memory for the weights, memory for the attention cache of every conversation in progress (the KV cache, which keeps the keys and values that attentionClick to read tutorial: Attention Is All You Need: The Transformer Paper Explained needs for every earlier token in the context windowClick to read tutorial: LLM Tokens, Context Windows, Embeddings and Hallucinations), and a server which batches requests. Tools such as Ollama, llama.cpp and vLLM do the serving, and they expose an HTTP API shaped like the hosted ones, so a Java application can call a local model with the same client code as a hosted one. The questions are about memory and money, which the program below works out. It uses the 4.25 bits per weight of a 4-bit block format from the quantizationClick to read tutorial: LLM Quantization Explained: 8-bit, 4-bit and GGUF post, and an attention cache like that of a 70-billion-parameter model with 80 layers and 8 key-value heads. The GPU price and speed are assumptions to replace with your own.Memory for open-weight models, and when your own GPU beats an API
//SelfHosting.java
public class SelfHosting {
static final double[] BITS = {16, 8, 4};
// Memory for the weights: parameters x bits / 8. A quantized format also stores a 16-bit scale for every block of
// 64 weights, which adds 0.25 bits per weight.
static double weightsGb(double billionParams, double bits) {
double bitsPerWeight = bits < 16 ? bits + 16.0 / 64 : bits;
return billionParams * bitsPerWeight / 8;
}
public static void main(String[] args) {
// 1. Which open-weight model fits on which hardware?
String[] names = {"7B model", "70B model", "671B model (a mixture of experts)"};
double[] sizes = {7, 70, 671};
System.out.printf("%-36s %10s %10s %10s%n", "Memory for the weights", "16-bit", "8-bit", "4-bit");
for (int i = 0; i < names.length; i++) {
System.out.printf("%-36s", names[i]);
for (double bits : BITS) System.out.printf(" %7.0f GB", weightsGb(sizes[i], bits));
System.out.println();
}
// The attention cache grows with every token of context: 2 (keys and values) x layers x KV heads x head size x 2 bytes.
// The numbers are those of a 70B model with 80 layers and 8 KV heads of size 128, 16-bit values.
long bytesPerToken = 2L * 80 * 8 * 128 * 2;
System.out.printf("%nKV cache of a 70B model: %,d bytes per token%n", bytesPerToken);
for (int context : new int[] {8_192, 32_768, 131_072})
System.out.printf(" %,7d tokens of context: %5.1f GB per conversation%n", context, bytesPerToken * (double) context / 1e9);
// 2. API or your own GPU? Break-even for one 80 GB GPU rented around the clock (all numbers are assumptions).
double gpuPerHour = 4.00, hoursPerMonth = 730, tokensPerSecond = 1_000;
double serverPerMonth = gpuPerHour * hoursPerMonth;
double capacity = tokensPerSecond * hoursPerMonth * 3600; // tokens a month at full load
System.out.printf("%nOwn GPU: $%.2f per hour x %.0f hours = $%,.0f a month; at %,.0f tokens per second it can produce %,.1f billion tokens a month.%n",
gpuPerHour, hoursPerMonth, serverPerMonth, tokensPerSecond, capacity / 1e9);
System.out.printf("%-28s %-26s %s%n", "API price per million tokens", "break-even tokens a month", "share of GPU capacity needed");
for (double apiPrice : new double[] {0.50, 2.00, 10.00}) {
double breakEven = serverPerMonth / apiPrice * 1_000_000;
System.out.printf("$%-27.2f %-26s %.0f%%%s%n", apiPrice, String.format("%,.1f billion", breakEven / 1e9),
100 * breakEven / capacity, breakEven > capacity ? " (never: one GPU cannot produce that many)" : "");
}
}
}
Memory for the weights 16-bit 8-bit 4-bit
7B model 14 GB 7 GB 4 GB
70B model 140 GB 72 GB 37 GB
671B model (a mixture of experts) 1342 GB 692 GB 356 GB
KV cache of a 70B model: 327,680 bytes per token
8,192 tokens of context: 2.7 GB per conversation
32,768 tokens of context: 10.7 GB per conversation
131,072 tokens of context: 42.9 GB per conversation
Own GPU: $4.00 per hour x 730 hours = $2,920 a month; at 1,000 tokens per second it can produce 2.6 billion tokens a month.
API price per million tokens break-even tokens a month share of GPU capacity needed
$0.50 5.8 billion 222% (never: one GPU cannot produce that many)
$2.00 1.5 billion 56%
$10.00 0.3 billion 11%
Explanation of the code
- A 7B model needs 14 GB at 16 bits and about 4 GB at 4 bits, so it runs on a laptop. A 70B model needs 140 GB at 16 bits - two 80 GB GPUs - but 37 GB at 4 bits, which fits a single 48 GB GPU. The 671B model, a mixture of expertsClick to read tutorial: Mixture of Experts (MoE) Explained: Routers and Experts like DeepSeek-R1, needs 1,342 GB at 16 bits and still 356 GB at 4 bits: it fits eight 80 GB GPUs (640 GB), because all its experts must be loaded even though only a few work on each token.
- The weights are not the only memory. The attention cache of this 70B model takes 327,680 bytes per token, which is 2.7 GB for a conversation of 8,192 tokens, 10.7 GB for 32,768 tokens and 42.9 GB for 131,072 tokens - for each conversation running at the same time. A 4-bit 70B model (37 GB) with four conversations of 32,000 tokens already fills an 80 GB GPU.
- The break-even compares a rented GPU, $4.00 an hour for 730 hours, which is $2,920 a month, with an API's price per million tokens. At $2.00 per million tokens the GPU pays for itself at 1.5 billion tokens a month - 56% of what it could produce at the assumed 1,000 tokens a second. At $10.00 per million it needs only 11% of its capacity.
- At $0.50 per million tokens the answer is "never": the break-even of 5.8 billion tokens is 222% of what one GPU can produce, so under these assumptions the API is cheaper than the hardware at any volume. Cheap APIs are hard to beat on price.
- The model's real speed depends on the model, its quantization and how well requests are batched, and the numbers leave out the cost of the people who keep a server running. Self-hosting often wins on other grounds - data which cannot leave, a fixed version, low latency on your own network - even when it costs more.
Circles mark the break-evens: the GPU is cheaper to the right of them. Against the $0.50 API it never is.
Choosing between them
A first pass at the choice. Real decisions also weigh licences, quality on your task and your team's skills.
Open weights also open options which an API does not. They let you fine-tune cheaply with LoRA, shrink a model with quantization or distillationClick to read tutorial: Knowledge Distillation Explained: Teacher and Student Models, and use the new reasoning modelsClick to read tutorial: Reasoning Models and Test-Time Compute Explained without sending a byte of data outside - which is why DeepSeek-R1's release under the MIT licence mattered so much.
Summary
In this post we went through open-weight, closed-weight and open-source LLMs. We saw that the labels differ in what is released: closed-weight models offer only an API, open-weight models release the weights under a licence which varies, and open-source models, in the strict sense, add training code and data information. We compared what each allows - running on your own hardware, keeping data inside your network, fine-tuning, inspecting, keeping a version forever - and went through licences, privacy, file-format and safety points. In Java, we worked out the memory which open-weight models need, including the attention cache of every conversation, and found that one rented GPU beats an API only at high, steady volume - and never against a very cheap one. Lastly, we went through a decision flow and three habits: start with an API, keep the application behind an interface, and measure on your own task. To put an open-weight model to work, the posts on LoRAClick to read tutorial: LoRA Paper Explained: Low-Rank Adaptation of LLMs, for adapting it, and quantizationClick to read tutorial: LLM Quantization Explained: 8-bit, 4-bit and GGUF, for running it on modest hardware, are the natural next reads.Continue Reading: AI and LLM Papers Explained
Reasoning Models and Test-Time Compute ExplainedClick to read tutorial: Reasoning Models and Test-Time Compute Explained Attention Is All You Need: The Transformer Paper ExplainedClick to read tutorial: Attention Is All You Need: The Transformer Paper Explained BERT Paper Explained: Bidirectional TransformersClick to read tutorial: BERT Paper Explained: Bidirectional Transformers Scaling Laws for Language Models (and Chinchilla) ExplainedClick to read tutorial: Scaling Laws for Language Models (and Chinchilla) Explained GPT-3 Paper Explained: Language Models Are Few-Shot LearnersClick to read tutorial: GPT-3 Paper Explained: Language Models Are Few-Shot Learners RAG Paper Explained: Retrieval-Augmented GenerationClick to read tutorial: RAG Paper Explained: Retrieval-Augmented Generation LoRA Paper Explained: Low-Rank Adaptation of LLMsClick to read tutorial: LoRA Paper Explained: Low-Rank Adaptation of LLMs Chain-of-Thought Prompting Paper ExplainedClick to read tutorial: Chain-of-Thought Prompting Paper Explained InstructGPT Paper Explained: Training LLMs with RLHFClick to read tutorial: InstructGPT Paper Explained: Training LLMs with RLHF
Reasoning Models and Test-Time Compute ExplainedClick to read tutorial: Reasoning Models and Test-Time Compute Explained Attention Is All You Need: The Transformer Paper ExplainedClick to read tutorial: Attention Is All You Need: The Transformer Paper Explained BERT Paper Explained: Bidirectional TransformersClick to read tutorial: BERT Paper Explained: Bidirectional Transformers Scaling Laws for Language Models (and Chinchilla) ExplainedClick to read tutorial: Scaling Laws for Language Models (and Chinchilla) Explained GPT-3 Paper Explained: Language Models Are Few-Shot LearnersClick to read tutorial: GPT-3 Paper Explained: Language Models Are Few-Shot Learners RAG Paper Explained: Retrieval-Augmented GenerationClick to read tutorial: RAG Paper Explained: Retrieval-Augmented Generation LoRA Paper Explained: Low-Rank Adaptation of LLMsClick to read tutorial: LoRA Paper Explained: Low-Rank Adaptation of LLMs Chain-of-Thought Prompting Paper ExplainedClick to read tutorial: Chain-of-Thought Prompting Paper Explained InstructGPT Paper Explained: Training LLMs with RLHFClick to read tutorial: InstructGPT Paper Explained: Training LLMs with RLHF