Open-weight models for companies: how to choose
Updated 2026-11-03
An open-weight model is a language model whose trained weights you can download and run on your own hardware. That is what makes a private, self-hosted AI possible: the model runs inside your network, and no prompt or document goes to the company that trained it.
Open-weight is not the same as open source
“Open weights” means the parameters are published. It does not mean the training data, the training code or unrestricted use are published too. The Open Source Initiative’s Open Source AI Definition (version 1.0, October 2024) asks for enough information about the data and the code to rebuild the model, and few popular models meet it. For a company the practical question is narrower: does the licence let us use it for our purpose?
Main model families and their licences
Model versions change every few months, so pick a family and then test its current release. Licence terms below are as published by each family for its widely used releases; read the licence of the exact model you deploy.
| Family | Made by | Typical licence | Notes |
|---|---|---|---|
| gpt-oss (20b, 120b) | OpenAI | Apache 2.0 | Mixture-of-experts; the 120b runs on one 80 GB GPU, the 20b on a 16 GB card |
| Qwen | Alibaba | Apache 2.0 for most sizes | Wide range from under 1B to large mixture-of-experts models; strong multilingual support |
| Llama | Meta | Llama community licence | Permissive for most companies; extra terms for very large user bases and an acceptable-use policy |
| Mistral | Mistral AI (France) | Apache 2.0 for several models; others under research or commercial licences | European provider |
| Gemma | Gemma terms of use | Small and mid sizes, runs well on modest hardware | |
| DeepSeek | DeepSeek | MIT for recent releases | Very large mixture-of-experts models; need server-class hardware |
How to choose one
- Write down the task. Answering from internal documents, extracting fields from forms, classifying tickets, summarising calls and assisting with code each favour different models.
- Collect 30 to 50 real examples with the answer you would accept. This test set decides more than any public leaderboard.
- Shortlist two or three models per size class you can afford to run: a small one (8B to 14B), a mid one (around 30B) and, if hardware allows, a large one.
- Test at the precision you will deploy. A 4-bit model can behave differently from the 16-bit original. Test the quantized version.
- Check the licence for your use case, including any user-count limits or field-of-use restrictions.
- Size the hardware for the winner with the LLM VRAM calculator.
Size versus speed
Bigger models answer harder questions better but need more memory and generate more slowly. Mixture-of-experts models hold many parameters in memory yet activate only a fraction per token, so they combine the knowledge of a large model with the speed of a smaller one, at the price of more memory. For many company tasks a well-chosen 14B to 32B model with retrieval over your documents does the job; see RAG or fine-tuning for how to add your own knowledge.
Keeping up with new releases
New open-weight models appear every month. Because your test set is fixed, swapping in a new model is a matter of rerunning the tests and comparing scores. Build the system so the model is a configuration setting, not a rewrite.