A private LLM on your own servers
We set up open-weight models on hardware you own or rent, connect them to your documents and systems, and hand over a system your team runs. Your data never goes to an outside AI provider.
Get a private LLM plan- On-premise, private cloud or air-gapped
- Model choice tested on your own examples
- GPU sizing and cost compared with API pricing
- Pilot on rented GPUs before you buy hardware
When a private LLM makes sense
- Your data cannot leave
- Client files, patient records, contracts, source code or designs that legal or security will not send to an outside AI provider.
- You use AI every day
- Per-token bills grow with every employee and every document. A server you own costs the same at ten users or a thousand requests an hour.
- Regulation applies
- GDPR, HIPAA, the EU AI Act or client contracts ask where data is processed and who can see it. On your servers the answer is simple.
- The network is closed
- Factories, labs and secure sites that run offline or air-gapped still get a working assistant.
How we set it up
Assessment
We pick one process with you, test open-weight models on your own examples, size the hardware and compare the cost with API pricing. You get a written plan and a fixed quote.
Pilot
We run the chosen model on rented GPUs, connect it to your documents or systems and measure it against your baseline before you buy anything.
Installation
We install the serving stack on your servers or private cloud, with access control, logging and monitoring, and train the people who will run it.
How much GPU memory a self-hosted LLM needs
Rough memory for the model weights alone. Long documents and many users at once need more on top, so we size from measured load.
| Model size | 4-bit | 16-bit | Typical use |
|---|---|---|---|
| 7–8B parameters | ~5 GB | ~16 GB | Classification, extraction, simple Q&A over documents |
| 14B | ~9 GB | ~28 GB | Stronger drafting and summarising, coding help |
| 30–32B | ~18–20 GB | ~64 GB | Most company assistants and retrieval (RAG) setups |
| 70B | ~40 GB | ~140 GB | Harder reasoning; usually a multi-GPU server |
Self-hosted LLM or an API?
| Self-hosted | Public API | |
|---|---|---|
| Where data goes | Stays on your servers | Sent to the provider |
| Cost model | Hardware once, then power and upkeep | Per token, grows with use |
| Quality on a narrow task | Close to frontier once tuned on your data | Strong out of the box |
| Open-ended reasoning | Behind the best frontier models | Best available |
| Who runs it | Your team, after handover | The provider |
Questions
What is a private LLM?
A large language model that runs on infrastructure you control, such as your own servers or a private cloud account, so prompts, documents and answers never pass through a public AI service. Most private deployments use open-weight models such as Llama, Qwen, Mistral or Gemma.
Is a self-hosted LLM as good as ChatGPT?
On open-ended reasoning, no. On a defined task with your own data, such as answering from your documents or extracting fields from forms, a tuned open-weight model with retrieval often comes close enough. We test both on your examples before you buy hardware.
How much hardware do we need?
It depends on the model size, how many people use it at once and how long the documents are. Many single-team setups run on one server with one or two GPUs; the table above gives rough memory needs. We size it from load measured in the pilot.
Can we start without buying servers?
Yes. The pilot runs on rented GPUs in a region you choose. You buy or lease hardware only after the pilot shows the numbers.
Where do you work?
With companies in Europe and the US. Engineers come on site for the assessment and the installation when the work needs it.
Tell us what you want to run
Describe the process and the data in two sentences. We reply with questions and times for a 30-minute call, and you get a straight answer on whether a private LLM fits.