Open-source & private LLMs
Powerful AI.
On hardware you own.
Open-weight models such as Llama, Qwen, Mistral, Gemma and gpt-oss now handle most everyday business tasks. We select, size, deploy and tune them on your servers or private cloud, so your data never leaves your control.
- Your people & appsChat, email, CRM, internal tools
- Secure gatewaySign-in, quotas, logs
- Inference serverOpenAI-compatible API
- Open model on your GPULlama, Qwen, Mistral, Gemma, gpt-oss
No prompts, documents or answers sent to the internet
Why open-source
Your model.
Your rules.
With an open-weight model, the AI is a piece of software you own and run, not a service you rent by the request.
Private
Client files, health records, financials and source code stay on your hardware.
Predictable cost
Hardware is a fixed cost. Heavy use doesn’t raise the bill.
Customisable
Fine-tune on your vocabulary and tasks; choose the exact version you run.
No lock-in
Swap in a better model when one comes out, without changing your apps.
Open-source or a cloud API
An honest comparison.
The largest cloud models still lead on the hardest reasoning tasks. A common setup uses both: a local model for sensitive data and routine work, a cloud model for the rest.
| Criterion | Open-source on your hardware | Cloud AI API |
|---|---|---|
| Where your data goes | Stays in your network | To the provider’s servers |
| Cost model | Hardware up front, low running cost | Pay per use, grows with volume |
| Quality | Strong on summaries, drafting, extraction and document Q&A | Best available on complex reasoning |
| Customisation | Full: fine-tune, quantise, pin versions | Limited to the provider’s options |
| Works offline | Yes | No |
| Vendor lock-in | None | Tied to one provider’s models and terms |
Hardware
Smaller than
you might think.
Typical setups for a small or mid-sized team, with 4-bit quantised models. We size yours from your users, tasks and document lengths.
| Model size | Typical hardware | Good for |
|---|---|---|
| Small7–14B parameters | One GPU with 16–24 GB, or a Mac with 32 GB unified memory | Classification, extraction, short drafts, chat for a small team |
| Medium24–35B parameters | One GPU with 48 GB, or a Mac with 64 GB or more | Document Q&A (RAG), reports, coding help for a department |
| Large70B+ or mixture-of-experts | One or more 80 GB data-centre GPUs, on site or in a private cloud | Company-wide assistant, complex documents, agents |
Rule of thumb: a model needs about 2 bytes of memory per parameter at 16-bit precision and about half a byte at 4-bit, plus room for the conversation. A 70B model needs about 140 GB at 16-bit and about 40 GB at 4-bit. The number of simultaneous users and the length of documents change the numbers, which is why we benchmark before you buy.
How it works
Measured first.
Bought second.
You see how open models perform on your own work before you spend anything on hardware.
-
Benchmark on your tasks
We run several open models on real examples from your business and compare them with a leading cloud model, so you know the quality you’ll get.
-
Size the hardware
GPU, memory and storage from your user count, tasks and context length, with options to buy, rent or reuse what you have.
-
Deploy
An inference server with an OpenAI-compatible API, so existing tools plug in, behind a gateway with sign-in, quotas and logs.
-
Connect & tune
A chat interface for staff, RAG over your documents, MCP access to your systems, and fine-tuning when the benchmark shows it helps.
-
Operate
Monitoring, security patches and upgrades to newer models after we test them on your benchmark.
What you get
Everything to run it yourself.
- Benchmark reportOpen models against a cloud model, on your tasks.
- Hardware specificationWith buy, rent and private-cloud options.
- Model serverInstalled, secured and reachable through an OpenAI-compatible API.
- Chat for your teamA private, ChatGPT-style interface with sign-in.
- MonitoringUsage, speed and errors, with alerts.
- RunbookUpdates, backups and model upgrades, step by step.
Technology
The open-source stack.
Models
- Llama
- Qwen
- Mistral
- Gemma
- gpt-oss
- DeepSeek
- Phi
Serving
- vLLM
- SGLang
- llama.cpp
- Ollama
Interfaces
- Open WebUI
- OpenAI-compatible API
- MCP
Fine-tuning
- LoRA & QLoRA
- Hugging Face
- Unsloth
Hardware
- NVIDIA GPUs
- Apple silicon
- Private-cloud GPUs
When open-source
isn’t the right choice.
- Your volume is low and the data isn’t sensitive. A cloud API under zero-data-retention terms costs less than a GPU.
- You need the strongest reasoning available today. The leading cloud models are still ahead. Use them for that part and keep the rest local.
- No one can look after a server. Choose a private-cloud setup we manage for you, or an API.
How we work together
Start small.
Decide on facts.
Every project goes through the same three stages. You decide at the end of each one whether to continue, and you can stop after any stage and keep everything built so far.
-
Stage 1
Assessment
We look at your process, your data and your systems, then tell you plainly whether AI is worth it, which approach fits, and what it would cost to build and run.
You get a written recommendation with scope, timeline and cost.
-
Stage 2
Pilot
A working version on your real data, used by a few of your people, with results measured against questions and cases from your own business.
You get a working pilot and an evaluation report.
-
Stage 3
Production & support
We secure it, connect it to your systems, roll it out to your team and keep it running: monitoring, model updates and improvements.
You get a production system, documentation and ongoing support.
Yours to keep
- Source code, in your own repository
- Prompts, configuration and infrastructure setup
- Test sets and evaluation reports
- Fine-tuned model weights, if any
- Documentation and runbooks
Not included in our fees
- Hardware, if you buy or rent GPUs
- Cloud hosting
- AI model usage, if you use a cloud API
- Third-party software licences
We list them in the assessment, so the full cost is clear before you commit.
Questions
Are open-source models good enough?
For most business tasks, such as summarising, drafting, extracting data from documents and answering questions over your files, current open models do well. For the hardest reasoning tasks, the leading cloud models are still ahead. The benchmark tells you which case you are in before you buy anything.
What hardware do we need?
Often a single GPU workstation or a high-memory Mac is enough for a team. Larger deployments use one or more data-centre GPUs, on site or rented in a private cloud. The sizing table above gives typical setups; we size yours from your usage.
Can we use these models commercially?
Most can, but licences differ: some are fully permissive (Apache 2.0 or MIT), others add conditions, such as Llama’s community licence. We check the licence of every model we recommend against your use.
Does running AI on site make us HIPAA or SOC 2 compliant?
No system is compliant on its own. Running the model inside your network keeps data under the controls you already have, such as access control, encryption and audit logs, which supports your compliance program. We document the setup so your auditors can review it.
Can we start in the cloud and move on site later?
Yes. The same open model and serving software run on a rented private-cloud GPU or on your own server, so you can start without buying hardware and move when it makes sense.
Can it be trained on our data?
Yes, with fine-tuning (LoRA), when the benchmark shows a gain. For facts that change, RAG over your documents usually works better than training, and we often combine the two.
Who keeps it running?
Either your team, with our runbook and training, or us, under a support agreement covering monitoring, patches and model upgrades.
Other AI solutions
Start with
a conversation.
No call needed to start. Send a short brief and we’ll reply in writing with what we’d do and what it would take.
Houston, Texas · Working with clients worldwide