Private AI and local LLMs: AI that stays inside your walls

Why private AI?

• GDPR: with on-premise or a European host, prompts and documents stay in the EU. That avoids third-country transfers and limits US CLOUD Act exposure.
• Trade secrets: your data is never used to train someone else’s model.
• Predictable costs: no per-token bill that grows with usage. A fixed server and fixed costs.
• Works offline: in the factory, in the vehicle or on site without internet.

What we set up

• Open-weight models such as Mistral, Llama and Qwen, served with vLLM or Ollama on your own GPU server or with an EU host.
• A knowledge assistant (RAG) over your own documents, with per-user permissions and sources cited in every answer.
• A hybrid setup where that fits better: sensitive data stays local, and generic tasks go to the EU endpoint of a frontier model.
• Monitoring, evaluation sets and logging, so you can show what the system does.

The EU AI Act: what SMEs need to know now

The AI Act applies in phases. Prohibited practices and the AI-literacy duty for staff have applied since February 2025. General-purpose AI rules have applied since August 2025. Through the Digital Omnibus package, the EU is delaying the heavy high-risk obligations, expected to December 2027 (standalone systems) and August 2028 (AI embedded in products). Transparency duties, such as telling people they’re talking to AI, still matter.

Every project comes with a short AI Act classification: which risk class the use case falls into, which duties apply, and how the design accounts for them.

This is general information, not legal advice. Regulation changes, and we check the current status at the start of every project.