Building GDPR-Compliant Enterprise RAG Systems on EU Infrastructure

The Compliance Problem

Enterprise organizations processing regulated data—contracts, compliance documentation, personnel records, client communications—face a hard constraint: data cannot cross borders without formal legal justification. The moment you send a prompt containing personal data to a US-based LLM API, you trigger GDPR Article 44, which prohibits transfers to third countries unless specific safeguards apply.

Data protection authorities increasingly expect organisations to be able to document their AI data flows — what personal data an AI system processed, on what legal basis, what technical protections were in place, and who the processor was and where the data physically resided. (The EDPB runs a yearly Coordinated Enforcement Framework; AI and international transfers have been recurring themes.)

For most regulated enterprises, the practical answer is the same: build your inference on EU infrastructure. This guide walks through the architecture, code patterns, and production design that power a real compliance-verification product—and how to integrate EU-hosted Qwen for RAG into that system.

Start building: Get a free JuiceFactory API key — no credit card required.

Why General Cloud LLMs Don't Work

Standard cloud LLM APIs (OpenAI, Anthropic, Google) store queries and responses for model improvement, safety monitoring, or have data residency that cannot be proven to align with GDPR Chapter V transfer mechanisms. Even with data privacy agreements, the legal path is:

• Option A: Claim "Standard Contractual Clauses" (SCC) cover the transfer
• Option B: Rely on an "adequacy decision" for your jurisdiction
• Option C: Implement "supplementary measures" (encryption, anonymization, access control) that demonstrably mitigate the third-country risk

Option C is the safest but complex. Option A+B is simpler if your LLM provider is on EU soil, zero-knowledge architecture, or both.

This is where JuiceFactory fits: EU-hosted Qwen inference with a stateless, zero-retention design and bring-your-own-key (BYOK) support, so prompts stay on EU infrastructure under keys you control. (Confirm the exact data-processing and retention terms against your own DPA/SLA before you cite them to an auditor.) But the system around the LLM matters just as much as the LLM itself.

Real-World Architecture: securityguru's RAG Engine

The best way to understand GDPR-compliant RAG is to see it in production. securityguru is an EU-based compliance-verification product that ingests customer policies, control documentation, and audit trails, then reasons over them using an LLM to map coverage against ISO 27001, PCI DSS, and GDPR frameworks.

Here's how the system handles compliance constraints:

1. Embedding: Local, Deterministic, EU-Only

from sentence_transformers import SentenceTransformer

EMBED_MODEL = "all-MiniLM-L6-v2"   # a small, CPU-friendly, open-source embedding model
_embedder = SentenceTransformer(EMBED_MODEL)

# Customer's documents → chunks → embeddings (all on-host, no API calls)
texts = [f"{c['heading']} {c['text']}" for c in chunks]
vectors = _embedder.encode(texts, batch_size=32, show_progress_bar=False).tolist()

Key design choice: Embeddings are computed locally using sentence-transformers with a small open-source embedding model such as all-MiniLM (open-source, runs on CPU). No external API call. No data leaves the EU.

This is not about raw quality—hosted embedding APIs would be slightly better—but about data sovereignty. By accepting a small accuracy trade-off, you avoid an entire category of compliance risk.

2. Vector Storage: Qdrant on EU Infrastructure

from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams

QDRANT_URL = os.environ.get("SG_QDRANT_URL", "http://100.97.237.108:30633")
qdrant = QdrantClient(url=QDRANT_URL, timeout=10, check_compatibility=False)

# Per-customer collection (data isolation)
def ensure_collection(customer_id: str):
    name = re.sub(r"[^a-z0-9_]", "_", customer_id.lower())[:50]
    qdrant.create_collection(
        collection_name=name,
        vectors_config=VectorParams(size=384, distance=Distance.COSINE),
    )

Qdrant runs on EU infrastructure (a Kubernetes cluster on Hetzner, physically located in EU data centers). Customer documents, embeddings, and metadata stay there.

Per-customer isolation: Each organization's data lives in its own Qdrant collection. No cross-customer bleed.

3. Retrieval: Multi-Query Deduplication

def _build_evidence_queries(req_id: str, req_title: str, evidence_meta: dict) -> list[str]:
    """Multi-query retrieval: 2-3 search angles per requirement."""
    queries = [req_title]
    queries.append(f"{req_title} policy procedur dokumentation")
    desc = (evidence_meta or {}).get("evidence_description", "")
    if desc and not re.search(r"\.md|\.pdf|\.docx|/workspace/", desc):
        queries.append(desc[:200])
    return queries

def search_chunks(customer_id: str, query: str, top_k: int = 8) -> list[dict]:
    embedder = _get_embedder()
    vector = embedder.encode([query], show_progress_bar=False)[0].tolist()
    results = qdrant.query_points(
        collection_name=_collection_name(customer_id),
        query=vector,
        limit=top_k,
        score_threshold=0.2,
    ).points
    return [{"score": r.score, "heading": r.payload.get("heading"),
             "text": r.payload.get("text"), "source": r.payload.get("source")} 
            for r in results]

Why multi-query? Different search angles catch different relevant documents. A requirement like "data encryption" needs hits on "cryptography," "AES-256," "TLS," not just exact matches.

Local deduplication prevents the LLM from reasoning over the same evidence twice.

4. LLM Reasoning: EU-Hosted Qwen via JuiceFactory

JUICEFACTORY_BASE = "https://api.juicefactory.ai/v1"

def _get_llm_client(model: str | None = None):
    from openai import OpenAI
    cfg = _PROVIDER_CONFIG["juicefactory"]
    api_key = os.environ.get(cfg["env_var"], "")
    
    client = OpenAI(base_url=cfg["base_url"], api_key=api_key)
    _clients_cache[provider] = client
    return client

def _chat_with_retry(model: str, messages: list, max_tokens: int = 400, 
                     temperature: float = 0.0, max_attempts: int = 3):
    """Retry + circuit-breaker for LLM reliability."""
    for attempt in range(1, max_attempts + 1):
        try:
            client = _get_llm_client(model)
            resp = client.chat.completions.create(
                model=model,
                messages=messages,
                max_tokens=max_tokens,
                temperature=temperature,
            )
            _record_health(_provider_for_model(model), was_error=False)
            return resp.choices[0].message.content.strip(), model
        except Exception as e:
            _record_health(_provider_for_model(model), was_error=True)
            if not _is_retryable(e) or attempt == max_attempts:
                break
            backoff = 2 ** (attempt - 1)  # 1, 2, 4 seconds
            _time.sleep(backoff)
    raise last_exc

The chat client is OpenAI-compatible, which means JuiceFactory's EU-hosted Qwen models drop in seamlessly. The code never changes; only the base_url and API key differ from OpenAI.

No data logging: Prompts sent to JuiceFactory contain only the compliance requirement and the retrieved chunks—personal data is excluded by design.

5. Per-Chunk Verification: Layer 7 LLM Validation

Here's where GDPR compliance becomes active. Instead of trusting a single LLM pass over a large context window, securityguru verifies each retrieved chunk individually:

VERIFY_CHUNK_PROMPT = """Du verifierar om ett dokumentavsnitt utgör bevis för ett compliance-krav.

KRAV ({framework} {req_id}): {req_title}

DOKUMENTAVSNITT
Källa: {source}
Rubrik: {heading}
---
{text}
---

Fråga: Utgör detta avsnitt bevis för att kravet uppfylls?

Svara EXAKT med ett av:
  YES     — Avsnittet bevisar tydligt att kravet uppfylls
  PARTIAL — Avsnittet är relaterat men ofullständigt eller indirekt
  NO      — Avsnittet är inte relevant eller motbevisar kravet

Format (exakt):
VERDICT: <YES|PARTIAL|NO>
REASON: <en mening på svenska som motiverar verdict>"""

def verify_chunk(chunk: dict, req_id: str, req_title: str, framework_id: str, 
                 model: str = "qwen3.5-27b") -> dict:
    """LLM decides: is this chunk evidence for the requirement?"""
    prompt = VERIFY_CHUNK_PROMPT.format(
        framework=framework_id, req_id=req_id, req_title=req_title,
        source=chunk.get("source", ""), heading=chunk.get("heading", ""),
        text=(chunk.get("text", "") or "")[:CHUNK_TEXT_MAX],
    )
    
    answer, _ = _chat_with_retry(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        max_tokens=400, temperature=0.0,
    )
    
    # Parse YES/PARTIAL/NO
    verdict = "NO"
    for line in answer.splitlines():
        if line.startswith("VERDICT:"):
            raw = line.split(":", 1)[1].strip().upper()
            if "YES" in raw:
                verdict = "YES"
            elif "PARTIAL" in raw:
                verdict = "PARTIAL"
    
    return {"verdict": verdict, "reason": reason_text}

Why per-chunk? In compliance work, you must prove your conclusions, not just assert them. If an auditor asks "Where does the documentation show encryption is enabled?"—you point to a specific paragraph, not a vague LLM summary. This architecture forces that discipline.

6. Assessment Aggregation: Evidence-Driven Status

def assess_requirement_v2(customer_id: str, framework_id: str, req_id: str, 
                          req_title: str, model: str = "qwen3.5-27b") -> dict:
    """Multi-query retrieval → per-chunk verification → aggregated verdict."""
    
    # 1. Multi-query retrieval, deduplication
    queries = _build_evidence_queries(req_id, req_title, meta)
    seen: dict[str, dict] = {}
    for q in queries:
        for c in search_chunks(customer_id, q, top_k=8):
            key = f"{c.get('source')}::{c.get('heading')}::{(c.get('text') or '')[:60]}"
            if key not in seen or c["score"] > seen[key]["score"]:
                seen[key] = c
    chunks = sorted(seen.values(), key=lambda x: -x["score"])[:8]
    
    # 2. LLM verifies each chunk
    verdicts = []
    for c in chunks:
        v = verify_chunk(c, req_id, req_title, framework_id, model)
        verdicts.append({
            "source": c.get("source", ""),
            "heading": c.get("heading", ""),
            "score": c.get("score", 0.0),
            "verdict": v["verdict"],
            "reason": v["reason"],
        })
    
    # 3. Aggregate verdicts → status
    yes_n = sum(1 for v in verdicts if v["verdict"] == "YES")
    partial_n = sum(1 for v in verdicts if v["verdict"] == "PARTIAL")
    
    if yes_n >= 1:
        status = "COVERED"
        motivering = f"{yes_n} document sections confirm the requirement ({partial_n} partially relevant)."
    elif partial_n >= 2:
        status = "PARTIAL"
        motivering = f"{partial_n} sections mention the topic but lack complete proof."
    else:
        status = "NOT_COVERED"
        motivering = "No document sections verified as evidence (LLM rejected all matches)."
    
    return {
        "status": status,
        "motivering": motivering,
        "verdicts": verdicts,
        "chunks_used": len(chunks),
        "confidence": _confidence(best_score, yes_n, partial_n),
    }

The result is auditable, not opaque. You can trace back every compliance verdict to specific document excerpts and LLM reasoning.

Resilience: Production Patterns

Running compliance analysis at scale requires robust error handling. securityguru's retry + circuit-breaker pattern is worth studying:

Health Tracking

import time
from collections import deque

_provider_health: dict[str, deque] = {}
_HEALTH_WINDOW_S = 300        # 5-minute sliding window
_HEALTH_MIN_SAMPLES = 5       # Need 5 samples before circuit breaks
_HEALTH_FAIL_THRESHOLD = 0.5  # >50% failure → switch provider

def _record_health(provider: str, was_error: bool) -> None:
    now = time.time()
    dq = _provider_health.setdefault(provider, deque())
    dq.append((now, was_error))
    
    # Trim old samples outside window
    cutoff = now - _HEALTH_WINDOW_S
    while dq and dq[0][0] < cutoff:
        dq.popleft()

def _is_provider_unhealthy(provider: str) -> bool:
    dq = _provider_health.get(provider)
    if not dq or len(dq) < _HEALTH_MIN_SAMPLES:
        return False
    failures = sum(1 for _, err in dq if err)
    return (failures / len(dq)) > _HEALTH_FAIL_THRESHOLD

Real-world scenario: JuiceFactory is temporarily unavailable (maintenance, overload). The system detects >50% errors over 5 minutes and automatically falls back to a secondary model. Users don't lose assessments; they complete with a degraded but functional fallback.

Retry with Exponential Backoff

def _is_retryable(exc: Exception) -> bool:
    """Transient errors (429, 5xx, timeouts) → retry. 4xx else → fail."""
    msg = str(exc).lower()
    if "429" in msg or "rate limit" in msg:
        return True
    if any(c in msg for c in ("500", "502", "503", "504", "timeout")):
        return True
    return False

# In _chat_with_retry:
for attempt in range(1, max_attempts + 1):
    try:
        # LLM call
        return answer, model
    except Exception as e:
        if not _is_retryable(e) or attempt == max_attempts:
            break
        backoff = 2 ** (attempt - 1)  # 1, 2, 4 seconds
        time.sleep(backoff)

Production benefit: Rate-limited or briefly overloaded LLM providers don't cascade into customer failures. Three retry attempts with exponential backoff handle the vast majority of transient issues.

Data Minimization & Compliance Trade-offs

The architecture above makes specific choices to favor compliance over raw quality:

Each trade-off is intentional — driven by GDPR Article 5 (data minimisation) and Article 32 (security), and by a simple auditor-facing principle: auditable compliance beats an elegant black box.

Where JuiceFactory Fits

JuiceFactory provides the reasoning layer: EU-hosted Qwen inference accessed via an OpenAI-compatible API. In securityguru, it's the LLM powering both:

1. Chunk verification (per-chunk YES/PARTIAL/NO verdicts)
2. Requirement assessment (aggregated compliance status)

The key properties that make JuiceFactory suitable here:

• EU data residency: inference stays in the EU, which removes the Article 44 third-country-transfer problem at the root
• OpenAI-compatible API: drop-in replacement, no code rewrites
• Stateless, zero-retention design + BYOK: prompts run under keys you control (confirm the exact logging/retention terms in your DPA)
• Multi-modal support: Qwen-VL can reason over scanned policy documents (optional Layer 2 enrichment)

For organizations building RAG systems over sensitive documentation (contracts, audit trails, compliance frameworks), JuiceFactory's EU footprint removes the thorniest legal hurdle — the foundation of GDPR-safe AI inference.

Production Deployment Checklist

Before going live with a GDPR-compliant RAG system:

• Embedding model: Confirmed local-only, no external API calls
• Vector database: Deployed on EU infrastructure (Hetzner, Gaia-X, AWS EU)
• LLM provider: Confirmed EU data residency + BYOK option available
• Data isolation: Per-customer collections, no cross-org data leakage
• Audit trail: Every LLM call logged with timestamp, model, tokens, result
• Retry + circuit-breaker: Transient failures handled gracefully
• DPIA: Data Protection Impact Assessment completed (GDPR Article 35)
• DPA signed: Data Processing Agreement in place with all processors
• Testing: Compliance assessment validated against reference frameworks (ISO 27001, PCI DSS)

Related Reading

• GDPR Article 44: General Principle for Transfers — authoritative legal text on data transfer restrictions
• European Commission: Rules for Data Transfers Outside the EU
• EDPS: Generative AI guidance — EDPS guidance on LLM compliance
• GDPR Local: LLM Compliance Guide — practical compliance playbook

Start building: Get a free JuiceFactory API key — no credit card required.

Related guides

Related guides: RAG with Qwen · GDPR-Safe AI Inference · Self-Hosted RAG Without OpenAI · EU LLM API Comparison