🤖 RAG Systems - Pipelines vs MCP

Short answer: use pipelines for control, MCP for interoperability, and combine both if you’re building something serious. The “best” setup depends on whether you care more about tight integration and performance or flexibility and tool reuse.

Here’s how to think about it clearly:

🧠 1. Pipelines (best for full control + performance)

In OpenWebUI, pipelines are the most direct and powerful way to integrate a RAG system.

Why pipelines work well for RAG:

You control the full flow: query → retrieval → reranking → prompt → LLM → response
Easy to plug in:
- your vector DB (FAISS, Weaviate, Chroma, etc.)
- custom chunking / embedding logic
- hybrid search (BM25 + vectors)
Lower latency than tool-based approaches
Easier to debug and optimize

When to choose pipelines:

You’re building a production RAG system
You want fine-tuned retrieval quality
You need custom ranking or filtering logic
You care about speed

👉 In practice, most serious RAG setups in OpenWebUI end up here.

🔌 2. MCP (Model Context Protocol) (best for modularity)

Model Context Protocol (MCP) is more like a universal plugin system for LLMs.

Instead of embedding RAG directly, you expose it as a tool/service.

Why MCP is useful:

Decouples your RAG system from the UI
Lets multiple models/tools reuse the same backend
Works across ecosystems (not just OpenWebUI)
Cleaner architecture if you have many tools

Downsides for RAG:

More overhead (tool calls, serialization)
Less control over prompt composition
Slightly worse latency vs pipelines

When to choose MCP:

You already have a separate RAG service
You want multi-client access (not just OpenWebUI)
You’re building a tool ecosystem, not just chat

⚙️ 3. Hybrid approach (what advanced setups do)

This is often the actual “best” answer.

Use:

Pipelines → for core RAG flow
MCP → for optional tools or external knowledge sources

Example:

Pipeline handles:
- embeddings
- vector search
- reranking
MCP tools provide:
- SQL access
- document fetchers
- APIs

This gives you:

speed where it matters
flexibility where it helps

🧩 4. Other options (less ideal but sometimes useful)

Built-in knowledge / simple RAG

Quick to set up
Not very customizable
Fine for prototypes

External orchestration frameworks

LangChain
LlamaIndex

These can:

sit inside a pipeline
or act as your RAG backend

But:

add abstraction overhead
sometimes fight OpenWebUI instead of helping

🏁 Recommendation (practical)

If you’re deciding right now:

Start with pipelines
Add MCP only if you need cross-system reuse

Simple rule:

Want best RAG quality? → pipelines
Want reusable tools/services? → MCP
Want both? → hybrid

If you want, tell me:

what vector DB you’re using
whether you need multi-user / multi-agent
latency constraints

…and I can sketch a concrete architecture for your setup. You’re seeing the core tradeoff correctly: pipelines live inside OpenWebUI, so ownership naturally falls to that team—which becomes a bottleneck if multiple teams are iterating on RAG.

If you keep everything in pipelines, you’ll end up with:

tight coupling
slow iteration across teams
“please merge my retrieval tweak” PRs to the UI team

That doesn’t scale well organizationally.

🧭 A better division of responsibilities

Instead of pushing RAG logic into pipelines, treat RAG as a product/service owned by its own teams.

🔌 Option 1 — RAG-as-a-service (recommended baseline)

Have RAG teams expose APIs like:

/retrieve
/answer
/rerank

Then OpenWebUI pipelines become thin orchestration layers:

User query → call RAG API → inject into prompt → LLM

Why this works:

RAG teams deploy independently
OpenWebUI team stays focused on UX + orchestration
Clear contracts (API schemas instead of shared Python code)

🧠 Option 2 — MCP for clean ownership boundaries

This is where Model Context Protocol becomes very attractive.

Each team exposes their system as an MCP tool:

knowledge_base.search
legal_docs.query
codebase.lookup

Then OpenWebUI just consumes tools, not code.

Big advantages:

No pipeline ownership conflict
Plug-and-play tools across teams
Teams can version independently

Subtle bonus:

You can let the model choose which RAG system to query instead of hardcoding it.

⚙️ Option 3 — Hybrid (what mature setups converge to)

This solves both organizational and performance issues:

Pipelines:
- minimal orchestration
- prompt shaping
- caching
External RAG services (or MCP tools):
- retrieval
- ranking
- domain logic

So the pipeline becomes something like:

if domain == "finance":
    call finance-rag
elif domain == "legal":
    call legal-rag

Or via MCP:

LLM decides → calls appropriate tool

🧱 Practical architecture pattern

Here’s a setup that works well in multi-team environments:

1. Define a strict RAG contract

All RAG services return something like:

{
  "documents": [...],
  "citations": [...],
  "confidence": 0.87
}

2. Standardize embeddings (optional but helpful)

Avoid each team doing completely incompatible things unless necessary.

3. Keep pipelines “thin”

In OpenWebUI:

no heavy retrieval logic
no indexing logic
no business rules

Just:

routing
prompt assembly
response formatting

4. Decide MCP vs API per use case

Use MCP when:

discoverability matters
multiple tools exist
model-driven selection is useful

Use direct APIs when:

you need strict control
latency is critical
flow must be deterministic

🚨 What to avoid

Putting vector DB logic inside pipelines
Letting OpenWebUI team “own” retrieval quality
Tight coupling to frameworks like LangChain unless all teams agree
One giant “universal RAG” system (these usually become unmaintainable)

🏁 Bottom line

You’re right to push back on pipeline ownership.

Best organizational model:

RAG teams → own retrieval services (API or MCP)
OpenWebUI team → owns orchestration layer (thin pipelines)

That separation gives you:

independent deployability
faster iteration
fewer cross-team dependencies

If you want, I can sketch:

a concrete MCP tool schema
or a minimal pipeline that cleanly calls external RAG services

just tell me your stack (FastAPI? gRPC? vector DB?). Authorization changes things quite a bit—it’s not just an add-on, it shapes where your RAG logic is allowed to live.

If you ignore it early, you’ll end up leaking data across teams or rebuilding everything later.

🧠 First principle: where is access control enforced?

You have three possible layers:

1. In OpenWebUI (UI-layer auth)

Easy to implement
Works for simple setups

Problem: The moment you have multiple RAG services, this becomes fragile:

UI must understand all permissions
Easy to bypass if services are called elsewhere
Not reusable outside OpenWebUI

👉 This should not be your primary enforcement point.

2. Inside each RAG service (recommended baseline)

Each team’s RAG system enforces:

document-level access
tenant isolation
role-based filtering

Example:

User → RAG API (with JWT) → filtered retrieval → results

Why this works:

Security travels with the data
Each team owns its own access rules
Works across all clients (not just OpenWebUI)

👉 This is the most important layer.

3. Gateway / policy layer (for larger orgs)

You add a central layer that:

validates identity
injects claims
routes requests

This can sit in front of:

MCP tools
RAG APIs

🔌 How this affects MCP vs pipelines

With pipelines only

If pipelines call RAG APIs:

Pass user identity (JWT / headers)
RAG service enforces access

✅ Clean separation ❌ Pipeline must correctly forward identity (easy to mess up)

With Model Context Protocol

MCP adds a twist:

Tools need user context
The protocol must propagate identity

If done right:

User → OpenWebUI → MCP tool → RAG service (with user context)

If done wrong:

Tool runs with system privileges → 🔥 data leak risk

👉 The key question: Does your MCP implementation support per-user auth context propagation?

If not, you’ll need to extend it or wrap it.

🔐 Recommended architecture (secure + scalable)

✅ Pattern: “Auth travels with the request”

User logs into OpenWebUI
OpenWebUI gets identity token (JWT/OIDC)
Pipeline or MCP forwards token
RAG service enforces access

🧱 Concretely:

In OpenWebUI:

Do NOT decide what docs a user can see
ONLY forward identity:
- JWT
- user ID
- roles/claims

In RAG services:

Enforce:
- row-level security
- metadata filtering (e.g. department=finance)
- document ACLs

In vector DB:

Use one of:

filtered search (metadata filters)
per-tenant indexes
hybrid (tenant + attribute filtering)

⚖️ Design choices you’ll need to make

1. Multi-tenant vs shared index

Shared index + filters → flexible, harder to secure
Per-tenant index → safer, more ops overhead

2. Pre-filter vs post-filter

Pre-filter (best): restrict retrieval query
Post-filter: remove docs after retrieval (can leak signals)

👉 Always prefer pre-filtering inside the retrieval step

3. Caching implications

If you cache:

cache must be user-aware
otherwise: cross-user data leaks

🚨 Common failure modes

“We’ll filter in the UI” → ❌ insecure
“Service trusts OpenWebUI completely” → ❌ brittle
“One API key for all users” → ❌ no isolation
“LLM decides what user can see” → ❌ absolutely not

🏁 Practical recommendation

Given your multi-team setup:

👉 Best structure

RAG teams:
- own retrieval + authorization
- expose secured APIs or MCP tools
OpenWebUI team:
- passes identity
- orchestrates only

👉 If you want simplicity

Start with:

API-based RAG services + JWT auth
Thin pipelines

Then later:

introduce MCP for discoverability

🔍 One key question for you

How are users authenticated today?

OIDC (e.g. Keycloak, Auth0, Azure AD)?
Internal tokens?
Something else?

That determines:

how you propagate identity
how hard MCP integration will be

If you tell me that, I can give you a concrete request flow (headers, token shape, etc.). Yes — OpenWebUI can integrate with Microsoft Entra ID, and it’s actually one of the first-class supported SSO options.

✅ What’s supported (short answer)

OpenWebUI supports:

OAuth2
OpenID Connect (OIDC)
Microsoft login out of the box

That means you can use Microsoft Entra ID as your identity provider via standard OIDC/OAuth flows. (Open WebUI)

🧠 How it works (architecture-wise)

Under the hood, the integration looks like this:

User → OpenWebUI → Entra ID (login) → token → OpenWebUI → your backend

Microsoft Entra ID acts as the Identity Provider (IdP)
OpenWebUI acts as the client application
Authentication uses OpenID Connect on top of OAuth2 (Microsoft Learn)

⚙️ What you actually configure

In practice, it’s just environment variables in OpenWebUI:

MICROSOFT_CLIENT_ID
MICROSOFT_CLIENT_SECRET
MICROSOFT_CLIENT_TENANT_ID
OPENID_PROVIDER_URL (OIDC discovery endpoint)
WEBUI_URL (your app URL)

Example (simplified):

MICROSOFT_CLIENT_ID=...
MICROSOFT_CLIENT_SECRET=...
MICROSOFT_CLIENT_TENANT_ID=...
OPENID_PROVIDER_URL=https://login.microsoftonline.com/<tenant>/v2.0/.well-known/openid-configuration

👉 This is standard OIDC wiring—nothing OpenWebUI-specific magic here. (Open WebUI)

🔐 Important capabilities (relevant to your architecture)

1. Group / role propagation

You can pass Entra groups into OpenWebUI:

via groups claim in the token
map to OpenWebUI roles/groups

This enables:

role-based access (e.g. admin vs user)
multi-team separation

But:

Entra often sends group IDs, not names by default
you may need claim customization (Open WebUI)

2. Single provider limitation

Out of the box:

only one OAuth/OIDC provider at a time

So if you want:

Entra + something else → needs workaround (Open WebUI)

3. Token handling limitation (important!)

OpenWebUI:

authenticates users
does NOT act as a full identity proxy

Meaning:

you must explicitly forward tokens to your RAG services
or re-issue your own backend tokens

👉 This ties directly to your earlier architecture question.

⚠️ Where people get it wrong

This is where your auth + RAG design matters:

❌ Mistake

“OpenWebUI handles auth, so backend is safe”

→ Wrong OpenWebUI is just the entry point, not the enforcement layer.

✅ Correct model

Use Entra ID as the source of truth, but:

User logs in (Entra ID)
        ↓
OpenWebUI gets ID token
        ↓
Pipeline / MCP forwards token
        ↓
RAG service enforces authorization

🧩 How this influences your earlier question (pipelines vs MCP)

With Entra ID in play:

Pipelines

must forward JWT tokens manually
simple but easy to forget

MCP

must support user-context propagation
otherwise tools run with system privileges (dangerous)

🏁 Practical recommendation

Given your setup (multiple teams + auth requirements):

👉 Do this:

Use Microsoft Entra ID via OIDC in OpenWebUI
Treat OpenWebUI as:
- authentication entry point
- NOT authorization authority
Pass tokens downstream:
- pipelines → forward JWT
- MCP → ensure per-user context
Enforce access in:
- RAG services
- data layer (vector DB filters, ACLs)

🔍 One key design decision for you

Do you want:

pure Entra tokens everywhere (simpler, but leaks coupling), or
internal tokens (token exchange) between services (cleaner, more work)

That decision will define:

how your pipelines are written
how MCP tools authenticate

If you want, I can sketch:

a concrete JWT flow (headers, claims, validation)
or how to map Entra groups → RAG document filters

Those are usually the next pain points.

Great—these two pieces (JWT flow + group mapping) are exactly where most multi-team RAG systems either become clean… or quietly break.

I’ll sketch both in a practical, implementation-ready way, assuming you’re using OpenWebUI with Microsoft Entra ID.

🔐 1. JWT Flow (End-to-End)

🧠 Goal

User authenticates via Entra ID
Token flows through OpenWebUI → RAG services
RAG enforces access (not OpenWebUI)

🧱 Flow Overview

[User Browser]
      ↓ (login)
[Entra ID]
      ↓ (ID token + access token)
[OpenWebUI]
      ↓ (forward token)
[Pipeline / MCP]
      ↓
[RAG Service]
      ↓
[Vector DB / Documents]

🔑 Step-by-step

1. User logs in via Entra

OpenWebUI receives:

ID Token (who the user is)
optionally Access Token

Typical JWT claims:

{
  "sub": "user-id",
  "email": "user@company.com",
  "name": "Jane Doe",
  "groups": ["group-id-1", "group-id-2"],
  "tid": "tenant-id",
  "iss": "https://login.microsoftonline.com/...",
  "aud": "your-client-id"
}

2. OpenWebUI stores session

But here’s the key:

👉 Do NOT stop here You must propagate identity downstream.

3. Forward token to RAG

Option A — Pass Entra token directly (simplest)

Pipeline adds header:

Authorization: Bearer <entra_jwt>

Option B — Token exchange (cleaner, more scalable)

OpenWebUI calls an internal auth service:

Entra Token → Auth Service → Internal JWT

Internal token example:

{
  "user_id": "123",
  "roles": ["finance"],
  "permissions": ["read:docs"],
  "source": "entra",
  "exp": 1712345678
}

👉 This decouples your backend from Entra specifics.

4. RAG service validates JWT

At minimum, validate:

signature (JWKS endpoint from Entra)
aud (audience)
iss (issuer)
expiration

For Entra:

JWKS: https://login.microsoftonline.com/<tenant>/discovery/v2.0/keys

5. Enforce authorization in RAG

This is the critical step:

Extract claims → apply filters → retrieve documents

⚠️ Common mistake

Validating JWT but not using claims for filtering

That’s authentication, not authorization.

👥 2. Entra Group Mapping → RAG Filtering

🧠 Goal

Map Entra groups → document access rules

🔑 Problem

Entra sends:

group IDs (GUIDs) by default Not human-readable names.

Example:

"groups": [
  "3f5c2a1e-...",
  "9a8b7c6d-..."
]

🧩 Step 1 — Decide mapping strategy

Option A — Static mapping (simple, fast)

In your RAG service:

GROUP_MAP = {
  "3f5c2a1e-...": "finance",
  "9a8b7c6d-...": "legal"
}

Option B — Dynamic lookup (scalable)

Use:

Microsoft Graph API
cache results

Option C — Custom claims (cleanest)

Configure Entra to emit:

"groups": ["finance", "legal"]

👉 This avoids lookup entirely.

🧱 Step 2 — Attach metadata to documents

When indexing documents:

{
  "text": "...",
  "department": "finance",
  "access_groups": ["finance"]
}

🔍 Step 3 — Filter at query time (critical)

GOOD (pre-filtering)

allowed_groups = user.groups

results = vector_db.search(
    query,
    filters={"access_groups": {"$in": allowed_groups}}
)

BAD (post-filtering)

results = vector_db.search(query)
results = [r for r in results if r.group in allowed_groups]

❌ This can leak:

embeddings influence
ranking signals

🧠 Advanced patterns

1. Hierarchical access

"user_groups": ["finance.europe"]
"doc_groups": ["finance.*"]

2. Multi-tenant isolation

Add:

"tenant_id": "company-a"

Filter:

filters = {
  "tenant_id": user.tenant_id,
  "access_groups": {"$in": user.groups}
}

3. Hybrid RBAC + ABAC

Combine:

roles (admin, viewer)
attributes (department, region)

🔐 Where this runs

Always inside the RAG service, not in:

OpenWebUI
pipelines
MCP layer

🏁 Putting it together

✅ Clean architecture

[User]
  ↓
[Entra ID]
  ↓
[OpenWebUI]
  ↓ (JWT)
[Pipeline / MCP]
  ↓ (JWT)
[RAG Service]
  ↓ (enforces groups + tenant)
[Vector DB]

🧭 Key principles

AuthN (who you are) → Entra
AuthZ (what you can see) → RAG service
OpenWebUI → just passes identity

🚨 Final sanity checks

Before going live, verify:

JWT is forwarded to every RAG call
RAG rejects requests without valid token
Retrieval uses pre-filtering
Caches are user-aware
Group mapping is deterministic

If you want, I can next:

sketch a FastAPI middleware for JWT validation
or show how to wire this into a specific vector DB (Weaviate, Elastic, pgvector, etc.)

That’s usually where implementation gets tricky.

🤖 RAG Systems - Pipelines vs MCP#

🧠 1. Pipelines (best for full control + performance)#

Why pipelines work well for RAG:#

When to choose pipelines:#

🔌 2. MCP (Model Context Protocol) (best for modularity)#

Why MCP is useful:#

Downsides for RAG:#

When to choose MCP:#

⚙️ 3. Hybrid approach (what advanced setups do)#

🧩 4. Other options (less ideal but sometimes useful)#

Built-in knowledge / simple RAG#

External orchestration frameworks#

🏁 Recommendation (practical)#

🧭 A better division of responsibilities#

🔌 Option 1 — RAG-as-a-service (recommended baseline)#

Why this works:#

🧠 Option 2 — MCP for clean ownership boundaries#

Big advantages:#

Subtle bonus:#

⚙️ Option 3 — Hybrid (what mature setups converge to)#

🧱 Practical architecture pattern#

1. Define a strict RAG contract#

2. Standardize embeddings (optional but helpful)#

3. Keep pipelines “thin”#

4. Decide MCP vs API per use case#

🚨 What to avoid#

🏁 Bottom line#

🧠 First principle: where is access control enforced?#

1. In OpenWebUI (UI-layer auth)#

2. Inside each RAG service (recommended baseline)#

Why this works:#

3. Gateway / policy layer (for larger orgs)#

🔌 How this affects MCP vs pipelines#

With pipelines only#

With Model Context Protocol#

🔐 Recommended architecture (secure + scalable)#

✅ Pattern: “Auth travels with the request”#

🧱 Concretely:#

In OpenWebUI:#

In RAG services:#

In vector DB:#

⚖️ Design choices you’ll need to make#

1. Multi-tenant vs shared index#

2. Pre-filter vs post-filter#

3. Caching implications#

🚨 Common failure modes#

🏁 Practical recommendation#

👉 Best structure#

👉 If you want simplicity#

🔍 One key question for you#

✅ What’s supported (short answer)#

🧠 How it works (architecture-wise)#

⚙️ What you actually configure#

🔐 Important capabilities (relevant to your architecture)#

1. Group / role propagation#

2. Single provider limitation#

3. Token handling limitation (important!)#

⚠️ Where people get it wrong#

❌ Mistake#

✅ Correct model#

🧩 How this influences your earlier question (pipelines vs MCP)#

Pipelines#

MCP#

🏁 Practical recommendation#

👉 Do this:#

🔍 One key design decision for you#

🔐 1. JWT Flow (End-to-End)#

🧠 Goal#

🧱 Flow Overview#

🔑 Step-by-step#

1. User logs in via Entra#

2. OpenWebUI stores session#

3. Forward token to RAG#

Option A — Pass Entra token directly (simplest)#

Option B — Token exchange (cleaner, more scalable)#

4. RAG service validates JWT#

5. Enforce authorization in RAG#

⚠️ Common mistake#

👥 2. Entra Group Mapping → RAG Filtering#

🧠 Goal#

🤖 RAG Systems - Pipelines vs MCP

🧠 1. Pipelines (best for full control + performance)

Why pipelines work well for RAG:

When to choose pipelines:

🔌 2. MCP (Model Context Protocol) (best for modularity)

Why MCP is useful:

Downsides for RAG:

When to choose MCP:

⚙️ 3. Hybrid approach (what advanced setups do)

🧩 4. Other options (less ideal but sometimes useful)

Built-in knowledge / simple RAG

External orchestration frameworks

🏁 Recommendation (practical)

🧭 A better division of responsibilities

🔌 Option 1 — RAG-as-a-service (recommended baseline)

Why this works:

🧠 Option 2 — MCP for clean ownership boundaries

Big advantages:

Subtle bonus:

⚙️ Option 3 — Hybrid (what mature setups converge to)

🧱 Practical architecture pattern

1. Define a strict RAG contract

2. Standardize embeddings (optional but helpful)

3. Keep pipelines “thin”

4. Decide MCP vs API per use case

🚨 What to avoid

🏁 Bottom line

🧠 First principle: where is access control enforced?

1. In OpenWebUI (UI-layer auth)

2. Inside each RAG service (recommended baseline)

Why this works:

3. Gateway / policy layer (for larger orgs)

🔌 How this affects MCP vs pipelines

With pipelines only

With Model Context Protocol

🔐 Recommended architecture (secure + scalable)

✅ Pattern: “Auth travels with the request”

🧱 Concretely:

In OpenWebUI:

In RAG services:

In vector DB:

⚖️ Design choices you’ll need to make

1. Multi-tenant vs shared index

2. Pre-filter vs post-filter

3. Caching implications

🚨 Common failure modes

🏁 Practical recommendation

👉 Best structure

👉 If you want simplicity

🔍 One key question for you

✅ What’s supported (short answer)

🧠 How it works (architecture-wise)

⚙️ What you actually configure

🔐 Important capabilities (relevant to your architecture)

1. Group / role propagation

2. Single provider limitation

3. Token handling limitation (important!)

⚠️ Where people get it wrong

❌ Mistake

✅ Correct model

🧩 How this influences your earlier question (pipelines vs MCP)

Pipelines

MCP

🏁 Practical recommendation

👉 Do this:

🔍 One key design decision for you

🔐 1. JWT Flow (End-to-End)

🧠 Goal

🧱 Flow Overview

🔑 Step-by-step

1. User logs in via Entra

2. OpenWebUI stores session

3. Forward token to RAG

Option A — Pass Entra token directly (simplest)

Option B — Token exchange (cleaner, more scalable)

4. RAG service validates JWT

5. Enforce authorization in RAG

⚠️ Common mistake

👥 2. Entra Group Mapping → RAG Filtering

🧠 Goal