Skip to content
ContextRouter

Getting Started

Welcome to ContextRouter! This guide will help you understand what ContextRouter is and get you up and running quickly.

What is ContextRouter?

ContextRouter is a modular AI agent framework designed for building production-ready RAG (Retrieval-Augmented Generation) systems. It’s built on top of LangGraph and provides a clean separation between your agent’s decision logic and the technical implementation details.

Think of it as a “shared brain” that can:

  • Orchestrate multiple LLM providers (OpenAI, Anthropic, Vertex AI, local models)
  • Search across diverse data sources (databases, web, files, APIs)
  • Track data provenance so you always know where information came from
  • Scale from prototype to production without rewrites

Key Features

🧩 Truly Modular

Every component in ContextRouter can be swapped without changing your agent logic:

  • Switch from OpenAI to Anthropic with a config change
  • Move from Postgres to Vertex AI Search without code changes
  • Add new data sources by registering a connector

🔍 Advanced RAG

Go beyond simple vector search:

  • Hybrid search combining semantic and keyword matching
  • Intelligent reranking with Vertex AI or MMR
  • Knowledge graphs for relationship-aware retrieval
  • Automatic citations with source provenance

🛡️ Production Ready

Built for real-world deployments:

  • Bisquit Protocol for data provenance and audit trails
  • Blue/green deployments for zero-downtime updates
  • Langfuse integration for observability
  • Rate limiting and caching built-in

🌐 Universal Model Support

Use any LLM provider:

  • Commercial: OpenAI, Anthropic, Google Vertex AI, Groq
  • Aggregators: OpenRouter (hundreds of models)
  • Local: Ollama, vLLM, HuggingFace Transformers

How It Works

Your App → ContextRouter Cortex → Modules → Response
              (Decision Logic)    (Implementation)
  1. Your application sends a message to ContextRouter
  2. The Cortex (powered by LangGraph) decides what to do:
    • Does this need retrieval? From where?
    • Which LLM should generate the response?
    • Should we suggest follow-up questions?
  3. Modules execute the actual work:
    • Models call LLM APIs
    • Providers search your knowledge base
    • Connectors fetch live data
  4. Response flows back with full provenance and citations

Quick Example

Here’s a minimal example to give you a taste:

from contextrouter.cortex.runners import ChatRunner
from contextrouter.core import get_core_config


# Load configuration from settings.toml
config = get_core_config()


# Create a chat runner
runner = ChatRunner(config)


# Stream a response
async for event in runner.stream("What is RAG and why is it useful?"):
    if hasattr(event, 'content'):
        print(event.content, end="", flush=True)

That’s it! The runner handles:

  • Intent detection
  • Deciding if retrieval is needed
  • Searching your knowledge base
  • Generating a response with citations
  • Suggesting follow-up questions

Next Steps

Ready to dive in? Here’s your path:

  1. Installation — Get ContextRouter installed
  2. Quick Start — Build your first agent
  3. Core Concepts — Understand the architecture
  4. Models — Configure your LLM provider

Requirements

  • Python 3.11+
  • pip or uv for package management
  • At least one LLM provider configured (Vertex AI, OpenAI, or local)