Your documents.
Your database.
Grounded answers.
Point docpipe at your files and your Postgres. Vectors stay in your database; each app passes its own connection_string. Pick a runtime preset (fast, balanced, quality, agents) or wire explicit parsers, chunkers, and rerankers. Use the SDK locally, run docpipe serve as a shared internal API, or pull a profile-tagged Docker image.
$ pip install "docpipe-sdk[profile-balanced]"Quickstart
Profiles, presets, CLI, or Docker — shared internal API with per-app Postgres.
# Install profile (recommended)
$ pip install "docpipe-sdk[profile-balanced]"
# Parse a document
$ docpipe parse invoice.pdf --format markdown
# Discover plugins & presets
$ docpipe plugins list
$ docpipe profiles list
$ docpipe resolve invoice.pdf --goal ingest
# Ingest into your vector DB (each app passes its own --db URL)
$ docpipe ingest report.pdf \
--db "postgresql://..." \
--table docs \
--embedding-provider openai \
--embedding-model text-embedding-3-small \
--incremental
# Start API server ([server] for /admin + Alembic; [observability] for OTEL)
$ docpipe serve --port 8000
# curl -u admin:pass http://localhost:8000/profiles
# curl -u admin:pass http://localhost:8000/admin
# curl http://localhost:8000/metricsCapabilities at a glance
Parse → ingest → RAG around a shared service core — plugins, presets, pgvector, and observability on the outer ring. Tap a node to jump to details.
Orbit paused · tap wheel to spin · tap a node to explore
Composable Pipelines
Eight workflows — four composable stages plus extract, full chain, shared-service deployment, and agent/MCP integration. Use independently or chain end-to-end.
Documents
PDF, DOCX, images...Parse
MarkItDown · Docling · autoExtract
LangExtract · LangChainIngest
your pgvector · presetsRAG Query
6 strategies · agentsObserve
OTEL · /health · metricsDocuments
PDF, DOCX, images...Parse
MarkItDown · Docling · autoExtract
LangExtract · LangChainIngest
your pgvector · presetsRAG Query
6 strategies · agentsObserve
OTEL · /health · metrics1. Parse Only
Convert PDFs, Office files, and images to markdown. Use parser=auto, markitdown, docling, or glm-ocr.
import docpipe
doc = docpipe.parse("report.pdf") # auto (tier balanced)
doc = docpipe.parse("memo.docx", parser="markitdown")
doc = docpipe.parse("scan.pdf", parser="glm-ocr")
print(doc.markdown)2. Extract Only (LangExtract)
Extract structured entities from any text with LLMs.
schema = docpipe.ExtractionSchema(
description="Extract people and ages",
model_id="gemini-2.5-flash",
)
results = docpipe.extract(text, schema)3. Parse + Extract
Full pipeline: document to structured data in one call.
result = docpipe.run(
"invoice.pdf", schema
)
print(result.extractions)4. Parse + Ingest
Chunk, embed, and store in your Postgres. Pass connection_string per call — docpipe does not centralize RAG data.
config = docpipe.IngestionConfig(
connection_string="postgresql://myapp:pass@db:5432/myapp",
table_name="assistant_docs",
embedding_provider="openai",
embedding_model="text-embedding-3-small",
)
docpipe.ingest("report.pdf", config=config)
# API: preset=balanced picks parser + chunker bundle
# POST /ingest/stream for SSE progress on long jobs5. Full Pipeline
Parse, extract, and ingest - all in one call.
result = docpipe.run(
"contract.pdf", schema,
ingestion_config=config,
)6. RAG Query
Ask questions against your ingested documents with grounded answers and source citations.
rag_cfg = docpipe.RAGConfig(
connection_string="postgresql://...",
table_name="docs",
embedding_provider="openai",
embedding_model="text-embedding-3-small",
llm_provider="openai",
llm_model="gpt-4o",
strategy="hyde",
)
result = docpipe.query(
"What is the invoice total?",
config=rag_cfg,
)
print(result.answer) # grounded answer with citations
print(result.sources) # ["invoice.pdf"]
print(result.usage) # TokenUsage when provider reports counts7. Shared internal API
Deploy once on Kubernetes or Docker; multiple apps call the same docpipe with their own DB URLs and presets.
# Discover install profile + presets
curl -u admin:pass http://docpipe:8000/profiles
curl -u admin:pass http://docpipe:8000/plugins
# Ingest into *this app's* Postgres (not docpipe's)
curl -u admin:pass -X POST http://docpipe:8000/ingest \
-H "Content-Type: application/json" \
-d '{
"source": "s3://bucket/invoice.pdf",
"connection_string": "postgresql://delegate@delegate-db/delegate",
"table_name": "assistant_docs",
"embedding_provider": "openai",
"embedding_model": "text-embedding-3-small",
"preset": "balanced"
}'8. Agents & MCP
Tool-using RAG for agent frameworks. MCP endpoints expose parse and query tools.
# preset=agents on profile-agents image
curl -u admin:pass -X POST http://docpipe:8000/agents/query \
-H "Content-Type: application/json" \
-d '{"question":"...","preset":"agents",...}'
curl -u admin:pass http://docpipe:8000/mcp/tools
# POST /mcp/call — docpipe_parse, docpipe_rag_query6 Retrieval Strategies — Pick What Fits
Pick a retrieval strategy and optional reranker, or pass preset=balanced on the API to bundle chunking + rerank with the strategy. Stream via SSE; agents use preset=agents.
Pick your retrieval strategy
Six strategies — hover or tap a node to see when to use it.
RAG
auto
LLM classifies the question and dispatches to the optimal strategy automatically.
When to use: Mixed workloads, zero tuning{ "query": "What is the invoice total?", "strategy": "hyde" }
Standard cosine similarity search. Fast, reliable baseline for well-formed queries.
Best for: well-formed queries, fast responsesLLM generates a hypothetical answer first, embeds it, then retrieves real matching docs. Highest accuracy in benchmarks.
Best for: complex / technical queriesExpands your query into N variants via LLM, retrieves for each, then deduplicates and ranks results.
Best for: vague or short queriesRetrieves seed chunks, then expands context by fetching additional chunks from the same source documents.
Best for: long documents, context coherenceCombines dense vector search with sparse BM25 keyword retrieval via EnsembleRetriever. Best of both worlds.
Best for: exact terms, proper nouns, IDsLLM classifies your question and dispatches to the optimal strategy automatically. Best accuracy with zero tuning.
Best for: mixed workloads, unknown query types# Runtime presets bundle parser, chunker, reranker
# fast · balanced · quality · agents
curl -u admin:pass http://localhost:8000/profiles
# Pass preset on /ingest, /rag/query, /agents/query:
{"preset": "quality", "connection_string": "postgresql://...", ...}rag_cfg = docpipe.RAGConfig(
...,
strategy="hyde",
reranker="flashrank", # or "bge" on profile-quality
rerank_top_n=5,
)
# preset=quality sets semantic chunks + BGE automaticallyclass Invoice(BaseModel):
total: float
currency: str
result = docpipe.query(
"What is the total?",
config=docpipe.RAGConfig(
..., output_model=Invoice
),
)
invoice = result.structured
# Invoice(total=4250.0, currency='USD')# Stream tokens via SDK or POST /rag/stream (SSE)
for token in docpipe.stream_query(
"What is the total?",
config=rag_config, # stream=True
):
print(token, end="", flush=True)
# Before data: [DONE], optional metadata event:
# event: metadata
# data: {"type":"usage","usage":{"input_tokens":123,...}}What docpipe does
A composable document-to-RAG stack you embed in apps or run as a shared service — with explicit plugins, presets, and per-tenant Postgres isolation.
Your data stays yours
Vectors live in the Postgres you pass on each /ingest call. One shared docpipe instance can serve many apps — each with its own connection_string and table.
Profiles & presets
Install profile-slim … profile-agents (or Docker :balanced). Runtime presets fast/balanced/quality/agents pick parser, chunker, and reranker bundles per request.
Plugin registry
Parsers, chunkers, rerankers, and evaluators as plugins. GET /plugins and /profiles for discovery; operator allowlists via DOCPIPE_ENABLED_* env vars.
Multi-parser ingest
MarkItDown for speed, Docling for layout, GLM-OCR for scans, plus mineru/paddleocr on GPU profiles. parser=auto routes by tier.
6 RAG strategies
naive, HyDE, multi-query, parent-document, hybrid BM25+vector, and auto. FlashRank or BGE rerank; SSE streaming and token usage on responses.
SDK · CLI · HTTP API
import docpipe locally, script with the CLI, or deploy docpipe serve. SSE on /ingest/stream and /rag/stream; MCP tools for agent frameworks.
pgvector or turbovec
Production default is pgvector in your Postgres. Optional turbovec extra writes compressed on-disk indices for local or air-gapped prototypes.
Extract & agents
LangExtract and LangChain structured output for entities. preset=agents enables /agents/query with AutoGen tool-using RAG.
Observability
Prometheus /metrics, OTLP traces, JSON logs, enriched /health. Optional control-plane DB with /admin audit UI for operators.
Evaluation
Builtin metrics plus RAGAS and deepeval plugins. POST /evaluate/run and docpipe evaluate run for hit rate, faithfulness, and regression CI.
Pipeline modes — one card, four shapes
Parse pipeline
Docling or GLM-OCR → markdown
PDF → answer
One file → markdown → chunks in your database → cited answer.
invoice.pdf Parsed with parser=auto → Docling. Tables and headers preserved as markdown.