Architecture Guide — LLM-Powered Knowledge Graphs
A complete implementation reference for building domain-specific GraphRAG systems.
Architecture Overview
This system is a GraphRAG stack: it combines dense and sparse retrieval with a structured knowledge graph so the model can answer both direct lookup questions and multi-hop domain reasoning tasks.
FastAPI API
The API layer is the orchestration entrypoint. It validates requests, initializes the streaming channel, and coordinates routing, retrieval, and synthesis without embedding frontend concerns into backend logic.
Query Router
The router converts an open-ended question into a retrieval plan. It identifies the query category, extracts entities when possible, and decides how much to rely on graph traversal, lexical search, and semantic recall.
Neo4j + Weaviate
Neo4j stores typed entities and relations for explainable reasoning, while Weaviate stores chunk embeddings for broad semantic coverage. Together they balance structured precision and unstructured recall.
Fusion Retriever
Fusion retrieval collects evidence from dense search, BM25, and graph expansion in parallel. It combines these signals into a unified candidate pool so strong evidence is not missed just because it came from a different retrieval channel.
Reranker
The reranker is the precision gate. It uses a cross-encoder to rescore retrieved passages and graph facts against the full user question, lifting the most actionable evidence to the top of the context window.
Claude Synthesis
The final model stage turns ranked evidence into a concise grounded answer. It cites retrieved facts in the prompt context, produces a readable engineering response, and streams tokens immediately back to the client.
Why this architecture works: pure vector search is strong for semantic similarity, but it struggles with version differences, entity relationships, and multi-step fault chains. Adding a knowledge graph makes the retrieval layer reasoning-aware before the LLM starts writing.
Data Pipeline
The corpus pipeline transforms raw domain materials into clean, deduplicated, chunked retrieval units that can feed both the vector store and the graph extraction workflow.
Four-tier corpus design
- Structured code: parsable code, configs, command snippets, and structured text that exposes directly executable system knowledge.
- PDK artifacts: process documentation, rule references, versioned notes, and domain-specific constraint materials.
- Tool docs: command help, usage docs, flow guidance, and engineering references that explain expected behavior.
- Synthetic Q&A: generated training-style pairs that convert dense documentation into user-like question formats for better retrieval coverage.
Deduplication and normalization
Before embedding, the pipeline performs MinHash-based near-duplicate detection to suppress repeated content across overlapping sources. This reduces embedding waste, lowers retrieval noise, and prevents the ranker from repeatedly surfacing the same idea from slightly different files.
Documents are also normalized into a shared schema with source IDs, version tags, document families, and chunk lineage metadata. That shared metadata becomes critical later for graph extraction, debugging, and evaluation slicing.
| Stage | Purpose | Implementation Notes |
|---|---|---|
| Ingest | Load heterogeneous domain material into a unified corpus object | Preserve source type, title, version tag, and lineage metadata from day one. |
| Deduplicate | Remove exact and near duplicates using MinHash | Keep a canonical record and track which sources collapsed into it. |
| Chunk | Split long material into retrieval-sized segments | Use structure-aware boundaries instead of raw fixed-size slicing where possible. |
| Embed | Generate dense vectors with Voyage AI | Embed both retrieval chunks and synthetic Q&A prompts if they support search quality. |
| Load | Store vectors and metadata in Weaviate | Index chunk text, source tags, categories, versions, and document family IDs. |
Chunking strategy
Chunk around natural boundaries such as headings, rule blocks, error/fix pairs, config sections, and API examples. Add slight overlap between adjacent chunks so short dependency chains survive splitting, but keep chunks focused enough that reranking can still separate signal from context padding.
Embedding and vector loading
Use Voyage AI embeddings for semantic retrieval, then push each chunk plus metadata into Weaviate. Store enough metadata to support post-retrieval filtering by source family, version, and document role without rebuilding the index.
for doc in load_corpus():
normalized = normalize_document(doc)
canonical = minhash_deduplicate(normalized)
chunks = chunk_document(canonical, mode="structure_aware")
for chunk in chunks:
vector = voyage_embed(chunk["text"])
weaviate_upsert({
"text": chunk["text"],
"vector": vector,
"source_type": chunk["source_type"],
"version": chunk["version"],
"lineage_id": chunk["lineage_id"]
})Knowledge Graph
The graph stores domain knowledge as entities and typed relationships, enabling version-aware retrieval and multi-hop reasoning that dense search alone cannot recover reliably.
Ontology design
The ontology should reflect the way experts talk about the domain. Typical node types include Design, Version, Error, Fix, DRCRule, Constraint, FlowStage, Tool, Metric, and Artifact.
Representative relation types include HAS_ERROR, RESOLVED_BY, AFFECTS_STAGE, USES_TOOL, VIOLATES_RULE, GENERATES_CONSTRAINT, MEASURES, REFERENCES, RUNS_ON, and DIVERGES_FROM.
Triple extraction
Triple extraction converts unstructured text into graph facts by identifying entities, linking aliases, and assigning typed relations. The extraction pass should preserve provenance so every node or edge can be traced back to the document chunk that created it.
In practice, this lets you ground answers with graph-supported evidence instead of treating graph facts as opaque derived metadata.
| Capability | How the graph helps | Why it matters |
|---|---|---|
| Version awareness | Relations like RUNS_ON and DIVERGES_FROM connect tools, designs, rules, and fixes to specific versions. | Many engineering failures are not generic; they only appear after a version change. |
| Entity-centric retrieval | Detected entities trigger targeted subgraph expansion around the most relevant nodes. | Retrieval becomes context-aware before prompting the model. |
| Multi-hop reasoning | Traversal can follow chains like Design → Error → Fix → Version. | This supports diagnosis and comparison questions that span multiple documents. |
MATCH (d:Design)-[:HAS_ERROR]->(e:Error)
-[:RESOLVED_BY]->(f:Fix)
-[:RUNS_ON]->(v:Version)
WHERE d.name = $design_name
RETURN e, f, v
ORDER BY v.release_date DESCImplementation principle: make version edges first-class citizens. In operational domains, the difference between “works” and “fails” is often a branch, tool release, or rule-set divergence rather than the design itself.
Hybrid Retrieval (GraphRAG)
Hybrid retrieval combines semantic search, lexical search, and graph expansion so the system can answer both fuzzy conceptual questions and exact engineering lookups with the same API.
Query classification
The router classifies each question into one of five task families: error_diagnosis, rtl_qa, drc_rule_lookup, constraint_generation, and cross_tool_knowledge. This category determines which retrieval channels should dominate and how the final answer should be structured.
Retrieval channels
Dense retrieval uses Voyage embeddings against Weaviate, sparse retrieval uses BM25, and graph expansion queries Neo4j around detected entities. The fusion layer merges these candidate sets, then a cross-encoder reranker picks the most useful context for synthesis.
| Category | Primary evidence | Typical output shape |
|---|---|---|
| error_diagnosis | Graph neighborhoods + relevant chunks | Cause, affected version context, remediation path |
| rtl_qa | Vector retrieval + LoRA fallback | RTL design guidance, coding patterns, best practices |
| drc_rule_lookup | Lexical matches + structured rule nodes | Rule meaning, trigger conditions, design implications |
| constraint_generation | Examples, patterns, and graph-linked artifacts | Suggested constraint template with rationale |
| cross_tool_knowledge | GraphRAG-only retrieval across tool boundaries | Cross-tool interactions, compatibility, workflow guidance |
category = classify_query(query)
entities = detect_entities(query)
dense_hits = weaviate_search(query)
sparse_hits = bm25_search(query)
graph_hits = neo4j_expand(entities, category)
candidates = fuse_results(dense_hits, sparse_hits, graph_hits)
ranked = cross_encoder_rerank(query, candidates)
context = ranked[:k]
answer = claude_synthesize(query, category, context)Key design insight: graph expansion is not a replacement for vector retrieval. It is a targeted precision boost that recovers entity relationships and version-specific paths the embedding model may never place close together in vector space.
Streaming API
The backend streams response progress immediately using Server-Sent Events so the frontend can display metadata, partial text, and completion stats in real time.
FastAPI + SSE
Use FastAPI to expose an HTTP endpoint that returns an text/event-stream response. SSE is a good fit when the client only needs one-way live updates from the server during generation.
Event model
Emit three event types: meta for instant metadata, token for incremental answer text, and done for completion metrics such as total latency and token counts.
Reverse proxy behavior
When deployed behind Nginx, set proxy_buffering off so clients receive tokens as soon as they are generated. Pair the API with HTTPS via Let’s Encrypt and configure CORS for the expected frontend origins.
event: meta
data: {"category":"error_diagnosis","graph_facts":6,"chunks":9}
event: token
data: {"text":"The likely root cause is ..."}
event: done
data: {"latency_ms":4200,"prompt_tokens":3780,"answer_tokens":612}Nginx deployment note
Ensure the reverse proxy forwards long-lived responses correctly and keeps buffering disabled for the streaming endpoint. Without that setting, the user sees a delayed block of text instead of a live answer.
Frontend integration
The frontend only needs a lightweight event parser and a few UI states: pending, streaming, complete, and error. This keeps the dashboard portable enough to host separately from the API.
Frontend Dashboard
The frontend is intentionally simple: one polished single-page HTML application that visualizes the system, benchmarks, and live demo without introducing heavy client-side dependencies.
Information architecture
The dashboard is a single-page HTML interface with four tabs: Overview, Evaluation, Try Copilot, and Reproducibility. That split lets one file serve as a product page, experiment report, demo surface, and replication guide.
Interactive answer experience
The demo tab includes an SSE parser, a blinking cursor during generation, seed query pills for rapid testing, a debug strip, and a copy-answer button. The interface is minimal, but it exposes enough retrieval telemetry to make system behavior legible.
| UI element | Purpose | Recommended behavior |
|---|---|---|
| Overview tab | Summarize architecture, findings, and cost story | Lead with the core claim and the end-to-end system diagram. |
| Evaluation tab | Show benchmark results and sample cases | Use charts, tables, and expandable examples for depth. |
| Try Copilot tab | Interactive live query experience | Support streaming updates, seed prompts, and copy answer. |
| Reproducibility tab | Explain data, methods, and deployment | Keep this precise enough that another builder can recreate the stack. |
| Debug strip | Expose system internals during demo | Show Category, Graph facts, Chunks, Prompt tokens, Answer tokens, and Latency. |
const source = new EventSource(endpoint);
source.addEventListener("meta", evt => {
const meta = JSON.parse(evt.data);
renderDebugStrip(meta);
});
source.addEventListener("token", evt => {
const payload = JSON.parse(evt.data);
answerBuffer += payload.text;
renderStreamingAnswer(answerBuffer, true);
});
source.addEventListener("done", evt => {
const done = JSON.parse(evt.data);
renderFinalStats(done);
source.close();
});Deployment advantage: a single-file dashboard is easy to host, easy to archive with a paper or report, and easy to mirror on static hosting without a frontend build system.
Fine-Tuning (QLoRA)
Fine-tuning was used as a comparison track rather than the production serving stack, making it possible to benchmark retrieval-heavy prompting against a compact adapted open-weight model.
Training setup
- Base model: Mistral-7B.
- Method: QLoRA with 4-bit quantization to reduce memory cost during training.
- Data: synthetic Q&A pairs generated from the domain corpus.
- Output: adapter weights merged for deployment-style evaluation.
Role in the overall system
The fine-tuned model was useful for controlled evaluation and ablation-style comparison. Production answer generation still used the Claude API because the retrieved-context synthesis quality remained stronger and more reliable in the live system.
base_model = "Mistral-7B"
train_set = build_synthetic_qa_pairs(corpus)
qlora_adapter = train_qlora(base_model, train_set, quantization="4-bit")
merged_model = merge_adapter(base_model, qlora_adapter)
results = evaluate_on_edabench(merged_model)Evaluation (EDABench)
The evaluation framework measures whether the architecture improves practical answer quality, not just retrieval accuracy. The benchmark mixes real engineering cases with structured category coverage.
Benchmark composition
EDABench contains 120 samples distributed across the five main query categories. Seven of those are verified seed cases drawn from real ORFS experiments, anchoring the benchmark in practical failure and optimization scenarios rather than synthetic prompts alone.
Judge-based scoring
Answers are scored on factual accuracy, completeness, actionability, and specificity. This makes the metric closer to what an engineer values when using a system for troubleshooting or design guidance.
| Key result | Value | Interpretation |
|---|---|---|
| Quality improvement over standalone LLM | 6.7× | Hybrid retrieval plus graph grounding materially improves useful answer quality. |
| Graph hit rate | 67.5% | Most benchmark questions benefited from graph-retrieved evidence. |
| Mean answer quality | 0.482 | The stack achieved consistent mid-to-high usefulness across mixed categories. |
Evaluation takeaway: the gain came from architecture, not just prompting. Routing, graph expansion, fusion retrieval, and reranking together changed answer quality by an order of magnitude relative to a standalone model baseline.
Deployment Checklist
The deployment stack separates the static frontend from the streaming backend, keeping hosting simple while still supporting real-time inference and reproducible operations.
Backend deployment
- Provision an Ubuntu VPS for the API service.
- Create a Python virtual environment and install backend dependencies.
- Run the API through a
systemdservice for automatic startup and restarts. - Place Nginx in front of the API as the reverse proxy.
- Set
proxy_buffering offon the streaming route. - Issue an SSL certificate with Let’s Encrypt and automate renewals with certbot.
- Store secrets in environment variables rather than in source files.
- Include general input validation and abuse prevention controls.
Frontend and operations
- Host the single-page frontend on GitHub Pages.
- Point the frontend to the API domain over HTTPS.
- Configure CORS for the allowed frontend origins.
- Add a GitHub Actions keepalive ping if the backend platform benefits from periodic traffic.
- Document environment setup, data loading, and benchmark steps for reproducibility.
- Keep the dashboard self-contained so the public demo is easy to redeploy.
# Backend service lifecycle
1. create_virtualenv()
2. install_backend_dependencies()
3. configure_environment_variables()
4. register_systemd_service()
5. place_nginx_reverse_proxy_in_front()
6. enable_https_with_lets_encrypt()
7. publish_frontend_to_github_pages()Cost Analysis
A key result of the project is that a polished domain-specific GraphRAG system can be built and operated at modest cost when storage, serving, and evaluation are engineered carefully.
| Cost component | Value | Notes |
|---|---|---|
| Total build cost | $240 over 16 weeks | Includes experimentation, evaluation iteration, and deployment work. |
| Per-query inference cost | ~$0.01 | Approximate input and output token cost on Claude Sonnet. |
| Monthly infrastructure | ~$3 VPS | Low-cost backend hosting for the streaming API. |
| Vector store | Weaviate sandbox | Free tier suitable for early-stage experimentation. |
| Graph database | Neo4j Aura | Free tier sufficient for this graph scale. |
Budget lesson: most of the value came from architecture and evaluation discipline rather than expensive infrastructure. Thoughtful corpus design and retrieval strategy mattered more than larger compute budgets.
Lessons Learned
The strongest outcomes came from decisions that improved groundedness, observability, and deployment simplicity rather than from adding more model complexity.
Version-aware graph edges
Version-aware edges were essential for error diagnosis. They let the system connect symptoms, fixes, and behavioral changes to the correct release context rather than blending incompatible evidence.
Hybrid retrieval wins
Hybrid retrieval outperformed pure vector search by 139% in answer quality (0.482 vs 0.202). The graph made retrieval smarter, while dense and sparse channels preserved broad recall.
Streaming changes perception
SSE streaming improved perceived latency dramatically, with users seeing the first words in roughly 2–3 seconds instead of waiting around 15 seconds for a full response block.
Secret scanning matters
Pre-commit secret scanning is worth adopting early. It lowers the chance that operational credentials leak during rapid iteration, especially when code and deployment config evolve together.
Single-file frontend
A single-file HTML dashboard simplified deployment, review, and archiving. It also made the project easy to host on static infrastructure with almost no frontend operational overhead.
Evaluation drives design
Clear benchmark categories and judge criteria made architectural choices easier. Without a targeted benchmark, it is difficult to tell whether a retrieval change actually improves engineering usefulness.
Next Project Template
This architecture is portable. To adapt it to a new domain, keep the runtime skeleton and evaluation discipline, then swap in new corpus sources, ontology decisions, and benchmark tasks.
What to replace
- Corpus sources and ingest connectors
- Knowledge graph ontology and entity aliases
- Query categories used by the router
- Seed benchmarks and judge rubrics tied to the new domain
- Domain-specific answer templates and prompt framing
What to keep
- FastAPI + SSE API structure
- Hybrid retrieval pipeline with dense, sparse, and graph channels
- Fusion and reranking stage
- Evaluation framework and benchmark-first iteration loop
- Static frontend deployment and lightweight backend operations