Engineering notes / 03 decisions
Decision Log
Every architecture carries a tradeoff. These notes connect a project's constraints to the shape of its implementation, and the costs that come with it.
Based on the architectures documented in my project overviews. Alternatives describe other possible approaches; they are not a record of benchmarked comparisons.
Behind the choices(3)
Expand a decision to follow its constraint, alternatives, choice, and tradeoff.
CONSTRAINT
A question about a legal PDF may paraphrase an obligation while also naming a precise clause or legal term. The retrieval layer needs to surface useful passages for both kinds of query before the answer can cite them.
ALTERNATIVES
- Dense retrieval alone
- Captures semantic similarity, but similarity does not always preserve the importance of a specific term or clause identifier.
- Sparse retrieval alone
- Emphasizes term matches, but can miss a relevant passage when the question expresses the same idea in different words.
CHOICE
DocsAI stores OpenAI dense embeddings and SPLADE sparse vectors in Qdrant and combines the retrieval paths. A Cohere reranking step then orders the candidate passages before they inform the generated answer.
TRADEOFF
Two representations add ingestion work and storage. Reranking introduces another service call, with additional latency and cost. The extra stages also mean more configuration to inspect when an irrelevant passage reaches the answer.
WHEN TO REVISIT
Compare dense-only, sparse-only, and hybrid retrieval on the same representative questions. Check passage relevance and citation support alongside latency and cost before deciding whether every query needs the full pipeline.