Engineering Journal: Building Charlie’s Retrieval Foundation (RAG)

Stream A (Charlie Platform) | Sprint 5

Sprint 5 in the Stream „Charlie Platform“ marked Charlie’s transition from a conversational assistant into an AI platform capable of processing external knowledge. Rather than focusing on adding features, the sprint focused on designing a clean retrieval architecture that will support Retrieval-Augmented Generation (RAG) in future iterations. The most significant outcome was not the code itself but the architectural decisions that established clear responsibilities between document loading, chunking, embedding, storage and retrieval.

The Engineering Challenge

The initial question was deceptively simple: „How should Charlie process documents?“ The first ideas tended toward a single service responsible for loading, chunking and embedding documents. During the sprint, repeated architectural discussions showed that this approach would create a monolithic component with mixed responsibilities and poor extensibility.

The Result

The final architecture consists of a pipeline of focused components: DocumentLoader → Document → Chunker → EmbeddingService → Embedding → VectorStore → Retriever. Each component owns a single responsibility and communicates through well-defined interfaces.

Architecture Evolution

Documents
    │
DocumentLoader
    │
Document
    │
Chunker
    │
Chunks
    │
EmbeddingService
    │
Embeddings
    │
VectorStore
    │
Retriever
    │
Relevant Chunks

Engineering Journey

The defining moments of Sprint 5 happened during architectural discussions rather than implementation. The team repeatedly questioned whether objects should remain primitive types or become explicit domain models. Similar discussions led to separating orchestration from storage and to postponing a production vector database until the surrounding abstractions had stabilized.

Behind the Decisions

Decision 1 — Document instead of String

Initial Idea

Pass plain text through the pipeline.

Problem

Strings cannot carry metadata or evolve without changing multiple interfaces. Though, during retrieval, the text alone is often not sufficient. Metadata such as the source, title, document type, or timestamps provide essential context for retrieval, filtering, ranking, and source attribution. A dedicated Document object preserves this context throughout the pipeline.

Alternatives Considered

• Raw strings
• Dictionaries
• Rich domain model

Final Decision

Introduce a dedicated Document domain model.

Engineering Reasoning

Domain models create stable boundaries and allow metadata, provenance and future extensions without breaking downstream services.

Impact on Charlie

The pipeline now processes semantic objects rather than primitive values.

Decision 2 — Separate Chunking

Chunking became its own component instead of being embedded in the document loader. During the design of the ingestion pipeline, it became clear that there is no single „correct“ chunking strategy for RAG. Approaches range from simple fixed-size or token-based chunking to overlapping chunks, semantic chunking, hierarchical chunking, or strategies that enrich each chunk with additional context such as document summaries. Keeping chunking as an independent component allows different strategies to be evaluated, compared, and replaced without affecting the rest of the ingestion pipeline.

Decision 3 — EmbeddingService Owns Orchestration

Embedding generation belongs in a dedicated service, while the VectorStore focuses exclusively on persistence and similarity search.

• The EmbeddingService is responsible for orchestrating the embedding workflow. It receives the generated chunks, iterates over them, creates embeddings, and persists the resulting vector records through the VectorStore.
• The VectorStore is intentionally limited to persistence and similarity search, without knowledge of documents, chunking, or embedding generation.

This separation keeps infrastructure concerns isolated and allows embedding providers and vector store implementations to be replaced independently.

Exemplary flow:

Document
    │
    ▼
Chunker
    │
    ▼
List[Chunk]
    │
    ▼
EmbeddingService
        │
        ├─ iterate over chunks
        ├─ generate embedding for each chunk
        ├─ create vector records
        └─ persist them via VectorStore
                    │
                    ▼
              VectorStore

Representative Code

documents = loader.load_directory(path)
chunks = chunker.chunk_documents(documents)
embeddings = embedding_service.embed_all(chunks)
vector_store.store(embeddings)
results = retriever.retrieve(query)

Decision 4 — Delay the Vector Database

Instead of introducing Pinecone or another database immediately, the sprint focused on stable abstractions first. This reduced coupling and ensured that future storage technologies can be introduced without redesigning the application.

Trade-offs

• More classes in exchange for cleaner responsibilities.
• Slightly higher initial complexity in exchange for long-term extensibility.

Concepts Learned

• Rich domain models outperform primitive types for evolving AI systems.
• Dependency Injection simplifies experimentation.
• Orchestration and implementation should remain separate.
• Architecture should anticipate change without implementing it prematurely.

Engineering Insights

• Good AI architecture emerges through discussion as much as through coding.
• Explicit boundaries make future experimentation inexpensive.
• Documentation preserves architectural reasoning that source code alone cannot capture.
• Clean abstractions reduce vendor lock-in.

Lessons Learned

The sprint demonstrated that investing time in architectural conversations before expanding functionality produces a cleaner, more maintainable platform. The resulting design is ready for future RAG capabilities without major refactoring.

Future Evolution

• Persistent vector databases
• Multiple embedding providers
• Advanced chunking strategies
• Complete Retrieval-Augmented Generation

References

See Sprint Documentation, Sprint Planning and Architecture Documentation.

Conclusion

Sprint 5 was less about adding features than about establishing architectural foundations. By transforming document processing into a pipeline of focused components, Charlie became prepared for scalable retrieval, future RAG workflows and continued experimentation while preserving a clean architecture.