Artificial intelligence

A useful RAG is 80% about data and evaluation, not the model. Concretely: a versioned ingestion pipeline that splits documents along their structure (headings, sections, tables) and keeps metadata (source, date, access rights); hybrid retrieval (vectors and keywords) followed by reranking; an evaluation set of 50 to 200 real questions with expected answers, replayed on every change; answers that cite their sources and can say “I don’t know”. Access rights are enforced at retrieval time, never delegated to the model.

If the path of the task is known in advance (extract, classify, summarise, route), a workflow with a few model calls is more reliable, cheaper and testable. An agent that decides its own steps is justified when the path varies with every case and errors are recoverable (human validation, reversible actions). We almost always start with the workflow and add autonomy where it proves useful.

With an evaluation that looks like software testing: a set of representative cases built with business teams, explicit criteria (accuracy, faithfulness to sources, format, justified refusal), a score computed automatically on every change of prompt, model or data, then a sample reviewed by humans. In production, user feedback and failure cases feed that set continuously.

Yes, at several levels: models served in your own cloud region (Amazon Bedrock, Azure OpenAI, Vertex AI) with no training on your data; open models (Mistral, Llama, Qwen) deployed in your infrastructure; or sensitive data filtered before the call. The right level depends on your data classification and the operating cost you accept. We write it down explicitly during scoping.

Three items: model calls (predictable once volume is known, and reducible through caching, a smaller model for simple steps and short prompts), the retrieval and ingestion infrastructure, and above all the human time spent on evaluation and data upkeep. A pilot measures all three on your real volumes before any commitment.

Rarely first. A good prompt, examples and a RAG cover most needs. Post-training becomes relevant when the output format is tightly constrained, when volume justifies a smaller and cheaper model, or when the domain has vocabulary that general models handle poorly. The decision rests on an evaluation, not an intuition.