TL;DR : When every team is building RAG applications, is assembling your own retrieval stack always the right call? A practitioner’s guide.
With the rise of AI, many organizations and business units within have started realizing the need for intelligent search on their stored documents. Maybe to create a customer facing question and answer application, internal policy tools, or conversational chat applications with knowledge derived from organization’s documents. The purpose is clear, retrieve the relevant content, feed it to the LLM, and generate an answer.
But how you build that retrieval layer is where the decision matters, and where the decisions get expensive.
The short answer is if you need to validate search in weeks, Kendra wins. If you need cost efficiency, explainability, or custom ranking at scale, build the RAG pipeline. This article provides guidance on what to evaluate before making that decision.
This article covers two distinct paths and provides a concrete decision framework based on real implementation experience. This article refers to AWS services for demonstration purposes, however this architecture decision is cloud platform agnostic. Azure’s equivalent is Azure AI Search (formerly Cognitive Search), and Google offers Vertex AI Search. All three follow the same managed-search philosophy, and the tradeoffs discussed in this article apply broadly across all three, with Kendra used here as the primary reference.
Path A – Build a RAG pipeline from scratch.
Path B – Use Amazon Kendra
Path A – Build a RAG Pipeline
When you decide to build a RAG, typical architecture in AWS would look something like this –
A tailored RAG pipeline on AWS involves five distinct engineering concerns. Document ingestion and storage in S3, content chunking via Lambda, embedding generation using a model like Amazon Titan, vector indexing in OpenSearch, and finally semantic similarity search during retrieval where the user’s phrase is embedded and matched against the stored index to fetch relevant content. Each of these is a component you design, build, and operate independently.
[Text Wrapping Break]Apart from the mentioned components there will be a requirement to create multiple other Lambda functions to make the API calls to these services, prompt engineering, and also the entire thing needs to be orchestrated with something like AWS Step functions or Apache Airflow. Additionally there will be a requirement of establishing security control, monitoring, application metadata management, and building DevOPS pipeline for CI/CD.
This would also require efforts to choose correct technologies, and correct models to ensure that the solution meets performance and budgetary needs.
Equivalent components exist on Azure (Azure Functions, Azure OpenAI, Azure AI Search, Logic Apps) and GCP (Cloud Functions, Vertex AI Embeddings, Vertex AI Vector Search, Cloud Composer). The architectural concerns are identical across platforms only the managed service names differ.
Components you build:
| Layer | What You Build | Tools/Services |
| Ingestion | Document parsing, cleaning, chunking | Lambda, Textract, custom code |
| Embedding | Vector generation for each chunk | Amazon Titan Embeddings, Cohere, OpenAI |
| Storage | Index design, shard strategy, k-NN config | Amazon OpenSearch (k-NN plugin, HNSW) |
| Retrieval | Hybrid search logic, scoring, filtering | OpenSearch queries + custom scoring |
| Generation | Prompt engineering, context assembly | Amazon Bedrock (Claude), LangChain |
| Operations | Monitoring, Orchestration etc. | Amazon CloudWatch, AWS Step Function/Airflow |
Path B – Amazon Kendra
With Kendra this architecture looks really simple. Kendra collapses the entire data engineering as a managed service.
What Kendra handles for us:
- Document parsing/chunking from PDF, Word, HTML, PPT with no need to code
- Out of the box, zero code ML-based relevance ranking
- Support for natural language which interpret intent, not just keywords
- Search results returns precise passages, not just document links
- Hybrid search can blend keyword with semantic automatically
- 40+ native data connectors with incremental sync to keep the index hydrated
What Kendra does not do –
It does not provide a conversational interface which the RAG architecture provides. For this you need to build an application layer which leverages LLMs to process Kendra’s search output and creates a conversational interface.
Compare and Contrast
After understanding both the options let’s compare these solutions against some of the design lenses.
-
The Architectural Decision Making
| Dimension | RAG Pipeline | Kendra |
| Time to working search | 6–10 weeks | Days |
| Control over retrieval | Unlimited | Limited to Kendra’s model |
| Operational overhead | High (cluster, embeddings, reindexing) | Near-zero |
| Architectural choices | You choose: vector Database, embedding model, LLM, reranker | Opinionated: Kendra’s stack |
| Failure modes | Many (each component can fail independently) | Few (managed service, built-in high availability) |
| Vendor lock-in | Low (swap components freely) | High (Kendra-specific API) |
The above table shows a build vs buy comparison. With the RAG pipeline you build everything from scratch. Whereas in the case of Kendra you get similar features out of the box.
-
Search Capabilities: What Actually Matters in Practice
Where Kendra Wins Immediately
| Capability | RAG Pipeline | Kendra | Impact |
| Answer extraction | Requires development of LLM prompt, context window management | Built-in : returns answers with confidence | Weeks of prompt engineering saved |
| Document parsing | Build it (Textract/Tika + error handling) | Built-in (PDF, Word, HTML, PPT) | 2–3 weeks saved |
| Hybrid search | Build it (BM25 + vector + RRF fusion) | Automatic | 2–3 weeks of tuning saved |
| Multi-language | Configure analyzers + translate + re-embed | Configure language setting | Days saved |
| Incremental sync | Build change detection logic | Native connector feature | Ongoing maintenance saved |
Where Custom RAG Wins
| Capability | RAG Pipeline | Kendra | Impact |
| Mathematical scoring | Full control (combine signals with weights) | Categorical only (VERY_HIGH → LOW) | Critical if business need numeric scores/percentage match |
| Custom ranking functions | script_score, function_score, decay functions | Limited to relevance tuning boosting | Critical for credibility / recency weighting |
| Embedding model choice | Pick best model for your domain | Opaque internal model | Matters for specialized domains |
| Chunk size optimization | Tune for your document types | Fixed (Kendra decides) | Impacts answer quality |
| Analytics on search data | Full aggregation engine | Not supported | Critical if search doubles as analytics |
| Streaming ingestion | Real-time (seconds) | Batch (minutes to hours) | Critical for rapidly changing content |
-
Cost at Different Scales
| Dimension | Custom RAG (OpenSearch) | Kendra |
| Search infrastructure/month | ~$150–400 | ~$7,665 |
| Initial bulk embedding (70K docs) | ~$8–14 | Not needed |
| Ongoing embedding/month | ~$3–10 | Not needed |
| Ongoing ops/month (human cost) | ~$10,400–20,800 | ~$3,200 |
| POC scale (1K docs) | ~$78/month | ~$810/month (10×) |
| Production (70K docs) | ~$400/month | ~$7,665/month (19×) |
| Enterprise (1M+ docs) | ~$1,200–3,000/month | ~$15,000+/month (5–12×) |
Looking at the cost number it may appear that Kendra does not stand a chance in this competition. However Infrastructure cost is not the only factor, you need to account for the Build cost that adds up to this number. A cross-functional team (engineers, app developers, PM, DevOps, UX, QA) will typically cost ~$139K for an optimistic 6-week RAG build, rising to ~$278K for a realistic 12-week production build and ~$418K for a complex 18-week program.
Kendra’s equivalent build effort runs ~$32–96K depending on complexity, as work is largely configuration and integration with no embedding pipeline, cluster management, or scoring infrastructure to build.
On pure recurring infrastructure, RAG becomes cheaper than Kendra at production scale but when ongoing support is factored in (~$10–20K/month for RAG vs ~$3K/month for Kendra), the operational cost gap is significant and the break-even timeline extends well beyond the infrastructure savings alone.
Note – Validate AWS Service cost against current pricing before budgeting. Use AWS Cost calculator with correct region and usage. This calculation is based on 2025 pricing.
Azure AI Search and Vertex AI Search follow broadly similar pricing tiers to Kendra at enterprise scale. Validate against current pricing for your cloud provider before budgeting.
-
Responsible AI : who owns the risk?
| Dimension | RAG Pipeline (Build) | Kendra (Buy) | Matters Most For |
| Explainability | Full | Limited | Regulated industries, audits |
| Auditability | Complete chain | Input/output only | Compliance, legal review |
| Bias detection | Auditable | Black-box | Fairness requirements |
| Bias introduction risk | Higher (more custom code) | Lower (less surface area) | Teams without AI ethics expertise |
| Model governance | Full control | Cloud provider managed, no user intervention | Model risk management |
| Reproducibility | Deterministic | May drift with updates | Audit trails, regression testing |
| Hallucination safety | Requires guardrails | Inbuilt Safety | User-facing applications |
| Prompt injection | Attack surface exists, needs coding effort to prevent | No LLM = no risk | Public-facing search |
This is a metric where building RAG is a clear winner. In many regulated industries where explainability, auditability, fairness, and other safe AI have higher priority, building RAG is a much better option. If you’re optimizing for safety and simplicity with acceptable opacity, Kendra is acceptable.
-
The Decision Framework: Three Questions
In one of my engagements, I was architecting for a Regulated industry, during the POC phase I implemented the solution with Amazon Kendra, technical SMEs of customer side was happy with the outcome, however when we demoed it to business it faced outright rejection because we could not justify the result or sort order as that is not transparent or explainable with the Kendra. For Production implementation we had to build the custom RAG pipeline.
On another occasion I built the solution of a detailed FAQ platform for a ticket booking platform, the organization had a variety of FAQs and policy docs for different events, which they wanted to consolidate into a single retrieval platform. Due to the smaller size of the organization and the nature of the business they wanted to build the platform quickly. Needless to say, Amazon Kendra served the purpose well in that case.
After building search on both paths across multiple engagements, these are the questions that I ask before suggesting architecture for a search platform.
Question 1 – Timeline – How fast does this need to go-live?
- You have 2 weeks -> Go for Kendra
- You have a quarter to make this -> RAG Pipeline
- Long Term Platform investment with Agentic use cases in future -> RAG Pipeline
Question 2 – Result – What should your search look like?
- Just show the most relevant results : Kendra works perfectly, ML ranking is excellent out of the box
- We need to justify the search result, based on relevance, credibility, recency : Building RAG pipeline gives better control on explainability
Question 3 – Scale – How many documents and how many searches?
- < 50K documents, < 10K queries/day – Kendra is viable option, also saves the engineering cost
- >100K documents, high query volume, complex querying – RAG is better for long term, also cost effective
Conclusion:
At its core, this is a build vs. buy decision, and like most build vs. buy decisions, the right answer depends entirely on the current needs and future aspirations.
If you are early in validating whether intelligent search truly solves a problem for your organization, start with Kendra. Connect your S3 bucket, demo it to stakeholders, and productionalize for a few months or few key users. You will spend $500–$1,000 on infrastructure for the first month. Validate user engagement, and ask for feedback on what their search requirements actually are before starting a longer engineering program.
If that validation succeeds or you are hitting Kendra’s limits such as explainability requirements from a compliance team, some ranking needs that Kendra’s categorical scoring cannot support, query volumes that make the cost spike, that is your signal to build the RAG pipeline. By then you will have real user feedback to inform your embedding strategy, chunking approach, and relevance tuning. Build it in parallel, validate against the same queries Kendra was serving, and cut over once quality is proven. Plan for a parallel-run period before full cutover. Reindexing at scale and validating user behavior takes more time than most teams budget for.
The one pattern I would caution against is building the full RAG pipeline first to avoid being locked into Kendra, without first confirming that intelligent search delivers value for your users. I have seen that approach produce sophisticated infrastructure that solves a problem nobody was willing to pay for.
Start small. Validate the need. Then build for scale.
