Welcome to Sanjoy Kumar Malik’s AI & Software Development Resources
Sanjoy is a senior engineering and technology leader with over 20.7 years of experience in corporate IT, specializing in multi-tenant SaaS architecture, distributed systems design, cloud-native architecture, and Agentic AI. His is AWS Certified Solutions Architect – Professional and TOGAF certified. He architects scalable, resilient, secure, and cost-efficient enterprise platforms, taking complex technology initiatives from architectural vision to production. This site brings together the reference architectures, decision frameworks, books, and technical insights that shape his approach to enterprise architecture and engineering leadership. Let’s talk.
Agentic AI architecture in the enterprise
Reference architectures, decision frameworks and build guides for AI platform teams: agents, context, retrieval, model routing and governance, designed for production on AWS.
For CTOs, VPs of Engineering, Heads of AI, and Solution and Enterprise Architects, AI Architects
Consumers
Models and data
AI engineering
AI engineering is the discipline that makes AI features dependable: the practices, tooling and tests that turn a model call into a product component.
- Prompts, context and model choice treated as versioned, tested code.
- Evaluation sets and quality gates before every release.
- A gateway and serving layer, so teams can swap models without rewrites.
- Production monitoring for quality, latency and cost.
CASES = load_cases("evals/refund_policy.jsonl")
def test_answer_quality():
scores = [grade(answer(c.question), c.expected) for c in CASES]
assert mean(scores) >= 0.90, "Quality gate failed: do not release"Agentic AI architecture in the enterprise
Agents change what a platform must provide. Design the runtime, the boundaries and the controls first, then let teams build on them.
- Orchestration patterns and where to set autonomy limits.
- Context and memory as a platform service, not per-agent code.
- Tool access through APIs and MCP servers, with scoped permissions.
- Tracing and approvals so every agent action can be reviewed.
| Step | Agent | Action | Status |
|---|---|---|---|
| plan | support-agent | Break request into steps | Done |
| retrieve | support-agent | Search policy knowledge base | Done |
| tool | billing-agent | crm.get_account (MCP) | Done |
| approve | support-agent | Refund over limit | Human review |
| respond | support-agent | Draft reply with citations | Done |
Creating RAG solutions on AWS
From documents in S3 to grounded answers with citations. Choose each stage deliberately and measure retrieval before you tune the model.
- Ingestion, parsing and chunking that fit your document types.
- Embedding and vector store choices on AWS, with trade-offs.
- Retrieval, reranking and grounding to reduce wrong answers.
- Guardrails, evaluation and cost controls from day one.
import boto3
client = boto3.client("bedrock-agent-runtime")
answer = client.retrieve_and_generate(
input={"text": "What is our refund policy for enterprise plans?"},
retrieveAndGenerateConfiguration={
"type": "KNOWLEDGE_BASE",
"knowledgeBaseConfiguration": {
"knowledgeBaseId": "KB_ID",
"modelArn": "arn:aws:bedrock:<region>::foundation-model/<model-id>",
},
},
)
print(answer["output"]["text"])Ready for production, not only for the demo
An agent that works in a notebook still needs evaluation, guardrails, observability and cost control before it reaches users.
- A review checklist you can apply to any agent or RAG system.
- Model routing that sends simple requests to cheaper models.
- Governance that satisfies security and risk teams.
| Control | What it covers | Level |
|---|---|---|
| Evaluation | Golden sets, regression gates | Required |
| Guardrails | PII, prompt injection, topic limits | Required |
| Observability | Traces, token and cost per request | Required |
| Model routing | Cheap model first, escalate on need | Recommended |
| Human oversight | Approval on high-impact actions | Required |
Modern AI-driven SDLC
AI can speed up every stage of delivery. The gain holds only when each stage has clear inputs, checks and a person accountable for the outcome.
- Where AI helps across requirements, design, build, test, release and operations.
- Coding agents with scoped access, required tests and review before merge.
- Quality, security and cost checks built into the pipeline, not added later.
- Roles and skills your delivery teams need as the work changes.
| Stage | AI does | Human gate |
|---|---|---|
| Requirements | Draft stories and acceptance criteria | Product owner signs off |
| Design | Propose options against architecture decisions | Architect approves |
| Build | Coding agents work in scoped repos | Tests must pass |
| Test | Generate and expand test suites | Engineer reviews coverage |
| Release | Summarise changes, check policy | Human approves merge |
| Operate | Triage incidents, suggest fixes | On-call decides |
Featured posts
Architecture thinking for AI and technology leaders.
From Opportunity to Architecture: A Framework for AI & Technology Leaders
Most AI initiatives stall between the whiteboard and the build team. Here is the framework I use to carry an idea from business case to a design an engineering team can execute.
Read the post
Thinking in Enterprise LLM Application Architecture
A practical framework for production-grade LLM applications, from context and reasoning to security, evaluation and operations.
Thinking in Hardware Infrastructure for Production LLM, GenAI & Agentic AI Systems
Size and design AI infrastructure by connecting workload to tokens, memory, compute, networking, serving and economics.
Thinking in Python Runtime: How an Enterprise AI Application Occupies a Multi-Core System
How Python AI applications use multi-core CPUs, NUMA memory, threads, processes, the GIL and GPUs to scale.
Frameworks, books and writing
Go deeper on the thinking behind the architectures.
Questions architects ask
What is agentic AI architecture, and how is it different from a chatbot?
A chatbot answers questions. An agentic system plans, calls tools, uses context and takes actions toward a goal, so it needs orchestration, scoped permissions, evaluation and human oversight designed into the architecture, not added afterwards.
What is an enterprise agent platform?
A shared layer that gives every team the same way to run agents: orchestration, access to context and tools, guardrails, evaluation and cost control. Teams build agents on it instead of rebuilding the plumbing each time.
How much autonomy should an AI agent have?
As much as the business risk allows and no more. Set autonomy limits per action: let agents act freely on low-impact, reversible steps, and require human approval for high-impact or irreversible ones. Review those limits as evaluation results improve.
Why does RAG on AWS need an architecture, not just a vector store?
Retrieval quality depends on ingestion, chunking, embeddings, access control and evaluation as much as on the index. The guide covers each stage and the trade-offs between managed and custom options.
When should I use RAG, and when should I fine-tune?
Use RAG when answers depend on private or frequently changing knowledge, because you can update the data without retraining. Consider fine-tuning when you need consistent style, format or behavior, or a smaller and cheaper model to do a narrow task. Many systems combine both.
What does “production ready” mean for an AI agent or RAG system?
Beyond a working demo, it needs evaluation sets with quality gates, guardrails, observability for quality, latency and cost, security and access control, and human oversight on high-impact actions.
How do you keep the cost of LLM applications under control?
Treat cost as an architecture concern. Route simple requests to smaller models and escalate only when needed, cache where it is safe, set token budgets, and track cost per request alongside quality so you can see what each outcome costs.
What is the 28-CAADF framework, and when should I use it?
The 28-Category AI Architecture Decision Framework is a structured way to make connected AI architecture decisions across intelligence, knowledge, retrieval, agentic execution, trust, quality, economics and evolution. Use it when designing a new AI system or assessing an existing one.
What is an AI Architectural North Star?
The architectural intent that bridges business ambition and detailed engineering. It sets the capabilities, boundaries, principles and trade-offs that must stay coherent while implementation evolves, so teams share a direction without being told every detail.
How does AI change the software development lifecycle?
AI can assist at every stage, from requirements and design to code, tests, release and operations. The gains hold only when each stage has clear inputs, quality checks and a named person accountable for the result.
Do I need to be on AWS to use these architectures?
No. The reference examples use AWS services, but the underlying patterns, such as retrieval design, orchestration, evaluation and governance, apply on other clouds and on-premises with different building blocks.
Who is this for?
CTOs, VPs of engineering, heads of AI, Solution and enterprise architects, and AI architects who have to move AI work from prototype to production.
Plan your agent platform with a clear architecture.
Tell me what you are building and which decisions are blocking you.