The AI Architectural North Star
The AI Architectural North Star: Moving Generative AI from Fragile PoC to Enterprise Production
✍️ About the Author
Sanjoy Kumar Malik — Senior Engineering & Technology Leader with over two decades of distinguished corporate IT experience bridging complex systems engineering, cloud-native architecture, and production-grade AI strategy. A TOGAF 10 Certified Enterprise Architecture Practitioner and AWS Professional Certified leader.
Sanjoy focuses on turning AI possibilities into scalable products. His work connects business opportunity and product vision with enterprise architecture to establish an architectural foundation for engineering teams.
Launching late September 2026 (this month)
Your LLM PoC worked. Now comes the hard part.
Building an LLM application is no longer the difficult part.
A capable engineering team can connect an API, add an orchestration framework, integrate a vector database, and produce an impressive AI demo in days.
But a demo is not an enterprise system.
The moment that AI application meets real users, real enterprise data, real workloads, and real business expectations, a different set of problems emerges:
- Runaway token costs.
- Unpredictable hallucinations.
- Latency spikes.
- Data exfiltration risks.
- Uncontrolled agent behavior.
- Inadequate evaluation.
- Poor observability.
- Weak governance.
- Fragile architectures.
The challenge is no longer:
"Can we build it?"
The challenge becomes:
"Can we operate it reliably, securely, economically, and at scale?"
The PoC-to-Production Gap
Most AI experimentation is optimized for one objective:
Prove that the idea works.
Enterprise production requires something fundamentally different:
Prove that the system can survive.
Traditional software architecture was largely built around deterministic systems.
Generative AI introduces probabilistic behavior, model variability, context dependency, token-based economics, evolving foundation models, and new security boundaries around prompts, data, tools, and agents.
That changes the architectural equation.
An LLM wrapper may demonstrate technical feasibility.
It does not establish production viability.
And this is where The AI Architectural North Star begins.
A Practitioner’s Playbook for Engineering Enterprise AI
The AI Architectural North Star: Moving Generative AI from Fragile PoC to Enterprise Production provides a systematic path for moving from an AI idea to a production-grade enterprise capability.
This book connects the discipline of Enterprise Architecture and TOGAF 10 with the realities of LLM-powered and probabilistic AI systems.
It gives technology leaders a framework for answering four fundamental questions:
01 — What should we build?
Identify the AI opportunity that is genuinely worth pursuing, translate it into an AI product vision, and determine the intelligence the product actually requires.
02 — What should the architecture look like?
Translate product and intelligence strategy into an AI Architectural North Star — the target architecture that aligns business, application, data, intelligence, integration, security, and infrastructure concerns.
03 — How do we make it production-ready?
Engineer the system for security, governance, evaluation, reliability, observability, and economic sustainability.
04 — How do we scale it across the enterprise?
Move beyond individual AI projects toward reusable platforms, enterprise knowledge architecture, operating models, governance, and an industrialized AI production lifecycle.
The Three-Part Journey
PART I — From AI Idea to Architectural North Star
Decide before you build.
The first challenge isn't choosing an LLM.
It's determining what AI capability the business actually needs.
Part I takes you from:
AI Opportunity → Product Vision → Intelligence Strategy → Architectural North Star
You'll learn how to:
- Identify high-value AI opportunities
- Decompose workflows and business processes
- Prioritize AI use cases
- Define AI product visions
- Design human + AI interaction models
- Select between LLMs, traditional ML, and deterministic logic
- Evaluate foundation models
- Choose between prompting, RAG, fine-tuning, and structured generation
- Design model routing and escalation strategies
- Translate intelligence strategy into target architecture
- Apply enterprise architecture principles to probabilistic AI
- Connect AI architecture with the TOGAF 10 ADM
Key deliverables
- AI PoC-to-Production Readiness Scorecard
- AI Opportunity Assessment Matrix
- AI Product Vision Canvas
- Enterprise Model & Intelligence Decision Matrix
- AI Architectural North Star Blueprint
Outcome: A clearly defined architectural destination, not another AI experiment.
PART II — Engineering the Production AI System
Engineer for trust.
Once the Architectural North Star is established, the next question is:
How do we make the system production-grade?
Part II turns architecture into an operationally viable AI system.
You'll learn how to engineer for:
Security & Governance
Protect enterprise data, control AI behavior, manage permissions, enforce policies, and establish human approval gates.
Evaluation
Test systems whose outputs aren't always deterministic using golden datasets, regression testing, LLM-as-a-Judge, human evaluation, RAG evaluation, agent evaluation, safety evaluation, and task-success measurement.
Reliability & Observability
Design for unpredictable model behavior using fallbacks, provider failover, circuit breakers, rate limiting, asynchronous inference, streaming, queues, distributed tracing, AI telemetry, drift detection, and AI SRE practices.
AI FinOps
Understand the economics of intelligence from token consumption and model routing to semantic caching, retrieval optimization, cost attribution, capacity planning, and cost per successful task.
Key deliverables
- Enterprise AI Governance & Guardrail Matrix
- Enterprise AI Evaluation Framework
- AI Reliability & Observability Reference Architecture
- AI Unit Economics & FinOps Framework
Outcome: A secure, testable, observable, reliable, and economically viable production AI system.
PART III — Building the AI Enterprise
Scale beyond individual AI projects.
One production AI application is a project.
Dozens or hundreds of AI applications become an enterprise architecture problem.
Part III addresses what happens next.
You'll learn how to build the capabilities required to scale AI across products, teams, and business units.
Enterprise Knowledge Architecture
Design production-grade RAG and agentic architectures covering ingestion, chunking, embeddings, vector search, hybrid retrieval, reranking, context engineering, knowledge freshness, tool use, agent boundaries, and multi-agent systems.
And critically:
When not to use agents.
Enterprise AI Platform
Understand the architecture of an internal AI platform incorporating:
- Model gateways
- AI/API gateways
- Model routing
- Prompt management
- RAG services
- Vector infrastructure
- Guardrail services
- Evaluation services
- Observability
- Identity and access management
- Secrets management
- Reusable AI capabilities
- Multi-model architecture
- Cloud and hybrid deployment
Enterprise AI Operating Model
Move from isolated teams to an operating model spanning:
AI Product Management
AI/ML Engineering
Data Engineering
Platform Engineering
MLOps/LLMOps
Application Engineering
Security & Governance
Architecture
Explore centralized versus federated models, RACI structures, governance forums, architecture review, model approval, production approval, team topology, and enterprise-scale AI organizations.
The AI Production Factory
Finally, connect everything into one repeatable enterprise lifecycle:
Discover → Define → Strategize → Architect → Build → Evaluate → Secure → Deploy → Observe → Optimize → Govern → Scale
Key deliverables
- RAG & Agent Architecture Decision Framework
- Enterprise AI Platform Reference Architecture
- Enterprise AI Target Operating Model
- AI Production Factory Blueprint
Outcome: An enterprise capability for continuously building, operating, governing, and scaling AI.
From Experimentation to Industrialized AI
The transformation this book describes is not simply:
PoC → Production
It is:
AI Idea
↓
AI Opportunity
↓
Product Vision
↓
Intelligence Strategy
↓
Architectural North Star
↓
Production Engineering
↓
Enterprise AI Platform
↓
AI Operating Model
↓
AI Production Factory
↓
Continuous Scale
This is the architectural journey from one promising experiment to an enterprise AI capability.
Who Is This Book For?
This book is written for technology leaders who are responsible for making AI work beyond the demo.
CTOs
Establish an enterprise-wide architectural direction for AI while balancing innovation, risk, reliability, and economics.
VPs of Engineering
Turn AI initiatives into scalable engineering capabilities rather than disconnected experiments.
Heads of AI Engineering / AI Practice
Create repeatable architecture, engineering, governance, and delivery patterns for enterprise AI.
Chief Data Officers
Connect enterprise data, knowledge, governance, and AI into a coherent architecture.
Enterprise Solution Architects
Extend enterprise architecture practices into the world of LLMs, RAG, agents, and probabilistic systems.
This Is Not Another LLM Tutorial
You won't find a book here focused primarily on writing prompts or wrapping an API.
You don't need another tutorial showing you how to call an LLM.
The technology to build an AI PoC is increasingly commoditized.
The harder and more valuable capability is knowing:
- What to build.
- What not to build.
- How much intelligence the product actually needs.
- How to architect it.
- How to secure it.
- How to evaluate it.
- How to make it reliable.
- How to control its economics.
- How to operate it.
- And how to scale it across the enterprise.
That is the territory of this book.
Your AI PoC Is Not the Destination.
It is the starting point.
The organizations that win with Generative AI will not simply be the ones that build the most impressive demonstrations.
They will be the ones that can repeatedly transform promising AI ideas into production-grade, enterprise-scale capabilities.
That requires more than models.
It requires more than prompts.
It requires more than frameworks.
It requires an Architectural North Star.
Stop building AI demos that are destined to remain PoCs.
Start engineering AI capabilities designed for production.
The Book Will Be Available Soon
The AI Architectural North Star: Moving Generative AI from Fragile PoC to Enterprise Production will be available to read for free on my website at the end of September 2026.
Move beyond the PoC.
Architect for production.
Build AI that can scale.
Follow Sanjoy on LinkedIn and receive updates about AI Architectural North Star: Moving Generative AI from Fragile PoC to Enterprise Production through his LinkedIn post.