Future-Ready AI Ecosystem Architecture

This blog is an excerpt from the co-authored whitepaper, ‘Crafting your Future-Ready Enterprise AI Strategy’ aiming to support organizations on their journey in the Age of AI.

Foundational Considerations

So it is that we’ve been spending significant time and mental energy thinking about a proper “AI strategy” for organizations that wish to escape the sad fate of the frog, or that of the world’s organizations being left behind by the pace of technological change.

Let’s start by defining what we’re talking about when we discuss “artificial intelligence.” There’s an argument that we could trace the lineage of AI back to the Turing Machine of the 1940s, and then pull that string through the history of computing to include such logic-based “if this, then that” applications all the way through robotic process automation (RPA) technologies of recent years. These technologies lacked real intelligence, given that they were rather extraordinarily intelligent machines created by extraordinarily intelligent people. These machines were designed to make more efficient (or make possible) the intelligence of their creators at scale. 

Generative AI is what we have in mind when we think about artificial intelligence today. Here we define 'AI' as the ability of the machine to think independently of its creators or of the parameters its creators have set forth. Generative AI describes the ability of AI to generate unique and original responses - be they textual, imagery, or in some other medium - based on its index of accumulated knowledge. In other words, for the machine to assimilate data in unpredictable ways that conjure new responses, rather than to simply navigate a logical process for which it has been pre-programmed.

It's worth noting that we’ve also seen what we call “multi-modal” AI capabilities markedly mature in 2024. Multi-modal extends previously existing abilities to learn from and generate new text, imagery, video, and other mediums as separate scenarios such that single models can understand and generate across multiple modes. This is a significant development in the ability of an AI workload to wrap its inorganic head around a vastly expanded variety of mediums, which in turn expands both its comprehension and its generative abilities.

All of this is new. It is groundbreaking. Historically speaking, we are in a very different era in late 2024 than we were two years ago, a fact that causes me to encounter two foundational realities time and again.

You are (probably) not ready; almost nobody is

First, very few - if any - organizations are truly prepared to make the most of the AI wave crashing on their shore. Very few have done the hard work to build the kind of proper, modern data platform required to make AI work at scale across their organization. For this reason, nearly everyone’s “AI strategy” will at this point look very much like most everyone else’s AI strategy, as organizations across the economy and around the world scurry to get their house in order.

Future-ready, not future-proof

Second, nobody truly knows exactly what a mature AI capability will look like or exactly how this will play out in practice. Today we’re at a point with AI reminiscent of where we were with the “world wide web” in the late 1990s when organizations rushed to digitize their physical identity. Back then we - predictably - experienced a lot of websites that looked like someone had relocated their back-of-the-phonebook advertising to a screen. We went through a similar chain of events with the advent of the smartphone when developers first tried to cram desktop apps into a smaller form factor, and then again with smart watches when developers tried to cram smartphone apps onto our wrists. Comparably, many today are busy making yesterday’s business processes incrementally more efficient by grafting AI onto them. Each of these cases illustrates our tendency to transpose legacy paradigms to new technologies. That is, at least until we gain a sufficient understanding of the new capability to grasp how transformative it really is… and how best to exploit it.

Guiding principles

Any future-ready AI strategy must be flexible, meaning it is able to absorb tomorrow what we don’t fully grasp today. Your strategy should also offer immediate value to the organization beyond specific AI-driven workloads because the nature and value of these workloads will remain unclear for some time. For example, “data readiness” is indispensable to your AI strategy because it is likely to yield better AI workloads in the future, and because it offers real value in terms of security, accuracy, discoverability, and analytical integrity separate and apart from AI today.

This explains my fondness for the phrase future-ready, and why I cringe when I hear people say “future-proof.” The former describes a cloud ecosystem built with modern technologies using best practices that are most likely to absorb whatever future innovations come our way. The latter is unachievable in all circumstances.

How generative AI acts on enterprise data

Let’s establish a basic understanding of how AI uses and acts on enterprise data. We will define 'enterprise data' as data that is proprietary to a specific organization, kept and (I certainly hope!) secured inside the boundary of the organization’s data estate.

You may be familiar with the term “RAG”, an acronym for “retrieval augmented generation”. While this is not the sole means through which AI acts on organizational data - and new and evolving patterns now emerge regularly - RAG represents a good baseline for the general concept through which nearly all AI workloads essentially augment an existing model with an organization’s proprietary data.

Figure 1: A basic RAG pattern through which enterprise data (raw knowledge) is stored such that it can be (a) indexed and (b) accessed by AI. Capabilities such as Azure AI services act on that knowledge to produce a response before passing that response back to the user (via the App UX.)

In the top-right of the diagram we’re looking at various data sources sitting in a modern data platform (Azure SQL, OneLake, and Blob Storage are shown top to bottom for representative purposes). Blob Storage is a highly efficient way to store unstructured data, that is, files, images, videos, documents, etc. In this simple scenario we’ll say that unstructured data is drawn from Blob. 

These data sources are indexed by Azure AI Search (formerly called “Cognitive Search”), which also provides an enterprise-wide single search capability. Moving to the far left we see an application user experience (UX) e.g., a mobile, tablet, or web app that provides an end user the ability to interact with our workload.

The application sitting beneath the UX queries the knowledge contained in Cognitive Search’s index (as derived from the data sources on the right). It then passes that prompt and knowledge to Azure AI services to generate an appropriate response to be fed back to the user.

CIOs and enterprise architects need not be experts in the technical mechanics of AI to formulate and execute an effective AI strategy. That said, it is critical that leaders driving this strategy must understand this basic concept of how institutional AI - that is to say, AI workloads specific to your organization - both requires and acts on enterprise data.   

Without that data, it’s just AI, unspecific to the organization it is serving. 

Ecosystem Architecture

Think about the way most IT has evolved over the course of several decades. For much of this history, organizations acquired software by ginning up a specific need or “use case,” which was often followed by a basket of requirements pertaining to that use case either given to a team for development or turning into a procurement. Infrastructure, whether on-premise or cloud, was then deployed to accommodate the specific, forthcoming, solution.

Ecosystem-oriented architecture (EOA) inverts this approach. Ecosystem architects seek first to build a cloud ecosystem, that is, a collection of interconnected technical services that are flexible or “composable,” re-usable, and highly scalable. The ecosystem then expands, contracts, and is adapted over time to accommodate the workloads deployed within it.

EOA is ideal for scaling AI because it promotes data consolidation as a first principle, avoiding the de-consolidation that point solutions tend to promote via the use of data services tied specifically to the application itself and point-to-point data integrations with other point solutions.

Earlier we shared a sample ecosystem map (see the Ecosystem Map dimension in the Strategy and Vision pillar). Let’s now dig into the concept of EOA a bit further, as it is essential to understanding the Ecosystem Architecture pillar.

We’ve created what we call the “Reference Ecosystem”, essentially a composite of dozens of enterprise organizations’ cloud ecosystems that we’ve studied across many industry sectors and geographies.

Figure 2: The “Reference Ecosystem” is a composite of ecosystem-oriented architecture (EOA) across many different organizations in many different industries. Consider it the “North Star” for an effective EOA across an entire cloud estate and use it as a reference to build your own.

To orient you, the map makes a clear analogy between the cloud ecosystem and a city divided into “neighborhoods.” These neighborhoods are conceptual, in other words, one should not necessarily construe them as hard logical boundaries such as environments, subscriptions, or tenants. Rather, the ”neighborhood” concept helps us understand the categories, relationships, and boundaries of technical services and workloads, in what can be a vast cloud ecosystem, in a relatable, clear way.

Adopting an ecosystem-oriented architecture across an enterprise organization supports your AI strategy in many ways. Fully adopted EOA is the pinnacle of many of the strategies we’ve already discussed, the enterprise architecture “North Star,” if you will, in the era of AI:

• Use this Reference Ecosystem to orient you to the first four dimensions of our Ecosystem Architecture pillar, specifically Core Platform Services, Data Distribution, Integration, and Business Applications (which combines the Core Business Systems and Application Portfolio neighborhoods shown in the reference);

• EOA speeds the deployment of AI workloads, creating the conditions for those “quick wins” that many think they can achieve with AI only to find out that they’ve not done the work necessary around data consolidation, readiness, and scaling to make this work. Time to value is much shorter when your data is already consolidated, indexed, governed, secured, and you have supporting services such as application lifecycle management ALM (and MLOps) in place;

• EOA organizes and integrates “traditional” workloads found primarily in the Core Business Systems, Application Portfolio, and Unstructured Data neighborhoods in a way that supports AI workloads downstream. This integration mitigates the struggles many will have preventing new silos of unconsolidated data emerging as they scale;

• Undertaking a transition to ecosystem-oriented architecture aligns well with the two core principles that underpin the AI strategy:

  • Your AI strategy must be flexible, able to absorb tomorrow what we don’t fully grasp today . Cloud ecosystems are designed to be metaphorically living, breathing entities that evolve to meet today’s needs and achieve the promise of future innovation;

  • Your strategy should offer value to the organization beyond specific AI-driven workloads because the nature and value of these workloads will remain unclear for some time. Your transition to EOA is a great investment in AI, but also brings value to the organization in terms of time to value, resolution of technical debt, retirement of legacy licensing and capacity costs, reduction of organizational risk around data governance and security, and in the form of workloads such as search, outside integration, analytics, and reporting.

Read the whitepaper for a deep dive into the five dimensions of Enterprise Architecture: Core Platform Services, Data Distribution, Integration, Business Applications, AI Development Tools.