Data Management Fundamentals
This guide represents insights from Citisoft’s Data Practice, with key contributions from Broc Kerr, Mike Maltby, Jay Meador, Paul McInnis, Derek Pratt and Adam Taylor.
Short on time? Download the PDF to read at your convenience.
Download PDFDeveloping a Data Strategy and Data Operating Model that Deliver Value
Talk of data is nothing new, especially in the investment management industry where gathering and synthesizing data has always been at the core of the business model.
Yet, as topics like machine learning and artificial intelligence take data into thrilling new realms, it is all too common to hear about delivery delays, bloated budgets, and failed projects due to shortcomings in the foundational elements of data initiatives.
So where is the disconnect? Why are we often talking about the immense business value that data offers organizations and yet unable to deliver on the initial steps necessary to pursue that value?
To answer those questions, we return to the overlooked fundamentals, the relationships between the parts of data organizations, and the principles that are common to initiatives that successfully deliver business value.
But First, Is Data Worthy of the Hype?
Unquestionably, yes. Apart from the fact that we can surely agree that decision making is improved by having robust and accurate inputs (i.e., data), you would likely be more convinced if we were to cite a more specific and definitive source—in other words, you might say, “show me the data.”
While many studies point to the perceived value of data for businesses, in 2022, Harvard Business Review Analytic Services surveyed over 350 members of the Harvard Business Review audience to deliver findings in a report titled, Transforming Data into Business Value through Analytics and AI. The result: The 45% of respondents that graded their organization’s ability to derive business value from data between 7 and 10 on a scale of 0 to 10 were more likely than their 1-to-6 counterparts to also report improvements to revenue (+16%), profitability (+22%), customer retention (+32%), as well as employee satisfaction (+29%) over the year preceding the survey.
Back to the Fundamentals
The challenge is getting to the place where you can confidently say your organization is leveraging data for business value. It has never been easy, but evolving investment strategies, client expectations, and business requirements continue to increase the demands on investment data teams, making the road to value still more challenging. Though technology continues to advance in leaps and bounds, if not aligned within a thoughtfully constructed and managed data ecosystem, even the most powerful tools can exacerbate issues rather than provide solutions.
“First, your approach must be driven by business value. Second, your approach must be iterative. Third, the approach must be holistic.”
To navigate the daunting and growing demands on data teams and access the proven value of data, you need a data ecosystem with three core values. Successful data initiatives are defined by an approach that is:
- Driven by business value
- Iterative
- Holistic
They may seem obvious, but these three keys are the threads that wind through the various concepts, teams, and projects that turn data into value.
First, your approach must be driven by business value. When you begin digging into the tactical functions of a data ecosystem, it’s all too common to drift from the targeted business value that will ultimately make the project a success. Second, your approach must be iterative. Talk to anyone about turning data into value and they’ll start by saying it wasn’t easy. While it’s true that data projects are complex and take time, the solution is to employ an iterative approach where you continuously define, measure, analyze, and improve. Third, the approach must be holistic.
Why are we talking about strategy, governance, architecture, operations and analytics all in one place? Because the connections between elements are just as important as the separate parts. A lack of consideration or flaws in one area will likely lead to issues in others. As we explore the elements of the data ecosystem, these three traits will continue to come up and demonstrate their value.
The Data Ecosystem
From here on we’ll begin peeling back the layers. This begins with the all-encompassing data ecosystem, the term we use to describe the collection of all the plans, technologies, functions, people, and policies relating to data in your organization. Though data ecosystems can contain many people and responsibilities there are two fundamental components:
Data Strategy
The starting point and most important element, your data strategy is the guiding star for everything that follows in the data ecosystem. It must be aligned with the overall business strategy and consensus on the data strategy among involved parties is essential.
Data Operating Model
If the data strategy sets out your goal for using data, the data operating model is where you consider what is needed to deliver and support that target state.
The plan set out by the data operating model can then be broken down into four underylying elements:
- Data Governance
- Data Operations
- Data Architecture
- Data Analytics
Where All Data Projects (Should) Begin: The Data Strategy
Before delving into the elements of the operating model, it’s essential to align on the definition and role of the data strategy. A data strategy is the vision and long-term plan for how your company will utilize people, processes, and technology to derive business value from data.
Before setting off in pursuit of data-driven business value, every effort begins with an igniting spark, a vision for what data could bring to the business. The vision is then developed into a high-level plan, depicting with broad brushstrokes the people, processes, and technology that are thought to be necessary. This is the first of many instances in the data journey where it’s important to keep in mind the value of an iterative approach. The data strategy, like all aspects of the data ecosystem, will continuously evolve—whether we like it or not.
Though it is the starting point for a successful data ecosystem, the data strategy will not and should not contain all the answers for the transformation of vision into value. Rather, it’s important at this early stage to consider the general direction of work, high-level components thought to be necessary, and guardrails that will keep the effort focused on value while managing risk.
The final point regarding guardrails is where data governance enters the picture. Data initiatives begin with the data strategy and governance is an element of the data operating model that is built based on the strategy—yet the process is not so cleanly linear. Governance is a unique element of the operating model in that it should be considered at the beginning, alongside the data strategy. Data governance will get the spotlight in the following section but it’s important to note that successful projects begin developing data governance as a part of the initial data strategy work.
Overall, success during this first stage of the data journey is found by regarding the data strategy as not only a canonical artifact but also as a process—it will continue to evolve as you ask questions, learn more, and refine. The process behind data strategy formulation relies on two phases, familiar to all who have engaged in transformation projects: a current state assessment and target state definition.
- Current State Assessment
- Target State Definition
The current state assessment
The current state assessment considers how your organization ingests, stores, transforms, and consumes data. In other words, what are the entry points for data into your organization, where is it stored, what is done to prepare it for use, and how is it used to support business activity? Most organizations have multiple streams in which data flows through the organization, so it is here that data domains become a helpful concept. Data domains divide what can be a complex web of systems, processes, and data into underlying role-based categories. Examples of domains within investment management include the transaction, holdings, and reference data.
Looking at the way data moves through your organization and the different domains that group those flows, you’ll build an increasingly detailed understanding of the people, processes, and technology that make up your data organization while also revealing pain points, alignment to business needs, and the controls (or lack thereof) in place to instill standards and quality into data outputs. Before starting to build the data ecosystem you want, it’s essential to know where you stand today.
Defining the target state
One of the data strategy’s most important contributions is that it sets the stage for alignment of the entire data ecosystem with the broader business strategy. The process of aligning the data strategy to the business strategy begins with the definition of a target state, or the final form that you want your value-driving data ecosystem to take. By considering business goals, commercial prioritization, and any other organizational needs while depicting the target state, you begin to chart a path towards deriving material business value from data.
In addition to the needs of the business, the target state should consider the organizational realities that will affect the initiative. Though the target state is fundamentally aspirational, this is the time to instill your goals and plan with pragmatism. Candidly looking at the people, processes, and technology in place today, it’s important to ask if the targets for supporting business needs are achievable, and as a part of that, whether the foundation for the necessary data exists.
It may seem like these questions would stop most projects before they’ve started but rather than getting the team to scrap the plan altogether, the intent behind these target state considerations is to help focus efforts in the right areas and build a collective understanding of prioritization.
Organizations are often eager to deploy flashy solutions that promise fast insight, but all of today’s impressive analytics functionality requires a foundation of architecture, operations, and governance to bring quality and scalability to even the easiest-to-implement solutions.
Where data strategies fall short
Despite the foundational importance of data strategies, it is an often-neglected piece of data initiatives. Foremost, no matter how clear your objectives or the path to achieving them may seem, always take the time to develop a data strategy. Even if the exercise does not reveal new information around needs and goals, it’s a valuable opportunity in the early stages of work to build consensus among stakeholders. That said, a thorough exploration of the current state and target state almost always brings to light facts about your organization or the planned project that are better to know and consider before work begins.
A commitment to developing a data strategy is an invaluable start, but the next common pitfall to avoid is making sure that the data strategy is pragmatic and grounded in commercial impact and actual achievability.
As touched on when discussing target state definition, it is all too common to embark on an initiative that is doomed from the start because it does not consider the realities of the organization. The delayed, overbudget projects that give data initiatives a bad name often have their roots in articulate plans that seem carefully constructed but suffer from the fact that while developing the data strategy no one seriously asked, “Is this plan achievable for our firm today?”
Making the data strategy real
By assessing the current state, defining the target state, and grounding it all in pragmatism, the data strategy process and resulting high-level plan play an essential role in laying the foundation for a successful data initiative. A thoughtful data strategy drives a strong project plan, but its greatest value might be that its presence brings resiliency to the project, when the inevitable challenges and changes are encountered. It orients all the ensuing work in the same direction and takes the first steps towards instilling quality, integrity, and scalability in the data operating model.
The organizations suffering under the costly weight of siloed data that requires frequent reconciliation across platforms and makes the prospect of ingesting new data sources or adding new technology frightening? It’s usually because their data journey didn’t begin with a thoughtful, realistic data strategy.
To build a modern data ecosystem that is driven by business value, iterative and holistic, the data strategy is a vital starting point but it’s through the four elements of the data operating model that the aspirations of the strategy are transformed into something real.
That in mind, if we want to understand why data projects fail and why more than half of the respondents to the Harvard Business Review survey were not able to say their company effectively creates value from its data, it’s the data operating model where we need to look for answers.2
In the following four sections, we’ll look at governance, operations, architecture, and analytics—what they entail, keys to success, and the connections between them that drive value.
What is data governance?
Data governance is the structured approach to managing data as an organizational asset. It underpins sound data management from planning through execution, defining roles and responsibilities, setting data quality benchmarks, and ensuring secure, compliant data practices. This oversight mitigates risks related to data breaches, inaccuracies, and regulatory non-compliance, while also enhancing transparency, accountability, and trust in the data.
Data governance first takes shape along with the initial data strategy and sets the stage for a data operating model that is on the offense in its pursuit of business value.
When data governance begins as part of a well-constructed data strategy, it ensures that data initiatives are integrated within the broader business strategy rather than pursued in isolation. As it then guides operations, architecture, and analytics, governance plays an integral role in uniting the elements of the data operating model.
The holistic approach facilitated by data governance promotes cross-department collaboration and consensus building both inside and beyond the data organization.
Data governance in the cloud
Today, with cloud solutions spanning data management, architecture, and analytics, data ecosystems with cloud components have unique architectures that must be well understood and properly managed.
For example, a cloud provider is responsible for security of the cloud, but users are ultimately responsible for security in the cloud. Understanding that distinction and the implications for maintaining good governance practices while using cloud platforms is paramount to information security.
Further, cloud transmission costs have the potential to quickly grow. Effectively governing the flow of data to the cloud enables the organization to monitor and better manage those costs, delivering material cost benefits to good data governance.
Common factors driving success in data governance
All too often, the importance of data governance is not recognized when considering the data operating model or goals of the overall ecosystem. Even when it is regarded as a key component, it tends to be viewed as a purely risk-mitigating function. Both approaches neglect the value that data governance can bring and, in the end, almost always result in issues with the operating model’s ability to deliver value.
From our extensive experience, we’ve identified three key principles to help orient you towards data governance success and with that, a data operating model that delivers material business value.
- Start with data governance
- Consider organizational culture and align data governance to it
- Support offensive innovation in addition to defensive mitigation
Principle #1 – Start with data governance
The foundational role of data governance means that success follows when it is incorporated in data initiatives from the very beginning. Data strategy is the starting point, but data governance should be considered as a part of those early strategy conversations and continue to evolve with the broader data ecosystem. From that point forward, data governance provides the solid ground and guardrails for further work by data operations, architecture, and analytics.
That said, governance must not be considered a set-it-and-forget-it exercise. Instead, it becomes a living, breathing thing that is core to all data efforts moving forward. Early engagement with stakeholders to establish clear policies, standards, and procedures is essential, and in doing so, data governance becomes a key partner in the ensuing projects, rather than a bureaucratic obstacle.
Governance principle #2 will discuss how to guard against misalignment with the broader organization, but this principle aims at empowering data governance to unite the elements of the data operating model so that the collective effort can be as effective as possible in its pursuit of business value.
Principle #2 – Consider organizational culture and align data governance to it
Successful transformation programs often hinge on the nature of an organization’s culture, and data governance is no exception. To deliver on the operating model’s ultimate aim of business value, even a robust, thoughtful data governance function needs an organizational culture that is supportive of its policies and procedures. Designing a tailored data governance framework out of the gates that accounts for an organization’s unique strengths and culture provides the structural foundation to guide projects the right way.
This alignment between business and governance promotes a holistic approach to data management, facilitating collaboration across departments and aligning data initiatives with important business objectives. In this way, governance directs data investments toward projects that deliver measurable value by enhancing your decision-making and competitive advantage.
Organizational culture is a far-ranging topic that varies with the unique complexities of every business, so this topic brings with it a caveat rather than a clear solution: Before embarking on a data initiative, consider whether the company culture will be supportive or inhibitive to good data governance—with reality likely being somewhere in between. Whether the adjustments are large or small, to be successful, data governance must be constructed within the context of the encompassing organizational culture.
Principle #3 – Support offensive innovation in addition to defensive mitigation
Data governance traditionally has a strong defensive mandate, focusing on data quality, protection, privacy, and compliance in addition to its role safeguarding operational resiliency. However, modern governance must also focus on enabling offensive opportunities to drive business value.
On the defensive side, governance ensures compliance with evolving regulations, establishes policies that align with organizational values, and fortifies adherence across the organization. On the offensive side, governance needs to always maintain one eye on the invaluable creative frontiers of analytics that turn data into actionable insight. Governance done right is a balancing act—actively facilitating the exploration that delivers unique business value while avoiding the data issues that delay projects and diminish value.
With governance promoting a data-driven culture, organizations can unlock valuable insights and create new opportunities for growth and efficiency at lower cost and speed to market. For example, if a client wants to deploy a new reporting tool but they have to analyze and clean up the data platform to make it work, that will take much longer than a firm with good architecture diagrams, documented data lineage, and data quality practices—solid governance—in place.
Important across all elements of the data ecosystem, the continuous feedback and growth of iterative improvement is particularly valuable to a data governance function tasked with mitigating ever-evolving risks while also supporting the essentially creative analytics function. Keep in mind that the offense is just as important as the defense and recognize that governance (like the overall ecosystem) must continuously evolve.
Secure Your Cloud Data Journey with Confidence
Don't let evolving risks derail your cloud data strategy.
Discover how Citisoft's expertise ensures secure,
well-governed cloud data solutions.
A holistic view—How data governance connects to the other elements
While all elements of the data operating model are interconnected, data governance fills a uniquely keystone role as it dictates the policies, procedures, and boundaries that make up the landscape in which operations, architecture, and analytics exist. Governance can be referred to as both the foundation for and the binding glue between the elements. Role alignment and open communication between governance and all other elements of the data operating model are key to overall success.
A strong data governance function—thoughtfully constructed within the context of the organizational culture and incorporated early in data strategy conversations—reduces the likelihood of issues in all other elements of the ecosystem, but the connection between governance and analytics is noteworthy. Data analytics, a broad term for the various outputs of the data operating model, is the stage of production where the curtain drops, and the shiny new data product is delivered to data consumers. Finally in the hands of the end user, analytics becomes a common spot where an upstream governance issue finally bubbles to the surface.
Though flaws in data should be expected—and continually addressed through an iterative approach—ideally, they should be discovered and managed by the data operations function to be discussed in the next section, before they have a chance to reach the data consumers via analytics. Building on the typical defensive mandate, an effective approach to data governance also fosters collaboration throughout data initiatives and sets the stage for analytics exploration and innovation.
What is data operations?
Data operations serves as the linchpin connecting strategy and architecture with execution in a modern data ecosystem. It bridges the gap between conceptual data product management and the practical, day-to-day delivery of value-driving analytics. With governance laying out the rules of engagement, the stage is set for operations to determine the technology, infrastructure, and processes that will deliver the business’ data needs.
Because of operations’ vital role in turning a conceptual aim into real data, this section will refer to the specific role that embodies that synthesizing and steering function: the data product owner. This individual sets the data product vision, serves as intermediary between the data and business domains of the organization, and manages the data product towards the goal of continuous, iterative improvement.
Perhaps more than any other element, operations owns the primary responsibility for ensuring that your data operating model is driven by business value and designed with a holistic mindset. The ability for the data operating model to deliver business value largely hinges on how well data operations coordinates the functional areas of the data operating model and charts a collective path based on the data strategy.
Common factors driving success in data operations
Each element of the data operating model plays an essential role in deriving value from data. Sitting at the pivot point where the vision set out in the data strategy begins to translate into real technology and processes, data operations owns the primary responsibility for steering that translation effort.
Data initiatives may have start and end dates, but the effort to distill insight and value from your data is a journey of continuous evolution. Beyond operations’ role in delivering the initial target state, effective data product owners are vital to maintaining alignment between the business and data operating model as markets, people, and needs inevitably change. The following three principles are founded in our experience with the goal of setting you up for both initial and ongoing success.
- Begin with a clear data strategy
- Treat your data as a product and sweat the small stuff in its production
- Find strong data product owners and empower them to lead
Principle #1 - Begin with a clear data strategy
The foundation of successful data operations—and a value-driving data operating model with it—is a well-defined data strategy. This strategy must address the specific problems each data domain aims to solve and outline how data will be used to drive meaningful business outcomes. Without a clearly defined data strategy, the data product owners are doomed to deliver data that lacks business value.
Before orchestrating the operations that will deliver your data, verify that the aim of the data strategy is clear. Then, appoint data product owners to oversee each domain. With the data strategy as their North Star, each product owner can design an operational path with clear alignment between technology and the data as it flows from ingestion to consumption.
A data product owner has a unique and broad role, serving as the horizontal ‘CEO’ of their domain. They must have a complete, end-to-end understanding of how their data is obtained, transformed, and consumed by the entire organization. The execution of their strategy must align to the overarching business strategy and supporting products, whether it be in the front office investment decision making or their downstream consumers.
Data domains, in the asset management space, are somewhat standardized, but the granularity you get when defining data domain ownership depends on your organization and investment strategies. We commonly separate entity reference data, security reference data, and the various holdings-based books of records (also known as the BORS) as their own data domains, and then further separate them as needed. For example, a firm may elect to assign a data product owner to more specialized domains like ESG, performance measurement, investment analytics, and risk.
With data domains and their respective data product owners identified, you can start constructing a bottom-up strategy. Armed with the data strategy, each product owner can set forth on architecting their own model that will align to and bring to life the overall investment data strategy, roadmap, and plan.
A simplified data org chart showing where data product owners fit within reporting structures
The number of data product owners should be determined by the number of data domains, as should the number of data stewards reporting to the data product owners. Keep in mind, there is no one ‘correct’ org structure.
Principle #2 – Treat your data as a product and sweat the small stuff in its production
For operations to successfully turn a conceptual strategy into real, tactically useful data, the team’s approach and structure need to support the challenging task at hand. By treating your data as a consumable product—or more often, products—and accordingly, organizing your teams by domain, you will be poised to manage the full lifecycle of your data as well as the details necessary to instill quality throughout the production process.
In contrast to the passive mindset of the past that led data teams to sit within IT, treating data as a product orients operations towards an approach that is driven by business value, iterative, and holistic—our three keys successful data initiatives, as laid out in the introduction. This proactive approach ensures that the data product owners are engaged with the business, poised to act on the feedback that feeds continuous improvement, and positioned to understand where changes in the data lifecycle should be made.
A note on the many options for organizational structure
It’s worth taking a moment to connect this approach for a hypothetical data organizational structure to the real-world. Ideally, there is a distinct data, operations, and technology group (as depicted in operations principle #1) to both separate and align the three closely related functions—but that’s not practical for all organizations.
Some roles like the data stewards can sit in either data or operations and be effective. On the other hand, to have successful data product owners, it’s important that they sit alongside IT and operations, rather than being a part of either team. Allowing for pragmatic variability elsewhere, this key separation for data product owners grants them the independence and authority necessary to effectively guide their product towards the stated business goals.
There are endless ways to construct data teams and to determine how they fit into the broader organization. Ultimately, the ‘right way’ is whatever your organization finds effective and efficient.
Structurally, treating your data as a product requires organizing your teams around data domains so that operations has the perspective and authority to enact change in the right place as data moves from ingestion to consumption. This holistic perspective also facilitates attention to the details that are the unsung heroes of the data operating model. When focusing on business value and the data outputs that deliver it, it’s easy to neglect the reference data and metadata that are integral to consistent, quality, and timely data. From the holistic vantage of being oriented around data domains, operations’ data product teams understand the importance of all components, even if they exist in the background.
Principle #3 – Find strong data product owners and empower them to lead
Even if operations principles #1 and #2 are executed well, the success of data operations will frequently come down to the data product owners. The challenge for operations is to bring together all the elements of the operating model and make real the vision laid out by the data strategy.
This is no small task and though it takes a thoughtfully constructed team, as with any time you’re being guided by a ‘vision’, success requires strong leadership. Within data operations, those leaders are the data product owners. Find data product owners that see the vision of your data strategy, balance strategic and tactical knowledge, and understand that their product needs to continue evolving with the business. Then, empower them to lead and deliver. Operations’ leading role provides the opportunity to further ingrain the three keys to successful data management—driven by business value, iterative, holistic—as core values across the teams of the data operating model.
A holistic view—How data operations connects to the other elements
Though operations is responsible for ‘steering’ the data operating model, if operations is the driver, architecture is the car. As such, close alignment and smooth collaboration between operations and architecture are integral to this phase of the operating model where the value set out in the data strategy begins to materialize. Even the most effective data product owner will falter if they don’t have capable data architects and a strong working relationship with them.
Yet, as mentioned throughout this section, data operations play the important role of bringing together all the elements of the data operating model to ensure that they are working together towards the same goal. This role of connecting and synthesizing means that it’s important for operations to be closely tied to all elements.
Operations translates the business’ data needs into the architectural and operational requirements that can deliver valuable analytics, all the while considering the governance that mitigates risk and imbues the operating model with resiliency. If there is one area of the data operating model where communication and interpersonal skills are essential, it’s on the data operations team.
Data operations in the cloud
A cloud data architecture enables our data product owners to develop and deploy solutions to the business quicker, with higher quality, and with more informed, objective metrics. As modern cloud technology comes with improved data telemetry that can aggregate uses of data, monitor the health of data, and even pinpoint areas for improvements, the data product owner can inform their roadmap with real-time production data served up to them from the cloud. It gives the data product owner a full understanding of the complete lineage of their data that in turn empowers them to make meaningful improvements to it for their consumers.
What is data architecture?
In today's data-driven world, senior leaders in tech, operations, and data are tasked with more than just managing tools—they must strategically architect their entire data ecosystem. The effectiveness of your data platform is deeply intertwined with how well data architecture supports your broader ecosystem.
Though the entire data ecosystem needs to be thoughtfully architected (i.e., organized or constructed), data architecture’s responsibility is to build the network of systems and dataflows that carry your data from initial ingestion to end-user consumption, transforming unrefined inputs into valuable outputs in the process. As with each element of the data operating model, a holistic approach to architecture is key to instilling efficiency in what is often a complex web of systems and dataflows.
A holistic approach to data architecture goes beyond the allure and flashy functionality of cutting-edge platforms, emphasizing the critical need to take ownership of your data strategy, prioritize risk management, and adopt a mesh-based approach (discussed further in architecture principle #3). These elements, consistently driven by business value, collectively forge the infrastructure of an optimized data ecosystem that empowers your organization to unlock the full potential of its data assets.
Common factors driving success in data architecture
More than any other element of the operating model, data architecture delves into a more technical world of software solutions and dataflows. Working closely with operations to transform the data strategy into the targeted outputs, it’s important that architecture remains connected to the business and aware of pragmatic concerns with data consumption.
The world of data architecture can also lead to more rigid solutions, if not avoided, hindering the operating model’s ability to grow and adapt in the future. To maintain this balance of technical expertise and business-oriented pragmatism and flexibility, our experience with data architecture successes and failures points us toward three principles.
- Keep your data architects informed on your business’ needs
- Build a data tech stack that is guided by your data strategy and mindful of data governance
- Embrace the interconnected nature of modern data ecosystems
Principle #1 – Keep your data architects informed on your business’ needs
Your data strategy should serve as the North Star for data architecture, as it does with all elements of the operating model. For architecture, it guides your pursuit of business value and prevents you from getting lost in a maze of technology and dataflows.
Of course, to be guided by the strategy, architecture needs to be aware of it, but that’s a level of alignment that will often get data ecosystems close but short of their full potential. To manifest the full value of your data, the data architects need to understand the business needs that the strategy aims to satisfy.
This advanced level of engagement between architects and the business will foster a holistic perspective and greater clarity of purpose, but remember that this is not a one-and-done exercise. As your business needs evolve, keep your data architecture team (along with their partners in data operations) in the loop so that they can support the iterative evolution of the operating model.
Principle #2 - Build a data tech stack that is guided by your data strategy and mindful of data governance
Data platforms are powerful tools, but they're not magic wands. It's the quality, accessibility, and business value of insights derived from your data that will give you an edge.
By building an understanding of business needs within your data architecture team—architecture principle #1—your architects will be poised to employ tech solutions that support your data strategy, rather than relying on the flashy functionality to deliver value. The allure of feature-rich data platforms is strong. Yet, relying solely on vendor promises can lead to a scattered, siloed data landscape. To truly optimize your ecosystem, you need to be the architect, not just the occupant.
In addition to the data strategy, data architecture needs to be well versed in your organization’s data governance practices. Then, when considering solutions, don’t be swayed by shiny features. Instead evaluate technology based on the full list of factors, weighted according to your organization’s priorities.
Technology risk management
In today's crowded data platform landscape, it's tempting to get swept away by the latest toolset. However, a feature-first approach can lead to overly complex, poorly integrated systems. Here's a risk-based approach to choosing and managing technology:
- Identify data vulnerabilities - Analyze your data landscape for potential security breaches, privacy violations, and data quality issues.
- Prioritize risk mitigation - Focus on solutions that address your most pressing data risks. This could involve robust access controls, data encryption tools, or data lineage tracking platforms.
- Integrate with existing infrastructure - Look for data platforms that integrate seamlessly with your existing systems. This minimizes data silos, streamlines workflows, and reduces disruption.
- Future-proof for agility - Opt for modular solutions that can be scaled and adapted as your data needs evolve.
- Bake In Data Privacy and Governance - Data privacy and governance are non-negotiable. Bake them into your ecosystem from the start. Ensure compliance with regulations like GDPR and CCPA. Trustworthy data is the cornerstone of success.
Remember, the best data platform is one that minimizes risk, optimizes your specific needs, and fosters a data-driven culture within your organization.
Principle #3 - Embrace the interconnected nature of modern data ecosystems
In alignment with operations principle #2, data-mesh architecture recognizes that data ecosystems are interconnected, treating data as a product and organizing data teams by domain. When taking this holistic view of the data’s journey, APIs, data exchanges, and interoperability all matter. Separate as they may seem, the technical underpinnings of data architecture and the target business use case are two sides of the same coin.
For the data architect this means gone are the days of centralized data warehouses. Instead, today's data ecosystem thrives on a mesh-based approach where data resides in various locations but can be accessed and analyzed seamlessly by parties across the ecosystem. Beyond the holistic perspective it provides the data architecture team, the benefits of a data-mesh approach can be seen across the organization:
- Decentralized decision making - Empowering teams to access and analyze relevant data fosters faster, more localized decision-making, crucial in today's fast-paced environment.
- Improved data quality - Distributing ownership encourages teams to actively curate and maintain data quality within their domain, resulting in a more reliable data set. This facilitates architecture, and the broader data model, practicing the key operating model attribute of being iterative and focused on continuous improvement.
- Enhanced innovation - A mesh-based approach allows diverse teams to combine data from different sources, sparking novel ideas and fostering creative problem-solving.
- Scalability and agility - The mesh architecture can easily scale to accommodate growing data volumes and evolving business needs. As seen above with data quality, this illustrates another facet of an iterative approach and the value it brings.
A holistic view—How data architecture connects to the other elements
Each element of the data operating model is essential but it’s with data architecture that the proverbial rubber meets the road—the vision of the strategy, which is turned into a governance-guided data product roadmap by operations begins delivering value-driving analytics data thanks to the data architecture function.
Because data architecture and operations need to work hand in hand to be successful, their connection is of prime importance. As mentioned in the ‘holistic view’ portion of the operations section, if operations is the driver, architecture is the car. From the data architecture perspective, data operations should be a valuable resource and collaborator. Though we recommend that architecture understands the business’ needs, operations holds the primary responsibility for bridging the business and the technical, allowing data architects to focus on building the right systems and dataflows.
Another important relationship with data architecture comes as a byproduct of the data-mesh approach. To prevent duplication, or worse yet, fragmentation of effort due to the decentralization of mesh architecture, strong data governance is required. In particular, well-defined communication channels need to be outlined by governance and practiced by architecture. A thorough understanding of governance is also important as the data architecture function explores and adopts technology that should support rather than impede the aim of on-time, high-quality, value-laden data.
Your organization’s success centers not just on the tools you deploy or even the vision you set, but ultimately on how well you construct the data ecosystem. Integrate data architecture into the data operating model—and with that the encompassing data ecosystem—and you will be well on your way to deriving value from your data.
What is data analytics?
Perhaps the first question any data operating model design process should start with is asking, “What is the data actually used for?”. This is the question answered by data analytics, an umbrella term referring to all outputs of the data operating model, including data feeds, standard reporting, self-service dashboards, and increasingly, data products utilized in data science, predictive models, and even generative AI.
Unfortunately, the question of “what is all this for?” is often overlooked until far too late in the process. Many data operating models falter by focusing too heavily on enabling technology or defensive data governance, neglecting how data will deliver tangible value to the business. Analytics cannot be treated as an afterthought—from the outset, it must be central to the entire data ecosystem, ensuring that data is ready and able to be leveraged to drive material business outcomes.
To do so, the analytics function—and the data operating model as a whole—must be based on a philosophy of iterative improvement, be deployed in a way that fits the culture and maturity of the organization, and continuously measure and review the effectiveness of its contributions to the business.
As analytics practitioners bring their expertise and often biases around specific tools and technologies, however, discussion of analytics operating models often turns into the weighing and debating of particular tools, technologies, and platform-specific practices. Getting the model for analytics deployment and operations right for the organization should come before any technology-specific considerations. In doing so, the foundation for success is laid regardless of the tools employed.
Common factors driving success in analytics
Analytics is the culmination of everything that’s come before it, including all other components of the operating model and, of course, the data strategy. That position as the output of so much preceding work means that for analytics to be successful, governance, operations, and architecture must remain oriented toward the same vision of business value—but analytics must also be flexible and continuously validate that it’s delivering the desired value and doing so in a manner that works fluidly for the business’s data consumers.
Data analytics teams can struggle in various ways, but in our experience, the successful teams practice a few key principles.
- Establish an agile and iterative analytics framework
- Deploy analytics in alignment with your organization’s abilities and needs
- Measure the return on your analytics investment
Principle #1 – Establish an agile and iterative analytics framework
At its heart, data analytics is an experimental, iterative practice. Embracing these characteristics can be challenging to an organization accustomed to managing projects with fixed targets, timelines, and expected outcomes. Difficult as it may be, establishing a framework that encourages innovation while accepting that business priorities can change and experiments will sometimes fail, is essential to unlocking the potential not just of data analytics tools and approaches, but also the real talents of your people.
Industry standard frameworks such as CRISP-DM and its variations have been defined and evolved over the past few decades as a means of organizing analytics activities. While these reference models each have their strengths and weaknesses, understanding their general principles can be useful in establishing an agile analytics framework fit for your organizations needs and aligned with existing methodologies:
The five steps to continuous analytics iteration
- Business Use Cases – The first step of any analytics effort must be focused on identifying the business needs to identify analytics use cases and their value and establishing a framework for prioritizing efforts and objectives.
- Understanding – Acquiring, profiling, and validating that the data available meets the business needs in terms of scope, timing, and quality.
- Implementation – Building out the analytics solution with a focus on essential aspects of the use case to ensure that implementation cycles can be kept short, and preliminary results can be evaluated with a minimal investment of effort.
- Evaluation – Considering whether the solution delivers the promised business value, while also confirming that the business need still exists.
- Deployment – Releasing the solution and integrating it with operational workflows through training and re-engineering processes of the target users
As analytics use cases iterate through this process, it is unlikely that all will move straight through to deployment. In an agile environment, it is expected that business demands will evolve, and that more will be learned about use cases and data as the process unfolds. Keeping cycles short and iterative means that this new knowledge can be fed back into the process and adjustments made. Making the decision to not deploy a solution should still be regarded as a successful experiment, and not a failure of the process.
Principle #2 – Deploy analytics in alignment with your organization’s abilities and needs
There are two main facets to how data analytics engages with the broader organization. First, how are analytics resources deployed, and second, what are the delivery modes. Misalignment between either of these and the analytics abilities and needs of the organization can result in unrealized value, even if the data itself is fit for purpose.
Though open analytics access has become more popular, deployment should be thoughtfully aligned to the business’ needs to realize value with minimal friction. The three primary styles of deployment are:
Centralized: Capabilities are owned by a dedicated analytics group that supports use cases across the organization. This approach, where skills and capability can be rapidly developed, is ideal for smaller organizations, or those looking to build a new analytics function.
Federated: Analytics resources sit within business teams but report via a dotted line to a central analytics team. As an organization matures and both the analytics capabilities and need for customization increase, this approach offers a balance of agile response to business needs with central management of technical direction and best practices.
Distributed: Empowered analytics teams sit within different business units and operate independently. This approach offers the greatest extent of agility and customization but to optimize efficiency, a central analytics team should still provide guiding frameworks and best practices while promoting knowledge sharing between the teams.
Beneath the organizational structure provided by the deployment approach, organizations have more options when it comes to their analytics delivery modes. In order of increasing complexity, delivery modes include standardized data products (e.g., static reports), self-service exploration (e.g., interactive visualizations), and data science (e.g., advanced predictive modeling).
An effective strategy should consider the diverse needs of different user groups and prioritize implementation accordingly. Different groups will have varying expectations on how they want to engage with analytics data. The analytics function should consider these needs in terms of different ‘user personas’ and accommodate them through a well-planned roadmap.
Modes of Data Delivery
Analytics technology and tools are integral to the overall data platform, but this doesn’t simply mean there is a need for tight controls and restrictions on the tools being used.
Analytics practitioners often utilize specialized tools outside those typically supported by the technology team, so the platform strategy should focus on enabling these tools in a secure and managed way, facilitating effective analytics while minimizing risks.
Data analytics in the cloud
Cloud-based analytics platforms and tools have played an important role in the rapid evolution of analytics and are an essential foundation to high-performance analytics practices for three key reasons:
- Agility – The ability to rapidly experiment with and adopt new tools, techniques, libraries and models. Cloud analytics platforms such as Databricks, Sagemaker and Microsoft Fabric are frequently updated and enhanced, but they also integrate with marketplaces that allow incorporation of 3rd party components into the analytics process that can greatly accelerate the capabilities of the base platforms.
- Scalability – The ability to rapidly and efficiently scale storage and compute resources allows analysts to experiment with and deploy solutions using large data sets or complex calculations in a way that was not possible without a cloud-based infrastructure. At present, Snowflake is an exemplar for easy-to-use, cloud data scalability.
- Collaboration – Having shared cloud platforms with integrated team features allows an analytics team to foster a culture of collaboration that can significantly increase the efficiency and reduce the risk associated with analytics practices. Because of its essential role, all the solutions mentioned under agility and scalability offer some form of built-in collaboration functionality.
Principle #3 – Measure the return on your analytics investment
Leading with a focus on analytics can quickly start delivering measurable benefits by engaging with key stakeholders to identify and implement use cases and then using these successes to build support and buy-in for the overall data program.
Just as every other part of the business is asked to validate its return on investment (ROI), analytics should also deliver the data to prove its value. That said, when defining KPIs for the analytics function, it’s important to remember that analytics are agile and experimental. Not every initiative will yield a direct, measurable business benefit, but the overall program should clearly contribute to business value. If fit-for-purpose analytics data isn’t being regularly utilized by the intended users, there is likely a disconnect somewhere across the data program terms of co-ordination or prioritization. Even timely, accurate, insightful data can’t deliver value if it’s not being used.
Finally, intangible benefits, like evolving the data culture, should also be considered. Challenging as they may be to quantify, there is clear value to increasing the data fluency and effectiveness of data-driven decision making within an organization. Though it might not be their primary objective, analytics often deserves credit for the progress seen in these areas as they work with the business on utilizing data functionality and outputs.
A holistic view—How data analytics connects to the other elements
Analytics delivers tangible outcomes that are relatively easy to assess as a proxy for the overall data program—are the results correct, and were they quick and efficient to deliver? But it’s important to do so with an understanding of how analytics fits into the overall data program—the governance, operations, and architecture that made this last step in the data journey possible, and how they integrate and interact to provide feedback and enable improvements.
For the operations function, analytics should be a close partner in identifying initial use cases, data issues, monitoring process performance, and assessing key metrics. At the same time, analytics relies on the accuracy and timeliness of data being generated by data operations.
Similarly, data architecture is an essential partner to analytics in addressing the always-evolving demands of the business. Often this manifests in architecture evaluating and, when appropriate, implementing new technologies to add or improve analytics tools or capabilities. In turn, analytics provides data architecture with valuable insight into utilization, spending, and performance of the technology and dataflow infrastructure.
To deliver on the potential value of data, analytics must be at the heart of your data operating model. This places data analytics in a frontline role both for receiving business feedback, and for collaboration with the other data functions to share and co-ordinate iterative improvement of the data platform.
Success is Built on the Fundamentals
The value of data is real, and the potential is only growing with the recent advances in AI and machine learning. Though the investment management industry and world at large have seen a flurry of talk and activity on these frontiers of data, many firms are still struggling with the basics.
Returning to the fundamentals of data management is an effort to cut through the complexity and unique challenges that each firm faces to focus on the hallmarks of success:
- Start with a thoughtfully defined data strategy that is aligned to your business strategy.
- Embrace data governance, not only for its ability to manage risk and quality but also for its ability to set the stage for a data operating model that is on the offense in pursuit of greater business value.
- Empower data product owners to transform the data strategy’s vision into a practical and always-evolving plan for delivering value.
- Instill your data architects with an understanding of the business use cases and value that they are integral to delivering.
- Focus analytics on the iterative pursuit of measurable business value.
- And finally—ensure that your approach across the entire data ecosystem is driven by business value, acknowledges the importance and necessity of continuous improvement, and understands that a holistic view of the people, processes, and technology is key to their interconnected success.
To deliver the value that your firm sees in data, the solution is in the approach that you take to constructing or transforming your data ecosystem. Tantalizing as they may be, before incorporating the latest data innovations, it’s vital to ensure that you are building on a solid foundation. No matter where you are in your data management journey, Citisoft has the battle-tested experience to support your pursuit of data-driven business value.
Partner with experts in data management
Learn how our Data Practice can help you tap into new insights,
design a data ecosystem to enable future growth, and provide
practical solutions to even the most complex data challenges.
We have the expertise and track record to design and deliver projects of any scope.
Discuss your needs with an expert and discover how we can help.