It is 2023 and data products (or Data Mesh in a broader sense) are a priority for a data-driven enterprise. Many organizations still need to work on clearly defining data products and how they create value for their organization and clients. I am not going to get into the technicalities of what is a data product or why it shouldn't be confused with data assets. Twain said something about fewer words, so I will skip words like domain-oriented, decentralized ownership, and self-serve for better readability. Instead, we will talk about key ideas derived from Citisoft clients who seem to get data products.
Data is the fuel of operations, and like fuel, it is non-renewable. It is not something that can be quickly recovered when lost or mismanaged. Therefore, in many cases, even when companies have a lot of data (especially in financial services), they feel they could make better use of it. To make use of the data, an organization must know how good the data is and to know that, the organization must know where it comes from—its supply chain, if you will.
We have been slowly getting there—I’d wager that every organization has at least been taking steps to ensure they are aware of their data supply chain. A clear view of their data supply chain tells companies where they stand and what they need to do. They need to be able to answer questions such as: What are the sources of our data? How is it created? Who owns it? How can we access it? And how do we get insights from it? These are all perennial questions. Today, financial institutes are expected to have a library of policies and procedures around them. These are also expensive questions to answer.
During a recent discussion on many of these topics with the Chief Data Office at a global bank, a familiar question arose (a favorite for the consulting types):
"What are we aiming for?"
Behind the “Data Product OKRs,” providing insights into business operations and customers lay a more straightforward answer: trust. Organizations need to trust their data to operate confidently. Yes, there are SOC rules that tell organizations what they must do, but being able to trust the data is the real goal.
Organizations thinking about data products are realizing that managing data in a federated manner is as much about technology as it is about organizational and mindset change.
Traditionally, most companies are project-driven when it comes to data. This means they focus on executing business requests and producing reports based on that data. This approach works but has some significant drawbacks, including slow time-to-delivery (because of the need for customization and manual effort), lack of reuse, and the risk of delivering incorrect data.
A product-driven organization focuses on creating a sustainable architecture (business and technology) for managing data and delivering value from it. Technology-wise, this means building an infrastructure that allows anyone in the company to access data as needed and use it to drive business decisions—without having a developer or DBA do so manually. From a business standpoint, it means standing up business teams with the technical aptitude to take on data stewardship by actively participating in its supply chain.
The product-driven approach advocates commonality, not just in the ability to store and access data, but also in how an organization thinks about data requirements. Data requirements must be repeatable rather than specified piecemeal by each user group (or worse yet, by individual users themselves!).
An even better approach would be to tie data requirements (and thereby data products) to business outcomes.
Data products are not data outputs, and they’re not just data. They aren’t just software, either. In fact, they may not even be visualizations (sounds horrible, I know!). Data products are simply trusted data deliverables that provide value through a specific business outcome they produce. They can be associated with corporate-level outcomes, like risk-weighted assets, or be something more business-line specific such as liquidity.
The alignment of business outcomes with data products should be a continuous iterative process throughout their lifecycle. Without this alignment, it can be a struggle to work backwards and justify budget spending. Anyone who has spent money on documenting data lineage would attest to that. More often than not, clearly defining the value and business outcome automatically makes it appealing in often-quoted functional dimensions like reusability or read-optimization.
Data products can be tricky to manage and measure because they are relatively "intangible" compared to other products. However, I’d argue that they shouldn’t be treated differently from any other product. And like any other products, clients (end users) need to be able to measure the impact of their data products. These clients need to be able to quantify the return on investment and demonstrate that their data products are either increasing revenue or reducing costs.