Citisoft Blog

Big Data...Really? Or Is It Time to Go on a "Data Diet"?

Written by Steve Young | Mar 25, 2015

The “Big Data” bandwagon shows no sign of slowing down, but should it be a primary data management concern for most asset managers?

As we move through Lent, we sometimes realise that our New Year’s resolutions have not achieved very much, so we stiffen our resolve to ‘do better’. For many of us, the New Year brought with it a new diet, as we counted the cost of consuming (and carrying) a few too many calories. Forty days of restraint is a good way to energise your plans for a new beginning. Within investment management circles, should we be applying the same rigour to our ‘data diet’?

Over the last three years we have seen a growing emphasis within investment management firms on data management. Whilst this is understandable and in most cases appropriate, data is a broad term which unless qualified may lead to greater confusion than clarity. It is easy in the current climate to lose focus and adhere to a data strategy that lacks the necessary definition to be useful. Without understanding the data requirements that will drive the growth of the business, and how different data consumers and manufacturers (both people and technologies) use and create this data, data programmes are likely to fail.

In an era of ill-defined and misunderstood industry buzzwords (such as ‘Big Data’ or ‘IBOR’), firms will have a tendency to build data architectures in order to consume and store vast amounts of information (much of which may be unnecessary). As we see regulators and data suppliers taking an increasing interest in what data is stored and how and where it is consumed, the overall cost of storing data is growing as volumes increase (despite the fact that unit costs of storage themselves are falling). Firms that are keeping swathes of redundant data will be leaking valuable budget and management resources on information that brings zero value to the organisation.

Furthermore, as the volume of data being consumed and stored rises, it becomes increasingly difficult to ‘see the wood from the trees’. Extracting precise data sets can take longer to execute and the need to regularly purge or archive redundant data becomes more acute. Derived data in particular can often be discarded so long as the source data remains accessible.   

This lack of data discipline, or ‘data obesity’, poses a growing threat to the investment management industry. As with food consumption, introducing a ‘data diet’ (where only the required data calories are consumed) will be a key competitive advantage in the future. Of course, this requires vision, discipline, a well conceived strategy and (sometimes) even courage. The easy option is to consume everything ‘just in case’, but in the fast changing world of investment management, firms need to decide whether they wish to be built like a sumo wrestler (big, strong and powerful, built to push their way forward in an unchanging and traditional manner) or like a gymnast (with the ability to quickly learn and execute new and more complex moves). Once they have identified their approach, they must adjust their data consumption accordingly.

As with food, the diet alone will not bring the skills required, but it is a critical constituent in enabling a new beginning.