Skip to main content

Another look at Data Warehouse management

Build a data ecosystem that streamlines rather than slows your daily operations.

The challenges around data propagation and data warehouse management are well documented in the investment management industry. Data underpins nearly every step in the value chain, yet its collection, management, storage, and analysis remain difficult. Cutting-edge technology provides part of the solution, but technology is only one component of a holistic approach. Breaking down silos and legacy systems is critical. But this requires a change in mindset if investment management firms want to remain nimble, innovative, and competitive.

Traditionally, data warehouse management glitches have been seen as the preserve of the technology team. Fixing data storage and interrogation problems was not easy. The advent of cloud-based solutions has enhanced agility, transparency, and data storage, but technology only goes so far in solving broader data challenges. A more effective long-term approach involves pivoting away from the IT team while increasing business user involvement to create a metadata-driven, user-friendly information management framework for your day-to-day operations.

Start with your business objectives in mind

The first step in developing an effective data warehouse strategy is gaining a better understanding of your challenges from an operational and developer perspective. For the former, the complexity around setting up data feeds, developing reports, and building dashboards are frequent stumbling blocks that we address in our client engagements. Equally as important is tackling the fact that data warehouses, which structure data in a standardized manner, can become rigid without proper long-term planning. If this happens, you may find it difficult to upgrade existing processing logic, much less introduce new asset types and address evolving regulatory and reporting requirements.

Portfolio managers and finance teams also face complications from the multiple accounting systems they need to handle various asset classes and investment vehicle types, as well as varied versions of net asset value (NAV). If these complexities are only unearthed after your data warehouse becomes operational, you will lose time and struggle to keep pace as your business needs change.

Developers, on the other hand, must be well versed in investment management domain knowledge, including how your business operates, the asset classes involved, trade and position management, and the web of third-party systems that offer point solutions to different steps in the fund lifecycle. If you look externally for support in developing your data warehouse, make sure your partner has relevant industry experience as well as technical know-how.

Focus on user experience in managing data warehouse operations

A deeper dive into these components enables firms to develop a software and data ecosystem that prioritizes ease-of-use over technical complexity. This will require technology changes that drive simplification and enhance traceability. Such technology changes will also need to encompass processes such as monitoring, governance, and support, as well as organizational training.

On the first point, abstracting technological complexity enables non-programmers to add or amend your data propagation, transformation, extraction, and visualization routines; this is similar to how Excel empowers analysts to convert text to data and add filters and charts. In addition, the metadata-driven nature of the data ecosystem allows the storage of linkages between components. This lets non-technical users debug issues without looking into the code.

From a process point of view, scheduling and exception-handling procedures, and a well-defined escalation matrix are critical. Provide support on how to escalate and resolve issues. Ensure strong governance that revolves around greater controls over reporting and changes to feed processing. If partnering with an external provider, choose one that possesses a deep knowledge of investment management operations, knows the right team members to involve and questions to ask, and can demonstrate the ability to deliver future-proof data warehouse and ecosystem solutions.

As for your people, they need to be well equipped and trained on the fundamentals of feed processing, reporting concepts and data modeling. Involvement throughout the solution development project is critical, not just to get critical input, but also to develop a new mindset around information management and build buy-in for new ways of working.

From data storage to information management

Investment firms can reap several benefits from this information management framework. Done successfully, you will reduce total cost of ownership (TCO) and increase efficiency, which in turn improves your return on investment (ROI). You’ll free up senior members of your development team to focus on strategic initiatives while queries are handled by experienced but non-technical users, whose job it is to look at end-to-end data processes. Last but certainly not least, reporting is simplified as day-to-day changes can easily be made to warehouse components.

Creating technological edge in private equity. Learn how private equity funds, led by their CTOs and leveraging new technologies,
are driving digital transformation in PE portfolio companies.


Case study: Linedata Services platform

Linedata relies on a data management ecosystem to deliver a broad range of front-office and middle-office outsourcing services. Our “Linedata Services” platform provides dozens of clients and scores of Linedata analysts with secure, on-demand access to up-to-date insights and reporting. It provides a range of functionalities including AI-driven Cognitive Investment Data Management, Investment data analytics, risk analysis and reporting, and Reconciliation and NAV reporting.

The platform’s development provided an ideal test case for the information management framework approach outlined above. We started with a traditional data warehouse approach built around data storage, SQL Server Integration Services (SSIS) for extract, transform, and load (ETL) activities, and SQL Server Reporting Services (SSRS).

Given the complexity of the myriad data integration, operational and reporting requirements of our clients, this model proved to have limitations. To address these challenges without disrupting client service, we deployed a metadata-driven, user-friendly “information management framework”. Our project team included data architects, software developers, and operations experts, and drew on two decades of experience providing software, data, and services solutions to the investment management industry.

The benefits of this approach quickly manifested themselves. These included:

  • Reducing developer hours for supporting data propagation and reports by 75%.
  • During the rapid onboarding of a global client, hundreds of data feeds were configured by non-technical members of the Outsourcing Services team without developer intervention.
  • Services team members can define their reports and adapt these rapidly as client needs evolve.

About the author, Vishal Patil

Vishal Patil is the Senior Director, Advisory & Product Development at Linedata. He has 20+ years of experience in designing technical and outsourcing solutions for global and regional investment managers. He and his teams have helped dozens of firms achieve efficiencies and scale in software development and support services. Vishal leads Linedata’s advisory and software development services group in India

Need to kickstart or progress digital transformation in your organization? Contact Linedata today about our Software Advisory services, including custom-built data warehouse solutions.

Connect with us today