Data boundaries and data quality in a BI reporting system

By Evan Schmidt

The following article discusses problems than can arise when a BI reporting system has data managed within the boundaries of the reporting system itself.

 

Ideally a BI reporting system is a consumer of data rather than a system of record. In most cases the BI system should receive data and should not become the system of record for maintaining data. However, to get the reporting system launched in a reasonable amount of time, it’s often the case that the reporting system does manage some data within the boundaries of the reporting platform.

Here are some of the common types of data and situations where the BI system becomes the system of record:

• Data mapping between systems
• Connecting systems that do not share the same exact values or IDs
• Master Taxonomy Data
•  Geography hierarchies, groupings of attributes, etc.
• Sales and Targets
• Added data that is small in scope and does not change often
• Rules for KPI Indicators
• These may be indicator thresholds
• Metric Definitions
• Rules of qualification for various measurements

 

First, it’s important to realize that BI developers can be lazy when the BI system becomes the system of record for certain items. It’s easy to overlook the following IT like rules for keeping track of data in a BI system:

·         Expose the data in the reporting environment so users can see it

·         Keep track of created date, updated date

·         Keep track of created by user, updated by user

 

Second, it’s important to notice that keeping data in the reporting system can become a bad habit and can easily grow until it’s out of control. Keep the following rules in mind:

·         Do not create unnecessary data sets in the reporting database

·         Keep it small and simple

·         Keep the data organized

 

Finally the best long term solution for a BI reporting platform is to transition the data management responsibilities to new systems such as a targeting system, a master data management system, or a integrate the data into an existing application.

Second, it’s important to notice that keeping data in the reporting system can become a bad habit and can easily grow until it’s out of control. Keep the following rules in mind:

·         Do not create unnecessary data sets in the reporting database

·         Keep it small and simple

·         Keep the data organized

 

Finally the best long term solution for a BI reporting platform is to transition the data management responsibilities to new systems such as a targeting system, a master data management system, or a integrate the data into an existing application.