Business Intelligence Reporting Framework, a Maturity Model for BI Best Practices


BusinessintelligenceThe Business Intelligence space is relatively young but it is the fastest growing part of the technology market place and based on data growth is shows no signs of slowing down.  Data is doubling every 2 years.  It impacts every part of any business eg CIO, business departments, even security auditors.  Many new jobs titles like Chief Data Officer (CDO) are now part of HR job description for high performance companies.  These new roles will focus on how to make data part of the future innovation and are on every  CDO’s agenda.  How you are able to dominate this space will determine your competitive edge in the coming years.  Many companies are sitting on mounds of data, secured in massive databases with poor insight and far too little prescribed actions that are data driven.  If we are going to get to the next level then the first thing we need to understand is where we are and then we can take the next step.  Let us proceed to look at the BI Reporting Framework proposed by Stephen Choo Quan, a lead practitioner in the BI space for over 20 years.

There are 5 Levels in the Framework and each level is gradually more mature as it progresses from Level 1 to Level 5.  The maturity will result in effective and efficient consumption of the data by the user.  This means the user will arrive at the exact answer faster and with less effort as the BI Framework matures.  The two units of measure are TIME and EFFORT.  These are extremely important to measure as we want to be actionable to impact business outcome and not just be analytical and book smart.  What is important is that at any level eg Level 4 is a sum of all levels below it in addition to its current level.

The user culture typically defines how far an organization maturity gets to.  Consumer type users tend to max out at level 2, analyst tend to be at level 3 and 4 and the data scientist tend to be at level 4 and 5 in the framework. Levels 1-3 tends to focus on data to monitor, control and regulate the business flow.  Levels 4-5 creates data architecture as an infrastructure to enable data to become the center for innovation.  Let us explore this journey together and as we do ask yourself the following:

  • Where are we in the maturity level?
  • Who are the constituents, their culture and where do they want to be?
  • What can we do today to get to the next level?
  • What talent is needed on your team to get to that level and how do you acquire/leverage it?
  • How can we expand from analogue revenue thinking to append digital revenue thinking to drive innovation?
  • What is the future of data governance for pushing innovation and using data as business assets? (are we owners or stewards to encourage data usage?)


the typical characteristic of a practice that is at Level 1 are easy to spot.  There are usually a lot of reports with about 90% of them not relevant to the user and are, at best un-interesting to the user.  The large amount of report is usually explained by the business function trying to cover all possible reporting aspects that it could think of and the usage of these report reflect that they are more of clutter than useful. 10% of the reports will be useful to the user but often the user will have to manually run, wait on render and download a few reports and then merge the many runs together on the desktop. Each taking significant time and effort each time removing unwanted data and linking data in context across data sources to arrive at a final desired output.   The main complaint is users need a custom report from the development team and that can be expensive and time consuming, this can be weeks or even months.  The data is in the database but its so hard to get it out and in the format that is useful to the user/department.  The conditions that lead to this level 1 type build is a green field reporting build with a strong development team, there is usually no architect or inexperienced architect.


Over time the report usage will drive the development team to make relevant reports, new custom reports.  These reports may be parameter-ized to reduce the number of similar reports that are just different by the content but NOT by the structure.  The usage on the reports will be much higher and of more value.  The downloaded content will be in a directly usable format that saves the user hours of work enhancing and filtering data, day over day or week over week.  The clutter will be reduced and the users can easily find useful, relevant reports.  If the user needs changes then they will have to go back to the development team to create another release.  This can be frustrating to the user.  The time to get the data is fairly quickly until changes are needed.   You can identify Level 2 if you are not able to respond to new questions quickly.  These changes are driven by the data begetting a new question to answer some trend or point of interest in the data.  The conditions that lead to this level 2 type build is a strong subject matter expert who is vocal and goal oriented.  The development is usually augmented with a business intelligence analytic person.


A Level 3 user can select/re-order report columns, create calculated measures and filter data as desired.  The ability of the end user to sort and format reports into dashboard will dramatically reduce explanation type analysis to the business.  These users at this level can become analyst or power users looking for trends in the data as they have the ability to ask ad-hoc questions and get relatively immediate answers.  Business definitions will start to get standardized as the reports will be securely shared with other users of similar data access needs.  Reports can be scheduled saving time and have multiple ways of delivery including authorized login, email or FTP. Information will be able to be pulled out of sources on demand or pushed to the user on named days and times.  You can identify Level 3 by the number of new reports, the number of scheduled reports running and the amount of data being pulled back as many new ad-hoc questions arise.  Users at this level 3 tend to feel empowered and satisfied.  The caveat with the report democracy is that there may have variances in calculations and segmentation criteria are not agreed to and standardized.  These can lead is data trust issues if not managed.  There is also a strong need to secure master boiler plate reports and the derived data as the reporting insights are now seen as valuable assets.  The conditions that lead to this level 3 type build is a strong subject matter expert and a strong business intelligence expert.  There is usually the starting of a data integration practice at this level.


Level 4 builds on Level 3 by standardizing many of the common and useful reports to create common nomenclature and dimension tables to support the business standards.  The standardized dimensions allow reports to be interconnected and build relationships and workflow.  If we create hyper connected data then more ideas will be generated by the organization. The concept of drilling across is a real level 4 trait.  Most tools make drilling up and down hierarchies possible but drilling across but in context of the dimension requires more maturity.  The reports start to share common dimensions as parameters.  The complex calculations are localized and made reusable to all reporters.

The most time consuming part of the visualization delivery cycle is the data preparation and becomes one of the key focus areas at Level 4.  This Level is characterized by more advanced data capture which leads to rich data sets in real time or pseudo real time.  Strong modelling powered by big historical data stores from many different sources is typical traits at level 4.  This is also accompanied by rich interactive widgets that can visualize the data points and make it human digestible, this is known as making human pre-attentive visualization.  Mature Data models are populated by sophisticated automated Extract Transform and Load (ETL) tools that speed up the visualization. In memory stores are popular in some tools that offer answers at the speed of thought on many device types.

Key Performance Indicators (KPI), KPI composition and leading indicators are all usually designed and built for real time monitoring.  Digital dashboards are common at this level, looking at multiple parts of the business in real time and monitoring for anything that seems out of the normal.  These thresholds are set by mature businesses to ensure they are aligned to the projected planned goals.  Alerts can be pushed to the users or can be pulled by visiting the dashboard when thresh holds are broken. The conditions for Level 4 is driven for 3 main factors the need for real time data (faster data refreshes), fast query times and the need for a larger scope of the data domain.  The analytics satisfy the explanatory aspects for consumers and now consumers  want to see the data changes faster and refresh happening as fast as possible.  The power users also want to do more data exploration now that many operational KPI are automated.


In the previous 4 Levels we are rear view mirror type of analysis with a focus on business monitoring, control and regulation, looking at in the best case real time or in most cases historical data.  Level 5 seeks to use data for innovation not just control.  It links level 4 output and flows but adds rules to those workflows to add a prescriptive actionable compliment. How can we predict our future if we do not understand our past?  Level 4 and Level 5 build on each other.   The Level 5 seeks to interrogate the data to see what else can data be used for.  This is the Level that pushes the term analytics beyond business intelligence.  It results in decision trees based rules on user dimensions (features and labels) to add prescriptive widgets to analytic applications. The decision trees can be rule based or machine language defined depending on how static the labels are in the dimension and the complexity of relationships. Google maps is a one such use case where the app can prescribe an alternative routes with changing traffic patterns and tell the amount of minutes that will be saved in real time.  This level 5 might engages systems machine learning to train based on historical cases to create predictive decision trees and retain better and more accurate business rules.  This information that is typically known only by the subject matter experts (SME) with many years of experience. SMEs are human capital that build and maintain rules based trees today but SME cannot scale but machine learning can scale with little cost overhead.

SME usually drive the business innovation but the next innovation idea will most likely come from a data pattern found at this level 5 processing.  SMEs today are now troubled with larger volumes of data than ever before and that volume continues to double every 2 years.  The Level 5 BI will augment SME knowledge when it finds more complex patterns in the data than the traditional ad-hoc queries.

If we have large amounts of data that can support revenue or cost facing processes then machine learning can help to find patterns to optimize the process.  More details on why this form of Artificial Intelligence will be critical augmentation to SME can be seen here.  The Level 5 utopia goal is to get to a segmentation of one.  That way we treat each person as unique and special but at the same time can find categories and influence behavior.  Sentiment analysis of the languages will allow structured and unstructured to be mashed together to make richer data sets for the future. I do not think I can cover this in just one paragraph so this Level will be continued on this article which includes big data.  My guess is that big data as a asset will possibly drive the future if formatted and packaged for commercial use.  The condition that drives Level 5 is using data as a asset via data exploration.  We are yet to really derive the true value of decision support from IT.  We have seen it in operations but we are yet to see it in effect for decisions.



Level 1: Many reports but few are consumer useful and need to be augmented or reworked on the desktop.Usually operational detailed downloads.

Level 2: Relevant useful consumer reports but still suffering long development time for new requests

Level 3: Self service and a growing number of rich analytics, KPI and dashboards that are non IT driven.

Level 4: Strong operations monitoring that impact business outcome using real time data and analytics

Level 5: Subscription actions based on predictive models of large historical data sets.  SME are augmented with AI in decision making.


Stephen Choo Quan

Stephen is a double threat holding both SAP business objects certified architect as well as being a certified IBM DB2 Database Developer. read More…