Download the entire article (.pdf file) by clicking the below icon:
[icon_list icon=”acrobat” size=”large” link=”/wp-content/uploads/2014/05/STILE-Point-14-Aligning-Knowledge-Management-Systems-to-Business-Strategy-v.final_.pdf” target=”blank”]
[clear]
In the modern enterprise, millions of bytes of information are generated every day. Within the enterprise are the information that is generated through transactional systems that are held in relational databases, network databases etc. A vast majority of information that is generated is also held in unstructured contents such as documents, email discussions, web page content, media such as audio and video. Apart from this, there is a chunk of information that is generated external to the enterprise such as social media sites and external websites etc.
The Knowledge Management systems in an Enterprise need to manage various kinds of data being generated in order to offer a competitive business advantage. In many organizations the unstructured data is not mined to provide the necessary business intelligence and hence most of the time goes under-utilized.
In this STILE Point, I shall explain some techniques and principles to use the data that is generated within the Enterprise (typically held in Structured and Relational Databases) to derive competitive business intelligence.
Strategy and Business Requirements
As stated in my previous STILE Point, there needs to be a good partnership between IT and the various lines of business. The Business Intelligence requirements must be driven by the company’s strategy and its vision. There should be a clear understanding of the stakeholders and the expectations from the system. Also it is important to clarify and understand the type of information that is requested by each of the stakeholder groups. The key performance indicators that drive the business at various levels must be stated and clarified. If there are benchmarks for these KPIs, it will help measuring the actuals vs. these benchmark values and report deviation.
This is an extremely important step, as this information will drive the data warehouse design.
As an example: the information that is requested by the executives of the company would be more to do with indicators such as new clients obtained in the last quarter; or income generated for last quarter vs. the income generated for the same quarter the previous fiscal year. Performance indicators related to the various business units of the firm should emanate from the top level and should allow the drill down to the lowest level of data. Now, in the case of Operational or Business Unit managers, the information would be specific to their line of operation along with their specific business indicators.
It would be the function of a good project manager or business analyst to work with the various teams and document business requirements from the system.
Figure: Business Intelligence (BI) Stack Architecture
Extraction, Transformation and Load Process (ETL)
The first part of designing such a system involves extracting data from disparate source systems. Based on the analysis of the business requirements, the analyst must try to understand the sources of data. As an example, if the requirement stated is to compare budget for cost centers and actual expended values; In the Enterprise, this information could be held in completely different systems. In yet another company that I was consulting for, the information about sales figures of the various retail outlets sent to the corporate by means of MS Excel files. The requirements were to collate these excel files from the sources and derive information out of it.
An important part of the extraction phase involves the parsing; cleansing and validation of the extracted data. There would be checks if the data meets the expected structure. If not, data may be rejected or based on logic can be accepted only part of it.
In the transformation stage, business rules are applied to the extracted source data to comply with the resultant target information. Various transformation types such as looking up for dependent values; or aggregation on certain fields can take place before the load can actually take place.
In the load phase, the extracted and transformed data is loaded to the target typically known as the Data warehouse. Based on the business requirements that were gathered, this process can vary widely. Based on needs, there could be separate data warehouse generated daily, weekly, monthly or annually. The load can also take place to a data mart, which is a simple form of a data warehouse focusing on a single functional area such as Sales; Finance or marketing. The specific department in the organization essentially controls data marts.
Multi-dimensional data store:
Typically the steps that I had discussed earlier, involves in the ETL process loading data to a data warehouse that is essentially a Relational Database Model or a Relational Database Management System (RDBMS). In a Multidimensional database or a Multidimensional database Management System (MDBMS) would allow the ability to rapidly process the data in the database so that answers can be generated rapidly. A MDBMS uses the idea of Cubes where dimensions of data are available to end-users. A typical example would be sales figures that are available in various dimensions of geography, time, cost centre or any other dimension that is desired. The end user can derive the measures for a single dimension or combination of dimensions.
In a typical architecture, data from the data warehouse are processed and loaded to the MDBMS structures. This operation can be timed and based on the requirements can be executed as well through routinely timed scheduled jobs.
As the data is already processed and pre-loaded, complex queries and analysis can be done from the cubes and the processing times are also much faster and responsive. In a typical scenario, there could be multiple cubes in the architecture with specific security constraints exercised on each of the cubes. So, the Marketing and Sales cube can be viewed only by the Marketing Department. Also, certain MDBMS allows the functionality to define Perspectives (or views) of a cube with independent access rights for smaller business teams.
Presentation Layer
The presentation layer in the architecture is where business users use client software. There are a variety of client tools that are used to present the data from the data warehouse or data marts to the end users. These include charting, tabular reports with drill down capabilities, canned reports with filters etc. Latest portal technology allows embedding more than one reporting tool in a single page to provide rich information for the end users.
In a typical BI implementation, the views that are available for different stakeholders can be different and can be personalized based on the user (and his/her role in the organization). For example, executives would like more visual representations and high-level overviews, but operational managers would like to see more of details and grid like reports on his/her page. This is why most of BI presentation software implementations provide a mixed bag of tools that is tailored to not only specific tool functionality but the audience as well.
In addition, the presentation layer provides an added level of security for different groups of users; and allows also rendering in a variety of different platforms such as web; mobiles and desktops.
In the next article, I shall delve into the more complex design and modeling aspects of providing intelligence out of the big data stores, semi and unstructured data that are generated within and external to the Enterprise.
[hr]
Mr. Subramanian has over 18 years of experience in the IT industry and provides consulting support in the following areas: Process Re-Engineering, Business-IT Alignment, Knowledge Management (KM) Portals and Information Systems, B2B web enabled systems, Business Intelligence (BI), Enterprise and Application Architecture; Systems Integration using Services Oriented Architectures (SOA); Cloud computing and Project Management.
Mr. Subramanian’s consultative engagements span across several firms globally in the Middle East, Europe, US, Australia and New Zealand, and India. He has also proven experience across diverse domain knowledge across sectors such as Governmental Initiatives; International Organizations; Manufacturing; Pharmacy/Health Care; CPG etc.
