Description
Sizing a Business Intelligence Environment
Sizing a Business Intelligence
Environment
InfoManagement Direct, J une 2004
Anand Bondre
What is Overgrowth?
Size becomes a concern at some point in the life of a business intelligence (BI) platform.
By their very nature, BI platforms have large volumes of data scattered throughout the
environment, but they can easily grow beyond "right" size quite unless attention is paid to
this challenge. An optimally sized BI platform not only performs better but also has many
cost and operational benefits. An overgrown infrastructure has business and technical
issues and will need to reduce the volume of data without affecting business operations.
ADVERTISEMENT
The size of a BI platform should be in proportion to the business requirements it is addressing. For example, terabytes
of data in an infrastructure can be justified if the business has large number of users who run hundreds of reports to
analyze tons of data. But a smaller business environment with a lesser volume of business activity, even with
terabytes of data, can have an overgrown infrastructure.
In general, the technical team should be capable of envisioning the right size BI
infrastructure for the business requirements that are being addressed. Sizing utilities
provided by a vendor help ensure that the platform does not overexpand.
Why Size a BI Platform?
There are many reasons to have a right-size BI platform. An overgrown BI infrastructure:
• Is not cost beneficial for the business - the larger volume of the infrastructure
increases costs of database maintenance and logistics (disk space etc);
• Leads to larger data load windows that negatively impact business operations;
• Results in indirect negative effects on production support costs and networking of
the infrastructure;
• Presents thorny administrative activities (e.g., system refreshes) with larger
volumes of data in the environment.
• Can have performance issues in business operations;
• Face difficulties with upgrades and enhancements in large infrastructures
(refreshing data in storage objects becomes time-consuming, required production
downtimes are longer etc.);
Page 1of 3 Sizing a Business Intelligence Environment
19/01/2010 http://www.information-management.com/infodirect/20040604/1004488-1.html?zkPrinta...
• Have negative impact on the three crucial elements of the business intelligence
infrastructure - time, money and human resources.
Right Size a BI Platform
Keeping a business intelligence infrastructure as small as possible in size and yet as
effective as it should be is sometimes a challenge and needs adequate design planning
and active support and maintenance.
• Design phase - Normally the flow of data from online transaction processing
(OLTP) to a given BI infrastructure begins with the first destination of persistent
storage area. Later it is made available in operational data store (ODS). The final
destination of data is either an ODS or a multidimensional data target or both. If
redundancy in data is going to be observed in number of ODSs or in multidimensional
database targets, it is best if it is addressed in design phase itself. In the cases where
data needs to be available in two or more data targets to meet analytical requirements
of one part of the business per data target, the design of the BI infrastructure should
incorporate needs of more than one element of business in one data target. For
example, if an ODS is being designed for point-of-sales division, it should be extended
to meet requirements of another division, such as product sales. Once two different
parts of business can access the same data, users can be authorized to access data
pertaining to their only business element. This not only helps satisfy more
requirements with less effort, but it also helps for future implementations as the data is
available to be used from one target.
• Active in business - The persistent staging area (PSA) largely affects the size of BI
platform as it holds large volumes of data which grow constantly. Data in a PSA is
mainly for contingency purposes. For the most current technical problems with
business data, administrators can fall back to the PSA (depending on certain other
terms). But getting data that is six months old from the PSA would not be optimal; but,
in fact, it would be a cumbersome task that requires meticulous planning to avoid data
integrity issues. (Businesses should not have historical data in the PSA.) If the BI
platform is adequately active, problems should be discovered within a day or a week
at the most. It is also important to note that the data in PSA is not reportable. That is, it
is there solely for those just-in-case situations. The bottom line - it does not make
sense to leave data history in the PSA because it is useless when it gets old. If you
ever need the data again, you could get it quickly and easily from the business
transaction systems. Only desirable data history should be stored in the PSA. Older
data should be purged regularly. This helps in reducing redundancy in a BI
infrastructure because the conditioned and formatted data that is used for analytical
purposes is actively available in necessary data targets.
• Long history - Archiving is another mechanism that helps reduce size and
redundancy in the BI environment. Although archiving is very useful, it is mostly
performed on historical data. For many legal and business requirements, the business
intelligence landscape is required to keep data active and available for certain period.
But at a determined time, data can be archived. Very old data does not add much
value from an analytical point of view as markets, industries, products and economies
tend to change over a period of time. Analysis of very large volume of data (for
Page 2of 3 Sizing a Business Intelligence Environment
19/01/2010 http://www.information-management.com/infodirect/20040604/1004488-1.html?zkPrinta...
example, last ten years) is quite rare and is also technically difficult. How long a
business needs data to be active in its landscape depends on what kind of industry
the business is in. For example, in the airline industry, tourists' destinations, seasons
and holidays do not change a lot. But a company in music industry probably would
see noticeable changes in their business in a short time. The depth of history that is
needed for airline industry is different from music industry. Hence their archiving
requirements are different. No matter what industry the business is in, once the data in
its BI landscape becomes outdated, it should be archived to relieve the landscape.
Many other factors can be considered to keep the size of a BI infrastructure optimum.
Indexes on flat tables (or dimensions) and aggregates on multidimensional database
structures contribute to the overall size of a BI platform. Formulae and spreadsheets
provided by the vendor of the BI platform must be used extensively to know the size of
the BI objects before they are active in business. A list of BI objects, list of other
elements (master data, software size, etc.) and anticipated growth will give a full picture
of the BI platform. Once this is done, it can be used as a reference to determine the
growth in the overall size of the platform.
Anand Bondre is a senior data warehouse consultant with Rapidigm, a consulting firm
specialized in information management and strategic implementation of business
intelligence solutions. He has more than 12 years of IT industry experience mainly on
SAP applications. His recent certified specialization is in the comprehensive execution of
business intelligence and analytical solutions of SAP. Bondre can be reached at
[email protected].
For more information on related topics, visit the following channels:
• Business Intelligence (BI)
©2010 Information Management and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.
Page 3of 3 Sizing a Business Intelligence Environment
19/01/2010 http://www.information-management.com/infodirect/20040604/1004488-1.html?zkPrinta...
doc_940754120.pdf
Sizing a Business Intelligence Environment
Sizing a Business Intelligence
Environment
InfoManagement Direct, J une 2004
Anand Bondre
What is Overgrowth?
Size becomes a concern at some point in the life of a business intelligence (BI) platform.
By their very nature, BI platforms have large volumes of data scattered throughout the
environment, but they can easily grow beyond "right" size quite unless attention is paid to
this challenge. An optimally sized BI platform not only performs better but also has many
cost and operational benefits. An overgrown infrastructure has business and technical
issues and will need to reduce the volume of data without affecting business operations.
ADVERTISEMENT
The size of a BI platform should be in proportion to the business requirements it is addressing. For example, terabytes
of data in an infrastructure can be justified if the business has large number of users who run hundreds of reports to
analyze tons of data. But a smaller business environment with a lesser volume of business activity, even with
terabytes of data, can have an overgrown infrastructure.
In general, the technical team should be capable of envisioning the right size BI
infrastructure for the business requirements that are being addressed. Sizing utilities
provided by a vendor help ensure that the platform does not overexpand.
Why Size a BI Platform?
There are many reasons to have a right-size BI platform. An overgrown BI infrastructure:
• Is not cost beneficial for the business - the larger volume of the infrastructure
increases costs of database maintenance and logistics (disk space etc);
• Leads to larger data load windows that negatively impact business operations;
• Results in indirect negative effects on production support costs and networking of
the infrastructure;
• Presents thorny administrative activities (e.g., system refreshes) with larger
volumes of data in the environment.
• Can have performance issues in business operations;
• Face difficulties with upgrades and enhancements in large infrastructures
(refreshing data in storage objects becomes time-consuming, required production
downtimes are longer etc.);
Page 1of 3 Sizing a Business Intelligence Environment
19/01/2010 http://www.information-management.com/infodirect/20040604/1004488-1.html?zkPrinta...
• Have negative impact on the three crucial elements of the business intelligence
infrastructure - time, money and human resources.
Right Size a BI Platform
Keeping a business intelligence infrastructure as small as possible in size and yet as
effective as it should be is sometimes a challenge and needs adequate design planning
and active support and maintenance.
• Design phase - Normally the flow of data from online transaction processing
(OLTP) to a given BI infrastructure begins with the first destination of persistent
storage area. Later it is made available in operational data store (ODS). The final
destination of data is either an ODS or a multidimensional data target or both. If
redundancy in data is going to be observed in number of ODSs or in multidimensional
database targets, it is best if it is addressed in design phase itself. In the cases where
data needs to be available in two or more data targets to meet analytical requirements
of one part of the business per data target, the design of the BI infrastructure should
incorporate needs of more than one element of business in one data target. For
example, if an ODS is being designed for point-of-sales division, it should be extended
to meet requirements of another division, such as product sales. Once two different
parts of business can access the same data, users can be authorized to access data
pertaining to their only business element. This not only helps satisfy more
requirements with less effort, but it also helps for future implementations as the data is
available to be used from one target.
• Active in business - The persistent staging area (PSA) largely affects the size of BI
platform as it holds large volumes of data which grow constantly. Data in a PSA is
mainly for contingency purposes. For the most current technical problems with
business data, administrators can fall back to the PSA (depending on certain other
terms). But getting data that is six months old from the PSA would not be optimal; but,
in fact, it would be a cumbersome task that requires meticulous planning to avoid data
integrity issues. (Businesses should not have historical data in the PSA.) If the BI
platform is adequately active, problems should be discovered within a day or a week
at the most. It is also important to note that the data in PSA is not reportable. That is, it
is there solely for those just-in-case situations. The bottom line - it does not make
sense to leave data history in the PSA because it is useless when it gets old. If you
ever need the data again, you could get it quickly and easily from the business
transaction systems. Only desirable data history should be stored in the PSA. Older
data should be purged regularly. This helps in reducing redundancy in a BI
infrastructure because the conditioned and formatted data that is used for analytical
purposes is actively available in necessary data targets.
• Long history - Archiving is another mechanism that helps reduce size and
redundancy in the BI environment. Although archiving is very useful, it is mostly
performed on historical data. For many legal and business requirements, the business
intelligence landscape is required to keep data active and available for certain period.
But at a determined time, data can be archived. Very old data does not add much
value from an analytical point of view as markets, industries, products and economies
tend to change over a period of time. Analysis of very large volume of data (for
Page 2of 3 Sizing a Business Intelligence Environment
19/01/2010 http://www.information-management.com/infodirect/20040604/1004488-1.html?zkPrinta...
example, last ten years) is quite rare and is also technically difficult. How long a
business needs data to be active in its landscape depends on what kind of industry
the business is in. For example, in the airline industry, tourists' destinations, seasons
and holidays do not change a lot. But a company in music industry probably would
see noticeable changes in their business in a short time. The depth of history that is
needed for airline industry is different from music industry. Hence their archiving
requirements are different. No matter what industry the business is in, once the data in
its BI landscape becomes outdated, it should be archived to relieve the landscape.
Many other factors can be considered to keep the size of a BI infrastructure optimum.
Indexes on flat tables (or dimensions) and aggregates on multidimensional database
structures contribute to the overall size of a BI platform. Formulae and spreadsheets
provided by the vendor of the BI platform must be used extensively to know the size of
the BI objects before they are active in business. A list of BI objects, list of other
elements (master data, software size, etc.) and anticipated growth will give a full picture
of the BI platform. Once this is done, it can be used as a reference to determine the
growth in the overall size of the platform.
Anand Bondre is a senior data warehouse consultant with Rapidigm, a consulting firm
specialized in information management and strategic implementation of business
intelligence solutions. He has more than 12 years of IT industry experience mainly on
SAP applications. His recent certified specialization is in the comprehensive execution of
business intelligence and analytical solutions of SAP. Bondre can be reached at
[email protected].
For more information on related topics, visit the following channels:
• Business Intelligence (BI)
©2010 Information Management and SourceMedia, Inc. All rights reserved.
SourceMedia is an Investcorp company.
Use, duplication, or sale of this service, or data contained herein, is strictly prohibited.
Page 3of 3 Sizing a Business Intelligence Environment
19/01/2010 http://www.information-management.com/infodirect/20040604/1004488-1.html?zkPrinta...
doc_940754120.pdf