Description
The following individuals from the Data Warehouse team within IACS participated in the selection process in a limited fashion.
Rensselaer Data Warehouse Project
- 1 -
Business Intelligence (BI) Tool Selection Process
Implementing an aggressive schedule during the months of October, November, and December,
2001, the Data Warehouse team within Integrated Administrative Computing Services (IACS)
organized and managed the Business Intelligence (BI) tool selection process. Key participants
were identified and the Purchasing Office was contacted to establish a sound selection process.
The BI Tool Selection Committee
On October 18
th
, 2001, the Data Warehouse Sponsorship Group nominated over a dozen business
users from across campus to serve on the BI Tool Selection Committee. This committee was
tasked with choosing the enterprise-wide BI tool that will be used to access information within
the Rensselaer data warehouse environment. BI tool members included: David Bohan,
Director, Alumni Relations; Craig Clawar, Assistant Director for Technical Operations, P&DE;
Jackie Ellsworth, Associate Director, Controller’s Office; Patricia Gray, Assistant Director,
Contracts & Grants; Helen Grzymala, Associate Director of Financial Planning & Budget;
Sharon Kunkel, Director of Student Records & Financial Services; Jack Mahoney, Director of
Enrollment & Institute Research; Keith Martens, Purchasing Agent, Procurement Services;
Eileen McLoughlin, Director of Financial Planning & Budget; Cindy Price, HRIS
Coordinator, Human Resources; Kim Scalzo, Director, Professional & Distance Education;
Steven Schwan, Manager of Financial Operations, P&AS; and Donna Tomlinson, Manager of
Financial Operations, School of Engineering.
In addition to the committee members, numerous additional business users participated in the
selection process, ensuring that a broad spectrum of Rensselaer business users were involved in
the decision-making processes.
Integrated Administrative Computing Services (IACS) Participants
The following individuals from the Data Warehouse team within IACS participated in the
selection process in a limited fashion (i.e. developed the prototype data models, coordinated
training sessions, provided technical support, etc.): David Goldschmidt, Business Systems
Analyst; Ora Fish, Project Manager; Raj Kahlon, Data Warehouse Architect; Kate Owens,
Business Systems Analyst; and Jeff Stark, Business Systems Analyst.
Request for Proposal
The BI Tool Selection Committee helped to develop a BI Tool Request for Proposal (RFP) that was
sent out to 7 vendors on November 7
th
. The vendors included in the initial RFP were: Brio
Software, Inc.; Business Objects; Cognos Corporation; Hummingbird Corporation; Hyperion
Solutions Corporation; Information Builders, Inc.; and Oracle Corporation.
A total of 6 vendors responded to the RFP before the November 21
st
deadline (Hummingbird
Corporation did not respond).
Rensselaer Data Warehouse Project
- 2 -
Initial Vendor Presentations
Each of the 6 vendors that responded to the RFP was given an opportunity to present its BI tools
to the committee in 3-hour sessions during the last week of November. Such presentations were
generic in the sense that they did not make use of Rensselaer data prototypes.
Through a structured evaluation process, the committee narrowed the field from 6 vendors down
to the following 3: Brio Software, Inc.; Cognos Corporation; and Hyperion Solutions Corporation.
The following matrix was used t0 evaluate the 6 vendors:
Rensselaer Data Warehouse Project
- 3 -
The Hands-On Trial Process
The 3 vendors who survived the first cut were then invited to provide the committee a hands-on
trial period during the month of December. Each vendor set up and integrated its product(s) with
an Oracle database prototype in preparation for the hands-on trial process. The prototypes were
developed by the Data Warehouse team and included 2 subject areas: finance and student data.
The Data Warehouse team also purchased an NT machine to accommodate the selection process.
Once installed and integrated with the database prototypes, each vendor then spent a day training
users and answering questions. With hands-on access to the vendor tools, users found that such
sessions were extremely valuable and helpful in evaluating the 3 vendor products.
The following matrix was used to evaluate the 3 vendors that participated in the hands-on trial
period:
The Final Decision
At the end of the hands-on trial period, on December 19
th
, the committee met to officially select a
BI tool vendor. After the 2-month evaluation process, Brio Software, Inc. was chosen as the BI
tool solution for the Rensselaer data warehouse environment.
The voting that took place at the December 19
th
meeting yielded the following results:
# of votes
for 1
st
place
# of votes
for 2
nd
place
# of votes
for 3
rd
place
Brio Software, Inc. 10 3 0
Cognos Corporation 4 6 2
Hyperion Solutions Corp. 0 3 9
Note that one member of the committee ended up with a tie for 1
st
place between Brio Software,
Inc. and Cognos Corporation. Also note that two committee members abstained from voting for
Hyperion and Cognos due to their respective absences from the presentations and trial periods.
Rensselaer Data Warehouse Project
- 4 -
References
Once the committee identified Brio Software, Inc. as the top choice, the Data Warehouse team
contacted several companies and Universities that currently make use of Brio’s products. In
general, all references seemed quite satisfied with Brio’s ability to respond to and address their
needs.
The following references were contacted either via email or conference call:
! Anne Bennett, University of Connecticut.
! Scott Coopee, University of Connecticut.
! Cindy Sedlacek, Cornell University.
! Natalie Vincent, Syracuse University.
Technical Assessment
To ensure that the BI Tool Selection Committee’s choice of Brio Software, Inc. was a sound and
reasonable selection from a technical and architectural perspective, the Data Warehouse team
evaluated the top two BI tools after the user evaluations and voting was complete. Thus, the
remaining sections focus on the BI product suites from Brio Software, Inc. and Cognos
Corporation.
The fundamental goal of this analysis was to ensure the feasibility of implementation. Primary
focus technical questions and topics included:
! Product integrates well with data warehouse database (i.e. Oracle).
! Product integrates well with Informatica’s ETL solution, taking full advantage of the
underlying metadata.
! Product integrates well with portals.
! Product’s architecture is sound and sensible.
! Product’s hardware requirements are reasonable.
! Product’s security model is sound and fits into Rensselaer’s environment well.
! Product contains an effective administrative component.
! Product supports the ability to avoid “harmful” or “runaway” queries.
! Product supports multiple output formats (e.g. Excel, PDF, HTML, XML, etc.) in a useful
and effective manner.
Rensselaer Data Warehouse Project
- 5 -
General Product Functionality (Technical Assessment)
Both the Cognos and Brio products provide numerous business intelligence components,
including managed reporting, ad hoc querying, analytical data processing, etc. Both product
suites contain web-based interfaces, as well as a BI web portal component.
Cognos’ web interfaces are entirely CGI-driven, producing HTML output for browser
consumption. Brio’s web-based interface consists of a browser plug-in, which therefore requires
minor installation steps by the end-users, as well as ongoing plug-in software management. The
plug-in allows end-users to create a dataset via an online query and then make use of the data
offline.
Data Warehouse Database Integration (Technical Assessment)
Not surprisingly, both the Cognos and Brio products support native access to the underlying data
warehouse database in Oracle.
Informatica Metadata Integration (Technical Assessment)
Both the Cognos and Brio products integrate with Informatica’s ETL products via Informatica’s
Metadata Exchange (MX2) API. Within the Cognos environment, it is not clear how deep the
level of integration goes beyond a simple import of the data. The Cognos environment maintains
its own metadata. For Cognos users, a “Description” field is available to view metadata details.
Within the Brio environment, the integration seems stronger in that there is an extensive amount
of metadata information available to the user in a series of dialog boxes. Further, Brio seems to
integrate with rather than import from Informatica. Since metadata is a key to the success of the
data warehouse project, using Brio as a BI tool seems to increase the likelihood of exposing the
users to the metadata (i.e. ensuring that users have immediate access to the business terms and
rules, etc.).
Institute-wide Web Portal Integration (Technical Assessment)
Both the Cognos and Brio products provide mechanisms for integrating with a third-party
Institute-wide web portal environment. From within the Cognos environment, HTML reports
and such are drawn out via a CGI interface; Cognos also includes support for report export via
XML with well-defined DTDs. From within the Brio environment, services are available via a
well-defined API.
General Product Architecture (Technical Assessment)
Both the Cognos and Brio products provide multi-component, extensible architectures. Cognos
supports a 3-tier architecture that offers scalability with multi-threaded processing and load
balancing. The Cognos architecture consists of a single Web Server component, a single Dispatch
Server component, and one or more BI Server components. Brio also supports a 3-tier
architecture that offers scalability with multi-threaded processing and load balancing, including a
Broadcast Server component and an On-Demand Server component.
A fundamental difference between Brio and Cognos is that the Cognos PowerPlay environment
supports the generation of data cubes that are then made available to end-users. Brio does not
natively support the generation of data cubes; instead, Brio is generally used to access the
underlying database directly. While Cognos cubes move basic query activity from the database to
a dedicated Cognos server machine, Cognos cubes consist of predefined hierarchies and
Rensselaer Data Warehouse Project
- 6 -
preprogrammed drill-back functionality; in other words, data is forced into hierarchies and drill-
back (or drill-through) capabilities need to be set up by a team member who is well-versed in the
Cognos product and multidimensional databases (MDD).
Within Brio, end-user analysis (i.e. slicing and dicing) of data generally occurs on the user’s
desktop; in effect, Brio users work with datasets on their desktop. Architecturally, distributing
the load of the users to their desktops is advantageous, especially as user desktops continue to
grow in terms of hardware (i.e. user desktop machines are becoming equivalent with small to
medium-size servers).
Note that Brio does in fact integrate with existing data cubes, assuming that the format of the
cube is open and non-proprietary (e.g. integration with a Hyperion Essbase cube is supported, yet
integration with Cognos data cubes is not supported).
The resulting architectural question is whether or not to make use of a data cube technology in
the Rensselaer data warehouse. This question is discussed in the following bulleted points:
! Cognos is correct in stating that much of the load involved in typical user behavior can be
captured in the data cube, and therefore off-loaded from the data warehouse database.
Such deferred processing requires centralized hardware to contain the cubes, as well as
support staff to create, configure, tune and manage the cubes (i.e. an entirely new skill set
is required).
! In the Brio model, Brio’s BI tools directly query the underlying data warehouse database,
thus the load involved in report generation and ad hoc querying occurs on the database
itself. The database must be properly configured, tuned and managed.
! An additional Brio model involves accessing and querying a Hyperion Essbase cube, in
which case the two products working in unison become in some ways architecturally
equivalent to the Cognos environment.
Considering that Rensselaer’s data warehouse database is an Oracle database, as well as Oracle’s
emergence in the data warehousing field, it seems likely that through proper organization and
tuning, the data warehouse database will scale to end-user needs. Both Oracle 8i and especially
Oracle 9i have functionality specific to supporting data warehouses. If through the course of the
Rensselaer Data Warehouse Project, a cube is deemed necessary to handle performance or other
types of problems, it seems likely that Oracle’s data cube technology will have matured to an
acceptable level, or that another solution involving Oracle’s backend technologies will become
evident.
Security Architecture and Integration (Technical Assessment)
Both the Cognos and Brio products support an array of security options, including respect for
underlying database logins and roles (i.e. end-users individually authenticate to the underlying
database), integration with LDAP services, integration with NT security systems, etc.
Administrative Functionality (Technical Assessment)
Both Cognos and Brio provide administrative tools for their products. Runaway queries are
avoided by allowing administrative users to limit the number of rows returned or the elapsed
execution time for a given query. Note that queries can be both limited and cancelled by end-
users, yet the corresponding runaway query when an end-user cancels a query actually remains
running (i.e. the user regains control, but the server may still be busy and require administrative
attention).
Rensselaer Data Warehouse Project
- 7 -
Output Formats (Technical Assessment)
Both the Cognos and Brio products support a multitude of export or output formats, including
view-ready HTML and print-ready PDF, as well as analyze-ready Excel.
doc_896290270.pdf
The following individuals from the Data Warehouse team within IACS participated in the selection process in a limited fashion.
Rensselaer Data Warehouse Project
- 1 -
Business Intelligence (BI) Tool Selection Process
Implementing an aggressive schedule during the months of October, November, and December,
2001, the Data Warehouse team within Integrated Administrative Computing Services (IACS)
organized and managed the Business Intelligence (BI) tool selection process. Key participants
were identified and the Purchasing Office was contacted to establish a sound selection process.
The BI Tool Selection Committee
On October 18
th
, 2001, the Data Warehouse Sponsorship Group nominated over a dozen business
users from across campus to serve on the BI Tool Selection Committee. This committee was
tasked with choosing the enterprise-wide BI tool that will be used to access information within
the Rensselaer data warehouse environment. BI tool members included: David Bohan,
Director, Alumni Relations; Craig Clawar, Assistant Director for Technical Operations, P&DE;
Jackie Ellsworth, Associate Director, Controller’s Office; Patricia Gray, Assistant Director,
Contracts & Grants; Helen Grzymala, Associate Director of Financial Planning & Budget;
Sharon Kunkel, Director of Student Records & Financial Services; Jack Mahoney, Director of
Enrollment & Institute Research; Keith Martens, Purchasing Agent, Procurement Services;
Eileen McLoughlin, Director of Financial Planning & Budget; Cindy Price, HRIS
Coordinator, Human Resources; Kim Scalzo, Director, Professional & Distance Education;
Steven Schwan, Manager of Financial Operations, P&AS; and Donna Tomlinson, Manager of
Financial Operations, School of Engineering.
In addition to the committee members, numerous additional business users participated in the
selection process, ensuring that a broad spectrum of Rensselaer business users were involved in
the decision-making processes.
Integrated Administrative Computing Services (IACS) Participants
The following individuals from the Data Warehouse team within IACS participated in the
selection process in a limited fashion (i.e. developed the prototype data models, coordinated
training sessions, provided technical support, etc.): David Goldschmidt, Business Systems
Analyst; Ora Fish, Project Manager; Raj Kahlon, Data Warehouse Architect; Kate Owens,
Business Systems Analyst; and Jeff Stark, Business Systems Analyst.
Request for Proposal
The BI Tool Selection Committee helped to develop a BI Tool Request for Proposal (RFP) that was
sent out to 7 vendors on November 7
th
. The vendors included in the initial RFP were: Brio
Software, Inc.; Business Objects; Cognos Corporation; Hummingbird Corporation; Hyperion
Solutions Corporation; Information Builders, Inc.; and Oracle Corporation.
A total of 6 vendors responded to the RFP before the November 21
st
deadline (Hummingbird
Corporation did not respond).
Rensselaer Data Warehouse Project
- 2 -
Initial Vendor Presentations
Each of the 6 vendors that responded to the RFP was given an opportunity to present its BI tools
to the committee in 3-hour sessions during the last week of November. Such presentations were
generic in the sense that they did not make use of Rensselaer data prototypes.
Through a structured evaluation process, the committee narrowed the field from 6 vendors down
to the following 3: Brio Software, Inc.; Cognos Corporation; and Hyperion Solutions Corporation.
The following matrix was used t0 evaluate the 6 vendors:
Rensselaer Data Warehouse Project
- 3 -
The Hands-On Trial Process
The 3 vendors who survived the first cut were then invited to provide the committee a hands-on
trial period during the month of December. Each vendor set up and integrated its product(s) with
an Oracle database prototype in preparation for the hands-on trial process. The prototypes were
developed by the Data Warehouse team and included 2 subject areas: finance and student data.
The Data Warehouse team also purchased an NT machine to accommodate the selection process.
Once installed and integrated with the database prototypes, each vendor then spent a day training
users and answering questions. With hands-on access to the vendor tools, users found that such
sessions were extremely valuable and helpful in evaluating the 3 vendor products.
The following matrix was used to evaluate the 3 vendors that participated in the hands-on trial
period:
The Final Decision
At the end of the hands-on trial period, on December 19
th
, the committee met to officially select a
BI tool vendor. After the 2-month evaluation process, Brio Software, Inc. was chosen as the BI
tool solution for the Rensselaer data warehouse environment.
The voting that took place at the December 19
th
meeting yielded the following results:
# of votes
for 1
st
place
# of votes
for 2
nd
place
# of votes
for 3
rd
place
Brio Software, Inc. 10 3 0
Cognos Corporation 4 6 2
Hyperion Solutions Corp. 0 3 9
Note that one member of the committee ended up with a tie for 1
st
place between Brio Software,
Inc. and Cognos Corporation. Also note that two committee members abstained from voting for
Hyperion and Cognos due to their respective absences from the presentations and trial periods.
Rensselaer Data Warehouse Project
- 4 -
References
Once the committee identified Brio Software, Inc. as the top choice, the Data Warehouse team
contacted several companies and Universities that currently make use of Brio’s products. In
general, all references seemed quite satisfied with Brio’s ability to respond to and address their
needs.
The following references were contacted either via email or conference call:
! Anne Bennett, University of Connecticut.
! Scott Coopee, University of Connecticut.
! Cindy Sedlacek, Cornell University.
! Natalie Vincent, Syracuse University.
Technical Assessment
To ensure that the BI Tool Selection Committee’s choice of Brio Software, Inc. was a sound and
reasonable selection from a technical and architectural perspective, the Data Warehouse team
evaluated the top two BI tools after the user evaluations and voting was complete. Thus, the
remaining sections focus on the BI product suites from Brio Software, Inc. and Cognos
Corporation.
The fundamental goal of this analysis was to ensure the feasibility of implementation. Primary
focus technical questions and topics included:
! Product integrates well with data warehouse database (i.e. Oracle).
! Product integrates well with Informatica’s ETL solution, taking full advantage of the
underlying metadata.
! Product integrates well with portals.
! Product’s architecture is sound and sensible.
! Product’s hardware requirements are reasonable.
! Product’s security model is sound and fits into Rensselaer’s environment well.
! Product contains an effective administrative component.
! Product supports the ability to avoid “harmful” or “runaway” queries.
! Product supports multiple output formats (e.g. Excel, PDF, HTML, XML, etc.) in a useful
and effective manner.
Rensselaer Data Warehouse Project
- 5 -
General Product Functionality (Technical Assessment)
Both the Cognos and Brio products provide numerous business intelligence components,
including managed reporting, ad hoc querying, analytical data processing, etc. Both product
suites contain web-based interfaces, as well as a BI web portal component.
Cognos’ web interfaces are entirely CGI-driven, producing HTML output for browser
consumption. Brio’s web-based interface consists of a browser plug-in, which therefore requires
minor installation steps by the end-users, as well as ongoing plug-in software management. The
plug-in allows end-users to create a dataset via an online query and then make use of the data
offline.
Data Warehouse Database Integration (Technical Assessment)
Not surprisingly, both the Cognos and Brio products support native access to the underlying data
warehouse database in Oracle.
Informatica Metadata Integration (Technical Assessment)
Both the Cognos and Brio products integrate with Informatica’s ETL products via Informatica’s
Metadata Exchange (MX2) API. Within the Cognos environment, it is not clear how deep the
level of integration goes beyond a simple import of the data. The Cognos environment maintains
its own metadata. For Cognos users, a “Description” field is available to view metadata details.
Within the Brio environment, the integration seems stronger in that there is an extensive amount
of metadata information available to the user in a series of dialog boxes. Further, Brio seems to
integrate with rather than import from Informatica. Since metadata is a key to the success of the
data warehouse project, using Brio as a BI tool seems to increase the likelihood of exposing the
users to the metadata (i.e. ensuring that users have immediate access to the business terms and
rules, etc.).
Institute-wide Web Portal Integration (Technical Assessment)
Both the Cognos and Brio products provide mechanisms for integrating with a third-party
Institute-wide web portal environment. From within the Cognos environment, HTML reports
and such are drawn out via a CGI interface; Cognos also includes support for report export via
XML with well-defined DTDs. From within the Brio environment, services are available via a
well-defined API.
General Product Architecture (Technical Assessment)
Both the Cognos and Brio products provide multi-component, extensible architectures. Cognos
supports a 3-tier architecture that offers scalability with multi-threaded processing and load
balancing. The Cognos architecture consists of a single Web Server component, a single Dispatch
Server component, and one or more BI Server components. Brio also supports a 3-tier
architecture that offers scalability with multi-threaded processing and load balancing, including a
Broadcast Server component and an On-Demand Server component.
A fundamental difference between Brio and Cognos is that the Cognos PowerPlay environment
supports the generation of data cubes that are then made available to end-users. Brio does not
natively support the generation of data cubes; instead, Brio is generally used to access the
underlying database directly. While Cognos cubes move basic query activity from the database to
a dedicated Cognos server machine, Cognos cubes consist of predefined hierarchies and
Rensselaer Data Warehouse Project
- 6 -
preprogrammed drill-back functionality; in other words, data is forced into hierarchies and drill-
back (or drill-through) capabilities need to be set up by a team member who is well-versed in the
Cognos product and multidimensional databases (MDD).
Within Brio, end-user analysis (i.e. slicing and dicing) of data generally occurs on the user’s
desktop; in effect, Brio users work with datasets on their desktop. Architecturally, distributing
the load of the users to their desktops is advantageous, especially as user desktops continue to
grow in terms of hardware (i.e. user desktop machines are becoming equivalent with small to
medium-size servers).
Note that Brio does in fact integrate with existing data cubes, assuming that the format of the
cube is open and non-proprietary (e.g. integration with a Hyperion Essbase cube is supported, yet
integration with Cognos data cubes is not supported).
The resulting architectural question is whether or not to make use of a data cube technology in
the Rensselaer data warehouse. This question is discussed in the following bulleted points:
! Cognos is correct in stating that much of the load involved in typical user behavior can be
captured in the data cube, and therefore off-loaded from the data warehouse database.
Such deferred processing requires centralized hardware to contain the cubes, as well as
support staff to create, configure, tune and manage the cubes (i.e. an entirely new skill set
is required).
! In the Brio model, Brio’s BI tools directly query the underlying data warehouse database,
thus the load involved in report generation and ad hoc querying occurs on the database
itself. The database must be properly configured, tuned and managed.
! An additional Brio model involves accessing and querying a Hyperion Essbase cube, in
which case the two products working in unison become in some ways architecturally
equivalent to the Cognos environment.
Considering that Rensselaer’s data warehouse database is an Oracle database, as well as Oracle’s
emergence in the data warehousing field, it seems likely that through proper organization and
tuning, the data warehouse database will scale to end-user needs. Both Oracle 8i and especially
Oracle 9i have functionality specific to supporting data warehouses. If through the course of the
Rensselaer Data Warehouse Project, a cube is deemed necessary to handle performance or other
types of problems, it seems likely that Oracle’s data cube technology will have matured to an
acceptable level, or that another solution involving Oracle’s backend technologies will become
evident.
Security Architecture and Integration (Technical Assessment)
Both the Cognos and Brio products support an array of security options, including respect for
underlying database logins and roles (i.e. end-users individually authenticate to the underlying
database), integration with LDAP services, integration with NT security systems, etc.
Administrative Functionality (Technical Assessment)
Both Cognos and Brio provide administrative tools for their products. Runaway queries are
avoided by allowing administrative users to limit the number of rows returned or the elapsed
execution time for a given query. Note that queries can be both limited and cancelled by end-
users, yet the corresponding runaway query when an end-user cancels a query actually remains
running (i.e. the user regains control, but the server may still be busy and require administrative
attention).
Rensselaer Data Warehouse Project
- 7 -
Output Formats (Technical Assessment)
Both the Cognos and Brio products support a multitude of export or output formats, including
view-ready HTML and print-ready PDF, as well as analyze-ready Excel.
doc_896290270.pdf