Business Performance Management . . . Meets Business Intelligence

Description
Business Performance Management . . . Meets Business Intelligence

ibm.com/redbooks
Business Performance
Management . . . Meets
Business Intelligence
Chuck Ballard
Colin White
Steve McDonald
Jussi Myllymaki
Scott McDowell
Otto Goerlich
Annie Neroda
Proactive monitoring and management
to maximize business performance
Continuous data workflows for
real-time business intelligence
Information integration
for an enterprise view
Front cover
Business Performance Management . . .
Meets Business Intelligence
July 2005
International Technical Support Organization
SG24-6340-00
© Copyright International Business Machines Corporation 2005. All rights reserved.
Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP
Schedule Contract with IBM Corp.
First Edition (July 2005)
This edition applies to Version 8.1 of DB2 UDB, Version 8.2 of DB2 Alphablox, Version 4.2.4 of
WebSphere Business Integration, Version 5.0 of WebSphere Portal Server, and Version 5.0 and
Version 5.1 of WebSphere Application Server, Version 3.5 of WebSphere MQ Workflow, Version
5.3 of WebSphere MQ, and Version 5.1 of WebSphere Studio Application Developer.
Note: Before using this information and the product it supports, read the information in
“Notices” on page vii.
© Copyright IBM Corp. 2004. All rights reserved. iii
Contents
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Business innovation and optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Business performance management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Optimizing business performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Contents abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Chapter 1. Understanding Business Performance Management . . . . . . . 11
1.1 The BPM imperative . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.2 Getting to the details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.2.1 What is BPM again? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.2.2 Trends driving BPM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.2.3 Developing a BPM solution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.3 Summary: The BPM advantage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Chapter 2. The role of business intelligence in BPM. . . . . . . . . . . . . . . . . 27
2.1 The relationship between BI and BPM . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.1.1 Decision making areas addressed by BPM . . . . . . . . . . . . . . . . . . . 29
2.1.2 BPM impact on the business. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.2 Actionable business intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.2.1 Key Performance Indicators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.2.2 Alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.2.3 Putting information in a business context . . . . . . . . . . . . . . . . . . . . . 35
2.2.4 Analytic applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.3 Data warehousing: An evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.3.1 The need for real-time information . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.3.2 Data warehousing infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.3.3 Data federation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
2.4 Business intelligence: The evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.4.1 Integrating BPM and BI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Chapter 3. IBM BPM enablers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
iv BPM Meets BI
3.1 IBM BPM Platform. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.1.1 User Access to Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.1.2 Analysis and Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.1.3 Business Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.1.4 Making Decisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.1.5 Event Infrastructure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
3.1.6 Enabling IT to help the business . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.1.7 Bringing it all together . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.2 Web services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.2.1 The promise of Web services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.2.2 Web services architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.2.3 IBM Web services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.2.4 Using DB2 as a Web services provider and consumer . . . . . . . . . . . 82
3.2.5 WebSphere Information Integrator and Web services . . . . . . . . . . . 86
Chapter 4. WebSphere: Enabling the solution integration . . . . . . . . . . . . 97
4.1 IBM Business Integration Reference Architecture. . . . . . . . . . . . . . . . . . . 98
4.1.1 BIRA components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
4.2 IBM WebSphere business integration. . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.2.1 WebSphere Business Integration Modeler . . . . . . . . . . . . . . . . . . . 104
4.2.2 WebSphere Business Integration Monitor. . . . . . . . . . . . . . . . . . . . 104
4.2.3 WebSphere Business Integration Server Foundation. . . . . . . . . . . 105
4.2.4 WebSphere Business Integration Server . . . . . . . . . . . . . . . . . . . . 111
4.2.5 IBM WebSphere MQ. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
4.2.6 WebSphere Business Integration Connect . . . . . . . . . . . . . . . . . . . 122
Chapter 5. DB2: Providing the infrastructure . . . . . . . . . . . . . . . . . . . . . . 127
5.1 Data warehousing: The base . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
5.1.1 Scalability for growth. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
5.1.2 Partitioning and parallelism for performance. . . . . . . . . . . . . . . . . . 130
5.1.3 High availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
5.2 Information integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
5.2.1 Data federation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
5.2.2 Access transparency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
5.3 DB2 and business intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
5.3.1 Continuous update of the data warehouse . . . . . . . . . . . . . . . . . . . 135
5.3.2 Concurrent update and user access . . . . . . . . . . . . . . . . . . . . . . . . 137
5.3.3 Configuration recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
Chapter 6. BPM and BI solution demonstration. . . . . . . . . . . . . . . . . . . . 143
6.1 Business scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
6.1.1 Extending the scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
6.1.2 Scenario product architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.1.3 Hardware and software configuration . . . . . . . . . . . . . . . . . . . . . . . 148
Contents v
6.2 Implementing the BPM scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
6.2.1 The business processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
6.3 Adding BI to the demonstration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
6.3.1 Federation through WebSphere Information Integrator . . . . . . . . . 161
6.3.2 Federation through DB2 XML Extender . . . . . . . . . . . . . . . . . . . . . 162
6.4 Adding DB2 Alphablox to the demonstration. . . . . . . . . . . . . . . . . . . . . . 166
6.4.1 Configuring the components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
6.5 Adding WebSphere Portal to the demonstration. . . . . . . . . . . . . . . . . . . 170
6.5.1 Configuring the components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
6.6 Completing the scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
6.7 Additional dashboard examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
Appendix A. Getting started with BPM . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
Getting started with BPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
Selecting measures and KPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
vi BPM Meets BI
© Copyright IBM Corp. 2004. All rights reserved. vii
Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area.
Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM
product, program, or service may be used. Any functionally equivalent product, program, or service that
does not infringe any intellectual property right may be used instead. However, it is the user's responsibility
to evaluate and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter described in this document.
The furnishing of this document does not give you any license to these patents. You can send license
inquiries, in writing, to:
IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.
The following paragraph does not apply to the United Kingdom or any other country where such provisions
are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES
THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer
of express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may
make improvements and/or changes in the product(s) and/or the program(s) described in this publication at
any time without notice.
Any references in this information to non-IBM Web sites are provided for convenience only and do not in any
manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the
materials for this IBM product and use of those Web sites is at your own risk.
IBM may use or distribute any of the information you supply in any way it believes appropriate without
incurring any obligation to you.
Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm
the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on
the capabilities of non-IBM products should be addressed to the suppliers of those products.
This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.
COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrates programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the
sample programs are written. These examples have not been thoroughly tested under all conditions. IBM,
therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy,
modify, and distribute these sample programs in any form without payment to IBM for the purposes of
developing, using, marketing, or distributing application programs conforming to IBM's application
programming interfaces.
viii BPM Meets BI
Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:
Eserver®
Redbooks (logo) ™
iSeries™
z/OS®
AIX®
Cube Views™
CICS®
Database 2™
Distributed Relational Database
Architecture™
Domino®
DB2 Connect™
DB2 OLAP Server™
DB2 Universal Database™
DB2®
DRDA®
Everyplace®
Hummingbird®
Informix®
Intelligent Miner™
IntelliStation®
IBM®
IMS™
Lotus Notes®
Lotus®
MQSeries®
Notes®
Redbooks™
SupportPac™
Tivoli Enterprise™
Tivoli®
WebSphere®
Workplace™
The following terms are trademarks of other companies:
Pentium, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel
Corporation or its subsidiaries in the United States, other countries, or both.
Excel, Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United
States, other countries, or both.
EJB, Java, Java Naming and Directory Interface, JavaServer, JavaServer Pages, JDBC, JDK, JSP, JVM,
J2EE, J2SE, Solaris, Sun, and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the
United States, other countries, or both.
Linux is a trademark of Linus Torvalds in the United States, other countries, or both.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Other company, product, and service names may be trademarks or service marks of others.
© Copyright IBM Corp. 2004. All rights reserved. ix
Preface
This IBM® Redbook is primarily intended for use by IBM Clients and IBM
Business Partners. In it we discuss and demonstrate technology, architectures,
techniques, and product capabilities for business performance management
(BPM).
BPM is a relatively new and evolving initiative that brings together a number of
technologies to monitor and manage the attainment of business measurements
and goals. However, this redbook is not intended to be a comprehensive treatise
on BPM, but more of an introduction to help you understand it and get started
with your implementation.
As the title implies, we also have a primary focus on the integration of BPM with
business intelligence. That is, we want to demonstrate how this integration can
better enable a more proactive, in addition to the more typical reactive, form of
business intelligence. And that is what enables fast action to be taken to resolve
issues and actually drive the attainment of goals and measurements rather than
passive monitoring of their status.
For example, we demonstrate the capability to actively monitor the business
processes and integrate their status data with the operational activity data in the
data warehouse. The combination of these two data sources provides an
enterprise-wide view of the business for decision making and reporting. With this
information, we can begin to manage and optimize business performance. This is
of significant value for the enterprise, business management, and business
shareholders.
BPM is itself a process developed to monitor, manage, and improve business
performance. It has the following three core categories of capability. These
capabilities are discussed in more detail throughout this redbook:
Information Management: including operational reporting, data federation,
data warehousing, and business intelligence
Process Management: including business processes, key performance
indicators (KPIs), alerts, process status, operational activities, and real-time
process monitoring
Business Service Management: including systems monitoring and
optimization of IT operations to meet the business goals
The results of these capabilities are brought together at the point of integration
for management and decision-makers - the business portal.
x BPM Meets BI
As businesses move forward in the evolution to real-time business intelligence,
there is a need to optimize the operational business activities. For example, they
must be modified to support real-time activity reporting, and the continuous flow
of data to the enterprise information repository - the DB2® data warehouse. One
major impact of this evolution is enhanced decision making, and proactive
avoidance of problems and issues in addition to more typical reactive measures
to minimize the impact of those problems.
Products such as WebSphere® Business Integration and DB2 Universal
Database™ (DB2 UDB) play a key role in BPM, and were used in the
development of this redbook. We have included example system architectures,
product installation and configuration examples and guidelines, examples of the
use of key performance indicators, and management dashboards to enable
improved business performance management. We believe this information and
our examples will be of great benefit as you continue to improve the management
of your business performance.
The team that wrote this redbook
This redbook was produced by a team of specialists from around the world
working at the International Technical Support Organization, San Jose Center.
The team is depicted below, along with a short biographical sketch of each:
Chuck Ballard is a Project Manager at the International
Technical Support Organization, in San Jose, California. He
has over 35 years experience, holding positions in the areas
of Product Engineering, Sales, Marketing, Technical Support,
and Management. His expertise is primarily in the areas of
database, data management, data warehousing, business
intelligence, and process re-engineering. He has written
extensively on these subjects, taught classes, and presented
at conferences and seminars worldwide. Chuck has both a
Bachelors degree and Masters degree in Industrial Engineering from Purdue
University.
Colin White is the President and Founder of BI Research.
He is well-known for his in-depth knowledge of leading-edge
Business Intelligence and Business Integration technologies,
and how they can be integrated into an IT infrastructure for
building and supporting the Smart Business. With over 35
years of IT experience, he has consulted for dozens of
companies throughout the world and is a frequent speaker at
leading IT events. Colin has co-authored several books, and
Preface xi
writes columns for DM Review and the B-EYE-Network on business intelligence
and enterprise business integration.
Steve McDonald is a Senior IT Specialist from Australia
supporting Business Integration and WebSphere. He has
nine years experience in the integration field holding
positions in Systems Programming, Technical Support, and
Business Integration Technical sales. Steve has worked at
IBM for four years. His areas of expertise include Process
Integration, Application Connectivity, Distributed Systems,
z/OS®, and Business Integration Patterns, and he is a
regular presenter at conferences in Australia. Steve has a
Bachelors Degree in Computing from Monash University and Postgraduate
Diploma in Management from Melbourne Business School.
Jussi Myllymaki is a Research Staff Member at the IBM
Almaden Research Center in San Jose, California. He
specializes in advanced data services for DB2, including
Web services and XML, extensible analytics, and information
integration. He has published extensively in the research
community and has filed 20 patents. He holds a Ph.D.
degree in Computer Science from the University of
Wisconsin at Madison.
Scott McDowell is a Certified Consulting I/T Specialist, and
uses his extensive knowledge in database administration to
help customers maximize their productivity using IBM
software. With over 14 years experience, he joined IBM as
an IT Specialist in 2000. His areas of expertise include High
Availability and Disaster Recovery, DB2 UDB, and BI tools.
Scott has given presentations on Disaster Recovery and
High Availability at IDUG and the High Availability
Conference. He has a Bachelors Degree in Management
from Bellevue University in Bellevue, Nebraska.
Otto Goerlich is a Certified Consulting IT Specialist
supporting business intelligence presales activities in the
EMEA Central Region. He has over 30 years of experience
in the IT field, holding positions in the areas of Software
Maintenance, Software Development, Technical Support,
and BI Technical Sales. His areas of expertise include DB2
UDB for large data warehouse implementations, BI tools,
data warehouse architectures, and competition. He has
written articles on these subjects and is a well accepted
presenter at technical conferences and seminars.
xii BPM Meets BI
Annie Neroda is a Senior Certified Business Intelligence
Software IT Specialist. She has been assisting customers in
implementing BI Solutions such as DB2 Cube Views™, DB2
Warehouse Manager, DB2 OLAP Server™, DB2 Intelligent
Miner™, and related BI tools from IBM's Business Partners
for the past ten years of her 30 year career. She has held
positions in DB2 technical instruction, software development,
and management. She has a Bachelors Degree in
Mathematics from Trinity College in Washington, D.C.
Special Acknowledgements:
Wayne Eckerson, Director of Research at The Data Warehousing Institute
(TDWI). Wayne is a highly respected thought leader, published author, speaker,
and consultant in the areas of Business Intelligence and Business Performance
Management. We have drawn upon his insights, and in some sections have used
graphics and text directly from his publicly available documents. We thank him for
his written consent to use these materials.
Dr. Barry Devlin, from IBM in Dublin, Ireland. Barry has long been know for his
leadership and expertise in data warehousing and information integration.
Thanks also to the following people for their contributions to this project:
From IBM Locations Worldwide
Sergio Arizpe, Tech Support - WBI Monitor and Modeler, Burlingame, CA.
Jasmine Basrai, WBI Monitor and Modeler, Burlingame, CA.
David Enyeart, Business Performance Management Architecture, Durham, NC.
Gil Lee, Information Integration Marketing, San Jose, CA.
Andrew Kumar, WebSphere Technical Sales, Melbourne, Australia.
John Medicke, Executive IT Architect and Chief Architect, SMB, Raleigh, NC.
Kramer Reeves, BPM Software Product Marketing, San Francisco, CA.
Gary Robinson, Business Intelligence Development, San Jose, CA.
Billy Rowe, Business Performance Management Architecture, Durham, NC.
Jon Rubin, Product Manager, IBM Silicon Valley Lab, San Jose, CA.
Guenter Sauter, Information Integration Solutions Architect, Somers, NY.
Robert Spory, Worldwide Sales - WBI Monitor and Modeler, Hazelwood, MO.
Jared Stehler, BI Analytics, Silicon Valley Lab, San Jose, CA.
Louis Thomason, STSM - Information Integration, Somers, NY.
Eric Wayne, Business Innovation and Optimization Architecture, Raleigh, NC.
Paul Wilms, Information Integration Technology Solutions, San Jose, CA.
Dirk Wollscheid, DB2 Information Integrator, San Jose, CA.
Kathryn Zeidenstein, WebSphere II Product Management, San Jose, CA.
From the International Technical Support Organization, San Jose Center
Mary Comianos - Operations and Communications.
Preface xiii
Deanna Polm - Residency Administration.
Emma Jacobs - Graphics.
Become a published author
Join us for a two- to six-week residency program! Help write an IBM Redbook
dealing with specific products or solutions, while getting hands-on experience
with leading-edge technologies. You'll team with IBM technical professionals,
Business Partners, and/or customers.
Your efforts will help increase product acceptance and customer satisfaction. As
a bonus, you'll develop a network of contacts in IBM development labs, and
increase your productivity and marketability.
Find out more about the residency program, browse the residency index, and
apply online at:
ibm.com/redbooks/residencies.html
Comments welcome
Your comments are important to us!
We want our IBM Redbooks™ to be as helpful as possible. Send us your
comments about this or other IBM Redbooks in one of the following ways:
Use the online Contact us review redbook form found at:
ibm.com/redbooks
Send your comments in an Internet note to:
[email protected]
Mail your comments to:
IBM Corporation, International Technical Support Organization
Dept. QXXE Building 80-E2
650 Harry Road
San Jose, California 95120-6099
xiv BPM Meets BI
© Copyright IBM Corp. 2004. All rights reserved. 1
Introduction
Before you dive into the detailed sections of this redbook, it is important to point
out the business performance management (BPM) initiative is currently in its
infancy and is changing rapidly. For example, there are many other terms being
used to describe this area of interest. They are used at many seminars and
conferences around the world. But there is indeed a lot of confusion and the
definitions given are not always clear or consistent. Some terms sound similar,
but are very different. Some may even have the same acronym, but mean
something very different. As examples, you may have heard terms such as
business activity monitoring, corporate process management, business activity
management, and business process management. With those terms come
acronyms such as BAM, BPM, and CPM. Though they sometimes sound similar,
they do not necessarily have the same meaning, and certainly not the same
solution capabilities.
This redbook is intended to provide you with an introduction to BPM, its
relationship to business intelligence, and an overview of the current IBM BPM
solution and product capabilities. However, as an industry leader, IBM continues
to evolve its BPM solution and supporting product set. As always, contact your
IBM representative to be sure you have all the details on the latest functions and
features of the IBM solution.
2 BPM Meets BI
Business innovation and optimization
As an example of the IBM BPM evolution, IBM is bringing a more precisely
defined initiative to this marketplace, comprised of a new name, messages, and
content. Central to this theme are two key values, competitive differentiation
through innovation and efficiency through optimization. The name for this
initiative is Business Innovation and Optimization (BIO). This new initiative will
also serve to differentiate the robust IBM offering, and avoid the confusion
generated by the many definitions and related terminology that have been
generated relative to the topic of business performance management.
IBM Business Innovation and Optimization is not a market category; rather, it
represents a collection of software technology capabilities, best practices, and
industry expertise to address needs and functionality identified in several market
categories, including:
Business Performance Management
Enterprise Performance Management
Corporate Performance Management
Business Intelligence
Business Services Management
Business Process Management
Business Activity Monitoring
More information on BIO will be forthcoming, but, for purposes of this redbook,
we will continue to use the term BPM. BPM is a paradigm that enables success
by proactively managing business performance and the attainment of business
measurements, in addition to reactively resolving individual business issues
when required.
Managing and optimizing business performance is a critical requirement for
maximizing business profitability and returning shareholder value. In fact, in
many situations, it is a critical requirement for remaining viable in today’s
fast-moving and competitive business environment. Company executives are
also feeling pressure from a growing intolerance for missed performance targets.
And with performance measurement periods becoming shorter, management
simply must have the capability to more proactively influence the outcome. To do
this requires monitoring and tracking capabilities that can generate current,
complete, and accurate information upon which they can immediately act. This
information provides the required business intelligence for proactively managing
business performance.
Business intelligence (BI) is a key enabler of BPM. Hence the title of this
redbook, Business Performance Management....Meets Business Intelligence.
Introduction 3
Business performance management
BPM is all about taking a holistic approach for managing business performance.
Businesses align strategic and operational objectives, and business activities, to
fully manage performance through more informed and proactive decision
making. The holistic approach enables the integration and use of business
intelligence, process management, business service management, activity
monitoring, and corporate performance management to achieve a single view of
their enterprise. The elements of that holistic approach are depicted in Figure 1.
Figure 1 A holistic approach
Then by putting that information and understanding in context against the
business goals and priorities, action can be taken quickly to improve execution,
successfully meet the business measurements, and truly begin to manage
business performance.
The following comparison may be helpful in furthering the understanding of BPM.
Consider enterprise systems management (ESM), which represents a set of
technologies focused on integrated monitoring and management of IT systems,
for example, networks, servers, storage systems, and databases. ESM has its
own set of technical performance metrics, for example, bandwidth, throughput,
processor utilization, memory usage, and disk I/O ratios, along with the concept
of predefined performance thresholds, operational alerts for performance outside
the threshold limits, and rules-based corrective actions. It also comprises a
performance management portal in the form of a consolidated management
console, such as Tivoli® Enterprise™ Console. The concepts for BPM are very
Business
Process
Management
Business
Process
Management
Corporate
Performance
Management
Corporate
Performance
Management
Business
Service
Management
Business
Service
Management
Business
Activity
Monitoring
Business
Activity
Monitoring
Business
Intelligence
Business
Intelligence
Business
Performance
Management
Business
Performance
Management
4 BPM Meets BI
similar, but focused on the overall business processes, and at a higher level. For
example, BPM actually incorporates ESM and maps the business process onto
IT resources. This higher level view and the goal to align strategy throughout the
enterprise provide the enterprise perspective that is required for BPM.
The requirement for BPM is not new. Management has always had a need to
manage business performance. Previously, however, they have not had the
technology to implement it. The technology is now here to satisfy the need for
BPM. This is the right solution at the right time. The demand to meet business
performance goals by aligning strategy throughout the enterprise has never been
more critical. And now, IBM can deliver the solutions!
A BPM solution is comprised of disparate disciplines such as business process
integration, enterprise application integration, information integration, business
intelligence, and enterprise systems management. A common thread here is
information. Do you have the information you need for BPM, or is it locked up in
disparate departmental or functional areas? Can you get a business view of your
enterprise, or only departmental or functional area views?
The key is to unleash the power of your information. The IBM approach to BPM is
to take a holistic view of those disparate departments and functional areas and
integrate them across the business value chain! This gives you a more
comprehensive view of your enterprise and can enable you to better manage
business performance.
Becoming better informed
To enable BPM, businesses must become better informed, flexible, and dynamic.
Becoming better informed more and more means evolving towards a real-time
business intelligence environment. To enable faster decision making in this fast
moving economy requires information that is more and more current. That
requirement directly impacts your business operations, because one source of
that critical information is the status feedback from the operational activities.
Therefore, one requirement is to have the operational activities integrated with
the business intelligence environment. This not only enables fast access to the
results of operational activity reporting, but satisfies the need for more dynamic
business processes. Static processes, or those that cannot adapt to changing
needs, are a liability. As businesses analyze their operational activities in an
effort to become more efficient and effective, they are looking more and more to
IT to identify and implement an improved and integrated business process
environment.
If you are looking for this business agility and the ability to manage your business
performance, information is the enabler and IBM is uniquely positioned to offer a
comprehensive solution!
Introduction 5
BPM: The IBM approach
The IBM approach to BPM is enabled by an overarching framework that ties
together key business and IT disciplines such as business process integration,
business intelligence, and enterprise application integration. It enables
businesses to:
Sense, and respond to customer needs, competitive threats, and regulatory
pressures
See, understand, and use business information to adapt their business
processes and IT infrastructure to optimize business performance
Business performance management solutions deliver continuous improvement
and innovation by:
Aligning strategy horizontally and vertically throughout your company
Enabling proactive and directed action
Providing real-time, contextual insight
Delivering role-based visibility into business operations and metrics
Improving team productivity and effectiveness
The integration of business and IT process management, and business
intelligence is a key enabler for BPM. It provides the ability for you to move more
quickly towards managing and meeting your business measurements and goals.
A high-level view of the framework and component integration is depicted in
Figure 2.
The BPM framework is an enabler for the integration of business and IT
processes. The underlying technology then enables the operations and
information flow, providing the capability for you to make the decisions
required to best manage the business. This is your competitive advantage!
6 BPM Meets BI
Figure 2 BPM framework
Optimizing business performance
There is detailed information in the subsequent chapters of this redbook on
managing and optimizing business performance, along with an explanation of the
IBM BPM framework components, and examples of the BPM optimization
processes. As a starting point, we briefly introduce some of these concepts here.
IBM has a model-based approach to BPM that is key to enabling you to optimize
business performance. The approach is represented by a cycle of integrated
processes, as depicted in Figure 3. It is through the execution and refinement of
this process cycle that we can optimize business performance.
Figure 3 BPM optimization cycle
Understand
the state of
key
business
process
operations
in realtime
Visualize IT
process
operations in
business
terms, and
manage
service levels
to business
objectives
Understand the status of business
processes across business and IT, in
context against goals and trends, and
enable fast action to improve execution
Business
process
view
IT
process
view
Business
Performance
Management
Analyze
Model
Deploy
Monitor
IT
Operations
Business
Operations
Act
B
u
s
i
n
e
s
s

p
r
o
c
e
s
s
e
s
Introduction 7
The BPM framework components depicted in Figure 2 map to the processes in
the BPM optimization cycle shown in Figure 3. For example, the Business
Process view in Figure 2 is equivalent to Business Operations in Figure 3. The
same relationship exists with the IT components. The core processes of model,
deploy, monitor, analyze, and act define the methodology to enable performance
optimization. These processes are discussed in more detail in Section 3.1, “IBM
BPM Platform” on page 51.
Business intelligence brings another set of specialized tools and infrastructure to
bear on analyzing the data collected in IT systems by automating business
processes. Traditional decision-support technologies, ranging from nightly batch
reporting and ad hoc queries to data mining modeling and multi-dimensional
cube analyses, are all directed towards gleaning insight from the results of
operational processes after-the-fact, without direct, systematic links back to the
underlying operational systems. The breakthrough perspective of BPM links the
analytic, model-based insight of BI with the dynamic monitoring and action-taking
capabilities of BPM, closing the loop on the optimization cycle.
A similar relationship exists with the IT components. Using this modeled
approach can then also result in optimizing the IT activities.
Solution processes
Monitoring and managing business performance may sound simple, but do not
take these terms too lightly. Monitoring the business processes can provide the
information to enable you to manage business performance. But, first you must
enable the capability to monitor them. That is, you must have well-defined
processes and a means of reporting on the process’s operational activities. And
then managing them also brings additional requirements. To clarify, let’s look at a
few examples. You must:
Define exception processing.
Establish key performance indicators for exception processing.
Generate alerts for any exception conditions.
Define processes to handle alerts and correct the exception conditions.
Define corrective actions for the processes to avoid, or at least minimize,
future exception conditions.
This enables businesses to evolve from an organization of many discrete batch
processes to an integrated system capable of supporting continuous flow
processing. That in turn results in more timely information, detection of variations
That says it! The holistic closed-loop approach is the enabler for improved
business performance, and having that is another competitive advantage!
8 BPM Meets BI
from the plan, generates alerts to call for action, promotes proactive
recommendations for alert resolution, and provides a dynamic user interface to
enable the business to respond appropriately.
To provide these types of BPM solutions, however, requires a vendor to marshal
resources and expertise spanning the broadest imaginable range of business
and IT disciplines. IBM is uniquely qualified to do this, owing to the scope of its
software portfolio, business and IT services, and industry sector practices.
Getting the BPM advantage
Industry analysts are advising companies to optimize the key activities and
business processes that drive their revenues and profits, as a means of meeting
their business measurements and goals. BPM is a key enabler, and this redbook
will help you move forward with that solution.
Business flexibility and agility require continuous monitoring of the business
processes and support of a robust business intelligence (BI) environment. By
robust, we mean information is available and sufficiently current to support the
requirements for both operational and strategic decision making. Although
certainly not required in all cases or scenarios, the term sufficiently current is
rapidly evolving to mean real-time (or something near real-time).
Want to learn more about evolving to a real-time environment? There is another
IBM Redbook available on the subject, Preparing for DB2 Near Real-time
Business Intelligence, SG24-607, and we suggest this redbook for additional
reading.
Contents abstract
This section includes a brief description of the topics presented in this redbook,
and how we organize them. The information ranges from high-level product and
solution overviews to detailed technical discussions. We encourage you to read
the entire redbook. However, depending on your interest, level of detail, and job
responsibility, you may select and focus on topics of primary interest. We have
organized this redbook to accommodate readers with a range of interests.
Our objective is to demonstrate the advantages of, and capabilities for,
implementing BPM with DB2 UDB and WebSphere Business Integration. We
structured the redbook to first introduce you to the architectures and functions of
The aim (and result) is proactive business performance management and
problem avoidance, in addition to a reactive approach focused on problem
impact minimization.
Introduction 9
the products used. Then we describe how to install, configure, and integrate the
products to work best together. With a focus on process monitoring and
management, we then demonstrate example scenarios, and the procedures
required to evolve to this environment.
Chapter summary
Let’s get started by looking at a brief overview of the contents of each chapter of
the redbook, as described in the following list:
Chapter 1 introduces BPM and presents key descriptions and definitions to
establish a common understanding and point of reference for the material in
the redbook. The trends driving BPM are discussed along with associated
benefits. We then present a methodology for implementing a BPM
environment as a means to help you continue with your evolution. We present
information that deals with gathering requirements, architecting a solution,
and creating systems that are valuable and beneficial to use.
Chapter 2 continues the BPM discussion, describing how BPM integrates
with, and supports, a business intelligence (BI) environment. In general, BI is
a user of BPM and the means whereby decision makers can better enable
management of the business. Here we include discussions of key concepts
and technologies such as key performance indicators and analytic
applications. We then discuss the implementation of BI and the linkages to
BPM, along with the evolution to more of a real-time environment. This is the
desired final solution that results in a significant competitive advantage.
Chapter 3 continues by establishing a common and consistent understanding
of BPM by describing a number of what we call BPM enablers. We describe
the IBM BPM framework and discuss the base structure of the framework,
namely the Reference Architecture and Model Driven Development. We also
look at a service-oriented architecture (SOA) and Web services. The
discussion will show you how the IBM BPM framework supports an enterprise
BPM solution and can integrate with your particular BI environment.
Chapter 4 provides an introduction to the IBM WebSphere Business
Integration (WBI) product, which also helps to establish an understanding of
the BPM capabilities. There is detailed discussion on the architecture, and the
concepts of integration and standardization inherent in WBI.
Chapter 5 is focused on DB2 UDB. We describe how it enables and supports
business intelligence by providing the infrastructure for a robust data
warehousing environment. Then we describe a number of key features,
including scalability, parallelism, data partitioning, and high availability.
Chapter 6 describes the testing environment used for this redbook project.
We describe the hardware and software products used, along with the project
architecture. We then describe and demonstrate a sample application
scenario that enabled us to show BPM in action. The scenario shows
10 BPM Meets BI
examples of creating the processes and process flows, monitoring them as
they execute, capturing their progress, and displaying alerts and status on the
user dashboard of a portal. Acting on alerts provided during the sample
execution demonstrates closed-loop capabilities of a BPM implementation.
The testing of this scenario is a clear demonstration of a working BPM
implementation.
Appendix A provides a brief set of considerations for getting started with a
BPM implementation. One of the biggest challenges with BPM is knowing
where to start. This section describes three approaches typically used in
many organizations.
That’s an overview of the contents of the redbook. Now it’s time to make your
reading selections and get started.
© Copyright IBM Corp. 2004. All rights reserved. 11
Chapter 1. Understanding Business
Performance Management
Business performance management (BPM) is a key business initiative that
enables companies to align strategic and operational objectives with business
activities in order to fully manage performance through better informed decision
making and action. BPM is generating a high level of interest and activity in the
business and IT community because it provides management with their best
opportunity to meet their business measurements and achieve their business
goals.
IBM uses the term BPM for business initiatives that emphasize aligning strategic
and operational objectives in addition to monitoring and managing business
processes and associated IT events. In this chapter, we describe BPM in detail,
review BPM trends, and introduce a BPM framework and implementation
methodology.
1
12 BPM Meets BI
1.1 The BPM imperative
In today’s dynamic business environment, increased stakeholder value has
become the primary means by which business executives are measured. The
ability to improve business performance is therefore a critical requirement for
organizations. Failure to improve business performance is not tolerated by
stakeholders, who are quick to exercise their power. One result of this is the
volatility of stock prices, which creates a tense roller coaster ride for executives.
Bringing more pressure to bear is the fact that business performance
measurement time frames are becoming ever shorter. Quarterly targets have
replaced annual ones, and the expectation of growth and success is there at
every quarter end.
To help smooth out the roller coaster ride, businesses need to react quickly to
accommodate changing marketplace demands and needs. Flexibility and
business agility are key to remaining competitive, and, in some cases, viable.
What is needed is a holistic approach that enables companies to align strategic
and operational objectives in order to fully manage achievement of their business
performance measurements.
To become more proactive and responsive, businesses need information to give
them a single view of their enterprise. With this type of information, they can:
Make more informed and effective decisions.
Manage business operations and minimize disruptions.
Align strategic objectives and priorities both vertically and horizontally
throughout the business.
Establish a business environment that fosters continuous innovation and
improvement.
The need to continuously refine business goals and strategies, however, requires
an IT system that can absorb these changes and help business users optimize
business processes to satisfy business objectives. BPM assists here by providing
performance metrics, or key performance indicators (KPIs), which can be
employed to evaluate business performance. A KPI is a performance metric for a
specific business activity that has an associated business goal or threshold. The
goal or threshold is used to determine whether or not the business activity is
performing within accepted limits. The tracking and analysis of KPIs provide
business users with the insight required for business performance optimization.
BPM also becomes a great guide for IT departments who are being asked to do
more with less. It helps them focus their resources in areas that will provide the
most support to enable management to meet their business goals. They can now
Chapter 1. Understanding Business Performance Management 13
prioritize their tasks and focus on those tasks aligned with meeting business
measurements and achieving the business goals.
BPM requires a common business and technical environment that can support
the many tasks associated with performance management. These tasks include
planning, budgeting, forecasting, modeling, monitoring, analysis, and so forth.
Business integration and business intelligence applications and tools work
together to provide the information required to develop and monitor business
KPIs. When an activity is outside the KPI limits, alerts can be generated to notify
business users that corrective action needs to be taken. Business intelligence
tools are used to display KPIs and alerts, and guide business users in taking
appropriate action to correct business problems. To enable a BPM environment,
organizations may need to improve their business integration and business
intelligence systems to provide proactive and personalized analytics and reports.
1.2 Getting to the details
Many enterprises have already begun implementing the underlying components
required for supporting a BPM environment. These include planning, process
modeling, process monitoring, and analysis. It is often the case, however, that
organizations have not integrated their business processes and underlying
applications in a consistent way. Often business processes and applications are
isolated from each other, which typically results in the generation of information
silos. That is, they have information located in independent data stores that is not
available at an enterprise level. These implementations may provide some
departmental or functional area benefit, but this is of limited value to the overall
business. BPM solves this problem by providing a common business and
technical framework that allows the departments and functional areas of an
enterprise to adhere to a common set of goals and objectives.
1.2.1 What is BPM again?
Simply stated, BPM is a process that enables you to meet your business
performance measurements and objectives. It enables you to proactively monitor
and manage your business processes, and take the appropriate actions that
result in you meeting your objectives.
There are a few words in that statement that will take some doing. Here are a few
examples:
Monitor your processes. This means you have well-defined processes, and a
way to monitor them. And hopefully the monitoring is continuous.
14 BPM Meets BI
Manage your processes. You must be aware of their status, on a continuous
basis. That means you need to be notified when the process is not performing
satisfactorily. In your well-defined process, you will need to define when
performance becomes unsatisfactory. Then you must get notified so you can
take action.
Appropriate action. You will need the knowledge, flexibility, and capability to
take the appropriate action to correct the problem.
However, things are not always simple. Make sure you understand the terms. For
example, the BPM acronym is sometimes used in IT to describe business
process management. This is not the same thing as business performance
management. As a result, instead of BPM, some people prefer the term
corporate performance management (CPM), while still others prefer enterprise
performance management (EPM).
Listening to speakers at conferences or reading papers, you might also hear
BPM equated to budgeting, or to financial consolidation, or to Sarbanes-Oxley
compliance. Others think it has to do with financial reporting, dashboards,
scorecards, or business intelligence. Many of these are components of a
complete BPM solution. However, BPM is more than simply a set of
technologies. It offers a complete business approach, or methodology, for
managing business performance. This methodology is implemented using a
variety of business integration and business intelligence technologies.
To do this, BPM requires information about ongoing events in the operational
business processes and activities, in business operations as well as in IT
operations. This is accomplished by a set of key processes. It is the ongoing
execution and refinement of those processes that enables the optimization of
business performance. That is depicted in Figure 1-1.
Figure 1-1 BPM integrated processes
Analyze
Model
Deploy
Monitor
IT
Operations
Business
Operations
Act
B
u
s
i
n
e
s
s

p
r
o
c
e
s
s
e
s
Chapter 1. Understanding Business Performance Management 15
We describe those key processes in detail in “IBM BPM Platform” on page 51.
BPM standards group
To help encourage a common understanding of BPM and to create solutions that
satisfy a wide range of customers, IBM became a member of the BPM Standards
Group. Members of this group include application package vendors, tools
vendors, implementation consultants, IT and data warehousing experts, industry
analysts, management consultants, and system integrators. The customer
perspective is represented through an affiliation with the BPM Forum, a
business-oriented thought leadership group of senior-level practitioners in
operations, finance, and information technology. BPM Forum members are
involved in testing and validating BPM Standards Group deliverables.
The BPM Standards Group has developed a common definition for BPM, which
contains the following principles:
BPM is a set of integrated, closed-loop management and analytic processes,
supported by technology, that addresses financial as well as operational
activities.
BPM is an enabler for businesses in defining strategic goals, and then
measuring and managing performance against those goals.
Core BPM processes include financial and operational planning,
consolidation and reporting, modeling, analysis, and monitoring of key
performance indicators (KPIs) linked to organizational strategy.
More information about the BPM Standards Group can be found on their Web
site (see “Online resources” on page 198 for the location of this Web site).
1.2.2 Trends driving BPM
BPM is gaining attention in the business world because it is seen as a valuable
approach for gaining better control over business operations. It is of particular
value in helping solve issues that occur when trying to tie business planning to
operational execution. Examples of these issues are outlined in Table 1-1.
Executives and managers put significant effort into developing business
strategies and goals, which are then distributed throughout the company. The
problem comes when trying to monitor and manage the execution of those
strategies. In many organizations, there are no processes or tools in place to
validate that the strategies are being implemented as planned.
Another issue in many companies is that planning and budgeting cycles are not
flexible, or fast enough to satisfy fast changing business requirements. Plans and
budgets are often out-of-date before they are completed. This happens for many
16 BPM Meets BI
reasons, including inappropriate and unintegrated tools and methods that have
not kept up with current practices. There are many companies, for example, that
base their complete planning process on a series of spreadsheets linked
together over different computers, and even departments.
Table 1-1 Planning and execution issues
To resolve these issues, a sound process monitoring and management system is
needed. BPM enables this by getting people working towards the same business
goals and priorities, and by supplying an integrated solution for obtaining
optimum business performance. BPM provides accurate and timely
enterprise-wide information, and alerts decision makers to take proactive actions
to efficiently manage, control, and improve enterprise operations.
BPM solutions offer many benefits to organizations. A summary of key solutions
and benefits are shown in Table 1-2:
Table 1-2 BPM benefits
Issue Description
Users cannot get access to the current
status of business processes.
Process status and business information
is often scattered throughout the
organization, making it difficult for
business users to understand the current
status of the business.
Users find it difficult to put the status of
business processes into a business
context.
If business users obtain a view of the
status of their business, it is difficult for
them to assess that view against
historical trends and current goals.
Users are unable to take action and
make rapid changes to business
processes.
Even if business users can put process
status into a business context, it
becomes difficult for them to work with
organizational silos and quickly take
action and make process changes.
Solution Benefit
Align the business strategy horizontally
and vertically throughout your enterprise.
Organizations are no longer fractured by
independent actions of the business units.
They now are aligned and working
towards the same business goals.
Enabling proactive and directed action. Resources are directed towards actions
that are consistent with meeting the
business goals. This enables improved
resource planning and prioritization.
Chapter 1. Understanding Business Performance Management 17
BPM enables all parts of the organization to understand how goals and
objectives are tied to the business models and performance metrics. Since BPM
is an enterprise-wide approach, it also improves collaboration and
communications between the groups responsible for business operations.
Gaining insight into the business
The U.S. Sarbanes-Oxley Act and the new European Basel Capital Accord
(Basel II) are major drivers for implementing BPM solutions.
Sarbanes-Oxley represents one of the most broad ranging pieces of legislation to
affect corporations and public accounting in recent years. IBM recently released
the results of a survey detailing the current state of preparedness by U.S.
companies for supporting Sarbanes-Oxley, and the challenges facing CFOs and
financial executives in supporting the Act. The majority of CFOs surveyed by IBM
view the Sarbanes-Oxley Act as an opportunity to streamline systems and
improve the efficiency of real-time business processes.
For European companies, Basel II is a critical piece of legislation that
organizations urgently need to consider. Basel II promises significant business
benefits, but will be a major change for most businesses. Supporting Basel II
requires companies to review how they support the regulatory process and the
broader framework implied by Basel II. In particular, it forces them to consider
how they will integrate data, and also processes, that are currently split between
finance, operational, and risk-management functions.
Basel II does, however, provide businesses with the opportunity to free
themselves from legacy problems and out-of-date management structures. Basel
II encourages companies to develop a more streamlined and responsive
organization, and to provide relevant information for better risk management,
performance management, and capital allocation decisions.
Providing real-time, contextual insight,
delivering role-based visibility into
business operations and metrics.
Results in a consolidated view of the
current state of the business, by
combining process state information,
operational activity status and IT status. It
puts the current state of the business in
context, enabling proactive problem
avoidance.
Improving team productivity and
effectiveness.
Enables improved and automated
processes and allocation of resources.
Provides collaboration across enterprise
teams and processes to accelerate
business performance improvements.
Solution Benefit
18 BPM Meets BI
BPM offers more than just support for regulatory compliance. It also provides an
opportunity for businesses to integrate people with business processes in order
to improve business performance. This is consistent with the BPM capabilities
offered by IBM, which believes BPM is an enabling initiative for regulatory
compliance and gaining greater insight into business operations.
BPM incorporates related business technologies and capabilities, notably
business process modeling, business process integration, business intelligence,
and business services management.
Business process modeling and integration choreographs the interactions
among people, processes, and information to enable the execution of
business operations. The resulting integrated and mediated environment
provides an end-to-end view of business operations that would otherwise not
be possible.
Data warehousing technologies are used to extract, transform, and load (ETL)
business transaction data into data warehouses, which is then used for
creating business intelligence.
Business services management helps monitor the performance and health of
the IT infrastructure in real-time to aid in optimizing system performance.
These technologies, however, only partially address the challenges facing
enterprises, because they are typically not well integrated. Business
performance management solutions solve these problems by providing an
integrated platform that enables an organization to understand the status of
operational business and IT processes, and to take timely action to improve
those processes. The result is improved business performance.
BPM places enormous demands on BI applications and their associated data
warehouses to deliver the right information at the right time to the right people. To
support this, companies are evolving their BI environments to provide (in addition
to historical data warehouse functionality) a high performance and
enterprise-wide analytic platform with real-time capabilities. For many
organizations this is a sizable challenge, but BPM can be the incentive to move
the BI environment to a new level of capability. We discuss this in more detail in
Chapter 2, “The role of business intelligence in BPM” on page 27.
1.2.3 Developing a BPM solution
In a BPM solution, the business strategy builds the foundation for modeling the
business processes and key performance indicators in a consistent and aligned
manner. The key performance indicators help track and control the progress
towards the business strategy and goals. However, the collection of information
to calculate the measurements and provide sufficient context for analysis in root
Chapter 1. Understanding Business Performance Management 19
causes or for the discovery of optimization opportunities can be a daunting task.
Some measurements can be derived from business process events.
In other cases, consolidation technologies extract, clean, and aggregate the
information from a variety of operational systems. A data warehouse is a scalable
platform for collecting this vast amount of information and keeping the history of
key metrics. Then, analytic tools and applications can access this historical
context from the data warehouse.
It puts information in context, including historical performance, real-time events,
and planned targets necessary to calculate performance metrics and assess
whether or not action is necessary. Real-time, contextual insight requires the
correlation and integration of historical context with real-time data such as
process-monitor events. It delivers inline analytics and business intelligence
services for applications and business processes.
When measurements deviate from planned targets, business analysts employ
business intelligence tools to investigate the root causes so necessary actions
can be taken. The data warehouse provides the necessary context for the
analysis, and the business intelligence services transform this information into
insight by providing analytic, visualization, and reporting capabilities. Inline
analytics can then provide actionable insight.
There are many tasks involved in developing a BPM solution (see Appendix A,
“Getting started with BPM” on page 179, for more detailed information about
these tasks). One key task is the definition of KPIs, and how they are used to
monitor and manage business and system operations. As we have already
discussed, a BPM system can detect threshold conditions for KPIs and generate
alerts that call for corrective actions. We depict this in a high-level system’s view
in Figure 1-2.
A BPM system receives business and IT event data and filters it through
business rules or policies. Compliance is checked and alerts sent as appropriate
to the user interface. Suitable action is then taken to resolve any business
exceptions. This is an example of what is often referred to as a closed-loop
system.
BPM solutions, based on the closed-loop system shown in Figure 1-2, allow an
organization to understand the status of business processes across the
enterprise, place that understanding in context against business goals and
priorities, and take timely action to improve business operations. This process
constitutes the management of business performance.
20 BPM Meets BI
Figure 1-2 A simplified BPM closed-loop system
IBM delivers BPM solutions with capabilities such as the following:
Integrated modeling of processes, policies, goals, and KPIs
Externalized business rules for deploying complex business strategies and
improving responsiveness to business rule changes
Event-driven management of business and IT operations across the extended
enterprise
Integration of event and business metrics with business intelligence for
assessing KPIs relative to business goals
Advanced business simulation and performance optimization based on
real-time monitoring of business activities
Proactive monitoring and business alerts
User-customized analytics and reporting of On Demand business and IT
performance information
Industry-specific business performance management solutions (financial
services, health care, and retail, for example) with methodologies and
dashboards for evaluating and improving industry processes and services
Consolidated and dynamic resource management
Event Filtering
Business Rules
Alerts &
Thresholds
Analysis
Modeling &
Planning
Prioritization
Data Mining
Portals
BI Tools
Applications
Dashboards
Escalation
Vehicles
Production
Operations
External
Environment
Data
Sources
Data
Warehouse
Business
Policies
Events
Data
Context
Response
Chapter 1. Understanding Business Performance Management 21
BPM strategy considerations
For maximum impact on business performance, BPM should be considered an
enterprise-wide strategy. Unfortunately, the term enterprise-wide brings with it
several negative connotations. The primary one is the perception of a large
implementation effort. Other issues include how and where to begin, and
whether or not to use top-down versus bottom-up development. These issues are
not new, and are similar to those encountered during the implementation of data
warehousing. As with data warehousing, the key to success with BPM is an
iterative and phased implementation effort.
BPM potentially requires data, processes, and cooperation from all the
organizations in the enterprise. This again parallels data warehousing projects,
because projects must deal with different data, heterogeneous systems, and
many different processes, all of which are required to deliver an enterprise
information environment.
BPM reference architecture
IBM has developed a BPM reference architecture that provides a flexible,
extensible, open-standards-based infrastructure, which is a key added value of
the IBM solution. The architecture specifies the required capabilities and
services, and the defined interfaces that enable integration of the solution
components.
The reference architecture is depicted in Figure 1-3. In addition to showing the
architectural components, we have included the products available that enable
the solution. The product names in parentheses are IBM BPM software offerings,
and those in quotation marks are other associated IBM software offerings.
22 BPM Meets BI
Figure 1-3 BPM reference architecture
The IBM BPM Platform
IBM has also developed a BPM Platform that enables the assembly of
components encompassing business partner products and IBM foundation
technologies. The platform includes a wide range of capabilities for modeling,
integrating, connecting, monitoring, and managing business operations within an
enterprise and across a value chain of trading partners and customers. The
unifying framework that accommodates the IBM platform is illustrated in
Figure 1-4. This framework identifies the functional components required for
real-time monitoring, analysis, and optimization of business operations and their
underlying IT infrastructure.
The IBM BPM Platform provides a set of associated interfaces for Business
Partners to plug in components and customize the platform. These interfaces
support facilities such as:
Business rules for dynamic process control and adaptive performance
management
Information management for analytics and reporting
A common event infrastructure for event-driven management of business and
IT operations
Infrastructure Management Services
Interaction Services
Partner Services
Connectivity Services
Application &
Information Assets
Development Services
Business Performance Management Services
(WebSphere BI Modeler)
(WebSphere BI Monitor)
(IBM Workplace)
(WebSphere Portal)
(DB2 Alphablox)
(WebSphere
Information Integrator)
(DB2 Alphablox)
(DB2 Data
Warehouse Edition)
Information Services
(Tivoli BSM, Tivoli SLA)
“WBI Adapters”
“WebSphere
BI Connect”
“Tivoli Intelligent Orchestrator” “Tivoli Monitoring” “Tivoli Access Manager”
“WebSphere BI Server”
“WebSphere BI
Server Foundation”
Process
Services
Process Services
Business
Application Services
“WebSphere
Application Server”
Chapter 1. Understanding Business Performance Management 23
Workplace™ capabilities for business performance management visualization
and collaboration
Business services management for consolidated and dynamic resource
management for aligning IT with business objectives
The IBM BPM platform enables IBM to efficiently assemble end-to-end solutions
for specific business environments. The foundational capability anchoring the
platform is the IBM extensible On Demand portfolio of technologies. The IBM
platform is described in more detail in the section, “IBM BPM Platform” on
page 51.
Figure 1-4 IBM BPM framework
Creating a unified data framework
Creating a unified framework is critical to the success of a BPM implementation.
BPM is a paradigm that succeeds from proactively managing the environment,
rather than reactively resolving individual business issues. There is a need,
therefore, for an architected solution that consolidates and integrates data
related to monitoring, events, alarms, and situation-related information across
the enterprise. Fracturing that enterprise view with isolated data silos defeats the
Business performance management suite of tools
Enterprise integrated development environment
Common event infrastructure
24 BPM Meets BI
purpose of BPM. It actually inhibits the ability to monitor and manage
enterprise-wide business performance.
The need for a unified solution can be satisfied by integrating the required data in
a DB2 data warehouse and using the data warehouse to feed the BPM
environment. If this is not feasible, information integration technologies, such as
IBM WebSphere Information Integrator, can be used to create an integrated
business view of disparate source data. Using dependent data marts that
leverage data from a data warehouse is another possible solution.
IBM BPM solutions are based on DB2 relational databases, as well as a
combination of DB2 relational databases and DB2 OLAP databases. In most
cases, the BPM environment will involve a multi-tier approach, consisting of
legacy applications, data warehouses that feed the BPM solution, and BPM tools
and packaged applications.
Implementing a BPM system results in making business performance data
available to everyone who needs it. Usually, most of this performance data has
not been available to business users prior to the BPM implementation. A BPM
solution is typically facilitated by providing the proactive distribution of the data
through graphical dashboards, rather than relying on users to search for data
they require. Most users respond positively to graphical dashboards embedded
in enterprise portals, and require little if any training in how to use them.
1.3 Summary: The BPM advantage
BPM is attracting attention from businesses because they recognize that it will
help them reduce costs, increase revenues, and provide a competitive
advantage. And more importantly, it will enable them to proactively monitor,
measure, and manage their business performance, and give them the business
intelligence that can enable them to meet their performance targets.
The ability to monitor business processes provides the opportunity to manage
them, and thus to improve them. Companies gain benefit by introducing new
efficiencies into business processes, and, ideally, this results in more effective
business processes.
Important: We do not recommend, however, building a BPM solution using
from data from independent data marts, or sourcing the data directly out of the
operational systems. These approaches involve isolated data silos that lead to
inconsistencies and inaccurate results.
Chapter 1. Understanding Business Performance Management 25
Improved business process effectiveness is achieved using feedback from the
BPM environment. A BPM solution tracks KPIs and alerts management when
results indicate business performance is nearing, or outside of, defined
thresholds. You can then take proactive steps to correct issues before problems
arise, rather than reactively trying to minimize the impact of the problems after
they begin. Business processes can be changed based on the feedback from the
BPM monitoring activity, enabling improved management and more effective
business processes and activities.
Business process effectiveness is further enhanced by unifying BPM processing
with the business intelligence environment. BPM information is integrated into BI
systems, and made available across the enterprise through the use of
information technology. Integration with the BI environment allows business
performance to be compared, managed, and aligned with the business
strategies, goals, and objectives of the organization.
A robust data management infrastructure is required for a successful BPM
implementation. IBM provides that infrastructure using DB2, which acts as the
cornerstone of the business intelligence environment. By integrating business
processes, operational activity monitoring and reporting, and business
intelligence, we can have a complete view of the business across the business
value chain. This is powerful! It is a capability much discussed and much sought
after by all business organizations.
Now that we have an overview, we can explore the relationship between BPM
and business intelligence in more detail.
26 BPM Meets BI
© Copyright IBM Corp. 2004. All rights reserved. 27
Chapter 2. The role of business
intelligence in BPM
Business intelligence (BI) applications gather information about business
processes and activities to make it available to business users, enabling them to
make more informed decisions and take more effective action. Increasingly, BI is
the key to business success and is becoming a key component of almost all new
business initiatives.
Business intelligence is a fairly well understood area and is implemented in some
form in all businesses. In this chapter we discuss the capabilities and evolution of
BI, focusing specifically on the role of BPM in the BI environment. It is the
combination of these two initiatives that provides significant benefits to
businesses.
2
28 BPM Meets BI
2.1 The relationship between BI and BPM
BI enables businesses to access, analyze, and use their data for decision
making. It is used for long-term strategic planning, short-term tactical analysis,
and managing daily operational business activities. Key developments in BI
usage include:
Tactical and strategic BI are moving closer together. This is because strategic
time frames (budgeting and forecasting cycles, for example) are shrinking to
enable companies to become more responsive to business needs and
customer requirements.
Analytic applications are used more and more for proactively delivering
business intelligence to users, rather than requiring them to discover it for
themselves. In many cases, these applications not only deliver information
about business operations, but also put actual business performance into
context by comparing it against business plans, budgets, and forecasts.
Dashboards are the becoming the preferred method for delivering and
displaying business intelligence to users. Dashboards are more visual and
intuitive, and typically provide linkages that enable immediate action to be
taken.
Business rules are a key requirement as companies implement so-called
closed-loop processing to use the results of business intelligence processing
to optimize business operations. This is particularly the case when there is a
requirement to support automated decisions, recommendations, and actions.
Close to real-time (near real-time) or low-latency business information is
becoming an important requirement as organizations increasingly make use
of business intelligence for managing and driving the daily business
operations.
Table 2-1 compares traditional BI with BI used for BPM. It demonstrates how
BPM forms the underpinnings for many of the BI developments we have outlined.
Specifically, BPM helps BI cause operational decision making to become more
proactive and timely, and support a wide range of business users.
Table 2-1 BI comparison (based on material created by Wayne Eckerson of TDWI)
Category Traditional BI BI for BPM
Implementation Departmental Enterprise-wide
Focus Historical Timely, right-time, or real-time
Decisions Strategic and tactical Strategic, tactical, and operational
Users Business analysts Everyone
Chapter 2. The role of business intelligence in BPM 29
2.1.1 Decision making areas addressed by BPM
There are two primary, and different, areas of any company where business
intelligence is required. They are:
Business planning. This area is concerned with strategic issues such as
increasing revenues, reducing costs, and deciding what new products and
services should be introduced. This is where strategic business intelligence is
used. If, for example, the business goal is to maximize product sales, then the
BI system needs to provide executives with information about what products
sell best, in what locations and for what price, who the competition is, and
what is the impact of advertising, sales, and specials.
BPM aids BI in supporting strategic planning by providing easy-to-use
dashboards that show executives how actual business performance
compares with business goals and help executives identify ways of improving
long term business performance.
Business execution. This area focuses on day-to-day operations of the
company. It is concerned with developing and executing efficient business
processes. Business goals, for example, are to meet sales and profitability
targets, and produce and deliver products quickly and at the lowest cost
possible. This area uses tactical and operational business intelligence to help
in decision making and taking action. The difference between tactical and
operational BI is time. Tactical BI analyzes operations over a period of days,
weeks, and months, where operational BI is more typically concerned with
Orientation Reactive Proactive
Output Analyses Recommendations and actions
Process Open-ended Closed-loop
Measures Metrics Key performance indicators (KPIs)
and actionable (in-context) metrics
Views Generic Personalized
Visuals Tables, charts, and
reports
Dashboard and scorecards
Collaboration Informal Built-in
Interaction Pull (ad hoc queries) Push (events and alerts)
Analysis Trends Exceptions
Data Structured Structured and unstructured
Category Traditional BI BI for BPM
30 BPM Meets BI
daily operations and analyzing the business over a period of a few hours or
days.
BI helps improve business execution in two ways. One is monitoring workflow
and reporting operational results. An example of this is tracking production
with the objective of meeting production schedules and cost targets. This is
where BI has traditionally been used. The second area where BI aids
business execution involves monitoring workflow with the objective of
improving and managing the overall operational business process. An
example is to reduce production costs and identify areas where product
quality could be improved. It is in this second area where BI and BPM work
together to aid in optimizing the execution of business processes.
Business planning and business execution are tightly intertwined. How well the
business executes is a critical by-product of business strategy and planning.
Success depends on critical things such as the products and markets selected,
and pricing. Achieving business goals and objectives is in turn heavily dependent
on how well the business plan is carried out. Long-term company revenue, for
example, can only be increased if short-term sales goals are achieved. Strategic,
tactical, and operational BI, when combined with BPM, help organizations
optimize both their business planning and business execution processes and
activities.
Although BPM helps a BI system support both business planning and business
execution, we will in this redbook focus on how you can use BPM and BI to
support business execution. It is beyond the scope of this redbook to address the
role of BI and BPM in strategic business planning. So we will turn our attention
instead to the area of business process execution, and how to monitor and
manage business processes by integrating them with BI.
2.1.2 BPM impact on the business
The objective of BPM is to help companies improve and optimize their operations
across all aspects of their business. Business requirements, therefore, determine
what type of BPM environment is needed. Implementing BPM, however, is more
than just about installing new technology, it also requires organizations to review
the business environment to determine if changes are required to existing
business processes to take advantage of the benefits that BPM can provide.
It is important to understand that creating and managing business processes is
not a one time effort. Company goals, products, and services will change, and
new technologies will be introduced. An organization must, therefore, be ready to
quickly modify its business processes as business needs change. To keep ahead
of the competition, companies must have a good understanding of their business
and their marketplace, so they can quickly make appropriate business decisions.
Chapter 2. The role of business intelligence in BPM 31
Business processes in turn must be flexible enough to accommodate these rapid
and ongoing changes.
To be successful with BPM, a company must thoroughly understand their own
business processes and activities that support each area of their business. The
company must also break down the traditional information sharing barriers that
often exist between different business functions. Both of these efforts aid in
planning and deployment of BPM projects, because they enable a company to
identify where BPM can bring the biggest benefit. These efforts also help a
company focus on improving the management of business processes across
related business areas such as campaign and risk management, customer sales
and service, and supply chain management. At the same time, to remain
competitive, the company must evolve to more of a real-time operating and
decision making environment. This environment uses timely information to make
its critical business processes more responsive and thus more competitive.
BPM enables a BI system to tap into business events flowing through business
processes, and to measure and monitor business performance. BPM extends
traditional operational transaction processing by relating measures of business
performance to specific business goals and objectives. Business users and
applications can then be informed by user alerts and application messages about
any situations that require attention. The integration of business process
monitoring with operational BI analytics enables a closed-loop solution.
Enterprise data involved in managing business performance includes:
Event data from business process operations
Event data from IT infrastructure operations
Historical business process analytics and metrics
Business plans, forecasts, and budgets
Data occurring from external events, for example, changes in marketplace
conditions
This data is used by BI applications to create actionable management
information that enables BPM. The actionable information produced includes:
Key performance indicators (KPIs)
Alerts
Analytic context reports
Recommendations for corrective action
We now look more closely at this actionable information to see how it is used in a
BI and BPM environment.
32 BPM Meets BI
2.2 Actionable business intelligence
Any BI implementation is aimed at turning available data into information and
putting it into the hands of decision makers. It might be easy to conclude
therefore that BI and BPM are one and the same. BPM is focused, however, on a
subset of the information delivered by a BI system. It is concerned with
information that shows business performance and indicates business success or
failure. This information subset enables organizations to focus on the important
task of optimizing business performance.
2.2.1 Key Performance Indicators
Measures or metrics of business performance help drive business decisions.
Metrics show how well the business is doing relative to a defined strategy and
operating plan. A metric can be something as simple as how many parts were
just completed in an operation, or it may be a more complex measurement that
tracks profitability by product, product type, location, and season.
Most businesses have a large number of metrics, and in any BPM project one of
the primary tasks is to determine the most important metrics and thresholds that
can help management determine how the business is doing. We refer to these
metrics as the key performance indicators (KPIs).
Just as any successful project begins by collecting and understanding the
business requirements it is meant to satisfy, a successful BPM project begins by
defining the KPIs that will drive it. This is not a simple task. Methodologies and
best practices for creating the right KPIs for a business are not in the scope of
this redbook. This will be an exercise for your in-house business analysts, with
perhaps some help from outside business consultants.
Regardless of the method used to create them, it is important to note that each
KPI should measure one or more of the top objectives of a business plan, so that
managing it has an impact on improving business performance. For example, a
KPI must measure something that drives real business value.
To be successful in a BPM project, business managers must identify the KPIs
that are appropriate for the part of the business they manage. Some examples of
KPIs are:
Profit margin per transaction
Margin by customer
Customer average days to pay
Returns count, quantity, and value
Contribution to profit by product
Late benefits enrollment
Expiring purchasing contracts
Chapter 2. The role of business intelligence in BPM 33
Percentage of deliveries on time
Delivery days late versus early
Incomplete deliveries
Freight price percentage of revenue
Top 10 back orders by customer
High severity product defects
Expiring sales contracts
Incomplete sales orders
Adjustment counts and value as a percentage of total
Discount taken count and amount
Discount taken versus refuse count
Customer credit exposure
Total value per buyer
Seasonal forecast index values
Customer average arrears analysis
Customer credit limit exceeded
Moving average levels and usage
Average stock level and value
Zero stock days
Inventory stock outs
Reserved quantities and values
Withdrawn quantities and values
Book and physical stock group currency value
Open days for RFQ, PO, and requisitions
Average PO and contract value
Days from requisition to PO
KPIs such as those shown above are often created as ratios and displayed to
decision makers on a dashboard.
Using KPIs to manage the business
KPIs should be used to manage the business. If you have chosen, for example,
the number of rejects per thousand as a KPI, then you must also design a way,
perhaps on a dashboard, for an analyst or manager to monitor that indicator and
to enable them to take action when an issue arises. You must also provide
enough analytic context information so the analyst or manager can discover what
caused the trouble and determine what corrective actions are required. The goal
of a BPM environment is to provide a way to change course when business
operations are not performing satisfactorily.
2.2.2 Alerts
How do you know when a business operation has a problem? Most performance
management applications have some form of exception management capability.
34 BPM Meets BI
When a business goal is missed, or a business threshold is crossed, the
exception management mechanism takes a specific action. This action may be
simply highlighting the information and its business rule in a display or report, or
it may involve generating an alert to send to the user. Alerting is an important
feature of a performance management solution.
It is important, however, to consider exception management and alerting as two
separate mechanisms:
The objective of the exception process is to evaluate business rules against
business measurements. If the evaluation results in an exception, then the
appropriate information has to be collected and sent to the user. To be of
value, the information should not only document that an exception has been
raised, but should also include details about where to find more detailed
information, or even suggest an appropriate action.
The role of the alert mechanism is to route the exception message to the
user. This mechanism should be able to send the message to the right user at
the right time and in the right format. It should also be flexible enough that
destinations and formats can be modified dynamically as users travel and use
different devices.
The reason why exception management and alerting need to be separate is
because exceptions may be raised in a variety of tools and applications, and a
common alert mechanism is required for managing and routing those exceptions.
Without a common alert facility, managing and coordinating exceptions becomes
difficult as exception activity increases. In many BPM applications, exception and
alert management is combined into a single facility. This can make it difficult to
integrate these applications into a common alert framework.
Alerts can be used in conjunction with KPIs. When a KPI is out of an acceptable
range, an alert can be generated and proactively pushed to the analyst or
manager. The use of alerts in this manner avoids the need for the analyst or
manager to take preemptive action to look for these troubled situations.
There are many BI tools that support the delivery of alerts via mechanisms such
as dashboards, scorecards, and portals. These tools do not require business
users to request KPI information. Rather, the alerts for these users are created
automatically. Determining who should receive alerts is a part of the BPM design
process, and will be determined during requirements gathering.
Another design consideration is concerned with exactly when should a business
user receive an alert about a problem situation? The answer to this depends on
the requirements of the business. Today, with IBM technology, it is possible to
deliver instant alerts in almost real-time. This may not, however, always be
necessary. Some alerts will serve business needs if they appear on the
dashboard first thing in the morning after a night of batch processing, while
Chapter 2. The role of business intelligence in BPM 35
others are most useful if delivered right away after the business event that
caused them.
Determining how quickly an alert must be delivered depends on how fast
corrective action is needed to respond to it. Learning, for example, that the
number of rejects per thousand has risen too high can be reported daily if it will
take several days or weeks to correct this issue. On the other hand, if the
business needs to improve this KPI within a few hours, the BPM application
should deliver the alert immediately. BPM applications should be designed to
deliver what is often called right-time information. Right-time means that
information is delivered and action taken at the right time to meet business needs
and requirements. In some cases right time means almost real-time, but in other
situations it may mean minutes, hours, or days.
2.2.3 Putting information in a business context
Delivering an alert to a dashboard gets attention. Seeing that the number of
rejects per thousand is beyond an acceptable level is an obvious call to action. As
impressive as that is, it is not much help if the recipient cannot quickly determine
why the KPI has risen into the danger zone. For this reason, any alert pushed out
to a decision maker needs to be accompanied by pointers to more detailed
analytic information. This analytic information must put the alert in context and
thereby give it more meaning. This information may be produced by an
independent analytic application, an analytic component of the BPM solution, or
by the IT system itself, as is the case with IBM Tivoli Business Systems Manager
(TBSM). The design of this analytic information will vary based on requirements.
In some cases the designer will know exactly what causes an alert and can point
the decision maker to the information required to take corrective action. In other
cases, the designer may offer the decision maker a list of analytic reports to
choose from, any one of which might yield the best context information for further
analysis and action.
Taking corrective action is the critical step that closes the loop and turns BI into
BPM. A good BPM system will at a minimum offer a list of possible corrective
actions to be evaluated. A better system would recommend the most appropriate
solution to correct the problem, and might even take the corrective action
automatically. Clearly, an enterprise-wide BPM solution is likely to contain all of
these possibilities.
2.2.4 Analytic applications
Analytic applications assist business users and analysts as they investigate
information provided by a BI system and its underlying data warehouses. These
applications can be manually or automatically executed, or can be running
continuously. Analytic applications monitor and analyze data from operational
36 BPM Meets BI
activities and business processes. An application could, for example, compare
KPI information to business thresholds, and generate an alert when required.
Analytic applications can also generate derived data. An application could
calculate the profit on a particular product, in a particular geography, and during
some specific time frame. A decision maker could then determine whether or not
a pricing action is needed to yield the desired profit, or decide whether or not to
even continue producing that product. In some situations, the decision making
process can be automated.
A key objective of analytic applications is to open analysis and decision making
to more and more users. This enables companies to realize the promise of
business intelligence and data warehousing and their inherent benefits.
Extending analytic processing to more users
The intent of BI and data warehousing is to enable problem analysis and
resolution. The fruits of this concept, however, are often never fully realized. This
is because many business users do not possess the skills and experience to use
the BI system to analyze data and take action. Another approach is therefore
needed to open BI to more users and gain full benefit from it.
One approach is known as guided analysis. It consists of documenting, as a set
of best practices, the steps followed by a skilled analyst in resolving an alert. The
objective of guided analysis is to extend the alert resolution process to users with
less experience and fewer skills. This is not a new concept, but one that is not yet
widely used, at least, not in a formal manner.
Creating guided analysis involves an experienced and skilled analyst
documenting the step-by-step process that is used to resolve an alert. Included
will be any indicators that are important in providing a clue to the problem. These
indicators in turn lead to the next action to be taken, perhaps looking at a
particular file or report. Information from this file or report may lead to the next
action to taken, and so on. Once these steps are documented, they are
instantiated in an analytic application. The objective of this process is to enable
other users to execute the analytic application and follow the programmed steps
to resolve the alert condition.
Guided analysis is an interactive development process that is enhanced and
expanded as new indicators and new actions are discovered. Over time, it
becomes more and more valuable as the new analytic components are added.
Chapter 2. The role of business intelligence in BPM 37
2.3 Data warehousing: An evolution
A data warehouse contains consolidated data from multiple internal and external
applications. It is one of the prime sources of data for the BPM environment. In
addition to providing information for KPIs, a data warehouse also provides
detailed data for analyzing KPI alerts and determining what actions need to be
taken to resolve business issues. A data warehouse also often provides
pre-aggregated information so that decision makers can see business trends that
help them better understand the KPIs that have been presented to them. KPIs
that measure only a single department or application area can be sourced from
something other than an enterprise data warehouse, but more often BPM
implementations are meant to help manage the entire enterprise.
2.3.1 The need for real-time information
Based on how responsive BPM applications need to be to meet business
requirements, a data warehouse may need to be updated more often than that
provided by the traditional batch update process. Suggestions for the
construction of a near real-time data warehouse can be found in the IBM
Redbook, Preparing for DB2 Near-Realtime Business Intelligence, SG24-6071.
Supplying a data warehouse with fresh real-time data, however, can be
expensive. Also, some data cannot, or need not, be kept in the data warehouse,
even though it may be of critical value. This can be due to its size, or because its
format is difficult to map to the data warehouse structure and user queries.
To address the business requirement for real-time data, organizations need
additional methods of integrating data and delivering information without
necessarily requiring all data be stored in the data warehouse. Current
information integration approaches must, therefore, be extended to provide a
common infrastructure that not only supports centralized and local access to
information using data warehousing, but also distributed access to other types of
remote data from within the same infrastructure. This infrastructure should make
data location and format transparent to the user or application. This new
approach to information integration is a natural and logical extension to current
approaches to data warehousing.
2.3.2 Data warehousing infrastructure
The original business rationale for data warehousing is well known. It was to
provide usable and understandable business information to users. Although
some of this information already existed in business transaction systems, it was
clear that large quantities of raw data were also present in these systems that
38 BPM Meets BI
could be converted into useful business information. This need was addressed
using the multi-layered data architecture shown in Figure 2-1 on page 38.
Why is a multi-layered architecture required? One reason is performance. If
complex user queries are allowed to run against operational business transaction
systems that have been designed and optimized for other purposes, they are
likely to impact the performance of those systems. In addition, response times for
user queries will also be affected because operational data sources are usually
designed for transaction-oriented (operational) access rather than query
(informational) access. To solve these problems, a two-layer data architecture is
required, one operational in nature and the other informational.
Figure 2-1 Data warehouse: three layer architecture
This two-layer data architecture is usually further expanded to three layers to
enable multiple business views to be built on the consistent information base
provided by the data warehouse. This requires a little explanation.
Operational business transaction systems have different views of the world,
defined at different points in time and for varying purposes. The definition of
customer in one system, for example, may be different from that in another. The
data in operational systems may overlap and be inconsistent. To provide a
consistent overall view of the business, the first step is to reconcile the basic
operational systems data. This reconciled data and its history is stored in the
data warehouse in a largely normalized form. Although this data is consistent, it
may still not be in a form required by the business or a form that can deliver the
best performance. The third layer in the data architecture, the data mart,
addresses these problems. Here, the reconciled data is further transformed into
information that supports user needs for different views of the business that can
be easily and speedily queried.
One trade-off in the three layer architecture is the introduction of a degree of
latency between data arriving in the operational systems and its appearance in
Operational Systems
Data Warehouse
Data Mart
Metadata
Data Mart Data Mart
Chapter 2. The role of business intelligence in BPM 39
the data marts. In the past, this was less important to most companies. In fact,
many organizations have been happy to achieve a one day latency that this
architecture can easily provide, as opposed to the multi-week reconciliation time
frames they often faced in the past. The emergence, however, of BPM and other
new BI initiatives demand even shorter latency periods, down to sub-minute
levels in some cases.
Supporting low-latency data
IT organizations responded to the need for low-latency data in BI and BPM
applications by introducing the operational data store (ODS) and the operational
data mart into the data warehouse architecture. These data stores contain
integrated low-latency operational data. The position of the ODS in the data
warehouse architecture is shown in Figure 2-2. Architecturally, the ODS can be
viewed as either an additional layer between the operational systems and the
data warehouse, or as a way of directly supplying data to a data mart.
Figure 2-2 Adding the operational data store
The creation of operational data stores in both data warehousing and in non-data
warehousing projects has accelerated over the last few years. As a result, the
complexity of these projects has increased, as designers struggle to ensure data
integrity in a highly duplicated data environment, while at the same trying to
reduce the latency of data movement between the layers. While many
enterprises have successfully done this with IBM DB2 Universal Database
(UDB), a so-called federated data architecture involving enterprise information
integration (EII) software is often easier and more cost-effective to use for some
applications.
Operational Systems
(Operational) Data Marts
Data Warehouse ODS
40 BPM Meets BI
Data integration
There are several ways of integrating data in an IT system. Two primary methods
are:
Providing access to distributed data using data federation
Moving and consolidating data in a location that is more efficient or consistent
for the application
In its simplest form, federated access involves breaking down a query into
subcomponents, and sending each subcomponent for processing to the location
where the required data resides. Data consolidation, on the other hand,
combines data from a variety of locations in one place, in advance, so that a
query does not need to be distributed. Data federation is provided by enterprise
information integration (EII) software, while data consolidation can be
implemented using ETL and data replication.
The objective of EII is to enable business users to see all of the information they
need as though it resides in a single database. EII shields business users and
applications from the complexities associated with retrieving data from diverse
locations, with differing semantics, formats, and access methods. In an EII
environment, requests for information are specified using standard languages,
structured query language (SQL), extensible markup language (XML), or using a
standard Web service or content API. This provides additional benefits in the
form of cost and resource savings as all requests begin to standardize on
languages, implement code reuse, and the number of recurring queries increase.
Both data federation and data consolidation may require underlying data
mapping and data transformation. Mapping describes the relationships between
required elements of data, while transformation combines the related data
through this mapping. Since the same data may, depending on business
requirements, need to be consolidated in some cases and federated in others, a
common or shared set of mapping and transformation capabilities for both
approaches helps maintain data consistency.
Data mapping and transformation depend on a detailed description of the data
environment in which they operate. This description includes business meaning,
relationships, location, and technical format. This metadata must remain
consistent from definition phase of an integration project through to the running
of a federated query. A comprehensive and logically consistent set of metadata,
whether materialized in a single physical store or distributed across multiple
stores, is fundamental for integrating data. Thus it becomes apparent to make
information integration easier for BPM, that the meaning or semantics of the
information is as important as the syntax or structure.
Chapter 2. The role of business intelligence in BPM 41
2.3.3 Data federation
Data federation using EII software helps companies support new business
requirements for low-latency data, reduce storage for rarely used data, and
provide access to remote and varied data sources. Data federation provides the
ability to present a logical view of information without the need to physically move
all of the data into a data warehousing environment.
Does this mean you should abandon the traditional approach to data
warehousing? Absolutely not. Data federation cannot replace the data
warehouse approach. We do not recommend you use a fully federated or virtual
data warehouse because it will impact performance, data consistency, and data
autonomy. Instead, use data federation to extend and enhance your data
warehousing environment to address specific business needs.
Federated access to real-time data
In a traditional data warehousing environment, if a query or report requires
up-to-the-minute data as well as consolidated historical and analytic data, the
real-time data must be fed continuously into the data warehouse, possibly
through an ODS. The problem with this approach is that not only must significant
quantities of near real-time data be stored in the data warehouse, but also the
ETL environment must be capable of supporting sustained near real-time
processing. Data federation helps solve this problem by providing access to a
combination of live operational business transaction data and the historical and
analytic data already resident in the data warehouse. This scenario is shown in
Figure 2-3.
Figure 2-3 Federated access to real-time data
Application
Operational Systems
Data
Warehouse
ODS
BI Tool
Client
DB2
Database
Information Integration
Federation Metadata
Existing Operational Systems
Data Mart Data Mart Data Mart
Wrapper Wrapper
Other
Database
Application
42 BPM Meets BI
With federated data access, when a user query is run, a request for a specific
piece of operational data can be sent to the appropriate operational system, and
the result combined with the information retrieved from the data warehouse. It is
important to note that the query sent to the operational system should be simple
and have few result records. That is, it should be the type of query that the
operational system was designed to handle efficiently. This way, any
performance impact on the operational system and network is minimized.
In an IBM environment, EII is supported using the IBM WebSphere Information
Integrator (WebSphere II). Operational data sources that can be accessed using
IBM WebSphere II include those based on DB2 Universal Database, third-party
relational DBMSs, non-relational databases, as well as IBM WebSphere MQ
queues and Web services.
IBM WebSphere II federated queries are coded using standard SQL statements.
The use of SQL allows the product to be used transparently with existing
business intelligence (BI) tools, which means that these tools can now be used to
access not only local data warehouse information, but also remote relational and
non-relational data. The use of SQL protects the business investment in existing
tools, and leverages existing IT developer skills and SQL expertise.
Data federation is not limited to accessing real-time data. Any type of data can be
accessed in this way, removing the need to store it in the data warehouse, or a
data mart. In many data warehouse deployments, as much as 20 to 50 percent of
the data is rarely accessed. Where data is used infrequently and is not required
for historical purposes, federated data access enables data to be accessed
directly from its original location.
Federated access to unstructured content
In a traditional data warehousing environment, unstructured content is loaded in
the data warehouse and queried like any other type of data. Unstructured
content, however, is usually voluminous. Even when an organization is willing to
have large amounts of data in a data warehouse, other issues may arise.
Unstructured content may be volatile, outside the organizational span of control,
on the Internet, or in a partner data store. Data federation helps solve these
problems by providing direct access to unstructured content. This access is
shown in Figure 2-4. Again, the benefit of federation here is that it allows access
to the content when and as required. When a report is run, a subquery is
dispatched to the original content source, which returns only the required
information in its most up-to-date form.
Chapter 2. The role of business intelligence in BPM 43
Figure 2-4 Federated access to unstructured content
Federated access to multiple data warehouses
Another area of data warehousing where using a federated data approach is
beneficial is when multiple data warehouses and data marts exist in an
organization. This is shown in Figure 2-5.
In an ideal world, a data warehousing system consists of a single enterprise data
warehouse with multiple underlying dependent data marts. However, this is not
the case in many companies. Mergers, acquisitions, uncoordinated investments
by different business units, and the use of application packages often lead to
multiple data warehouses and standalone independent data marts.
In an uncoordinated data warehousing environment, as separate data
warehouses are added, it becomes increasingly difficult to build enterprise BPM
applications that monitor cross-business unit operations. One solution is to
duplicate the required data across the multiple data warehouses. This is not an
ideal approach, not only because of the additional costs of creating and
maintaining redundant data, but also because of the additional complexity it adds
to the environment. The ideal approach is to consolidate and rationalize the
multiple data warehouses, although this can be expensive and time-consuming.
A federated data approach can be used to simplify an uncoordinated data
warehousing environment through the use of business views that present only
the data needed by BPM applications. While this approach cannot fully resolve
the differences between the data models of the various data warehouses, it does
provide a low-cost way to simplify data access.
Information Integration
Federation Metadata
Wrapper Wrapper
DBMS
Content
Existing Operational Systems
Operational Systems
Data
Warehouse
ODS
BI Tool
Client
Data Mart Data Mart Data Mart
Extended
Search
DBMS
Content
44 BPM Meets BI
This approach to access disparate information can evolve over time by iteratively
increasing the scope of the federation until all of the required information is
available. In this way, the inevitable inconsistencies in meaning or content that
arise among different data warehouses and data marts are gradually discovered.
An organization can then decide whether to physically combine the disparate
data warehouses and data marts, or continue with data federation, or use a
combination of both approaches. If the decision is to integrate certain data
warehouses and data marts, the design of the combined information store is
simpler because data inconsistencies have already been discovered.
Figure 2-5 Federated access to data marts and data warehouse
When to use data federation?
It is important to re-emphasize that we do not recommend eliminating data
warehouses and data marts, and moving BI query, reporting, and analytic
workloads to a purely federated data infrastructure. Virtual data warehousing
approaches have been tried many times before, and most have failed to deliver
the value business users require. Data federation does not replace data
warehousing, it complements it.
Data federation is a powerful approach for solving certain types of data access
problems, but it is essential for you to understand the trade-off of using federated
data. One issue to consider is that federated queries may need access to remote
data sources, for example, an operational business transaction system. We have
already discussed the potential impact on the performance of operational
applications of complex query processing. With the federated data approach, this
impact can be reduced by sending only simple and specific queries to an
operational system. In this way, performance issues can be predicted and
managed.
Operational Systems
Data
Warehouse
ODS
BI Tool
Client
Information Integration
Federation
Metadata
Operational Systems
Data Mart Data Mart Data Mart
Wrapper
RDBMS
Data Mart
Data
Warehouse
Chapter 2. The role of business intelligence in BPM 45
Another potential issue is how to logically and correctly relate data warehousing
information to the data in operational and remote systems. This is a similar
problem that must be addressed when designing the ETL processes for building
a data warehouse. The same detailed analysis and understanding of the data
sources and their relationships to the targets are required. Sometimes, it will be
clear that a data relationship is too complex, or the source data quality
inadequate to allow federated access. Data federation does not, in any way,
reduce the need for detailed modeling and analysis. It may in fact necessitate
more rigor in the design process because of the real-time and inline nature of any
required data transformation or data cleanup.
Data federation is more suited to occasional queries, rather than regular repeat
queries that can benefit from the preprocessing of the source data. Also,
federated data queries cannot easily handle complex transformations, or large
amounts of data cleansing. In these situations, cleansing and loading the data
into a data warehouse is often a better solution.
The following is a list of circumstances when data federation is the appropriate
approach to consider:
Real-time or near real-time access to rapidly changing data. Making copies of
rapidly changing data can be costly, and there is always be some latency in
the process. Through federation, the original data is accessed directly.
However, you must consider the performance, security, availability, and
privacy aspects of accessing the original data.
Direct immediate write access to the original data. Working on a data copy is
generally not advisable when there is a need to modify the data, as data
integrity issues between the original data and the copy can occur. Even when
a two-way data consolidation tool is available, two-phase locking schemes are
required.
It is technically difficult to use copies of the source data. When users require
access to widely heterogeneous data and content, it may be difficult to bring
all the structured and unstructured data together in a single local copy. Also,
when source data has a very specialized structure, or has dependencies on
other data sources, it may not be possible to query a local copy of the data.
The cost of copying the data exceeds that of accessing it remotely. The
performance impact and network costs associated with querying remote data
sets must be compared with the network, storage, and maintenance costs of
storing multiple copies of data. In some cases, there will be a clear case for a
federated data approach when:
– Data volumes in the data sources are too large to justify copying.
– A very small or unpredictable percentage of the data is ever used.
– Data has to accessed from many remote and distributed locations.
46 BPM Meets BI
It is illegal or forbidden to make copies of the source data. Creating a local
copy of source data that is controlled by another organization or that resides
on the Internet may be impractical due to security, privacy, or licensing
restrictions.
The users' needs are not known in advance. Allowing users immediate and ad
hoc access to needed data is an obvious argument in favor of data federation.
Caution is required here, however, because the potential for users to create
queries that give poor response times and negatively impact both source
system and network performance. In addition, because of semantic
inconsistencies across data stores within organizations, there is a risk that
these queries would return incorrect answers.
When to use data consolidation?
The alternative to data federation to is copy and consolidate source data into a
local data store. The arguments in favor of data consolidation are the opposite of
those for data federation, namely:
Read-only access to reasonably stable data is required. Creating regular
copies of data source isolates business users from the ongoing changes to
source data, and enables data to be fixed at a certain moment in time to allow
detailed analyses to be done.
Users need historical or trending data. Historical and trending data is seldom
available in operational data sources, but can be built up over time through
the data consolidation process.
Data access performance and availability are overriding requirements. Users
routinely want fast access to data for complex query processing.
User needs are repeatable and can be predicted in advance. When queries
are well-defined, repeated, and require access to only a known subset of the
source data, it makes sense to create a copy for local access and use. This is
particularly the case when a set of users need a view of the data that differs
substantially from the way the data is stored in the source systems.
Data transformation or join processing is complex or long-running. We do not
recommend that you do this type of processing inline as a part of a federated
data query because of performance and high processing costs.
2.4 Business intelligence: The evolution
A BI system enables businesses to analyze and manage business performance
through the access and use of consistent and timely business data and
information. In an ideal world, organizations would react instantly to business
needs. However, this is not always possible. Accurate business information is
Chapter 2. The role of business intelligence in BPM 47
required for effective decision making. Also, it takes a certain amount of time to
collect and deliver this information to business users, and for users to act on this
information. The delay between a business event occurring, and action being
taken, depends on many things, including the technologies used to collect,
analyze, and deliver information to decision makers.
BI technologies and products are evolving to support more effective and more
timely access to information. Key trends here include:
Easy access to any data, on any source, at any time for analysis.
Linking business process data to operational activity data for a complete view
of the enterprise.
Implementation of business rules and KPIs to enable consistent management
of the business activities.
Automatic alert generation for proactive problem avoidance rather than
reactive problem impact minimization.
Analytic applications to guide users through problem resolution.
Pushing appropriate data to users for analysis and problem resolution.
Extended analytic applications for automatic problem resolution without the
need for user involvement.
Real-time data flow to enable monitoring and proactive management of
business processes.
Evolving a BI environment to include these capabilities enables companies to
proactively manage their businesses, rather than just simply reacting to business
situations as they arise.
2.4.1 Integrating BPM and BI
BPM enables a business intelligence system to tap into and monitor business
process events flowing through operational systems. These monitored events are
used to measure and manage business performance. The integration of process
event monitoring with BI is a key component of the overall BPM platform that
enables a closed-loop solution.
Figure 2-6 shows an example of a process-driven and closed-loop application
environment. In this environment, business applications execute transactions in
support of business processes, performing activities such as receiving customer
orders, managing inventory, shipping products, and billing customers.
Transaction data and/or events are captured and integrated in a data warehouse
environment for reporting and analysis by BI applications.
48 BPM Meets BI
Figure 2-6 Business process event-driven BI
The use of BI-driven performance management applications to relate transaction
data to business goals and forecast converts the integrated, but raw, data
warehouse data into useful and actionable business information as depicted in
Figure 2-7.
Business users employ their business expertise and guided analysis to evaluate
the actionable business information produced by the BPM and BI systems to
determine what decisions, if any, need to be made to optimize business
operations and performance. Applying business expertise to business
information creates business knowledge. This knowledge can then be fed back to
the business processes that created the transaction data and business
information being analyzed, and the business processes enhanced as
appropriate. In some situations this feedback loop can be automated.
automated
feedback
action
analysis
action
analysis
action
analysis
guided
analysis
manual
feedback
monitor
payment
guided
analysis
monitor
sales
guided
analysis
monitor
order cycle
back order
check
inventory
receive
order
shipping
billing
customer customer
track
processes
actions
actionable BI actionable BI actionable BI data
warehouse
Business
intelligence
system
business
(process)
activities
Courtesy of Collin White BI Research
Chapter 2. The role of business intelligence in BPM 49
Figure 2-7 Creating actionable business information
The ability to make the BI applications process event-driven, rather than
data-driven, is especially important in operational right-time BI processing. This
is why right-time BI solutions should be integrated with the business performance
management capabilities that are used to design and deploy business
transaction applications. The process models of these business transaction
applications help identify points in the business process that need to be
monitored by the BI applications.
The need for a BI system to become business process event-driven is the key to
success in exploiting the business benefits offered by right-time business
intelligence processing. This is why the tight integration of BI and business
transaction systems and products is so crucial for this type of BI processing.
(Automated)
actions
Metrics
Actionable
metrics
User
alerts
action
rules
metric
rules
context
rules
exception
rules
transformation
rules
Transaction
data/events
Inventory for part xyz is 654 units
Inventory for part xyz is 146 units
Below today’s requirements
Alert: Inventory for part xyz is 146
units short + guided analysis workflow
Action: place urgent order for
Part xyz
Courtesy of Collin White BI Research
50 BPM Meets BI
© Copyright IBM Corp. 2004. All rights reserved. 51
Chapter 3. IBM BPM enablers
In this chapter, we discuss technologies and interfaces that enable the
development and deployment of a business performance management (BPM)
solution. Two key enablers are the IBM BPM Platform and Web services. Since
this redbook focuses on the integration of BPM with Business Intelligence, we
also introduce IBM products to support this integration, including WebSphere
Portal, Lotus® Workplace, WebSphere Business Integration (WBI), WebSphere
Information Integrator (WebSphere II), and DB2 Universal Database (UDB).
3.1 IBM BPM Platform
One of the primary enablers of BPM is the IBM BPM Platform, introduced earlier
in “The IBM BPM Platform” on page 22. Associated with this platform is a set of
five processes for BPM that provide the interfaces and tools to support a BPM
closed-loop system.
The processes are depicted in Figure 3-1.
3
52 BPM Meets BI
Figure 3-1 IBM Business Performance Management Platform
We now take a closer look at the details of the processes:
Model. Business modeling is a key process that helps capture what matters
to businesses, business policies, key performance indicators, business events
and situations, and the actions necessary to respond to events and optimize
performance. The business models serve as the basis for the instrumentation
of business processes for monitoring and optimizing business operations.
Monitoring enables visibility into business performance problems on a
real-time basis, and provides the ability to take timely actions to optimize
business operations.
Modeling allows us to align objectives and priorities between IT and business.
This BPM focus area ties well into the current industry push for Model Driven
Development (MDD). An example is a business process that can be modeled
by a business analyst, and then utilized by an IT department to modify the
same model to deploy into an IT environment.
IT models help capture the managed IT resources and services within an
organization. The topology of resources, hardware systems, software
infrastructure, and the properties and relationships across these resources,
can be reflected in models.
Capturing models of IT resources can offer significant benefits to IT
management activities, from service level agreement (SLA) enforcement to
transaction performance monitoring and problem determination. IT models
can help with causal analysis by correlating events back to the resources that
directly or indirectly caused them to occur.
Chapter 3. IBM BPM enablers 53
To align IT objectives and performance with business needs, IT models
should be related to business models. Modeled relationships between the
elements in a business model (processes, organizations, and services) and
the IT resources on which the business depends (systems and software) can
be used to provide shared context and can help ensure the health and
availability of IT infrastructures for optimal business performance.
Deploy. The deploy process transforms, packages, distributes, and installs
the models created during modeling. It converts platform-independent models
to technology-specific implementations. The deploy process must be capable
of handling virtualized resources as virtual teams for human resources and
workload management for infrastructure resources. It should also support the
deployment of new components into an existing deployed system (dynamic
redeploy) and into an currently executing solution (hot deploy).
Deploying the process delivers tools to integrate and manage the business
processes. The use of technologies such as information integration and
business intelligence specifies the interfaces used to access and analyze the
information that flows from all the other information sources in an enterprise
for dynamically controlling business processes, managing business, and IT
performance.The common event infrastructure provides a common
architecture for creating, transmitting, and distributing business process and
IT events.
Monitor. The monitor process enables administrators and managers to
access performance data and view the status of resources and business
processes and performance data in real-time.
The model-driven operations are run by linking the deployed models to the
execution environment. It supports business performance management
through run-time instrumentation that:
– Monitors business metrics, KPIs, and critical business situations in
real-time.
– Provides a feedback loop from monitoring and analysis back to model
improvement and redeployment.
– Injects executable business rules to allow dynamic changes to business
behavior.
The monitor process can also send alerts to users when particular business
patterns or exceptions occur. These alerts are delivered by e-mail, Web,
pager, or phone along with a hyperlink to make it easy for users to access
relevant information.
Monitoring enables managers to see how projects are performing and what
specific issues need resolution. Executives can easily view business metrics
and alerts tailored to their areas of responsibility. This tailoring can be both
role-based and industry-focused.
54 BPM Meets BI
Monitoring the deployed business process allows proactive and directed
action to be taken and provides a real-time contextual insight into process that
is running and gives users a personalized and role-based view of displaying
business information and managing business and IT operations.
The most common mechanisms for viewing performance data are
dashboards and scorecards. A dashboard provides a graphical user interface
that can be personalized to suit the needs of the user. A dashboard
graphically displays scorecards that show performance KPIs, together with a
comparison of these KPIs against business goals and objectives.
Analyze. The analyze process is used by the monitor process to calculate
predefined KPIs and perform ad hoc analyses. These KPIs and analyses can
be used with associated historical data to evaluate the performance of the
organization.
Analysis of the information is provided with context to the users who make
decisions on process metrics to detect anomalous situations, understand
causality, and take action to maintain alignment of processes with business
goals.
A key technology for monitoring and analyzing business performance is data
warehousing. A data warehouse brings together data from multiple systems
inside and outside the organization. A data warehouse provides access to
both summary data and detailed business transaction data. This data may be
historical in nature, or may reflect close to real-time business operations. The
availability of detailed data enables users to drill down and perform in-depth
analyses of business performance.
BI applications and tools can also play an important role in analyzing
performance data and historical data in a data warehouse. Analysis of
business event data and other historical business data is necessary for
diagnosing business performance problems. It is also crucial for evaluating
decision alternatives and planning appropriate corrective actions when
business performance management issues are detected.
The analysis process supports business services management by predicting
potential IT infrastructure problems before they occur. By analyzing historical
information about the health and performance of the infrastructure, analysis
can help predict potential violations of service level agreements or internal
resource performance thresholds before they actually materialize.
Analysis using ad hoc methods means that the monitoring process of BPM
must be designed in such a way that it remains dynamic and easily
changeable. Businesses will constantly want to view information about
performance in new ways that are more informative and more easily
understood. And IT must be positioned to provide that level of support.
Chapter 3. IBM BPM enablers 55
Act. This process can be either tactical or strategic in nature. The tactical
approach usually involves line-of-business users who react to a real-time
dashboard or alert. Exception conditions are flagged based on defined
exception rules. If an alert is triggered by a business exception, a user can
respond rapidly to handle the situation. In certain situations, this response
can be automated. Examples of actions performed in response to alerts
include reassigning work items or changing their priority, modifying process
structure, altering resource allocations, changing rules, or modifying trigger
conditions for business situations.
It also applies at the strategic level. Based on information from the analysis
process, managers can implement and prioritize major business initiatives,
such as adding new lines of business, redeploying assets and resources, and
making major acquisitions of technologies or business capabilities.
For well-documented and straightforward business processes, organizations
can implement intelligent processes that can automatically recommend or
take action in response to a predefined event. One travel-related e-business
firm, for example, alerts managers to expand the inventory of airline seats and
hotel rooms in response to customer purchasing trends.
Being able to act appropriately based on the information provided to resolve
problems based on business priorities and service level agreements is critical.
Again, linking IT and business strategies and goals is one of the key tenets of
business performance management.
The primary IBM products that support each process are shown in Table 3-1.
Table 3-1 IBM products supporting the BPM Platform
IBM BPM Platform IBM Product
Model WebSphere Business Integration
Modeler
Deploy WebSphere Business Integration
DB2 Alphablox
WebSphere Information Integrator
DB2 UDB Data Warehouse Edition
DB2 Content Manager
Monitor WebSphere Business Integration (WBI)
(including WBI Monitor)
WebSphere Portal
Lotus Workplace
56 BPM Meets BI
In the sections that follow we look each of the processes in turn, and introduce
the products outlined in the table. Subsequent chapters of this redbook discuss
the products in more detail.
3.1.1 User Access to Information
The Monitor and Analyze processes of the BPM Platform supply a user interface
for both business and IT users to access business information, and manage
business and IT operations. WebSphere Portal and Lotus Workplace serve as
the foundation for these areas. We will first review the role of an enterprise portal,
and then discuss how IBM technologies support enterprise portal development
and the IBM BPM Platform.
An enterprise portal provides both desktop and mobile business users with a
single integrated and personalized Web interface to the business content they
need to do their jobs. A portal also provides collaboration facilities that allow
portal users to exchange information and communicate with each other.
From a business intelligence perspective, a portal helps relate business
intelligence to many other types of business information that exist in an
organization. Figure 3-2 shows you an example of this in the context of a call
center application. The portal in the figure provides a single view of all the
operational and BI information and applications required by a call center support
person to do their job.
The portal is quickly becoming the primary business user interface to enterprise
systems because it personalizes business content to match each user’s role in
the organization, making it much easier to find and access information.
A portal can be used to display BPM dashboards that present role-based content
and context-sensitive business views of key business activities, events, and
Analyze DB2 Alphablox
WebSphere Information Integrator
DB2 UDB Data Warehouse Edition
DB2 Content Manager
Act WebSphere Application Server
WebSphere Business Integration
Server and Server Foundation
WebSphere Business Integration
Modeler
IBM BPM Platform IBM Product
Chapter 3. IBM BPM enablers 57
measurements to business users. The information presented through these
tailored business views is used to monitor and manage business performance.
Although BPM dashboards are a relatively recent innovation, the idea of using an
intelligent information system to supply relevant information to business users is
not. Management information systems (MIS) and executive information systems
(EIS) have been used for many years in organizations to display business
information. Business executives or managers primarily use these latter systems
to view financial data about the health of their business. This data is typically time
delayed in that it shows events after they have occurred.
Figure 3-2 Call center example
BPM dashboards represent a significant advance over MIS and EIS approaches
for displaying business information. In addition to strategic and tactical
information, a dashboard also delivers near real-time information, alerts, and
automated recommendations based on rules and thresholds defined by
line-of-business managers and users. This type of dashboard allows a business
user to monitor business events, detect business issues, and execute
appropriate business actions.
Dashboards have become mission critical workplaces for key CxO-level
executives (CIO or CEO, for example) to ensure that their leadership teams
execute effectively in the challenging business environments prevalent today.
Dashboards not only support business executives, but also assist a wider
audience of line-of-business (LOB) and systems management users.
User Query
Show me all the
customers with a
net worth > $ 1M
who hold IBM
DBMS DBMS
Data
Warehouse
DBMS
Business Integration
Workflow
Open New Brokerage
Account for Customer
Portal
Legacy
Data
58 BPM Meets BI
The popularity of dashboards is a facet of the growing importance of BPM and
BI. Dashboards offer significant business value in monitoring and managing
business performance. Formal studies of the information needs of executives
and managers indicate the importance of getting information quickly and
efficiently. There is also a strong desire by managers and executives to be able to
filter information to focus on key business objectives and performance indicators.
Dashboards are designed to satisfy these requirements. Dashboards provide a
graphical display of business critical information and use visual cues, such as
color, to improve the speed of problem determination and decision making.
You can have many types of BPM dashboards, intended for different users and
purposes. Some are designed for executives and LOB managers, while others
are intended for systems administrators. In the examples that follow, we discuss
the key types of dashboards used by organizations today.
An executive dashboard is used as a management tool to define strategic
business goals and responsibilities, and to manage business performance
against those goals. The information or scorecards shown on an executive
dashboard enables business users to compare and analyze KPIs and business
goals. This facilitates efficient and informed strategic decision making. A
scorecard is often associated with a formal management methodology known as
the balanced scorecard or six sigma. Figure 3-3 shows an example of the
balanced scorecard approach.
Figure 3-3 Balanced scorecard methodology for an executive dashboard
A tactical business dashboard is intended for line of business (LOB) managers to
monitor and manage short-term business initiatives, for example, sales and
marketing campaigns. Figure 3-4 shows an example of a business dashboard
for a retail operation. The dashboard provides a store map in the top left quadrant
Critical
Business Processes
Business Outcomes
Business Drivers
Customer Requirements
to be met
Organization
Policies and Structure
Financial Objectives
For shareholders
Scorecard
Perspectives
Process
Customer
Process
Organization
Chapter 3. IBM BPM enablers 59
of the display. Selecting a location reveals detailed performance information as a
collection of metrics. The dashboard has a number of visual cues. Anything
highlighted in red, for example, is a potential performance problem. Additionally,
the left side and bottom of the dashboard offer menu choices to view
performance information from other business perspectives such as business
promotions, the product line, and so forth.
Figure 3-4 Tactical business dashboard
An operational process dashboard is used to monitor and manage daily and
intra-day business operations, and to display the progress of specific business
processes. Figure 3-5 depicts a process dashboard. It shows the status of each
process, and also information regarding which steps in a business process have
been completed, and how many steps have not started yet. It also displays
process statistics that enable you to monitor process execution and determine
how efficiently a business process is being executed. This latter information
enables you to take proactive steps quickly during process execution if a
potential problem appears.
The process dashboard also displays process alerts that require action. These
alerts are based on KPI and other relevant event thresholds. The dashboard alert
offers the opportunity for users to take immediate action to correct the threshold
60 BPM Meets BI
condition. This also presents the opportunity to implement an analytic application
that could potentially correct the issue automatically.
Figure 3-5 Operational process dashboard
Dashboard architecture
Dashboards are an integral part of the IBM BPM Platform. Dashboards are
delivered using IBM products. For example, Lotus Workplace uses services and
components from WebSphere Portal.
Users typically interact with a dashboard using a desktop Web browser, but you
can use other modes of interaction, including a thin client or a pervasive mobile
device. A dashboard consists of multiple display views that are sequenced based
on user interaction with the dashboard. Each display view shows data for a
specific business state. Display views are associated with a collection of
business services that interact with the back-end data sources.
Dashboards are grouped together on a Web page, which can be customized for
each user based on specific business rules. A BPM workplace can involve many
pages. The appropriate page is displayed based on how the user interacts with
the Web page and its associated dashboards.
Chapter 3. IBM BPM enablers 61
Dashboards within a page can interact with and pass information to each other. A
dashboard that allows a user to select a particular performance KPI, for example,
can interact with another dashboard to display performance trends for the
selected KPI. The interaction between dashboards in a page is handled by a
property broker that maps the data between dashboards and provides the ability
for a dashboard to monitor user actions and data changes.
The underlying technologies for IBM BPM dashboards are provided by Lotus
Workplace and WebSphere Portal. Both employ J2EE™ and Web services
standards. Lotus Workplace technology brings together several collaborative
capabilities into a single easily managed platform. When combined with
WebSphere Portal, Lotus Workplace allows collaboration to be extended to
enterprise business applications, processes, and systems. The IBM BPM
Platform leverages the combined strengths of both Lotus Workplace and
WebSphere Portal.
3.1.2 Analysis and Monitoring
The IBM BPM Platform enables and encourages tracking, analyzing, and
presenting critical business information and metrics to enterprise portal users
through role-based workplaces and their underlying dashboards.
The three main components of access and management of information are:
User interface
Information services
Information asset management
Access and management of information are critical success factors for BPM. For
a graphical illustration of the information access and management components,
see Figure 3-6.
62 BPM Meets BI
Figure 3-6 Information access and management components
Industry standards are the basis for the interfaces between the components of
information access. These interfaces allow IBM and its Business Partners to
incorporate products which provide access, presentation, and control of
information with optimal cost and speed. Table 3-2 identifies some of the IBM
products that support these requirements.
These products provide access to a wide variety of data sources. These products
are also responsible for creating and analyzing business intelligence assets (for
example, data warehouses, data marts, data cubes, and report caches). The
results of these analyses are associated with the business processes that
created the data being analyzed. Linking the results back to source business
processes enables a closed-loop system.
The insight gained by information analysis enables business users to modify and
optimize business processes. The analysis loop is closed when the results of the
business process changes are reflected in new data, which is then fed back into
the information assets to begin a new analysis cycle.
Table 3-2 Products supporting Information Access, Analysis, and Monitoring
Requirement or Component Product
User interface component
(visualization and reporting)
DB2 Alphablox
WebSphere Portal Portlets
IBM Business Partners
Business Activity Workplace
Presentation and Reporting
Consolidation
&
Placement
Services
Federation
Services
Information Services
I
n
f
o
r
m
a
t
i
o
n

A
s
s
e
t

M
a
n
a
g
e
m
e
n
t
Data
Warehouse
ODS
ODS
ODS
Information Assets
Transaction Systems
User Interface Components
BPM
Data Store
Portlet APIs
SQL &
Web
Services
JDBC
ODBC
Chapter 3. IBM BPM enablers 63
SQL acts as the main interface to information, but associated technologies such
as Web Services, and JDBC™ and ODBC, are also supported. Visualization and
reporting functions enable connections to provide user access to information
using the portlet APIs of the enterprise portal. These latter APIs are discussed in
“User Access to Information” on page 56.
User interface component
User interface components provide presentation, analysis, and reporting
capabilities that offer integrated, interactive, and role-based displays of business
process metrics.
The user interface components provide capabilities to satisfy business needs
that vary from basic reporting to advanced visualization. A variety of analysis
functions can be applied to information supported by the underlying information
services components. These include:
Multidimensional analysis and KPI generation to review different perspectives
of business activities and drill down into detailed information
Statistical analysis for enhancing business insight into business processes
and their activities
Data and text mining capabilities:
– Clustering analysis for grouping and profiling information
– Association analysis for determining associations between business
activities and events
– Sequential pattern analysis for discovering repeated sequences of events
– Classification algorithms to categorize information
– Predictive algorithms for scoring business factors, for example, the
likelihood of fraud, credit risk, or propensity to buy
These analytic functions can be stored and run using the services of the
underlying database management system of the data warehouse. analytic
functions operating close to the data have the benefit of being able to take
advantage of the scale and power of the database system. DB2 Data Warehouse
Edition is an example of a database technology that provides integrated services
for creating and running analytic functions. Additional functions may be applied in
Information services (data
consolidation and federation)
WebSphere Information Integrator
DB2 Data Warehouse Edition
IBM Business Partners
Information asset management DB2 UDB & Data Warehouse Edition
DB2 Content Manager
Requirement or Component Product
64 BPM Meets BI
purpose-built tools at the user interface services layer. An example of an IBM
product here is DB2 Alphablox.
Information services component
The information services component provides technical facilities and interfaces
for accessing and integrating heterogeneous data (IBM, Oracle, and Microsoft®
databases, for example) on different operating platforms (mainframe, UNIX®,
Linux, or Windows®), and with different data formats (relational, XML, text, and
spreadsheets, as example). The information services component supports
multiple data integration disciplines including data federation and data
consolidation. These capabilities are available to IBM Business Partners using
open industry standard interfaces: Web services, SQL and XML, and JDBC and
ODBC.
Information services components (refer to Figure 3-6 on page 62) are
responsible for coordinating access to diverse information sources. These
components enable information to be analyzed while at the same time
minimizing the impact on source systems. They also allow analysis to be isolated
from the heterogeneity of the format, storage type, and location of the
information. This means analysis and reporting applications can be written once,
and then easily customized and deployed against different sources of
information.
To satisfy a broad range of performance and information latency requirements,
the data federation and data consolidation technologies of information services
can be used independently, or in combination, with each other. The approach
taken will depend on requirements such as data access performance and
availability, complexity of the data transformation or analysis required, data
currency characteristics, data volumes, and data update frequencies. It is likely
that most organizations will use a combination of both approaches.
Data consolidation can be used when a historical perspective of business
operations and trends, and/or sophisticated data analysis (data mining, for
example) are required. This approach is also particularly useful in cases where
complex data transformations and handling large data volumes are necessary.
Data consolidation involves periodic copying of business transaction data and
events from various parts of the business process environment to a data
warehouse. The periodic nature of this approach supports data currency
requirements that are oriented to a specific point-in-time, for example,
close-of-day or month-end. Facilities known traditionally as extract, transform,
and load (ETL) and data replication provide this capability. Recent advances in
ETL technology now support the near real-time updating of a data warehouse,
which allows the data consolidation approach to be used for situations where
low-latency data is required for analysis.
Chapter 3. IBM BPM enablers 65
Data federation can be used when data about current business activities or
business trends needs to be correlated with information in business transaction
databases, event logs, point-in-time data warehouses, or other related structured
and unstructured data stores. Instead of moving the required information into a
data warehouse, data federation accesses the live source data dynamically when
running queries, reports, and analyses. This approach lends itself to situations
where both current and historical data is required in the same application. It
yields a lower cost of implementation in projects where copying large amounts of
data is prohibited by cost. It is also used when copying data is prevented by
organizational policies, security needs, or licensing restrictions. Data federation
capabilities are supported by an emerging technology known as enterprise
information integration (EII). IBM WebSphere Information Integrator is an
example of an IBM product that supports EII.
A combination of data consolidation and data federation allows developers and
software vendors to configure a best practices BPM solution that provides
access to the business information required to gain informed insight into
business operations. These concepts were discussed in-depth in 2.3 “Data
warehousing: An evolution” on page 37.
WebSphere Information Integrator
WebSphere Information Integrator (formerly DB2 Information Integrator) provides
EII functionality for handling data that resides in a variety of different databases
and a number of different formats, including:
IBM and third-party relational DBMSs such as Microsoft SQL Server,
NCR/Teradata, Oracle, and Sybase
ODBC and OLE DB data sources
Legacy DBMSs (IMS™, CA-IDMS, for example) and flat files including VSAM,
XML, and Microsoft Excel®
WebSphere MQ, WebSphere MQ Workflow, and Web services
DB2 Content Manager, Lotus Notes®, Lotus Domino®, and many third-party
content managers (Documentum, FileNet, Hummingbird®, Interwoven, Open
Text, and Stellent, for example)
DB2 Net Search Extender and Lotus Extended Search
Third-party application packages from vendors such as PeopleSoft, SAP, and
Siebel (supported using WebSphere Business Integration Server and
appropriate application adapters)
WebSphere Information Integrator simplifies access to information by eliminating
the need to know multiple SQL dialects, by managing connections to multiple
data sources simultaneously, and by managing complex join logic to correlate
multiple data sources. This not only reduces program complexity, but also
66 BPM Meets BI
reduces the level of skill needed for the programmer to cope with a
heterogeneous IT environment.
Using DB2 Control Center, the WebSphere Information Integrator administrator
defines the data source locations and types, user-ID mappings, and mapping the
source fields into a relational schema (known as a nickname). The administrator
can also define simple transformations that allow data in one database to be
joined to information in another data source.
The following scenarios illustrate the use of WebSphere Information Integrator
capabilities in different application environments.
Scenario 1: Real-time account information
In this scenario, a call center in a bank provides customer information to agents
in a call center through a data warehousing system. This information includes
details of customer transactions over a period of time, as well as summarized or
derived information showing behavioral trends and marketing opportunities. The
most recent information available in the data warehouse is as of
close-of-business yesterday. Most of the trend and marketing information is
updated monthly, and so it can be a number of weeks out-of-date.
The bank has two business problems. The first is that agents are unable to
support customers who make inquiries about today's transactions. The second is
that agents find it difficult to use customer calls as potential sales opportunities
because of the lack of real-time information about recent customer transactions.
Using the existing traditional data warehouse environment, the bank could solve
these issues by running the data warehouse ETL processes in near real-time
mode. This solution would lead to storing large amounts of additional detailed
data in the data warehouse, of which only a small portion would be used when
specific customers call.
A federated data approach can more easily solve the bank’s problem. In this
solution, WebSphere Information Integrator enables access to current customer
data residing in IBM DB2 and other relational and non-relational databases.
When a customer calls with a problem relating to a recent transaction, the agent
can run a query that joins consolidated customer information in the data
warehouse with the relevant transactions in the branch, ATM, and Internet
banking systems. When running this query, WebSphere Information Integrator
maps the consolidated customer number in the data warehouse to the
appropriate customer numbers in the underlying operational systems, and
submits relatively simple queries to each of these systems. The results from
these queries are then joined together and presented to the agent. This
approach is likely to be simpler and less costly than upgrading the ETL
processes to handle near real-time information.
Chapter 3. IBM BPM enablers 67
WebSphere Information Integrator can also help the agent to make more
informed decisions when marketing to a customer who has called in for another
reason. In this situation, trend or opportunity data in the data warehouse is joined
with real-time indicators in operational systems such as the customer account
balance or recent major transactions. The business benefit here is achieved
without the need to store substantial amounts of data that may never be used in
the data warehouse.
Scenario 2: Supply chain optimization
A classic stress test for BI is optimizing the retail supply and distribution chain to
meet customer demand. This is usually done by monitoring and analyzing
inventory levels in hundreds of locations, and from hundreds of suppliers. To help
improve the optimization process, many retail distribution companies are now
adding operational process dashboards to monitor stock levels and help these
companies prevent out-of-stock situations.
In this type of scenario, because there are thousands of items to track in many
hundreds of stores, the operational process dashboard is built around exception
alerting. That is, the system alerts business users to take action rather than
requiring them to constantly monitor inventory levels. Based on a set of business
rules, when stock drops below a certain threshold for an item, an alert is sent to
the dashboard to warn you about the low stock situation. The dashboard also
provides you with access to additional information for analyzing the low-stock
situation and determining what action to take. This information may include, for
example:
Stock level history on the alerted item for the last three weeks compared to a
year ago for the same items (from a DB2 data warehouse)
The current invoices and shipment orders for the alerted item (from the ERP
procurement system)
A summary of stores within fast shipping distance which may have some
excess stock to offer (from a shipping data mart not based on DB2)
A list of promotions to see if one of these may be driving the stock level down
(from a content management system documenting coupon rebate offers and
advertisements)
To deliver all these facts to the business user, WebSphere Information Integrator
can gather this information from various heterogeneous databases, and make it
available to the portal used to display the dashboard. Within minutes, you could
determine, for example, that no shipments are likely to arrive soon, find the
reason for the low stock situation, locate the most likely sources of
replenishment, and route inventory from the stores that have excess items.
68 BPM Meets BI
Scenario 3: Executive dashboard
At present most executive dashboards focus largely, if not exclusively, on
providing structured data to business users. It is widely recognized, however, that
a substantial amount of executive decision making requires access to
unstructured content. Unstructured content is excluded from current executive
dashboards because of difficulties in accessing and relating it to structured data,
and because of the large increase in data volumes that would be caused by
storing the content in a data warehouse. Another reason is that unstructured
content is often sourced externally and can be volatile.
To help solve the problem of accessing unstructured data, WebSphere
Information Integrator provides federated access to a wide variety of
unstructured data content stores. Where appropriate, content can be cached
locally to enhance performance. WebSphere Information Integrator also provides
tools that enable unstructured content to be joined with relational data (providing
of course that the unstructured content has suitable key fields). Used in this
manner, data federation allows a BI system and its underlying data warehouse to
provide a continuous stream of relevant content to the executive dashboard
without the need to store all of the content locally in the data warehouse.
Information asset management component
The BPM information assets are stored and accessed using database
management systems (DBMSs) such as IBM DB2. The performance, scalability,
availability, security, and extensibility features of DB2 have made it our
recommended cornerstone for the information asset management layer for
managing data, and running analytic and business functions against that data.
Accessing data in a relational DBMS can be done using queries defined using
the Structured Query Language (SQL), or using a graphical user interface (GUI)
that hides SQL syntax from the user.
The vast majority of DB2 applications today use a GUI to hide data access
details from business users. Application developers, however, need to learn a
language independent technique for accessing DB2 data. COBOL programmers,
for example, embed SQL statements in their applications and use preprocessors
to convert SQL statements into native library calls that interact with the DB2
server. C programmers currently have a wide range of database access options
available to them, including embedded SQL, the SQL Call Level Interface (CLI),
ODBC, ADO, OLE-DB, and ADO.Net.
Java™ programmers use JDBC and SQLj as direct database interfaces to DB2.
The JDBC interface is defined in Java 2 Standard Edition (J2SE™). Java 2
Enterprise Edition (J2EE) contains a set of language extensions for building
distributed Java applications. These extensions include Java Server Pages,
Servlets, and Enterprise Java Beans (EJBs), among others. These Java
Chapter 3. IBM BPM enablers 69
language extensions require an application server environment and use either
JDBC or SQLj as the access interface to DB2 databases.
3.1.3 Business Processes
The IBM BPM platform provides you the capability to design, deploy, integrate,
and manage business processes with support from the WebSphere Business
Integration (WBI) product set.
For the BPM environment, the WBI product set supplies real-time data about the
operation and performance of business processes. This data can be supplied to
the information analysis and delivery components (WebSphere II using
WebSphere MQ Workflow or WBI Server Foundation), or directly to BPM
monitoring capabilities (using, for example, WBI Monitor). But, how does WBI
Monitor do that?
WBI Monitor uses real-time data to generate KPI dashboards that can display the
ongoing performance of business processes. This data may can come from a
variety of data sources, including WebSphere MQ Workflow, WBI Message
Broker, and database products (IBM DB2). The monitoring capability of WBI
monitor adds real-time information captured from business processes to the
BPM environment.
The KPI information in WBI Monitor is delivered through two standard
dashboards:
Workflow dashboard. This dashboard enables users to monitor business
processes at run time, from anywhere, regardless of a user’s machine
configuration. The WBI workflow dashboard provides an operational view of
business processes, displaying run-time business metrics for currently
running process instances of each monitored process. The dashboard
provides an alert system that can be used to pinpoint problems and
bottlenecks in a running process. An example workflow dashboard is depicted
in Figure 3-7.
70 BPM Meets BI
Figure 3-7 Sample of WBI workflow dashboard
Business dashboard. This dashboard manages historical process-instance
performance data. An example is depicted in Figure 3-8. The information
contained within the business dashboard provides important decision making
data from a cost, time, and utilization perspective. The metrics you generate
can be imported back into WBI Workbench for continued analysis of the
process.
The WBI Server Foundation also contains capabilities that can be used to
construct IT systems that enable BPM. As examples, the WBI Server Foundation
supports key technologies and standards (BPEL and CEI), as well as providing
workflow capability through Process Web-clients. The metrics gathered from
running processes in WBI Server Foundation can be gathered and presented in
a Portal by using, as an example, an IBM Alphablox user interface to build
customizable management dashboards. More information on this topic is
provided in Chapter 4. “WebSphere: Enabling the solution integration” on
page 97.
Chapter 3. IBM BPM enablers 71
Figure 3-8 Sample of WBI business dashboard
3.1.4 Making Decisions
To achieve true business insight, a comprehensive business performance
management system should include a set of flexible business rules that allows
users to optimize business processes without needing additional modeling and
deployment cycles.
Business rules externalize business policies so they can be managed
independently of the application software. As marketing plans change, for
example, businesses can easily modify the business rules used to determine
customer discounts. As policies regarding customer credit are updated, the
business rules that govern customer approval and rejection based on credit
status and history can be modified accordingly. In short, business rules support
on demand business because they enable agility and responsiveness in
business processes.
Implementing business rules within the IBM BPM Platform allows not only IBM
software, but also customer-developed and IBM Business Partner applications,
to participate in a comprehensive rules management system.
We show the business rules architecture in Figure 3-9.
72 BPM Meets BI
Figure 3-9 BPM business rules architecture
The IBM BPM toolset includes the IBM WBI Modeler, an intuitive business
process modeling tool that enables business consultants and developers to
define business processes. The tool supports the simulation of modeled
processes to help ensure they meet business requirements. WBI Modeler has a
number of export formats for deploying business processes in external process
flow engines, including the Flow Definition Language (FDL) and the
industry-standard business process execution language (BPEL).
3.1.5 Event Infrastructure
The components of a business system usually employ different formats for the
information they collect about events. This makes it difficult for organizations to
handle events from disparate components and integrate them into a cohesive
and efficient management environment. If an IT group, for example, needs to
determine what made a business-critical e-business application fail, they may
need to understand 40 different event log formats. Root cause analysis of the
problem could require input and analysis by numerous system administrators,
spanning network, Web, and database environments.
To solve this problem, the IBM BPM Platform contains a common event
infrastructure (CEI) for the creation, transmission, persistence, and distribution of
a wide range of business, system, and network events. By providing a common
infrastructure for sharing common event information between different
Business
Modeler
Policy/Rules
Editor
Process
Choreography
Rules
Engine
BPEL
/ FDL
Rule
XML
Business
Monitoring

X

XMI Metadata
Interchange

IBM Rule
XML
Web Service /
J2EE

CEI Event Emitter
API

CEI Event Consumer
API
Interfaces
BPM Activities
Partner Components
WBI Foundation

IBM Components
Adapt
Model
Deploy
Run
Monitor
Analyze
Common Event Infrastructure
Chapter 3. IBM BPM enablers 73
applications, it becomes possible to correlate a business system outage to the IT
resource that caused the outage.
The CEI employs a consistent format for recording information about events. This
is known as the common base event (CBE) format. IBM has proposed CBE for
consideration as a new standard to the Organization for the Advancement of
Structured Information Standards (OASIS).
We depict the CEI architecture in Figure 3-10. The CEI architecture consists of
five layers of event interaction:
All event consumers share the common event infrastructure and event format,
and may submit events to the CEI.
Correlated CBE events are delivered to consumers.
The CEI is shared by event sources and consumers, and provides CBE
persistence, distribution, and access.
CBEs flow into the CEI.
Event sources submit CBE events, and other events types are converted to
the CBE format.
Figure 3-10 The common event infrastructure interaction model
Application
CEI
Layer 1
WBI/BPM
Applications
IT
Applications
Business Process applications
Business System Management
applications
CBE Events
Common Event Infrastructure (CEI)
Common Base Event (CBE)
WBI/Process
Applications
Other
Applications
Other
Applications
Standard
Properties
Context Data
Extended Data
Layer 2
Layer 3
Layer 4
Layer 5
e-business Infrastructure
IT Infrastructure
CEI format
definiton
74 BPM Meets BI
The CEI has several public interfaces that are exposed to external applications
and tools. These interfaces generally fall into one of the following categories:
CEI event submission interfaces allow applications to create and send events
to the management server. These interfaces enable customers, Business
Partners, and ISVs to create new applications and drive event services.
CEI event subscription interfaces allow applications to subscribe to particular
types of events as they arrive in real-time at the management server.
CEI event query interfaces allow applications to query historical events.
The CEI offers an event management system for supporting BPM applications.
The following are some of the capabilities of CEI for gathering information to
support BPM:
The ability to investigate a particular sequence of events. Organizations can
discover, for example, why a particular customer order took three weeks
longer than expected to get delivered.
The means to analyze past historical performance and provide an indication
of future behavior. The time, for example, that a particular supplier usually
takes to fulfill an order can be a good indication of how long the supplier will
take to deliver the next order for supplies.
The ability to maintain a historical record of business commitments. Some
businesses are legally required to keep an audit trail of the key decisions and
promises made by the business. Although CEI does not provide a
non-repudiation logging facility, it can still act as a central point where the
required data can be assembled and passed on to an external tamper-proof
log.
The ability to trigger action when an unexpected event occurs. This can be
used, for example, to alert the business that a particular customer order has
taken three weeks longer to deliver to the customer than it should have.
The ability to support business dashboards that show cross-application data.
IBM products that implement and support CEI include IBM Tivoli, WebSphere
Application Server, and WebSphere Business Integration Server.
3.1.6 Enabling IT to help the business
So far we have focused primarily on the handling and managing of business
events and business performance in a BPM system. It is also important, however,
that BPM support the underlying IT system so that both IT and business systems
are aligned with the business goals of the organization. This part of the IBM BPM
Platform is provided by products that enable Business Service Management.
Chapter 3. IBM BPM enablers 75
The development of IT systems management parallels that of business
management. As in the business area, IT products have evolved to first handle a
distributed environment, and then automate the management of this environment
to improve IT operational efficiency and productivity. With the development of
so-called management frameworks, IT is now able to create centralized
operational IT intelligence that can report on the status of all systems and
networks throughout the enterprise. This is somewhat analogous to a BI system
and its centralized enterprise data warehouse.
To achieve the strategic business goals of a company, however, IT needs tools to
manage its operations from a business perspective and to implement strategies
to deal with business change. This is the role of business service management
(BSM).
Aligning IT with business goals
BSM enables businesses to increase operational agility by aligning IT operations
and the resources they manage with business priorities. Managing IT resources
in this manner differs from the traditional method of managing resources by
technology type (servers, networks, and databases, for example). BSM helps IT
staff understand the components that enable the successful delivery of a service,
and, when problems occur, prioritize the issues to resolve first.
BSM is part of an overall BPM strategy that focuses on monitoring and managing
IT service delivery and associated business processes. By coordinating both the
business and IT events in an integrated framework, enterprises are better able to
make decisions that align IT and human resources utilization to meet business
priorities. In the IBM BPM Platform, this capability is provided by Tivoli Business
Systems Manager (Tivoli BSM).
Tivoli BSM enables an organization to view all of its IT resources and the existing
relationships they have with each other within the real-world environment. While
most organizations derive significant value from cataloging IT resources, the real
benefit of Tivoli BSM comes from its ability to create collections of resources that
represent business systems, such as key business processes and applications.
When an organization understands the IT resources required to deliver a specific
service, support a business process, and provide customer support or increase
overall revenue, then the organization has the information it needs to use
technology as a competitive tool.
Resource discovery using Tivoli BSM
Tivoli BSM uses two methods to discover resources and their relationships. The
first method is a set of discovery routines that scans periodically a specific
environment and identifies the resources within that environment. The discovery
76 BPM Meets BI
routines that ship with Tivoli BSM support IBM DB2 and CICS®, and IBM and IP
networks.
The second method for discovering resources involves environments that do not
have a set of explicit discovery routines. Resource discovery in this case is done
by listening for, or processing, incoming events that identify new resources that
have been added to the environment. This method enables applications, and
software and hardware components, to participate in BSM.
Once resources are discovered, they are mapped in the Tivoli BSM database to
a predefined model that describes resources and their hierarchical relationships.
A discovered database, for example, is placed as a child resource of the server
where the database was found running. The server, in turn, is contained in the
network location where the server was found. This mapping of architecture of
entire IT system of an organization creates a catalog, or resource pool, that
becomes the source for business system construction and management. This
catalog can be accessed and displayed using Tivoli BSM console.
Business systems are collections of resources within Tivoli BSM that are
grouped by the business, or the service, they deliver. Unlike IT resources, which
follow the rules predicated by the BSM object model, business systems can
contain any type of resource, and can be organized in any manner that fits user
needs. A business system can model discrete items (a service or an application),
collections of items (resources within a given geography), or even vertical areas
of responsibility.
Understanding the resources that are required to deliver a service can be a
challenge for many organizations. Automated methods provided by Tivoli BSM
for doing this include identifying key transactions, and analyzing the resources
that those transactions utilize during execution.
Figure 3-11 Integrating applications in BSM
2. Defining
Object Types
3. Understanding
Additional
Management
Capabilities
4. Understanding
Interested
Parties
5. Defining Event
Cataloging and
Workflow Analysis
6. Creating
Business
System Views
1. Identifying the
Components
Chapter 3. IBM BPM enablers 77
Monitoring business systems using Tivoli BSM
The second task of a BSM project is to ensure the underlying monitoring
infrastructure is in place to send the required events to Tivoli BSM. Any event that
indicates a situation that threatens, or attempts to threaten, the ability to deliver a
service successfully should be forwarded to Tivoli BSM. An example here is an
availability event that indicates that a resource has failed, been stopped, or is
simply unavailable. If a given level of performance is needed for successful
delivery of the service, then events that indicate performance problems also
need to be forwarded. The recorded events should cover the complete
end-to-end infrastructure that supports a particular service, including servers,
software components, transaction processors, middleware, databases, and the
network.
In addition to availability and performance events, other types of events that can
be forwarded to Tivoli BSM include those that track and monitor business
performance. The Tivoli BSM can associate these events with the resources that
generated them. The events may not require Tivoli BSM to notify users of a
problem, but may be needed for adding context to a report in terms of severity
(low, medium, high) based not only on the importance, but also the frequency of
the event.
3.1.7 Bringing it all together
Any one of the various sources of BI KPI data can provide value by supplying
information to a business user dashboard or a balanced scorecard application.
Integrating several, or all of these sources, into role-based business user work
spaces provides a complete view of the entire enterprise. This integration fulfills
the promise of BPM by providing the right information to the right users for
managing the performance of the entire enterprise.
The key BI functionality for BPM is enabled by WebSphere Information Integrator.
This product supports the federation of multiple data sources for use by a single
application, KPI, or portlet. Using this technology, businesses can combine
performance information from across the enterprise to create KPIs that give a
broader view of business performance than those derived from a single source.
The relationship of information and BI to other capabilities in the IBM BPM
Platform is illustrated in Figure 3-12. This provides access to information
managed by a data warehouse, current business transaction data, and real-time
event sources. Information in the data warehouse (for example, business process
metrics for trend analysis) can be analyzed and integrated with real-time event
data, process-specific event logs, message queue data, and system IT
warehouse information, and presented to business managers in a role-based
dashboard.
78 BPM Meets BI
Figure 3-12 IBM BPM core capabilities
Another way the various aspects of BPM tie together is at the workstation
interface using the workplace. Dashboards supporting various BPM functions
allow users to interact directly with several components of the BPM framework as
shown in Figure 3-13.
Figure 3-13 Dashboard functional architecture
Analytics/reporting
Business Partner
Tools
Tivoli Business Systems
Manager (TBSM)
Tivoli Data
Warehouse
WebSphere Business
Integration
Tivoli
Enterprise
Console
WebSphere Portal
WebSphere Business
Integration Server
Tivoli Service Level
Advisor
DB2 Data
Warehouse
and
OLAP Server
Monitor
Modeler
Integrated Dashboard
Business Service Mgmt Process Mgmt Information Mgmt
WebSphere
Information Integrator
Event log
BPM
Models;
Business
Vocabulary
Business
Operations
Monitoring
Business
Operations
Monitoring
Business
Systems
Monitoring
Business
Systems
Monitoring
BPM
Information
Management
BPM
Information
Management
Notification
Service
Rules,
Workflow
***
Adaptive Actions
Adaptive Actions
Common Event Infrastructure
Portlets Dashboard Dashboard
Process
Integration
User
Interaction
Applications
(New & Legacy)
B2B
Interaction
System
Resources
e- business Solution Infrastructure
Network
Resources
• Business Users IT Users
KPI Mgmt
Query Situations OLAP Alerts Actions Reports KPIs Events Change Policies Access
Workplaces for Business Performance Management
M
o
d
e
l
i
n
g
T
o
o
l
s
Partner
Contributions
Chapter 3. IBM BPM enablers 79
3.2 Web services
The BPM environment and IBM BPM Platform involve many components, tools,
services, and applications interconnected in a distributed computing system.
Although organizations have been building distributed systems and applications
for a number of years, many of them are now moving towards the use of a
service-oriented architecture (SOA) as a more effective approach for developing
and maintaining a distributed environment. The primary technology used today to
implement a SOA is Web services.
The word Web in Web Services means that all operations are performed using
the technology and infrastructure of the World Wide Web. The word service
represents an activity or processing performed on behalf of a requestor, a person
or application. Web services have existed ever since the Web was invented. The
ability of a Web browser to access e-mail, or to order a product on the Internet,
are examples of Web services. More recently, however, Web services
increasingly make use of XML-based protocols and standards, and it is better to
think in terms of XML Web services than a Web Browser. In this redbook, for
simplicity, we use the term Web services to signify XML Web services.
Web services enable any form of distributed processing to be performed using a
set of standard Web- and XML-based protocols and technologies. For more
detailed information on Web services and related topics, see the work effort by
the World Wide Web Consortium at:http://www.w3c.org
In theory, the only requirements for implementing a Web service are:
A technique to format service requests and responses
A way to describe the service
A method to discover the existence of the service
The ability to transmit requests and responses to and from services across a
network
The main technologies used to implement these requirements in Web services
are XML (format), WSDL (describe), UDDI (discover), and SOAP (transmit).
There are, however, many more capabilities (authentication, security, and
transaction processing, for example) required to make Web services viable in the
enterprise, and there are numerous protocols in development to provide these
capabilities.
It is important to emphasize that one key characteristic of Web services is that
they are platform neutral and vendor independent. They are also somewhat
easier to understand and implement than earlier distributed processing efforts
such as Common Object Request Broker Architecture (CORBA). Of course, Web
80 BPM Meets BI
services will still need to be implemented in vendor specific environments, and
this is the focus of facilities such as IBM Web Services.
3.2.1 The promise of Web services
Web services technology is essentially a new programming paradigm to aid in
the development and deployment of loosely-coupled applications both within and
across enterprises. In the past, developers have tended to develop most of their
applications from the ground up. The term code reuse was used, but this was
often not put into practice because developers usually only trust the code they
develop. As software development has progressed as a discipline, and as
programming languages have also advanced, the ability to reuse application
code has increased. The Java programming language, for example, has many
built-in class libraries that developers use.
As applications grow, they need to be able to execute in a distributed
environment. Distributed applications provide unlimited scalability and other
benefits. Defining an interface for distributed applications has been a challenge
over the years. Language-independent technologies such as CORBA (Common
Object Request Broker Architecture) provide a complicated and powerful
programming model. Other distributed technologies work well within a single
language environment, such as Java RMI (Remote Method Invocation) and
Microsoft's DCOM (Distributed Common Object Model), but are not useful in a
heterogeneous systems environment.
In contrast, Web services provide a simple-to-understand interface between the
provider and consumer of application resources using a Web Service Description
Language (WSDL). Web services also provide the following technologies to help
simplify the implementation of distributed applications:
Application interface discovery using Universal Description, Discovery, and
Integration (UDDI)
Application interface description, again using UDDI
A standard message format using Simple Object Access Protocol (SOAP),
which is being developed as the XML Protocol specification by W3C
3.2.2 Web services architecture
We define the Web services architecture in several layers. Figure 3-14 illustrates
these layers.
Chapter 3. IBM BPM enablers 81
Figure 3-14 Web services layered architecture
The underpinnings of the Web services architecture are WSDL and SOAP.
WSDL is an XML vocabulary used to describe the interface of a Web service, its
protocol binding and encoding, and the endpoint of the service. SOAP is a
lightweight protocol for the exchange of information in a distributed environment,
and is used to access a Web service. It is transport-protocol independent. SOAP
messages can be transported over HTTP (HyperText Transfer Protocol), for
example, but other protocols are also supported. Examples include:
SOAP over WebSphere MQ (Message Queuing)
RMI (Remote Method Invocation) over IIOP (Internet Inter-ORB [Object
Request Broker] Protocol)
At present, the current SOAP standard only defines bindings for HTTP. SOAP is
rightfully seen as the base for Web application-to-application interoperability. The
fast availability of SOAP implementations, combined with wide industry backing,
has contributed to its quick adoption.
SOAP employs a XML-based RPC (Remote Procedure Call) mechanism with a
request/response message-exchange pattern. It is used by a service requestor
to send a request envelope to a service provider. The SOAP request envelope
contains either an RPC method call or a structured XML document. Input and
output parameters, and structured XML documents are described in XML
schema. The service provider acts on a request and then sends back a SOAP
response envelope.
SOAP
Routing
Transactions
Context
Attachments
Security
Reliability
Interface
Service
Composition
Agreements
XML Schema Inspection
Directory
XMLP
WSDL
BPEL
UDDI
Protocols Discovery Descriptions
BPEL = Business Process Execution Language
XMLP = XML Protocol
Quality of Service
82 BPM Meets BI
The existence of a Web service can be published and advertised in a public
UDDI registry. Publishing Web services in a public registry allows client
applications to discover and dynamically bind to Web services. UDDI helps
distributed application developers solve the maintenance problem caused by
constantly changing application interfaces. Developers can use internal private
registries, and public UDDI registries hosted on the Internet by companies such
as IBM and Microsoft, to publicize their application interfaces (as specified by
WSDL) and to discover other Web services. When a WSDL interface changes, a
developer can republish the new interface to the registry, and subsequent access
to the Web service will bind dynamically to the new interface.
3.2.3 IBM Web services
Two key IBM products for supporting Web services are WebSphere Studio and
WebSphere Application Server.
WebSphere Studio contains a set of development tools for creating and
maintaining Java applications that use Web services. WebSphere Studio is
based on an open development framework known as Eclipse. For more details,
see:http://www.eclipse.org
WebSphere Studio provides tools for creating WSDL interfaces to Java
applications and DB2 data. You can publish Web services defined using
WebSphere Studio to a UDDI registry directly from the WebSphere Studio
environment. WebSphere Studio also provides a UDDI browser.
IBM WebSphere Application Server is a J2EE-compliant Java Web Application
Server. It is an ideal platform for hosting DB2 Web service provider applications.
WebSphere Application Server includes the Apache SOAP server. For details,
see:http://xml.apache.org/soap/
3.2.4 Using DB2 as a Web services provider and consumer
IBM DB2 can participate in a Web services environment as a server provider or a
service consumer. This is shown in Figure 3-15. The DB2 Web service
infrastructure and tools shown in the figure are supplied with DB2 UDB Version 8.
Chapter 3. IBM BPM enablers 83
Figure 3-15 DB2 in a Web services environment
DB2 as a Web services provider
The left-hand side of Figure 3-15 outlines how DB2 acts as a service provider.
Applications access DB2 Web services using a WSDL interface that is created
using the DB2 Web services Object Runtime Framework (WORF).
The WSDL interface to DB2 consists of an XML file (a Document Access
Definition Extension, or DADx file) that defines a set of one or more DB2-related
operations. An operation can invoke a DB2 stored procedure, retrieve or store an
XML document, or run an SQL CREATE, SELECT, UPDATE, or DELETE
statement. Data returned from an operation may be formatted as an XML
document or a Java object.
In Example 3-1, we define a DADx operation called listMeetings for retrieving all
of the data from the DB2 calendar table for a specific date.
Example 3-1 listMeetings Web service

List meetings on calendar on a certain date.

SELECT * FROM CALENDAR
Important: It is important to note that each DADx operation is currently limited
to a single SQL statement and executes within a single unit of work.
H
T
T
P
/
G
E
T
WebSphere
Application Server
S
Q
L
SQL
Applications
DB2 providing Web Services DB2 consumes Web Services data
DB2
H
T
T
P
/
S
O
A
P
H
T
T
P
/
S
O
A
P
DB2 Web Service
Provider
Web Service
UDFs
Stored Procedures
Tables
XML Extender
Soap
Client
Web
Browser
Service
Providers
84 BPM Meets BI
WHERE DATE = :date

DB2 WORF automatically generates the WSDL interfaces, as depicted in
Figure 3-16, for each of the defined operations in the DADx XML file. It also
creates a Web services application for testing purposes. The test application may
use a plain HTTP or SOAP binding. The HTTP binding is useful for testing a DB2
Web service directly from a Web browser. Web services clients can use the
SOAP binding to create distributed applications. After testing and deploying the
DB2 Web service, any Web services client can start using it.
Figure 3-16 Development scenario for DB2 Web service provider applications
You can deploy the DB2 DADx XML file and its run-time environment (Apache
SOAP) on a Java Web application server such as Apache/Jakarta Tomcat or
WebSphere Application Server. Using WebSphere Application Server provides
additional benefits, for example, pooled database connections and centralized
administration. You can also deploy WebSphere Application Server using
horizontal and vertical scaling techniques to provide fault-tolerance and
high-transaction rates required for a popular DB2 Web service.
DB2 WS
Provider
WebSphere
Application Server WS client
5) SOAP
-Tables
-Stored
Procedures
-XML Extender
6)SQL
DB2
UDDI
registry
2) Publish WSDL
3) Find
WSDL
DB
programmer
or DBA
1) create
4) Develop
Client
Web
App
SELECT *
FROM
CALENDAR
DADX files
SELECT *
FROM
CALENDAR
Chapter 3. IBM BPM enablers 85
DB2 as a Web services consumer
The right-hand side of Figure 3-15 shows how DB2 uses user-defined functions
(UDFs) to operate as a Web services consumer. These SQL functions provide
the necessary language hooks for using Web services, and make it easier for
developers to create applications that consume and integrate Web services data.
We depict this in Figure 3-17. Another benefit of using SQL to access a Web
service is that the retrieved data can be manipulated within the SQL statement
before it is returned to the client application. DB2 Web services are consumed
via SQL statements, and so it is simple to test access to a Web service using a
tool such as the DB command line processor (CLP).
Figure 3-17 Development scenario for DB2 Web service provider applications
The WSDL for the listMeetings DB2 Web service provider shown in Example 3-1
could, for example, be defined as a DB2 UDF and then invoked from an SQL
statement in a client application. A non-SQL Web service client could run the
same listMeetings Web service without using a DB2 UDF, but this requires more
programming effort. An existing WSDL Web service interface can be converted
to a DB2 UDF using the plug-in provided by WebSphere Studio.
A Java application server is not required to access a Web service using DB2
SQL. During SQL statement execution, a direct connection with the Web service
provider is established, and the response is returned as either a relational table
or a scalar value.
Recommendations for using DB2 Web services
When you expose DB2 data to Web services clients using DB2 WORF, consider
embedding the data access logic in a DB2 stored procedure. These procedures
provide a very powerful technique for creating an abstraction layer for DB2 data
SQL
Applications
DB2
DB programmer
or DBA
1) Retrieve WSDL
WSDL
2) create
SQL
4) request/
response
3) execute
HTTP/SOAP
Web Service UDF
Service
Providers
86 BPM Meets BI
access. They can be created in various programming languages, including Java
and the standard SQL procedure language.
To make the development job easier, also consider using DB2 Development
Center, which is provided in DB2 UDB Version 8 as a replacement for DB2
Stored Procedure builder. You can use DB2 Development Center to create and
test stored procedures and to create other SQL extensions for DB2.
3.2.5 WebSphere Information Integrator and Web services
In this section we discuss the benefits of WebSphere II compared with DB2
UDFs, and look at how WebSphere II works in a Web services environment.
DB2 user-defined functions (UDFs) enable SQL-based applications to access
one or more remote data sources that have been defined as Web services. This
capability provides a basic level of data federation. Its limited support for data
source modeling and transaction integrity, however, restrict the use of DB2 UDFs
in more sophisticated data federation projects. For these types of projects, you
should use WebSphere II instead.
A WebSphere II federated system uses a wrapper to access and interact with
remote content. You can define this content as a Web service, relational
database, or XML file, as examples. A wrapper maps the remote content to a
table-like object known as a nickname. You can then use one or more of these
nicknames in a DB2 (federated) view and access and manipulate the nickname
like any other kind of relational data. Wrappers are more powerful than UDFs,
and are typically better at exploiting the more advanced features of the remote
content. Their definition, however, requires a more skilled developer.
If a server architecture is central to the development and deployment of an
application, then the WebSphere II wrapper architecture is likely to be the
appropriate solution to use. Suppose, for example, the goal of an application is to
integrate a set of Lotus Notes and/or BPM data sources. It is possible to write a
DB2 UDF to access multiple data sources, but the burden is on the UDF
developer to find the appropriate information to identify the data sources as
arguments, manage connections to the data sources, and use a scratchpad to
store any state information. Furthermore, the information in the scratchpad is
only valid for a single SQL statement, and so UDF invocations from separate
statements will each require their own connections and scratchpads. For this
type of integration, the multi-server and connection management support offered
by the wrapper architecture is better for handling access to multiple remote
content stores. On the other hand, an application that retrieves the temperature
from an on-line thermometer, for example, does not require the notion of a
server, and the wrapper solution may be overkill in this case, and the use of a
DB2 UDF may be more appropriate.
Chapter 3. IBM BPM enablers 87
The DB2 view mechanism when used with underlying WebSphere II wrappers
and nicknames provides a powerful mechanism for combining data from remote
sources and for shielding applications from the details of different content store
formats. The SQL-based interface to federated data sources has the additional
benefit of allowing database administration tools, development tools, and
object-oriented components to work transparently with remote heterogeneous
data.
A WebSphere II federated system
A WebSphere II federated system is a special type of distributed database
management system (DBMS). It consists of a DB2 instance that operates as a
federated database server, a federated database, one or more content sources,
and client applications that access the federated database and its underlying
content sources.
With a federated system, applications can send requests to multiple content
sources by issuing a single SQL statement against a federated view of the data.
An application can, for example, join information from a DB2 UDB table, a
third-party relational DBMS table, a Web service, and an XML tagged file in a
single SQL statement. Figure 3-18 shows the components of a WebSphere II
federated system.
Figure 3-18 WebSphere Information Integrator
Biological
Data and
Algorithms
Text
Sybase
Informix
SQL Server
Oracle
Teradata
WebSphere MQ
ODBC
IBM
Extended
Search
Excel
…
WWW, email,… XML
DB2 UDB
Software AG
Adabas
VSAM
CA-IDMS
CA-Datacom
IMS
DB2 II Classic
Federation
DB2 Family
Web
Services
WebSphere II
Classic
Federation
SQL, SQL/XML
Federation Server
Wrappers and Functions
O
D
B
C
Integrated SQL View
WebSphere
Information Integrator
88 BPM Meets BI
The power of WebSphere II lies in its ability to:
Join data from local tables and remote data sources, as if all the data is stored
locally in the federated database.
Update data in relational data sources, as if the data is stored in the federated
database.
Replicate data to and from relational data sources.
Take advantage of the processing strengths of the remote data source by
sending requests to the content source for processing.
Compensate for SQL limitations of the remote data source by processing
parts of a distributed request on the federated server.
The federated server
The DB2 instance in a WebSphere II system is called a federated server
because it responds to requests from multiple client applications for the
processing of federated data. Any number of DB2 instances can be configured to
function as federated servers. An existing DB2 instance can be used as a
federated server, or a new one can be created specifically for the federated
system.
A federated server may send parts of the requests it receives to the data sources
for processing. The DB2 instance that manages the federated system is still
referred to as the federated server, even though it acts as a client when it pushes
down a request to a remote data source.
Two key features of the federated server distinguish it from other application
servers:
You can configure a federated server to receive requests that might be
partially or entirely intended for remote data sources. In these cases, the
server distributes these requests to the remote data sources.
Like other application servers, a federated server uses DRDA®
communication protocols (over TCP/IP) to communicate with DB2 family
instances. However, unlike other application servers, a federated server uses
the native interface of a remote data source to access data. A federated
server, for example, uses the Sybase Open Client to access Sybase data and
a Microsoft SQL Server ODBC Driver to access Microsoft SQL Server data.
The federated database
To users and applications, data sources accessed using WebSphere II appear to
belong to a single federated DB2 database managed by a federated database
server. A system catalog documents the characteristics of the data sources that
comprise the federated database. The federated server consults the information
Chapter 3. IBM BPM enablers 89
in this system catalog and associated data source wrappers to determine the
best plan for processing SQL statements.
The federated server processes SQL statements as though the data sources
were regular relational tables or views. As a result:
The federated server can join relational data with data in non-relational
formats. This is true even when the data sources use different SQL dialects,
or do not support SQL at all.
The characteristics of the federated server take precedence when there are
differences between the characteristics of the federated server and those of
the data sources.
– Suppose, for example, the code page used by the federated server is
different from that used by the data source. In this case, when character
data from the data source is presented to a client application, it is
transformed using the code page of the federated server.
– Another example is when the collating sequence used by the federated
server is different from that used by the data source. In this case, any sort
operations on character data are performed by the federated server and
not at the data source.
Wrappers and nicknames
A wrapper is the mechanism that the federated server uses to communicate with
and retrieve data from a data source. To implement a wrapper, the federated
server uses routines stored in a library called a wrapper module. These routines
allow the federated server to perform operations such as connecting to a data
source and retrieving data from it. The federated server administrator uses the
CREATE WRAPPER statement to register a wrapper for each data source type
that is to be included in the federated system. This is depicted in Figure 3-19.
Suppose, for example, that you want to access three DB2 for z/OS tables, one
DB2 for iSeries™ table, and two Informix® tables. You need to create one DRDA
wrapper for the DB2 data objects, and one Informix wrapper for the Informix data
objects. Once these wrappers are registered in the federated database they can
be used to access other objects from those data sources. The DRDA wrapper, for
example, can be used with all DB2 family data source objects.
90 BPM Meets BI
Figure 3-19 DB2 in a consumer environment using the Web services wrapper
A wrapper performs many tasks, as in the following:
Connecting to a data source. The standard API of the data source is used by
a wrapper to connect to it.
Submitting queries to a data source. The user query is submitted in SQL for
SQL data sources, and is translated into native language statements, or API
calls, for non-SQL data sources.
Receiving results from the data source. The standard API of the data source
is used by a wrapper to receive results.
Responding to federated server requests for the data-type mappings of a data
source. Each wrapper contains default data-type mappings. For relational
wrappers, data-type mappings created by the administrator override those
default mappings. User-defined data-type mappings are stored in the system
catalog.
Responding to federated server queries about the default function mappings
of a data source. A wrapper contains information for determining if DB2
functions need to be mapped to the functions of the data source, and, if so,
how the mapping is to be done. This information is used by the SQL compiler
to determine if the data source is able to perform certain query operations.
For relational wrappers, the function mappings created by the administrator
override the default function mappings. User-defined function mappings are
stored in the system catalog.
A nickname is an identifier that is used in an SQL query to refer to an object at a
data source. These objects, for example, can be tables, views, synonyms,
table-structured files, search algorithms, business objects, Web services
Service
Providers
S
Q
L
SQL
Applications
DB2
H
T
T
P
/
S
O
A
P
Stored Procedures
Tables
XML Extender
Web Service
Wrapper
/Nicknames
DB2 providing Web Services DB2 consumes Web Services data
using Web service wrapper
H
T
T
P
/
G
E
T
WebSphere
H
T
T
P
/
S
O
A
P
DB2 Web Service
Provider
Soap
Client
Web
Browser
Chapter 3. IBM BPM enablers 91
operations, and so forth. Nicknames are registered to the federated server using
the CREATE NICKNAME statement. Each nickname is associated with a specific
wrapper.
Web services wrapper for a Web services provider
A Web services provider can act as a data source for WebSphere II. The provider
is accessed by a federated server using a Web services wrapper.
To configure the federated server to access a Web services provider, the
administrator must provide the federated server with information about the Web
services and operations that are to be accessed. The steps required to add a
Web services provider to a federated server are as follows:
1. Use the CREATE WRAPPER statement to register the appropriate Web
services wrapper.
2. Use the CREATE SERVER statement to register a server definition for each
Web service to be accessed.
3. Optional: Create any required user mappings.
4. Use the CREATE NICKNAME statement to register a nickname for each Web
service operation to be used.
5. Optional: Create a federated view of the Web services nicknames.
We briefly review the main steps below, but for detailed information on how to
define and create a Web services wrapper, please consult the data source
configuration guide for WebSphere II.
You can configure the federated server to access a Web services provider by
using the DB2 Control Center or the DB2 command line. The DB2 Control Center
includes a wizard to guide you through the steps required to configure the
federated server. This process is depicted in Figure 3-20, which shows a
development scenario using a WebSphere II Web services wrapper. This
process enables applications to access Web services via the SQL API.
92 BPM Meets BI
Figure 3-20 Adding Web services to a WebSphere II federated server
Registering a Web services wrapper
To register a wrapper, you issue a CREATE WRAPPER statement with the name
of the wrapper and the name of the wrapper library file. To register, for example, a
wrapper with the name my_wrapper on the federated server that uses the AIX®
operating system, issue the following statement:
CREATE WRAPPER my_wrapper LIBRARY 'libdb2ws.a';
The name of the wrapper library file that you specify depends on the operating
system of the federated server.
Registering the server definition
After you register the wrapper, you must register a corresponding server for each
Web service that you wish to access. To register, for example, a Web service
called my_server, issue the following statement:
CREATE SERVER my_server WRAPPER my_wrapper;
Registering nicknames for Web services
You create one nickname for each Web service operation you use. Define Web
service operations in a XML WSDL file. Create nicknames quickly using the DB2
Control Center Discover tool. You specify the URL of the WSDL file for which
nicknames you plan to create, and the Discover tool processes the file, and
creates the appropriate nicknames.
WSDL
WebSphere
Information
Integrator
1) Retrieve WSDL
2) create
SQL
Applications
4) request/
response
3) execute
Web Service
Wrapper
And Nicknames
SQL
HTTP/SOAP
DB programmer
or DBA
Service
Providers
Chapter 3. IBM BPM enablers 93
Example 3-2 shows a nickname ZIPTEMP that uses a Web service operation
called getTemp. The Web services operation reads a ZIP code and returns the
temperature at that location.
Example 3-2 ZIPTEMP nickname for the getTemp Web service operation
CREATE NICKNAME ZIPTEMP (
ZIPCODE VARCHAR (48) OPTIONS(TEMPLATE '&column'),
RETURN VARCHAR (48) OPTIONS(XPATH './return/text()')
)
FOR SERVER "EHPWSSERV"
OPTIONS(URL 'http://services.xmethods.net:80/soap/servlet/rpcrouter',
SOAPACTION ' ' ,
TEMPLATE '

&zipcode[1,1]

',
XPATH '/soapenv:Envelope/soapenv:Body/*' ,
NAMESPACES '
ns1="http://www.xmethods.net/sd/TemperatureService.wsdl",
ns2="urn:xmethods-Temperature" ,
soapenv="http://schemas.xmlsoap.org/soap/envelope/"');
Each nickname contains column definitions for the message values that sent to,
and received from, the Web service operation. The TEMPLATE option defines a
input column (ZIPCODE in the example), and the XPATH option defines an
output column (RETURN in the example). The nickname ZIPTEMP can be
referenced in an SQL query like any other data object:
SELECT * FROM ZIPTEMP WHERE ZIPCODE='97520';
The Web services WSDL file
Now that we understand the WebSphere II part of the definition, let’s take a look
at how the Web services wrapper interacts with a Web services provider, and at
the WSDL file that defines the operations that are supported by a provider.
A WebSphere II Web services wrapper communicates with a Web services
provider using SOAP messages. A SOAP message is a XML document that
defines the layout and contents of the message. A message can contain
document-oriented information, or a process-oriented remote procedure call
(RPC). The SOAP message is embedded in an envelope consisting of a
mandatory body containing the message itself, and an optional header. The
header has information about things such as encryption and authentication.
94 BPM Meets BI
The operations supported by a Web service are defined in a WSDL XML
document. This document defines a Web service operation in terms of the
messages that it sends and receives. A WSDL document can contain one or
more Web service operations.
The input and output messages of a Web services operation are mapped to the
columns defined in the associated WebSphere II nickname. The input messages
are mapped using the nickname TEMPLATE specification, and the output
messages are mapped to the hierarchical structure defined using the XPATH
specification of the nickname. You can define a separate output hierarchy for
each Web service operation in the WSDL document.
Example 3-3 shows a WSDL definition for the TemperatureService Web service,
which supports a single operation called getTemp. The input value is described
by the zipcode column. The output value is described by the return column. In
the WSDL document, these columns are identified in a message element, which
represents the logical definition of the data that is sent to and from the Web
service provider. If more explanation of the information in the message element
is needed, then the WSDL document can also contain a type element, which can
refer to predefined types that are based on the XML schema specifications, or
types that are defined by a user.
The TemperatureService example also shows how the WSDL binds the Web
service operation to use the SOAP protocol over HTTP, and how it identifies the
network address where the Web service is located.
Example 3-3 The TemperatureService Web service