Description
The goal of Business Activity Monitoring is to provide management with immediate awareness of business events across the enterprise as well as changing business conditions, so that appropriate and timely decisions can be made.
ENSEMBLE WHI TE PAPER
ENABLING THE REAL-TIME ENTERPRISE
BUSINESS ACTIVITY MONITORING WITH ENSEMBLE
InterSystems Corporation Page 1
ENABLING THE REAL-TIME ENTERPRISE
Executive Summary
Business Activity Monitoring (BAM) enhances
business intelligence by detecting events within
enterprise systems, filtering them for items of
interest, and presenting that information to an
executive in the form of a visual “dashboard” on
a computer screen.
Ensemble provides advanced Business Activity
Monitoring solutions, built atop a comprehensive
and extremely fast platform for application inte-
gration, composite application development, data
coordination, and business process/workflow
management.
Ensemble is designed around the precept that
successful business integration solutions are driv-
en by metadata. Metadata definitions represent,
and bring order to, all integration touchpoints,
including data, services, and business processes –
as well as the messages and data flows that are
the nerve impulses of a system. Thus Ensemble
takes a data management-centric approach to
integration.
That approach is realized with the single, shared
metadata repository and ultra-scalable message
warehouse that are at the core of Ensemble’s
architecture. Its data management capabilities and
data-centric approach, combined with a strong
Service Oriented Architecture, make data easily
available to any application that needs it.
Ensemble’s Business Activity Monitoring capabili-
ties exploit all these characteristics to enable rapid
construction of advanced and comprehensive set
of BAM solutions. Thanks to its unique fusion of
technologies, Ensemble taps a broad source of
enterprise business data, provides the tools that
turn business data into business intelligence, sup-
ports sophisticated analytics to spot trends and
exceptions, and enables automated responses to
specified conditions. And InterSystems’ proven
technology innovations enable these capabilities
to operate in real time, with the highest levels of
reliability and scalability.
1
•
INTRODUCTION
The goal of Business Activity Monitoring is to provide
management with immediate awareness of business
events across the enterprise as well as changing business
conditions, so that appropriate and timely decisions can
be made. By providing executives with this kind of real-
time information, BAM solutions reduce costs and
speed the execution of business strategies.
Ensemble’s Business Activity Monitoring capabilities
rely on its foundation of enabling technology. The
Ensemble technical enablers exploited by BAM include:
I
Full-Spectrum Integration and Development –
an environment that supports rapid development
and integration of: composite applications,
custom adapters, and business process orchestra-
tions – in support of BAM solutions
I
Universal Service Architecture – an advanced
and unique abstraction technology that enables
consistent and efficient object representation of
different programming models and data formats
for rapid access to application functions and data
I
Persistent Object Engine – a high-performance,
high-availability distributed object database, mes-
sage warehouse, and metadata repository that pro-
vides ultra-fast retrieval and manipulation of mes-
sages and events
I
Customizable Management Environment –
a comprehensive set of tailorable and extensible
monitoring and management facilities that are
tightly integrated with the messaging engine and
message warehouse, as well as with business
process management and development tools
Ensemble’s technology provides three major advantages
for BAM solutions. First, it provides a broad informa-
tion base. Ensemble’s Universal Service Architecture
and Persistent Object Engine allow the system to per-
sistently store all messages and events that pass through
the system, and to make them all accessible in the same
format and by way of the same mechanisms.
Second, Ensemble provides the ultra-high performance
necessary for the true “Real-Time Enterprise” (RTE).
Ensemble’s high-performance Persistent Object Engine
allows a great deal of complex processing to be per-
formed on business data, events, and messages in real
time, in order to transform raw data into business intel-
Page 2 InterSystems Corporation
ENABLING THE REAL-TIME ENTERPRISE
ligence. So Ensemble not only encompasses a broad
set of information from underlying applications and
business processes – it also analyzes and correlates this
information on-the-fly.
Third, Ensemble enables real-time reaction to busi-
ness intelligence and events. In addition to raising
dashboard alerts so that users can take action,
Ensemble’s BAM solutions can automatically trigger
sophisticated, programmatic actions in response to
the business intelligence it develops. In this way,
Ensemble’s BAM implementation enables an adaptive
closed-loop system for business process management,
making it possible for business processes to react
dynamically to changes in the business environment.
Ensemble’s Business Activity Monitoring builds upon,
integrates with, and exploits all of Ensemble’s
advanced development, management, and integration
capabilities, by fusing together previously
autonomous technologies in support of rapid and
comprehensive integration.
Thus Ensemble fuses business intelligence technology
with data transformation and routing technology. It
fuses event-driven architectures with service-oriented
architectures. It fuses business application develop-
ment with business process modeling and orchestra-
tion. It fuses all these things to provide and support
Business Activity Monitoring solutions. And it
achieves this with a single development environment,
a single management environment, and a single
architecture.
This unique fusion and comprehensive approach
results in extremely fast integration and composite
application development, as well as support for the
broadest set of application integration approaches and
solutions, including data coordination, business
process and workflow management, composite appli-
cation development, and, finally, Business Activity
Monitoring.
Figure 1: Ensemble, The Next Generation
Integration Platform
Speed is a common theme across all aspects of
Ensemble: speed of development, of integration, of
system performance, and of time-to-solution. Faster
reaction times result in lower costs, better service,
and higher customer satisfaction for any business.
But for some industries – for example, those that
deal in perishable information or goods – it is an
enormous competitive advantage.
Composite Composite
Applications Applications
Data Data
Coordination Coordination
Business Process Business Process
/ Work Flow / Work Flow
Business Activity Business Activity
Monitoring Monitoring
Ensemble Ensemble
Technology Technology Application Application Data Data
2
•
ENSEMBLE’S BAM ARCHITECTURE
OVERVIEW
Ensemble’s BAM architecture is implemented in
three logical layers:
I
Event delivery and display layer
I
Event processing and filtering layer
I
Event absorption layer
Event Delivery and Display
The delivery and display layer is responsible for
notifying recipients about an event once the nature
of that event (e.g., impact and severity) has been
analyzed and evaluated. Notification can mean both
the presentation of information to an executive by
way of a dashboard, or the triggering of an automat-
ed process.
you are building a dashboard widget, a complex
adapter, a business process definition, or a complete
business activity monitoring application.
As we have seen, BAM solutions exploit all aspects of
Ensemble’s underlying capabilities. Therefore, a BAM
development environment must be able to deal with
difficult problems at many different levels, including
the creation of complex adapters, the modeling/defi-
nition of both simple and complex business process-
es, and the development of full-blown composite
applications.
This unified environment has many associated advan-
tages, including greatly reduced complexity and train-
ing costs and greatly increased speed of development.
Full spectrum integration and development even pro-
vides insurance against vendor lock-in, because every-
thing in Ensemble is exposed equally well to both
Ensemble’s tools and to third-party tools of every
kind.
Because Ensemble abstracts system components and
represents them in a consistent form, and because
these representations can be projected into code or
other standards-based formats that developers can
take forward and embellish, Ensemble gives you bi-
directional interoperability between multiple different
development and business process definition tools.
Ensemble provides great flexibility in choosing a path
to arrive at a BAM solution, which can then be
deployed in Ensemble's high-performance execution
environment.
InterSystems Corporation Page 3
ENABLING THE REAL-TIME ENTERPRISE
A dashboard is a specialized web page that uses
meters to provide a real-time, graphical display of
measured values, as shown in Figure 2. The measured
value is associated with a specific property within a
Business Metric Service class.
Ensemble includes a complete set of meter styles,
including speedometers, odometers, fuel gauges, light
bars, traffic lights, indicator lights, line charts, and
bar charts – and it supports the development of cus-
tomized meters. Note that Ensemble dashboards are
implemented using standard HTML and SVG
(Structured Vector Graphics), so there are no propri-
etary ActiveX or Java components to deal with.
Thus Ensemble’s delivery and display capabilities pro-
vide business agility and flexibility. While many BAM
approaches give managers insight into transactions
and operations, they lack the ability to close the
process feedback loop, and to effect dynamic changes
to the processes that carry out business strategy. In
this way, Ensemble can notify recipients of an excep-
tion, and continuously feed information back into the
relevant business processes, causing them to react
dynamically to changing events. It is this adaptive
closed-loop system that turns your organization into
a real-time enterprise.
Now let us briefly examine how Ensemble’s develop-
ment environment aids the creation of BAM solu-
tions. Full Spectrum Integration and Development
simply means that you use the same “diagrammatic”
or GUI-based environment – The Studio – whether
Figure 2: Ensemble Dashboard and Meters
The delivery and display layer exploits all of
Ensemble’s notification and alert mechanisms to
notify recipients about an event. For example, you can
specify that when a Key Performance Indicator (KPI)
measurement goes out of range, Ensemble will auto-
matically display that information in the form of a
traffic light, send appropriate messages via email and
pager, and trigger a program or business process to
deal with the event.
InterSystems Corporation
Event Processing and Filtering
The event processing and filtering layer consists of a
set of Business Metric Services, which filter and/or
analyze message warehouse data in real time.
A business metric service gathers relevant information
in order to calculate a Key Performance Indicator
(KPI). A KPI calculation can analyze and correlate
events/messages by employing pre-built analytical
models as well as templates that can incorporate real-
time access to any number of back-end operational
systems.
For example, a business metric service class can calcu-
late values by invoking applications, by gathering data
from external database systems, by accessing (via
objects or SQL) data stored within Ensemble (such
as messages or Business Process information), or by
executing code.
A key advantage that Ensemble delivers for Business
Activity Monitoring is the power and speed it brings
to the development of sophisticated KPIs through
the abstraction and projection capabilities of its
Universal Service Architecture. Abstraction provides a
consistent object view of underlying systems and
applications—no matter what platforms, languages,
data models, storage architectures, network proto-
cols, or other technologies those underlying systems
use. In creating a performance indicator, Ensemble
abstraction enables a developer to rapidly access the
widest range of data, without becoming entangled in
the technical details of the underlying applications.
Ensemble projection makes that abstracted data avail-
able to the widest range of development tools and
technologies, from Java, C++ and .Net, to SQL,
XML and Web services. That frees developers to use
the most effective approach for building each KPI,
and eliminates retraining by virtue of using tools and
technologies that each developer already knows.
Once a KPI has been calculated, it is stored in the
BAM cache and becomes available to the event deliv-
ery and display layer. This high-performance, persist-
ent cache provides efficient and common retrieval,
such that multiple cache readers can use a given KPI
value without incurring the expense of recalculation
(recalculation occurs only at the intervals specified by
the business metric service that defines it).
Event Absorption
The event absorption layer is responsible for the
collection of events. This layer leverages Ensemble’s
transactional message warehouse, which persists and
manages all the messages/events that pass through the
system and make them available, as objects, for contin-
uous real-time analysis. The message warehouse is
implemented via the proven technology of
InterSystems’ Persistent Object Engine.
The ability to persistently store all of the messages that
pass through the system increases system reliability,
provides great flexibility in reporting and analysis, and
enables transaction compensation functions to be pre-
built and executed in real-time. And of course a mes-
sage warehouse is required in order to provide end-to-
end management of messages and the business
processes they drive.
ENABLING THE REAL-TIME ENTERPRISE
Business Metric Services
A business metric service is a specialized business
service that gathers, or otherwise calculates,
one or more numeric values. It is automatically
triggered at specified intervals to perform the
calculation. This calculation can incorporate:
I
A call to an external application
I
A query to an external database
I
An SQL query against Ensemble data, such
as stored messages and business process
information
I
Execution of metric class code
Ensemble places calculated values in a high-
performance, persistent cache so that multiple
users can retrieve these values without incurring
the expense of recalculation.
Business metric values can be displayed on
dashboards via one or more meters, and/or
can be used to trigger business operations and
business processes.
Ensemble provides a New Business Metric
Wizard, which makes it ease to create new
business metric classes.
Page 5 InterSystems Corporation
The Persistent Object Engine has two components:
a virtual machine, and an object store. They are tightly
coupled and operate in the same memory space and
environment. The virtual machine is where the
message broker and orchestration engine execute.
It is optimized for event-driven logic so that processes
can react rapidly to incoming events.
Persisting huge amounts of message traffic – without
impacting overall system performance – clearly
requires extremely high performance. The underlying
technology of the Persistent Object Engine is shared
with our Caché database product, which has a proven
capacity for terabytes of data, tens of thousands of
concurrent users, and hundreds of thousands of
database operations per second.
Ensemble also employs transactional bitmap indexing,
which can radically increase the performance of com-
plex queries, and allows very fast access to live data in
the message warehouse.
Thus, in the context of BAM, the Persistent Object
Engine provides more than “mere” transactional
speed. While the ability to handle huge transaction
volumes is impressive, it becomes even more useful
due to Ensemble’s ability to rapidly access and
analyze this information, both for process manage-
ment and BAM.
Managing BAM Solutions
We have briefly described what BAM can accomplish,
and how Ensemble’s underlying technology supports
BAM. Another of Ensemble’s technology enablers is
its customizable management environment, which
makes it easy to debug BAM solutions while they are
being developed, and to diagnose and fix any prob-
lems that may surface in an operational setting.
Integrated systems tend to be difficult to manage
because they are often loosely coupled. The nature of
asynchronous business processes and message-oriented
applications makes it difficult to trace their threads of
execution, and therefore to debug them. To address
this difficulty, an integration platform must provide
good end-to-end management capabilities, at develop-
ment time as well as after deployment. Ideally, the
integration platform should record every single mes-
sage that passes through the system and provide
ENABLING THE REAL-TIME ENTERPRISE
Transactional Bitmap Indexing
Database query performance is critically depend-
ent on having indexes on the properties used for
searches. Most databases use indexes that, for
each possible value of the column or property,
maintain a list of identifiers for the rows/objects
that have that value. A bitmap index contains a
separate bitmap for each possible value of a
column/property, with one bit for each
row/object that is stored. A bit set to 1 means
that the row/object has that value for the
column/property.
The advantage of bitmap indexes is that complex
queries can be processed by performing Boolean
operations (AND, OR) on the indexes, thus
determining exactly which instances fit the query
conditions without having to search through the
entire database. Bitmap indexes can often boost
response times for queries that search large
volumes of data by a factor of 100 or more.
But bitmap indexes suffer from two problems:
1) they can be painfully slow to update in a
relational database, and 2) they can take up far
too much storage. Thus with relational data-
bases, bitmap indexes are rarely used for transac-
tion processing applications.
InterSystems’ transactional bitmap indexes use
unique technology to eliminate these two
problems. Updating this new type of bitmap is
often faster than traditional indexes, and sophis-
ticated compression techniques radically reduce
storage requirements. The result is ultra-fast
bitmaps that can often be used to search millions
of records in a fraction of a second in on-line
transaction-processing database.
It is transactional bitmap indexing that allows
Ensemble’s BAM capability to perform real-time
analysis of live data, without impacting the
performance needed to store the data
persistently.
Page 6 InterSystems Corporation
the ability to analyze this stored information and
trace message paths – as is the case with Ensemble’s
message warehouse.
Ensemble’s management features also include
message warehouse maintenance, configuration con-
trol, queue and process monitoring, detailed event
logs, and usage histograms. Thus Ensemble provides
a wealth of diagnostic data, as well as the tools to
analyze it in real time. In addition, Ensemble man-
agement tools are extensible, and support standards-
based (e.g., SNMP) third-party management tools.
Ensemble gives you BAM solutions that are capable
and usable in every aspect, from development to
integration to operational deployment.
3
•
BAM SCENARIO
Consider a healthcare provider or institution that
uses multiple laboratories to perform tests for its
patients and clinicians. Each of its departmental
systems might interact with a single laboratory, or
use multiple laboratories, to obtain test results. But
whatever the mode of interaction between provider
and laboratory, it is a business process.
It is easy to imagine a straightforward Business
Process Management (BPM) approach to increasing
efficiency; for example, the business process might
route tests among these multiple laboratories based
on the availability of specific instrumentation or
equipment, or on geographic location, or on any
number of other possible criteria.
But the biggest process problem faced by this
healthcare provider is that laboratories can experi-
ence bottlenecks, and tests can become backed up.
For example, a lab might be overloaded on a given
day due to random fluctuations, or as a result of a
local outbreak of disease, or because of a blizzard
that shuts down the region. There is no BPM mech-
anism that allows the provider to do any kind of
dynamic "load balancing" for laboratories – and this
is where Ensemble’s BAM can provide a solution.
For example, BAM can enter the picture when the
provider and the labs agree on a Service Level
Agreement (SLA) that specifies how long it should
take for a particular type of test result to be returned.
This becomes a Key Performance Indicator (KPI).
BAM allows you to define a business metric service
that calculates this KPI – it might measure when the
test was sent, when the results were returned, and use
whatever other information might be relevant (e.g.,
time-of-day constraints in the SLA). The important
point is that there is now a real-time measurement
that reflects some aspect of that business process.
Because of this, the system can monitor this business
process. It can capture when every single test is sent,
when every set of results is returned, and so on. The
data points from this measurement can be processed
and the results monitored using a visual dashboard.
The dashboard might display a speedometer or traffic
light or whatever type of gauge is desired; but there is
now a visual representation of this KPI, which is recal-
culated continuously.
Now imagine that Laboratory “C” begins to fall
behind in the execution of their SLA. The Laboratory
“C” traffic light “goes red” and we know we have a
problem. This is the essence of BAM.
However, the fusion of BAM and BPM provided by
Ensemble means that the continuously-calculated KPI
can provide feedback to the business process. As long
as the Laboratory “C” KPI is within its specified
range, the process operates in its normal mode. But as
soon as the KPI goes out of range, the business
process can stop dispatching tests to Lab “C” and
refrain until the KPI is back in range. Suddenly, this
business process is reacting intelligently to events, in
real time.
But even this is only a first step. Suppose that we
define business process metrics that compare the
results of the lab tests with known medical norms,
rather than simply measuring conformance to the
SLA. For example, we might know the incidence level
for a particular disease in a particular location; that
the average number of flu cases in the month of
January for a given hospital over the last 50 years is X.
Armed with this knowledge, and with the ability of
Ensemble to manage messages based on their content,
we can define KPIs to monitor what is actually hap-
pening in that hospital. For example, we might have a
KPI that measures the incidence of flu cases and
defines an acceptable range up to 50% more than the
historical average. When that KPI “goes red,” it is a
ENABLING THE REAL-TIME ENTERPRISE
good indication that certain medical steps should be
taken, and that information could be sent to the hos-
pital emergency room or to public health authorities.
We have only begun to explore what BAM solutions
can be used to accomplish. Every enterprise wants
better business intelligence and greater agility in
responding to it. So the only difficult thing to imag-
ine is what kind company would not benefit from
real-time Business Activity Monitoring.
4
•
SUMMARY AND CONCLUSIONS
Ensemble’s BAM capabilities embody the conver-
gence of operational business intelligence (BI) and
real-time application integration. Its real-time data
analytics and notification mechanisms – coupled with
its connectivity, reliable messaging, ultra-fast/ultra-
scalable message warehouse, and integrated powerful
BPM tools – make Ensemble the perfect platform for
building Business Activity Monitoring solutions.
InterSystems Corporation
World Headquarters
One Memorial Drive
Cambridge, MA 02142-1356
Tel: +1.617.621.0600
Fax: +1.617.494.1631
www.InterSystems.com
InterSystems Ensemble is a trademark of InterSystems Corporation. Other product names are trademarks of their respective vendors.
© copyright 2004 InterSystems Corporation. All rights reserved. 11-04
Ensemble’s sophisticated message warehouse has
built-in business processes and primitives, bit-mapped
indexing, and the ability to have processes running in
the background in order to continually monitor the
live message stream as well as stored messages. These
capabilities let you spot trends and exceptions in time
to do something about them.
Ensemble’s unmatched performance and abstraction
capabilities enable BAM to operate in real time, and to
utilize business information from every corner of the
enterprise. This gives organizations the timely business
intelligence they need, and the ability to respond to
that intelligence quickly enough to make a difference.
Next Steps
To find out more about Ensemble or other
InterSystems products and/or to be contacted by an
InterSystems representative, please visit our website at:
www.InterSystems.com/Ensemble
doc_681587506.pdf
The goal of Business Activity Monitoring is to provide management with immediate awareness of business events across the enterprise as well as changing business conditions, so that appropriate and timely decisions can be made.
ENSEMBLE WHI TE PAPER
ENABLING THE REAL-TIME ENTERPRISE
BUSINESS ACTIVITY MONITORING WITH ENSEMBLE
InterSystems Corporation Page 1
ENABLING THE REAL-TIME ENTERPRISE
Executive Summary
Business Activity Monitoring (BAM) enhances
business intelligence by detecting events within
enterprise systems, filtering them for items of
interest, and presenting that information to an
executive in the form of a visual “dashboard” on
a computer screen.
Ensemble provides advanced Business Activity
Monitoring solutions, built atop a comprehensive
and extremely fast platform for application inte-
gration, composite application development, data
coordination, and business process/workflow
management.
Ensemble is designed around the precept that
successful business integration solutions are driv-
en by metadata. Metadata definitions represent,
and bring order to, all integration touchpoints,
including data, services, and business processes –
as well as the messages and data flows that are
the nerve impulses of a system. Thus Ensemble
takes a data management-centric approach to
integration.
That approach is realized with the single, shared
metadata repository and ultra-scalable message
warehouse that are at the core of Ensemble’s
architecture. Its data management capabilities and
data-centric approach, combined with a strong
Service Oriented Architecture, make data easily
available to any application that needs it.
Ensemble’s Business Activity Monitoring capabili-
ties exploit all these characteristics to enable rapid
construction of advanced and comprehensive set
of BAM solutions. Thanks to its unique fusion of
technologies, Ensemble taps a broad source of
enterprise business data, provides the tools that
turn business data into business intelligence, sup-
ports sophisticated analytics to spot trends and
exceptions, and enables automated responses to
specified conditions. And InterSystems’ proven
technology innovations enable these capabilities
to operate in real time, with the highest levels of
reliability and scalability.
1
•
INTRODUCTION
The goal of Business Activity Monitoring is to provide
management with immediate awareness of business
events across the enterprise as well as changing business
conditions, so that appropriate and timely decisions can
be made. By providing executives with this kind of real-
time information, BAM solutions reduce costs and
speed the execution of business strategies.
Ensemble’s Business Activity Monitoring capabilities
rely on its foundation of enabling technology. The
Ensemble technical enablers exploited by BAM include:
I
Full-Spectrum Integration and Development –
an environment that supports rapid development
and integration of: composite applications,
custom adapters, and business process orchestra-
tions – in support of BAM solutions
I
Universal Service Architecture – an advanced
and unique abstraction technology that enables
consistent and efficient object representation of
different programming models and data formats
for rapid access to application functions and data
I
Persistent Object Engine – a high-performance,
high-availability distributed object database, mes-
sage warehouse, and metadata repository that pro-
vides ultra-fast retrieval and manipulation of mes-
sages and events
I
Customizable Management Environment –
a comprehensive set of tailorable and extensible
monitoring and management facilities that are
tightly integrated with the messaging engine and
message warehouse, as well as with business
process management and development tools
Ensemble’s technology provides three major advantages
for BAM solutions. First, it provides a broad informa-
tion base. Ensemble’s Universal Service Architecture
and Persistent Object Engine allow the system to per-
sistently store all messages and events that pass through
the system, and to make them all accessible in the same
format and by way of the same mechanisms.
Second, Ensemble provides the ultra-high performance
necessary for the true “Real-Time Enterprise” (RTE).
Ensemble’s high-performance Persistent Object Engine
allows a great deal of complex processing to be per-
formed on business data, events, and messages in real
time, in order to transform raw data into business intel-
Page 2 InterSystems Corporation
ENABLING THE REAL-TIME ENTERPRISE
ligence. So Ensemble not only encompasses a broad
set of information from underlying applications and
business processes – it also analyzes and correlates this
information on-the-fly.
Third, Ensemble enables real-time reaction to busi-
ness intelligence and events. In addition to raising
dashboard alerts so that users can take action,
Ensemble’s BAM solutions can automatically trigger
sophisticated, programmatic actions in response to
the business intelligence it develops. In this way,
Ensemble’s BAM implementation enables an adaptive
closed-loop system for business process management,
making it possible for business processes to react
dynamically to changes in the business environment.
Ensemble’s Business Activity Monitoring builds upon,
integrates with, and exploits all of Ensemble’s
advanced development, management, and integration
capabilities, by fusing together previously
autonomous technologies in support of rapid and
comprehensive integration.
Thus Ensemble fuses business intelligence technology
with data transformation and routing technology. It
fuses event-driven architectures with service-oriented
architectures. It fuses business application develop-
ment with business process modeling and orchestra-
tion. It fuses all these things to provide and support
Business Activity Monitoring solutions. And it
achieves this with a single development environment,
a single management environment, and a single
architecture.
This unique fusion and comprehensive approach
results in extremely fast integration and composite
application development, as well as support for the
broadest set of application integration approaches and
solutions, including data coordination, business
process and workflow management, composite appli-
cation development, and, finally, Business Activity
Monitoring.
Figure 1: Ensemble, The Next Generation
Integration Platform
Speed is a common theme across all aspects of
Ensemble: speed of development, of integration, of
system performance, and of time-to-solution. Faster
reaction times result in lower costs, better service,
and higher customer satisfaction for any business.
But for some industries – for example, those that
deal in perishable information or goods – it is an
enormous competitive advantage.
Composite Composite
Applications Applications
Data Data
Coordination Coordination
Business Process Business Process
/ Work Flow / Work Flow
Business Activity Business Activity
Monitoring Monitoring
Ensemble Ensemble
Technology Technology Application Application Data Data
2
•
ENSEMBLE’S BAM ARCHITECTURE
OVERVIEW
Ensemble’s BAM architecture is implemented in
three logical layers:
I
Event delivery and display layer
I
Event processing and filtering layer
I
Event absorption layer
Event Delivery and Display
The delivery and display layer is responsible for
notifying recipients about an event once the nature
of that event (e.g., impact and severity) has been
analyzed and evaluated. Notification can mean both
the presentation of information to an executive by
way of a dashboard, or the triggering of an automat-
ed process.
you are building a dashboard widget, a complex
adapter, a business process definition, or a complete
business activity monitoring application.
As we have seen, BAM solutions exploit all aspects of
Ensemble’s underlying capabilities. Therefore, a BAM
development environment must be able to deal with
difficult problems at many different levels, including
the creation of complex adapters, the modeling/defi-
nition of both simple and complex business process-
es, and the development of full-blown composite
applications.
This unified environment has many associated advan-
tages, including greatly reduced complexity and train-
ing costs and greatly increased speed of development.
Full spectrum integration and development even pro-
vides insurance against vendor lock-in, because every-
thing in Ensemble is exposed equally well to both
Ensemble’s tools and to third-party tools of every
kind.
Because Ensemble abstracts system components and
represents them in a consistent form, and because
these representations can be projected into code or
other standards-based formats that developers can
take forward and embellish, Ensemble gives you bi-
directional interoperability between multiple different
development and business process definition tools.
Ensemble provides great flexibility in choosing a path
to arrive at a BAM solution, which can then be
deployed in Ensemble's high-performance execution
environment.
InterSystems Corporation Page 3
ENABLING THE REAL-TIME ENTERPRISE
A dashboard is a specialized web page that uses
meters to provide a real-time, graphical display of
measured values, as shown in Figure 2. The measured
value is associated with a specific property within a
Business Metric Service class.
Ensemble includes a complete set of meter styles,
including speedometers, odometers, fuel gauges, light
bars, traffic lights, indicator lights, line charts, and
bar charts – and it supports the development of cus-
tomized meters. Note that Ensemble dashboards are
implemented using standard HTML and SVG
(Structured Vector Graphics), so there are no propri-
etary ActiveX or Java components to deal with.
Thus Ensemble’s delivery and display capabilities pro-
vide business agility and flexibility. While many BAM
approaches give managers insight into transactions
and operations, they lack the ability to close the
process feedback loop, and to effect dynamic changes
to the processes that carry out business strategy. In
this way, Ensemble can notify recipients of an excep-
tion, and continuously feed information back into the
relevant business processes, causing them to react
dynamically to changing events. It is this adaptive
closed-loop system that turns your organization into
a real-time enterprise.
Now let us briefly examine how Ensemble’s develop-
ment environment aids the creation of BAM solu-
tions. Full Spectrum Integration and Development
simply means that you use the same “diagrammatic”
or GUI-based environment – The Studio – whether
Figure 2: Ensemble Dashboard and Meters
The delivery and display layer exploits all of
Ensemble’s notification and alert mechanisms to
notify recipients about an event. For example, you can
specify that when a Key Performance Indicator (KPI)
measurement goes out of range, Ensemble will auto-
matically display that information in the form of a
traffic light, send appropriate messages via email and
pager, and trigger a program or business process to
deal with the event.
InterSystems Corporation
Event Processing and Filtering
The event processing and filtering layer consists of a
set of Business Metric Services, which filter and/or
analyze message warehouse data in real time.
A business metric service gathers relevant information
in order to calculate a Key Performance Indicator
(KPI). A KPI calculation can analyze and correlate
events/messages by employing pre-built analytical
models as well as templates that can incorporate real-
time access to any number of back-end operational
systems.
For example, a business metric service class can calcu-
late values by invoking applications, by gathering data
from external database systems, by accessing (via
objects or SQL) data stored within Ensemble (such
as messages or Business Process information), or by
executing code.
A key advantage that Ensemble delivers for Business
Activity Monitoring is the power and speed it brings
to the development of sophisticated KPIs through
the abstraction and projection capabilities of its
Universal Service Architecture. Abstraction provides a
consistent object view of underlying systems and
applications—no matter what platforms, languages,
data models, storage architectures, network proto-
cols, or other technologies those underlying systems
use. In creating a performance indicator, Ensemble
abstraction enables a developer to rapidly access the
widest range of data, without becoming entangled in
the technical details of the underlying applications.
Ensemble projection makes that abstracted data avail-
able to the widest range of development tools and
technologies, from Java, C++ and .Net, to SQL,
XML and Web services. That frees developers to use
the most effective approach for building each KPI,
and eliminates retraining by virtue of using tools and
technologies that each developer already knows.
Once a KPI has been calculated, it is stored in the
BAM cache and becomes available to the event deliv-
ery and display layer. This high-performance, persist-
ent cache provides efficient and common retrieval,
such that multiple cache readers can use a given KPI
value without incurring the expense of recalculation
(recalculation occurs only at the intervals specified by
the business metric service that defines it).
Event Absorption
The event absorption layer is responsible for the
collection of events. This layer leverages Ensemble’s
transactional message warehouse, which persists and
manages all the messages/events that pass through the
system and make them available, as objects, for contin-
uous real-time analysis. The message warehouse is
implemented via the proven technology of
InterSystems’ Persistent Object Engine.
The ability to persistently store all of the messages that
pass through the system increases system reliability,
provides great flexibility in reporting and analysis, and
enables transaction compensation functions to be pre-
built and executed in real-time. And of course a mes-
sage warehouse is required in order to provide end-to-
end management of messages and the business
processes they drive.
ENABLING THE REAL-TIME ENTERPRISE
Business Metric Services
A business metric service is a specialized business
service that gathers, or otherwise calculates,
one or more numeric values. It is automatically
triggered at specified intervals to perform the
calculation. This calculation can incorporate:
I
A call to an external application
I
A query to an external database
I
An SQL query against Ensemble data, such
as stored messages and business process
information
I
Execution of metric class code
Ensemble places calculated values in a high-
performance, persistent cache so that multiple
users can retrieve these values without incurring
the expense of recalculation.
Business metric values can be displayed on
dashboards via one or more meters, and/or
can be used to trigger business operations and
business processes.
Ensemble provides a New Business Metric
Wizard, which makes it ease to create new
business metric classes.
Page 5 InterSystems Corporation
The Persistent Object Engine has two components:
a virtual machine, and an object store. They are tightly
coupled and operate in the same memory space and
environment. The virtual machine is where the
message broker and orchestration engine execute.
It is optimized for event-driven logic so that processes
can react rapidly to incoming events.
Persisting huge amounts of message traffic – without
impacting overall system performance – clearly
requires extremely high performance. The underlying
technology of the Persistent Object Engine is shared
with our Caché database product, which has a proven
capacity for terabytes of data, tens of thousands of
concurrent users, and hundreds of thousands of
database operations per second.
Ensemble also employs transactional bitmap indexing,
which can radically increase the performance of com-
plex queries, and allows very fast access to live data in
the message warehouse.
Thus, in the context of BAM, the Persistent Object
Engine provides more than “mere” transactional
speed. While the ability to handle huge transaction
volumes is impressive, it becomes even more useful
due to Ensemble’s ability to rapidly access and
analyze this information, both for process manage-
ment and BAM.
Managing BAM Solutions
We have briefly described what BAM can accomplish,
and how Ensemble’s underlying technology supports
BAM. Another of Ensemble’s technology enablers is
its customizable management environment, which
makes it easy to debug BAM solutions while they are
being developed, and to diagnose and fix any prob-
lems that may surface in an operational setting.
Integrated systems tend to be difficult to manage
because they are often loosely coupled. The nature of
asynchronous business processes and message-oriented
applications makes it difficult to trace their threads of
execution, and therefore to debug them. To address
this difficulty, an integration platform must provide
good end-to-end management capabilities, at develop-
ment time as well as after deployment. Ideally, the
integration platform should record every single mes-
sage that passes through the system and provide
ENABLING THE REAL-TIME ENTERPRISE
Transactional Bitmap Indexing
Database query performance is critically depend-
ent on having indexes on the properties used for
searches. Most databases use indexes that, for
each possible value of the column or property,
maintain a list of identifiers for the rows/objects
that have that value. A bitmap index contains a
separate bitmap for each possible value of a
column/property, with one bit for each
row/object that is stored. A bit set to 1 means
that the row/object has that value for the
column/property.
The advantage of bitmap indexes is that complex
queries can be processed by performing Boolean
operations (AND, OR) on the indexes, thus
determining exactly which instances fit the query
conditions without having to search through the
entire database. Bitmap indexes can often boost
response times for queries that search large
volumes of data by a factor of 100 or more.
But bitmap indexes suffer from two problems:
1) they can be painfully slow to update in a
relational database, and 2) they can take up far
too much storage. Thus with relational data-
bases, bitmap indexes are rarely used for transac-
tion processing applications.
InterSystems’ transactional bitmap indexes use
unique technology to eliminate these two
problems. Updating this new type of bitmap is
often faster than traditional indexes, and sophis-
ticated compression techniques radically reduce
storage requirements. The result is ultra-fast
bitmaps that can often be used to search millions
of records in a fraction of a second in on-line
transaction-processing database.
It is transactional bitmap indexing that allows
Ensemble’s BAM capability to perform real-time
analysis of live data, without impacting the
performance needed to store the data
persistently.
Page 6 InterSystems Corporation
the ability to analyze this stored information and
trace message paths – as is the case with Ensemble’s
message warehouse.
Ensemble’s management features also include
message warehouse maintenance, configuration con-
trol, queue and process monitoring, detailed event
logs, and usage histograms. Thus Ensemble provides
a wealth of diagnostic data, as well as the tools to
analyze it in real time. In addition, Ensemble man-
agement tools are extensible, and support standards-
based (e.g., SNMP) third-party management tools.
Ensemble gives you BAM solutions that are capable
and usable in every aspect, from development to
integration to operational deployment.
3
•
BAM SCENARIO
Consider a healthcare provider or institution that
uses multiple laboratories to perform tests for its
patients and clinicians. Each of its departmental
systems might interact with a single laboratory, or
use multiple laboratories, to obtain test results. But
whatever the mode of interaction between provider
and laboratory, it is a business process.
It is easy to imagine a straightforward Business
Process Management (BPM) approach to increasing
efficiency; for example, the business process might
route tests among these multiple laboratories based
on the availability of specific instrumentation or
equipment, or on geographic location, or on any
number of other possible criteria.
But the biggest process problem faced by this
healthcare provider is that laboratories can experi-
ence bottlenecks, and tests can become backed up.
For example, a lab might be overloaded on a given
day due to random fluctuations, or as a result of a
local outbreak of disease, or because of a blizzard
that shuts down the region. There is no BPM mech-
anism that allows the provider to do any kind of
dynamic "load balancing" for laboratories – and this
is where Ensemble’s BAM can provide a solution.
For example, BAM can enter the picture when the
provider and the labs agree on a Service Level
Agreement (SLA) that specifies how long it should
take for a particular type of test result to be returned.
This becomes a Key Performance Indicator (KPI).
BAM allows you to define a business metric service
that calculates this KPI – it might measure when the
test was sent, when the results were returned, and use
whatever other information might be relevant (e.g.,
time-of-day constraints in the SLA). The important
point is that there is now a real-time measurement
that reflects some aspect of that business process.
Because of this, the system can monitor this business
process. It can capture when every single test is sent,
when every set of results is returned, and so on. The
data points from this measurement can be processed
and the results monitored using a visual dashboard.
The dashboard might display a speedometer or traffic
light or whatever type of gauge is desired; but there is
now a visual representation of this KPI, which is recal-
culated continuously.
Now imagine that Laboratory “C” begins to fall
behind in the execution of their SLA. The Laboratory
“C” traffic light “goes red” and we know we have a
problem. This is the essence of BAM.
However, the fusion of BAM and BPM provided by
Ensemble means that the continuously-calculated KPI
can provide feedback to the business process. As long
as the Laboratory “C” KPI is within its specified
range, the process operates in its normal mode. But as
soon as the KPI goes out of range, the business
process can stop dispatching tests to Lab “C” and
refrain until the KPI is back in range. Suddenly, this
business process is reacting intelligently to events, in
real time.
But even this is only a first step. Suppose that we
define business process metrics that compare the
results of the lab tests with known medical norms,
rather than simply measuring conformance to the
SLA. For example, we might know the incidence level
for a particular disease in a particular location; that
the average number of flu cases in the month of
January for a given hospital over the last 50 years is X.
Armed with this knowledge, and with the ability of
Ensemble to manage messages based on their content,
we can define KPIs to monitor what is actually hap-
pening in that hospital. For example, we might have a
KPI that measures the incidence of flu cases and
defines an acceptable range up to 50% more than the
historical average. When that KPI “goes red,” it is a
ENABLING THE REAL-TIME ENTERPRISE
good indication that certain medical steps should be
taken, and that information could be sent to the hos-
pital emergency room or to public health authorities.
We have only begun to explore what BAM solutions
can be used to accomplish. Every enterprise wants
better business intelligence and greater agility in
responding to it. So the only difficult thing to imag-
ine is what kind company would not benefit from
real-time Business Activity Monitoring.
4
•
SUMMARY AND CONCLUSIONS
Ensemble’s BAM capabilities embody the conver-
gence of operational business intelligence (BI) and
real-time application integration. Its real-time data
analytics and notification mechanisms – coupled with
its connectivity, reliable messaging, ultra-fast/ultra-
scalable message warehouse, and integrated powerful
BPM tools – make Ensemble the perfect platform for
building Business Activity Monitoring solutions.
InterSystems Corporation
World Headquarters
One Memorial Drive
Cambridge, MA 02142-1356
Tel: +1.617.621.0600
Fax: +1.617.494.1631
www.InterSystems.com
InterSystems Ensemble is a trademark of InterSystems Corporation. Other product names are trademarks of their respective vendors.
© copyright 2004 InterSystems Corporation. All rights reserved. 11-04
Ensemble’s sophisticated message warehouse has
built-in business processes and primitives, bit-mapped
indexing, and the ability to have processes running in
the background in order to continually monitor the
live message stream as well as stored messages. These
capabilities let you spot trends and exceptions in time
to do something about them.
Ensemble’s unmatched performance and abstraction
capabilities enable BAM to operate in real time, and to
utilize business information from every corner of the
enterprise. This gives organizations the timely business
intelligence they need, and the ability to respond to
that intelligence quickly enough to make a difference.
Next Steps
To find out more about Ensemble or other
InterSystems products and/or to be contacted by an
InterSystems representative, please visit our website at:
www.InterSystems.com/Ensemble
doc_681587506.pdf