Description
Agile BI has reached a tipping point. Broadly recognized by analysts and technologists as a powerful alternative to traditional business intelligence (BI) development approaches.
Enabling Agile Business
Intelligence with Balanced
Insight Consensus®
2
Enabling Agile Business Intelligence with Balanced Insight Consensus®
Executive Summary
Agile BI has reached a tipping point. Broadly recognized by analysts and technologists as a powerful
alternative to traditional business intelligence (BI) development approaches, Agile BI ofers viable solutions
to many longstanding delivery challenges, including:
• Insufcient IT-business collaboration
• Excessively long development cycles
• Blown budgets
• Static documentation that is out-of-date before go-live
• Low end-user satisfaction and adoption rates.
The bottom line is that organizations in a wide range of industries have adopted Agile BI approaches to
reduce BI project costs, shorten delivery timelines, and promote stronger collaboration between IT and
business users.
As you will see, the question has become not if or whether to embrace Agile BI, but rather exactly when
and precisely how. Specifcally, that means choosing which projects or business units would beneft most
from the rapid development of an easy-to-consume BI application. And it also means selecting an Agile
BI delivery tool that enables efcient, cost-efective and low-risk migration to Agile BI. The ideal tool also
complements existing and future BI technology investments.
As highlighted on the ensuing pages, Balanced Insight Consensus® ofers unique advantages in BI project
delivery, as well as superior functionality and fexibility for Agile BI. The Consensus vision is for business-
driven BI – putting users (traditionally the missing link in BI development) at the heart of the process
through “top-down” requirements defnition and rapid prototyping.
This white paper will highlight the key business issues and typical project delivery challenges addressed by
Agile BI and Balanced Insight Consensus, and demonstrate why Consensus has been hailed as the leader in
the Agile BI space.
3
Introducing Agile BI
Given the difculty that many organizations have faced in delivering the BI applications their managers and
executives need to understand performance and make critical business decisions, it’s not surprising that an
alternative development approach is being embraced. Indeed, there is a broad and growing consensus that
Agile BI’s time has come. Signifcant momentum has been building, and Agile BI is on its way to becoming
the industry standard for BI project delivery. In April 2010, Forrester published “Agile BI Out of the Box,” which
defned key selection parameters for Agile BI tools and recommended that companies make the change.
Other analysts (including Gartner) issued reports that highlighted how best to apply Agile BI principles and
the specifc ways in which it is superior to traditional delivery approaches.
At the simplest level, Agile BI refers to the use of the agile software development methodology for BI
projects. (See sidebar for our holistic defnition of Agile BI.) BI delivery teams have discovered they can
realize signifcant benefts from these methods because the agile approach solves several fundamental and
longstanding challenges to efective BI delivery:
• Rising Demand: Demand for information about business
performance has risen dramatically. (The Information Age
could just as well be called the BI Age.) BI delivery teams
have a large backlog of projects from business users
looking for more information to support their decisions.
But it’s not just more information users want; it’s more
information faster. Agile BI helps IT meet the imperatives
for quantity and speed in unlocking the full value of
data assets.
• Flexibility: The agile methodology is designed to adjust
to changing requirements – and BI requirements change
more frequently and profoundly than those for nearly all
other types of software projects. In fact, in a 2010 survey
of 200 business and IT executives conducted by Forrester,
67% of respondents said that BI requirements change
at least monthly. A full 20% of respondents said their
BI requirements change on a daily basis. Such changes
wreak havoc on the traditional waterfall delivery cycle,
yet they are inevitable during the lifetime of any
BI project.
• User Engagement: The great strength of the agile
methodology is that it fosters collaboration between
IT and the business. While traditional approaches
have struggled to place user needs at the core of the
process. Agile BI is all about giving users faster access
to functionality and more opportunities to provide
feedback. Ultimately, user engagement equates to higher
user satisfaction and adoption rates.
For more than 20 years, custom
software development teams have
used a mature toolset of agile
delivery methods like Scrum and
Extreme Programming (XP) to deliver
a broad range of software in critical
business functions. Agile BI refers to
the application of the agile software
development methodology to the
project delivery and development of
business intelligence applications.
Specifcally, that means an iterative
process noted for rapid prototype
development through a series
of “Sprints” to produce specifc
functionality that is shared with users,
who are given full and repeated
opportunities to provide feedback.
But the principles of Agile BI extend
beyond the realm of software
development to enable operational
and organizational agility – the ability
to execute nimbly and efciently, and
respond rapidly to new markets and
opportunities. The point is, Agile BI is
as much about how BI applications
are used and what they enable
businesses to do, as it is about how
the applications themselves are built.
Defning Our Terms:
What is Agile BI Delivery?
4
• Manageable Scope: Budget overruns and blown schedules can damage IT’s credibility, besides
costing the company real money. Because Agile BI focuses on the delivery of smaller sets of
functionality in shorter time periods, projects are driven by business defned scope and value. Project
timelines and budgets can be tracked in smaller units, and users pay for the value defned. Avoiding
scope creep is good news, but it’s better news that these budgets are signifcantly smaller and the
project timelines much shorter.
• Lower Costs, Higher Value: Agile methods in BI have a strong track record in reducing project costs
and shortening timelines. Further, because project budgets are aligned to high-priority deliverables
and outcomes – that is, high-powered, easy-to-consume applications that users like and that meet
real and urgent business needs – overall technology ROI also increases.
Agile BI with Balanced Insight Consensus®
The key to driving an Agile BI project is minimizing project management overhead, reusing existing assets,
and automating inefcient manual tasks. Aligning your project budgets to deliverables and outcomes
generates more value – that is, high-powered, easy-to-understand, easy-to-consume business intelligence
solutions that users like and that meet real business needs.
The following high level workfow illustrates how an organization would use Balanced Insight Consensus® to
automate their Agile BI process. See fgure 1.
Figure 1. Balanced Insight Consensus automates the Agile BI process.
Now let’s look more closely at these process steps and the unique features and capabilities of Balanced
Insight Consensus, and how it fulflls the promise of Agile BI at each step of the development process, from
requirements gathering and engagement with the business, to scope management, to execution and
delivery. The drivers we will investigate are:
• User Stories & “Just in Time” Elaboration
• Product & Sprint Backlog Management
• Sprinting ahead to “Potentially Shippable Code”
5
User Stories & “Just-in-Time” Elaboration
User stories are project components consisting of two to three sentences that serve as a starting point
for a conversation between business sponsors and developers. The dialog that ensues between the key
stakeholders elaborates on the details and tasks involved to deliver the specifc functionality within a BI
solution. In Consensus, Requirement and Question Artifacts are leveraged to capture user stories and the
broad parameters of a feature set. See fgure 2.
Figure 2. User stories are captured in Balanced Insight Consensus Requirement and Question artifacts.
Compare this approach to traditional waterfall methods and you will fnd your team spending weeks
feshing out verbose requirements documents with massive amounts of detail. While the documents may
seem impressive, the time to develop them is not well spent, because the requirements have changed
before the documents can be printed or emailed. The documentation will be stale well before the team
writes a line of code. Agile BI, and Consensus, embraces this reality with the user story approach, allowing
the business sponsor to elaborate on requirements in a “just-in-time” manner, immediately prior to the
beginning of the new feature’s development. Further, User Stories codify the “voice of the business” in the
development process and, ultimately, in the fnal solution.
6
Product and Sprint Backlog Management
A BI project consists of a collection of user stories and the development team must manage them carefully,
both in terms of their importance to users and the degree of difculty in implementing them. User stories
need to be categorized in one of two ways: Product Backlog or Sprint Backlog.
A Product Backlog is simply a list of things that need to be done. A key tenet of the agile methodology, a
Sprint is a miniature delivery lifecycle designed to produce specifc functionality. The Sprint Backlog is a
highly visible “to-do” list of user stories prioritized for the team to deliver in the current or next Sprint.
Managing the backlog is all about speed to implementation. Based on a team’s velocity – that is, the
approximate number of story points historically possible for the team to complete during the multi-week
sprint timeframe – the highest priority user stories in the Product Backlog become the Sprint Backlog.
The Product Backlog in Consensus represents the set of defned, unimplemented user stories currently
prioritized for delivery. The user stories are ordered by their relative priority, and they are given a story point
estimate, which is a high-level, relative weighting that measures difculty of implementation. For example,
a user story that is difcult to implement might be rated with 3 story points, whereas a simple user story
might be rated only 1.
The Product Backlog and Sprint Backlog can be viewed and managed in Consensus by tagging Requirement
and Question artifacts with Consensus Organizational Unit Artifacts. See fgure 3.
Figure 3. Sprint and Product Backlogs are easily managed in Consensus.
7
After a Sprint and with every business sponsor and user interaction, the team can consider how changing
business priorities may warrant a re-prioritization of user stories in the Product Backlog. Further, refecting
on each previous Sprint, the team may also identify lessons learned and delivery innovations that can be
used to increase quality and speed of future Sprints and functionality.
The Product and Sprint backlogs within Consensus very simply keep BI project teams focused on delivering
the highest-priority functionality, even as priorities change, month to month and quarter to quarter. Further,
they ofer a record of ongoing dialogue with the business, making reprioritizations of functionality and
requirements just that much easier to manage. Waterfall methods, by contrast, try to fx priorities once
and for all in impractical and unrealistic fashion. The “development in a vacuum” approach that results is a
signifcant factor in low satisfaction rates with BI tools developed by traditional means. Lessons learned are
largely lost and continuous improvement becomes nearly impossible in the context of long development
cycles and irregular feedback loops.
Sprinting Ahead to “Potentially Shippable Code”
Each Sprint follows a series of clearly defned steps as the team:
1. takes a user story from the Sprint Backlog,
2. discusses and validates the user story with a business representative or user,
3. rapidly builds a prototype aligned to requirements and scope,
4. develops and tests potentially shippable code, and
5. reviews and gains acceptance from an end-user or representative of the business.
During the Sprint, user stories recorded in Consensus Requirements Artifacts are elaborated and refned
with the business representative. The design is feshed out in detail and modeled as the business sees them
in Consensus Topic and Information Package artifacts. A simple user story like “As a sales manager I need to
see total proft per salesperson” might be elaborated into an Information Package as shown in Figure 4.
Information Packages
An Information Package is a technology-agnostic requirements consolidation and modeling format unique
to Consensus. Information Packages are critical to organizing data so that users can ultimately view it in a
variety of helpful ways, supporting a group of related questions that the business representative is implying
based on their requirements. Figure 4 highlights how a user may answer questions regarding both proft
and revenue measures, by date and by salesperson. Further, Consensus supports aggregations of the
measure data by sales manager, geographical areas, and various time periods. Flexibility is built in.
8
Figure 4. Consensus Information Packages are multi-dimensional, so they can answer questions regarding proft
and revenue measures, by date and by salesperson, and then aggregate data by sales manager, geographical
areas, and various time periods.
Though users may articulate the lowest grain of the information they need, it is also important to
understand the navigation path users would likely to take to reach granular data. Conversely, it’s valuable
to consider how that data might be aggregated upward into broader reports. Consensus is designed to
support such logical navigation and aggregation through reusable Topic Artifacts. For instance, a senior
sales executive can view results by country, region, and/or territory, as well as by individual sales rep.
See fgure 5.
Consensus Topic and Information Package artifacts consolidate user requirements in a non-technical way
and serve as an excellent discussion tool to facilitate collaboration with business sponsors. Users can more
clearly see the relationship between data sets and types of information, and also gain a stronger sense of
what BI tools are capable of.
9
Figure 5: Consensus breaks down simple user stories into
broader sets of related questions and views during Sprints.
Rapid Prototyping
The heart of the agile methodology, particularly
as it’s used in BI, is rapid prototyping that enables
developers to take user stories, consolidate them
in business models, and create a working sample
solution within existing BI tools that can be reviewed
with a business representative. These prototypes
help development team members understand user
stories to the appropriate level of detail, and business
users recognize the technical challenges involved
in delivering efective BI apps. They are also useful
in garnering valuable buy-in and excitement from
business users by showing them that the solution
will answer the business questions they’ve defned,
as well as related secondary questions that they are
likely to ask in the future. (See sidebar, The Three
Immutable Laws of Rapid Prototyping.)
In our experience, rapid prototyping is the heart of
Agile BI and subject to three immutable laws, and
Consensus is designed to help organizations take full
advantage of them:
1. The Agile Law of Prototype Malleability: The value
of a prototype is directly proportional to how
easy it is to modify. If it takes efort to change
a prototype in response to user feedback, then
fewer prototyping iterations will be undertaken
and the value of the prototyping efort diminishes.
Prototype generation in Consensus requires only
a few clicks and a few minutes. Valuable feedback
from business representatives can then be
incorporated in Consensus and a new prototype
can be generated with the same small efort.
This approach allows for multiple iterations to
occur during a single meeting. Compare that to
traditional approaches which require long waits
between major “releases.”
2. The Agile Law of Prototype Commutability: A
prototype is only as valuable as it is representative
of the end solution. Users do not need the extra
challenge of imagining what given functionality
will look like in their standard tool. Creating a
prototype in the BI platform already in place for
users, giving them an opportunity to interact
with the prototype in the same way they will
interact with the end solution. The prototype has
all the functionality of the end solution, including
standard functionality provided by the BI tool (like
save as PDF and export to Excel). Thus, they will be
able to articulate exactly what works and which
features need improvement. After a few iterations,
if your users tell you they are happy with the
prototype, you can be confdent they will be happy
with the end solution. And the confdence levels
of the business sponsors and the BI delivery team
increase dramatically.
3. The Agile Law of Prototype Lucidity: A prototype
is only as valuable as the new insights it brings
to what ultimately needs to be built. Prototypes
created with Consensus allow delivery teams to
gain a new perspective and keener understanding
on requirements, and provide business users an
opportunity to see that some of their ideas may
have been strong in theory, but aren’t necessary
in practice.
These three prototyping laws are critical
guardrails to ensuring that your Agile BI
delivery approach delivers all the quality
and rigor that users expect.
The Three Immutable Laws
of Rapid Prototyping
10
Consensus automatically will produce these prototypes for you, as well as going a step further –
generation of a full working prototype with sample data in the BI tool used by the business every day.
See Figures 6-7
Figure 6: Consensus makes it easy to generate full working prototypes in the very BI tools and platforms used
by the business every day.
Figure 7: A solution generated by Consensus in Oracle OBIEE and Oracle Answers. With automation driving
rapid prototyping, BI delivery teams can quickly move from requirements to functionality.
11
Consensus allows the same Information Package(s) to be deployed to any number of other business
intelligence environments simply by selecting a new target BI tool, like Microsoft Analysis Services, IBM
Cognos, SAP Business Objects, Microstrategy or various other popular tools. In this way, Consensus bridges
the gap between requirements gathering and actual outputs. See fgure 8.
Figure 8. A solution generated by Consensus in Microsoft SQL Server Analysis Services utilizing the same
Information Package from Figures 6-7.
From a technical perspective, Consensus generates the star schema data mart tables and builds the
Business Intelligence tool semantics and potentially Online Analytical Processing (OLAP) Cube on top of
them. The rich metadata and well-defned hierarchies pushed from Consensus make self-service capabilities
available “out of the box” from tools such as Microsoft Excel or IBI WebFOCUS. This approach provides a
perfect tool to verify that all expected business questions can be answered correctly and all needed usability
features are present. Reports, interactive dashboards and other user-friendly features can be built upon this
environment to evolve the prototype into the end solution, ensuring traceability from prototype approval to
fnal solution.
Consensus ensures that code is reusable for new BI apps, or for other projects seeking to leverage solution
designs across platforms. This speaks to core of the Agile promise, and the unique benefts of Consensus. It’s
not architected as “rip and replace.” Existing information assets are reusable, as well as the new assets (User
Stories, Information Packages and code) created by Consensus. They can be quickly and efciently deployed
for new apps and functionalities across the business, diferent projects, and diferent tools.
The output of each Sprint is an iteration of the solution that is potentially production ready and could be
shipped to give end users a subset of target functionality. After a sprint, queries might be slow, full drill-
down capabilities may not yet be available and data not fully scrubbed. But it’s critical to remember that
partial functionality in a few weeks is better than no functionality for several months. The idea is to give
users a sense of the end product. This early view will help elucidate any data surprises that would have
otherwise shown up dangerously close to the full solution go-live date. But, to some extent, the sprint is just
the beginning, as further business-IT interaction enables the solution to keep getting better.
12
By automating the prototyping process, Consensus ofers a signifcant advantage over conventional BI
software and tools, and traditional waterfall approaches, where prototyping is often manual and occurs
very early in the requirements process, if at all. Consensus allows changes to be made easily at multiple
levels of the prototype (like at the Information Package level). This means new, updated prototypes can
be more quickly regenerated – literally, with only a few clicks. Prototypes with incremental enhancements
of functionality give the business a constant fow of new information support based on prioritized needs,
leading to stronger decision making as the project progresses. Strong buy-in and support from the business
usually follows as users experience frsthand the power and benefts of advanced BI applications.
When prototypes are created manually within a BI tool, developers must touch multiple systems – writing
new SQL scripts to generate data marts, generating those marts in the platform, refning the semantic
layer in the BI tools and so on. Consensus generates all of this, maximizing your staf productivity and
standardizing your technical architecture.
Consensus prototypes also look and feel like end solutions, with real data and interfaces that mimic the end
environment, where many traditionally developed prototypes are mocked-up wire frames with little or no
interactivity. The more accurately prototypes refect end solutions and all requirements, the more efective
the end solution will be. See sidebar, The Three Immutable Laws of Rapid Prototyping.
Bridging the Delivery Gap
Traditional waterfall methods are overwhelmed by changing requirements. The end result is a delivery gap
(see Figure 9), which occurs when IT delivers a solution that meets requirements as stated during previous
gathering phases, not what the business needs at the time of delivery. BI veterans are all too familiar with
this situation.
Figure 9. The delivery gap is a common problem in business intelligence.
Requirements gathering is complete at a point in time, while the business
continues to evolve during the months a project is being implemented.
Within Agile BI and Balanced Insight Consensus, this challenge is the engine by which solutions are
delivered. In a fxed and fnite world, where one set of requirements applies forever (or even for a year), there
would be no need for Agile BI. But Agile BI refects the reality that requirements change when business
conditions and informational needs change. The delivery of a series of controlled-scope releases in quick
succession puts practice over theory and gives users a chance to refne requirements, restate preferences
and make suggestions on usability based on hands-on experience with a working prototype.
BI Delivery Gap
BI Delivery Go live
13
Agile BI Rewards
A leading energy/utility company was focused on a simple but profound business question: What is the
efect of customer notifcation activities on revenue collection? The goal was to understand which channel –
email, snail mail or outbound calls – yielded the best results in terms of increased collections.
The company frst attempted to extend its existing business intelligence solution using traditional
approaches, but gave up on the project because it was too expensive and time consuming. Then, the
delivery team turned to Agile BI and Consensus.
With Consensus, the Balanced Insight Agile BI solution resulted in a clear defnition of the required
functionality. In just two weeks, operational managers had access to the data and insights necessary to
optimize collection activities and boost cash fow. The benefts of Agile BI and Consensus also included
reduced delivery cycle times and lower project costs.
Traditional Approach
10 people
8 weeks
3,600 hours
With Consensus
4 people
2 weeks
350 hours
vs
vs
People
Duration
Hours
Revenue Operations BI Solution
$218k
$21m
Project Costs
Reserves Removed
from Balance Sheet
Fast – 1/2 the time
Customers go live 50-75% faster than with traditional approaches
Engaging – IT-Business Collaboration -- Early and Often
With user involvement in the development process, adoption rates soar and
projected benefts get realized.
vs vs
Traditional Approach
25
$250k - $500k
With Consensus
48
$100k
# of Projects
Project Cost
Productive – 1/2 the cost
Business intelligence delivery productivity doubles,
while reducing budgets 50-75%
14
Next Steps: Making the Move to Agile BI
Organizations often want to know how to facilitate the move to Agile BI. Specifcally, what’s the frst step to
take? In our experience, a small-scale BI delivery project aligned to meet a discrete business need is the best
place to start.
When bringing Agile BI delivery to your organization, consider the following checklist:
• Assemble a team of developers and business representatives to work together
• Decide whether business representatives on the team should be business stakeholders or technical
leads/liaisons with a strong handle on business needs
• Identify and prioritize appropriate user stories or requirements to address during an initial project
• Select specifc functionality for a two- or three-week Sprint
• Assess various Agile BI delivery tools that can integrate with your existing data warehouse and
BI environment
• Move into development and prepare to iterate.
Certainly organizations new to Agile BI should recognize how it’s something of a way of life once you get
started. Many organizations never view traditional development the same way again. We believe this is all
the more reason to move forward with Agile BI.
The Industry on Balanced Insight Consensus
“Consensus is the “missing ingredient” for our Agile Data Warehousing(TM) methodology. Consensus gets
business users thinking and agreeing about warehouse requirements and designs instead of trading endless
emails. With Consensus we arrive at approved prototypes quickly, bypassing lengthy committee reviews
and getting us a green light to start development months earlier than before.”
–Ralph Hughes, Chief Systems Architect, Ceregenics, and lead author of Agile Data Warehousing.
“Consensus delivers an unbelievable value to the clients by really addressing data warehouse agility and
bridging the business/IT gap. Companies that take advantage of Consensus’ features will develop more
accurate scope based on the ability to assist business people in articulating their needs and then rapidly
deliver a mock up of the BI solution.”
– Wayne Eckerson, Director of Research, The Data Warehouse Institute (TDWI)
R
R
R
R
R
R
15
“Balanced Insight Consensus has had a profound impact on how ICC delivers business intelligence solutions.
We view Consensus as a true diferentiator for ICC and been able to drive down implementation timelines
for BI solutions.”
– Don Jackson, Director of Business Intelligence, Information Control Corporation
“I was blown away on how fast Balanced Insight Consensus translated my business requirements into the
proper data model. Literally this occurred with a push of a button.”
– Julie Shannon, Independent BI Consultant
About Balanced Insight
Balanced Insight is revolutionizing how IT and business users experience business intelligence delivery.
Delivering Agile BI capabilities “out of the box,” Balanced Insight Consensus® enables IT to provide
unprecedented value to business users, empowering them to engage in BI development like never
before. The results? Consensus enhances speed of deployment, collaboration and productivity
throughout the development process, increases end-user satisfaction, reduces costs, and complements
existing BI technologies. Balanced Insight was founded in 2008 by technologist, innovator and author
Tom Hammergren.
Balanced Insight Inc.
8170 Corporate Park Drive
Suite 200
Cincinnati, Ohio 45242
513.322.1646
www.balancedinsight.com
doc_551036253.pdf
Agile BI has reached a tipping point. Broadly recognized by analysts and technologists as a powerful alternative to traditional business intelligence (BI) development approaches.
Enabling Agile Business
Intelligence with Balanced
Insight Consensus®
2
Enabling Agile Business Intelligence with Balanced Insight Consensus®
Executive Summary
Agile BI has reached a tipping point. Broadly recognized by analysts and technologists as a powerful
alternative to traditional business intelligence (BI) development approaches, Agile BI ofers viable solutions
to many longstanding delivery challenges, including:
• Insufcient IT-business collaboration
• Excessively long development cycles
• Blown budgets
• Static documentation that is out-of-date before go-live
• Low end-user satisfaction and adoption rates.
The bottom line is that organizations in a wide range of industries have adopted Agile BI approaches to
reduce BI project costs, shorten delivery timelines, and promote stronger collaboration between IT and
business users.
As you will see, the question has become not if or whether to embrace Agile BI, but rather exactly when
and precisely how. Specifcally, that means choosing which projects or business units would beneft most
from the rapid development of an easy-to-consume BI application. And it also means selecting an Agile
BI delivery tool that enables efcient, cost-efective and low-risk migration to Agile BI. The ideal tool also
complements existing and future BI technology investments.
As highlighted on the ensuing pages, Balanced Insight Consensus® ofers unique advantages in BI project
delivery, as well as superior functionality and fexibility for Agile BI. The Consensus vision is for business-
driven BI – putting users (traditionally the missing link in BI development) at the heart of the process
through “top-down” requirements defnition and rapid prototyping.
This white paper will highlight the key business issues and typical project delivery challenges addressed by
Agile BI and Balanced Insight Consensus, and demonstrate why Consensus has been hailed as the leader in
the Agile BI space.
3
Introducing Agile BI
Given the difculty that many organizations have faced in delivering the BI applications their managers and
executives need to understand performance and make critical business decisions, it’s not surprising that an
alternative development approach is being embraced. Indeed, there is a broad and growing consensus that
Agile BI’s time has come. Signifcant momentum has been building, and Agile BI is on its way to becoming
the industry standard for BI project delivery. In April 2010, Forrester published “Agile BI Out of the Box,” which
defned key selection parameters for Agile BI tools and recommended that companies make the change.
Other analysts (including Gartner) issued reports that highlighted how best to apply Agile BI principles and
the specifc ways in which it is superior to traditional delivery approaches.
At the simplest level, Agile BI refers to the use of the agile software development methodology for BI
projects. (See sidebar for our holistic defnition of Agile BI.) BI delivery teams have discovered they can
realize signifcant benefts from these methods because the agile approach solves several fundamental and
longstanding challenges to efective BI delivery:
• Rising Demand: Demand for information about business
performance has risen dramatically. (The Information Age
could just as well be called the BI Age.) BI delivery teams
have a large backlog of projects from business users
looking for more information to support their decisions.
But it’s not just more information users want; it’s more
information faster. Agile BI helps IT meet the imperatives
for quantity and speed in unlocking the full value of
data assets.
• Flexibility: The agile methodology is designed to adjust
to changing requirements – and BI requirements change
more frequently and profoundly than those for nearly all
other types of software projects. In fact, in a 2010 survey
of 200 business and IT executives conducted by Forrester,
67% of respondents said that BI requirements change
at least monthly. A full 20% of respondents said their
BI requirements change on a daily basis. Such changes
wreak havoc on the traditional waterfall delivery cycle,
yet they are inevitable during the lifetime of any
BI project.
• User Engagement: The great strength of the agile
methodology is that it fosters collaboration between
IT and the business. While traditional approaches
have struggled to place user needs at the core of the
process. Agile BI is all about giving users faster access
to functionality and more opportunities to provide
feedback. Ultimately, user engagement equates to higher
user satisfaction and adoption rates.
For more than 20 years, custom
software development teams have
used a mature toolset of agile
delivery methods like Scrum and
Extreme Programming (XP) to deliver
a broad range of software in critical
business functions. Agile BI refers to
the application of the agile software
development methodology to the
project delivery and development of
business intelligence applications.
Specifcally, that means an iterative
process noted for rapid prototype
development through a series
of “Sprints” to produce specifc
functionality that is shared with users,
who are given full and repeated
opportunities to provide feedback.
But the principles of Agile BI extend
beyond the realm of software
development to enable operational
and organizational agility – the ability
to execute nimbly and efciently, and
respond rapidly to new markets and
opportunities. The point is, Agile BI is
as much about how BI applications
are used and what they enable
businesses to do, as it is about how
the applications themselves are built.
Defning Our Terms:
What is Agile BI Delivery?
4
• Manageable Scope: Budget overruns and blown schedules can damage IT’s credibility, besides
costing the company real money. Because Agile BI focuses on the delivery of smaller sets of
functionality in shorter time periods, projects are driven by business defned scope and value. Project
timelines and budgets can be tracked in smaller units, and users pay for the value defned. Avoiding
scope creep is good news, but it’s better news that these budgets are signifcantly smaller and the
project timelines much shorter.
• Lower Costs, Higher Value: Agile methods in BI have a strong track record in reducing project costs
and shortening timelines. Further, because project budgets are aligned to high-priority deliverables
and outcomes – that is, high-powered, easy-to-consume applications that users like and that meet
real and urgent business needs – overall technology ROI also increases.
Agile BI with Balanced Insight Consensus®
The key to driving an Agile BI project is minimizing project management overhead, reusing existing assets,
and automating inefcient manual tasks. Aligning your project budgets to deliverables and outcomes
generates more value – that is, high-powered, easy-to-understand, easy-to-consume business intelligence
solutions that users like and that meet real business needs.
The following high level workfow illustrates how an organization would use Balanced Insight Consensus® to
automate their Agile BI process. See fgure 1.
Figure 1. Balanced Insight Consensus automates the Agile BI process.
Now let’s look more closely at these process steps and the unique features and capabilities of Balanced
Insight Consensus, and how it fulflls the promise of Agile BI at each step of the development process, from
requirements gathering and engagement with the business, to scope management, to execution and
delivery. The drivers we will investigate are:
• User Stories & “Just in Time” Elaboration
• Product & Sprint Backlog Management
• Sprinting ahead to “Potentially Shippable Code”
5
User Stories & “Just-in-Time” Elaboration
User stories are project components consisting of two to three sentences that serve as a starting point
for a conversation between business sponsors and developers. The dialog that ensues between the key
stakeholders elaborates on the details and tasks involved to deliver the specifc functionality within a BI
solution. In Consensus, Requirement and Question Artifacts are leveraged to capture user stories and the
broad parameters of a feature set. See fgure 2.
Figure 2. User stories are captured in Balanced Insight Consensus Requirement and Question artifacts.
Compare this approach to traditional waterfall methods and you will fnd your team spending weeks
feshing out verbose requirements documents with massive amounts of detail. While the documents may
seem impressive, the time to develop them is not well spent, because the requirements have changed
before the documents can be printed or emailed. The documentation will be stale well before the team
writes a line of code. Agile BI, and Consensus, embraces this reality with the user story approach, allowing
the business sponsor to elaborate on requirements in a “just-in-time” manner, immediately prior to the
beginning of the new feature’s development. Further, User Stories codify the “voice of the business” in the
development process and, ultimately, in the fnal solution.
6
Product and Sprint Backlog Management
A BI project consists of a collection of user stories and the development team must manage them carefully,
both in terms of their importance to users and the degree of difculty in implementing them. User stories
need to be categorized in one of two ways: Product Backlog or Sprint Backlog.
A Product Backlog is simply a list of things that need to be done. A key tenet of the agile methodology, a
Sprint is a miniature delivery lifecycle designed to produce specifc functionality. The Sprint Backlog is a
highly visible “to-do” list of user stories prioritized for the team to deliver in the current or next Sprint.
Managing the backlog is all about speed to implementation. Based on a team’s velocity – that is, the
approximate number of story points historically possible for the team to complete during the multi-week
sprint timeframe – the highest priority user stories in the Product Backlog become the Sprint Backlog.
The Product Backlog in Consensus represents the set of defned, unimplemented user stories currently
prioritized for delivery. The user stories are ordered by their relative priority, and they are given a story point
estimate, which is a high-level, relative weighting that measures difculty of implementation. For example,
a user story that is difcult to implement might be rated with 3 story points, whereas a simple user story
might be rated only 1.
The Product Backlog and Sprint Backlog can be viewed and managed in Consensus by tagging Requirement
and Question artifacts with Consensus Organizational Unit Artifacts. See fgure 3.
Figure 3. Sprint and Product Backlogs are easily managed in Consensus.
7
After a Sprint and with every business sponsor and user interaction, the team can consider how changing
business priorities may warrant a re-prioritization of user stories in the Product Backlog. Further, refecting
on each previous Sprint, the team may also identify lessons learned and delivery innovations that can be
used to increase quality and speed of future Sprints and functionality.
The Product and Sprint backlogs within Consensus very simply keep BI project teams focused on delivering
the highest-priority functionality, even as priorities change, month to month and quarter to quarter. Further,
they ofer a record of ongoing dialogue with the business, making reprioritizations of functionality and
requirements just that much easier to manage. Waterfall methods, by contrast, try to fx priorities once
and for all in impractical and unrealistic fashion. The “development in a vacuum” approach that results is a
signifcant factor in low satisfaction rates with BI tools developed by traditional means. Lessons learned are
largely lost and continuous improvement becomes nearly impossible in the context of long development
cycles and irregular feedback loops.
Sprinting Ahead to “Potentially Shippable Code”
Each Sprint follows a series of clearly defned steps as the team:
1. takes a user story from the Sprint Backlog,
2. discusses and validates the user story with a business representative or user,
3. rapidly builds a prototype aligned to requirements and scope,
4. develops and tests potentially shippable code, and
5. reviews and gains acceptance from an end-user or representative of the business.
During the Sprint, user stories recorded in Consensus Requirements Artifacts are elaborated and refned
with the business representative. The design is feshed out in detail and modeled as the business sees them
in Consensus Topic and Information Package artifacts. A simple user story like “As a sales manager I need to
see total proft per salesperson” might be elaborated into an Information Package as shown in Figure 4.
Information Packages
An Information Package is a technology-agnostic requirements consolidation and modeling format unique
to Consensus. Information Packages are critical to organizing data so that users can ultimately view it in a
variety of helpful ways, supporting a group of related questions that the business representative is implying
based on their requirements. Figure 4 highlights how a user may answer questions regarding both proft
and revenue measures, by date and by salesperson. Further, Consensus supports aggregations of the
measure data by sales manager, geographical areas, and various time periods. Flexibility is built in.
8
Figure 4. Consensus Information Packages are multi-dimensional, so they can answer questions regarding proft
and revenue measures, by date and by salesperson, and then aggregate data by sales manager, geographical
areas, and various time periods.
Though users may articulate the lowest grain of the information they need, it is also important to
understand the navigation path users would likely to take to reach granular data. Conversely, it’s valuable
to consider how that data might be aggregated upward into broader reports. Consensus is designed to
support such logical navigation and aggregation through reusable Topic Artifacts. For instance, a senior
sales executive can view results by country, region, and/or territory, as well as by individual sales rep.
See fgure 5.
Consensus Topic and Information Package artifacts consolidate user requirements in a non-technical way
and serve as an excellent discussion tool to facilitate collaboration with business sponsors. Users can more
clearly see the relationship between data sets and types of information, and also gain a stronger sense of
what BI tools are capable of.
9
Figure 5: Consensus breaks down simple user stories into
broader sets of related questions and views during Sprints.
Rapid Prototyping
The heart of the agile methodology, particularly
as it’s used in BI, is rapid prototyping that enables
developers to take user stories, consolidate them
in business models, and create a working sample
solution within existing BI tools that can be reviewed
with a business representative. These prototypes
help development team members understand user
stories to the appropriate level of detail, and business
users recognize the technical challenges involved
in delivering efective BI apps. They are also useful
in garnering valuable buy-in and excitement from
business users by showing them that the solution
will answer the business questions they’ve defned,
as well as related secondary questions that they are
likely to ask in the future. (See sidebar, The Three
Immutable Laws of Rapid Prototyping.)
In our experience, rapid prototyping is the heart of
Agile BI and subject to three immutable laws, and
Consensus is designed to help organizations take full
advantage of them:
1. The Agile Law of Prototype Malleability: The value
of a prototype is directly proportional to how
easy it is to modify. If it takes efort to change
a prototype in response to user feedback, then
fewer prototyping iterations will be undertaken
and the value of the prototyping efort diminishes.
Prototype generation in Consensus requires only
a few clicks and a few minutes. Valuable feedback
from business representatives can then be
incorporated in Consensus and a new prototype
can be generated with the same small efort.
This approach allows for multiple iterations to
occur during a single meeting. Compare that to
traditional approaches which require long waits
between major “releases.”
2. The Agile Law of Prototype Commutability: A
prototype is only as valuable as it is representative
of the end solution. Users do not need the extra
challenge of imagining what given functionality
will look like in their standard tool. Creating a
prototype in the BI platform already in place for
users, giving them an opportunity to interact
with the prototype in the same way they will
interact with the end solution. The prototype has
all the functionality of the end solution, including
standard functionality provided by the BI tool (like
save as PDF and export to Excel). Thus, they will be
able to articulate exactly what works and which
features need improvement. After a few iterations,
if your users tell you they are happy with the
prototype, you can be confdent they will be happy
with the end solution. And the confdence levels
of the business sponsors and the BI delivery team
increase dramatically.
3. The Agile Law of Prototype Lucidity: A prototype
is only as valuable as the new insights it brings
to what ultimately needs to be built. Prototypes
created with Consensus allow delivery teams to
gain a new perspective and keener understanding
on requirements, and provide business users an
opportunity to see that some of their ideas may
have been strong in theory, but aren’t necessary
in practice.
These three prototyping laws are critical
guardrails to ensuring that your Agile BI
delivery approach delivers all the quality
and rigor that users expect.
The Three Immutable Laws
of Rapid Prototyping
10
Consensus automatically will produce these prototypes for you, as well as going a step further –
generation of a full working prototype with sample data in the BI tool used by the business every day.
See Figures 6-7
Figure 6: Consensus makes it easy to generate full working prototypes in the very BI tools and platforms used
by the business every day.
Figure 7: A solution generated by Consensus in Oracle OBIEE and Oracle Answers. With automation driving
rapid prototyping, BI delivery teams can quickly move from requirements to functionality.
11
Consensus allows the same Information Package(s) to be deployed to any number of other business
intelligence environments simply by selecting a new target BI tool, like Microsoft Analysis Services, IBM
Cognos, SAP Business Objects, Microstrategy or various other popular tools. In this way, Consensus bridges
the gap between requirements gathering and actual outputs. See fgure 8.
Figure 8. A solution generated by Consensus in Microsoft SQL Server Analysis Services utilizing the same
Information Package from Figures 6-7.
From a technical perspective, Consensus generates the star schema data mart tables and builds the
Business Intelligence tool semantics and potentially Online Analytical Processing (OLAP) Cube on top of
them. The rich metadata and well-defned hierarchies pushed from Consensus make self-service capabilities
available “out of the box” from tools such as Microsoft Excel or IBI WebFOCUS. This approach provides a
perfect tool to verify that all expected business questions can be answered correctly and all needed usability
features are present. Reports, interactive dashboards and other user-friendly features can be built upon this
environment to evolve the prototype into the end solution, ensuring traceability from prototype approval to
fnal solution.
Consensus ensures that code is reusable for new BI apps, or for other projects seeking to leverage solution
designs across platforms. This speaks to core of the Agile promise, and the unique benefts of Consensus. It’s
not architected as “rip and replace.” Existing information assets are reusable, as well as the new assets (User
Stories, Information Packages and code) created by Consensus. They can be quickly and efciently deployed
for new apps and functionalities across the business, diferent projects, and diferent tools.
The output of each Sprint is an iteration of the solution that is potentially production ready and could be
shipped to give end users a subset of target functionality. After a sprint, queries might be slow, full drill-
down capabilities may not yet be available and data not fully scrubbed. But it’s critical to remember that
partial functionality in a few weeks is better than no functionality for several months. The idea is to give
users a sense of the end product. This early view will help elucidate any data surprises that would have
otherwise shown up dangerously close to the full solution go-live date. But, to some extent, the sprint is just
the beginning, as further business-IT interaction enables the solution to keep getting better.
12
By automating the prototyping process, Consensus ofers a signifcant advantage over conventional BI
software and tools, and traditional waterfall approaches, where prototyping is often manual and occurs
very early in the requirements process, if at all. Consensus allows changes to be made easily at multiple
levels of the prototype (like at the Information Package level). This means new, updated prototypes can
be more quickly regenerated – literally, with only a few clicks. Prototypes with incremental enhancements
of functionality give the business a constant fow of new information support based on prioritized needs,
leading to stronger decision making as the project progresses. Strong buy-in and support from the business
usually follows as users experience frsthand the power and benefts of advanced BI applications.
When prototypes are created manually within a BI tool, developers must touch multiple systems – writing
new SQL scripts to generate data marts, generating those marts in the platform, refning the semantic
layer in the BI tools and so on. Consensus generates all of this, maximizing your staf productivity and
standardizing your technical architecture.
Consensus prototypes also look and feel like end solutions, with real data and interfaces that mimic the end
environment, where many traditionally developed prototypes are mocked-up wire frames with little or no
interactivity. The more accurately prototypes refect end solutions and all requirements, the more efective
the end solution will be. See sidebar, The Three Immutable Laws of Rapid Prototyping.
Bridging the Delivery Gap
Traditional waterfall methods are overwhelmed by changing requirements. The end result is a delivery gap
(see Figure 9), which occurs when IT delivers a solution that meets requirements as stated during previous
gathering phases, not what the business needs at the time of delivery. BI veterans are all too familiar with
this situation.
Figure 9. The delivery gap is a common problem in business intelligence.
Requirements gathering is complete at a point in time, while the business
continues to evolve during the months a project is being implemented.
Within Agile BI and Balanced Insight Consensus, this challenge is the engine by which solutions are
delivered. In a fxed and fnite world, where one set of requirements applies forever (or even for a year), there
would be no need for Agile BI. But Agile BI refects the reality that requirements change when business
conditions and informational needs change. The delivery of a series of controlled-scope releases in quick
succession puts practice over theory and gives users a chance to refne requirements, restate preferences
and make suggestions on usability based on hands-on experience with a working prototype.
BI Delivery Gap
BI Delivery Go live
13
Agile BI Rewards
A leading energy/utility company was focused on a simple but profound business question: What is the
efect of customer notifcation activities on revenue collection? The goal was to understand which channel –
email, snail mail or outbound calls – yielded the best results in terms of increased collections.
The company frst attempted to extend its existing business intelligence solution using traditional
approaches, but gave up on the project because it was too expensive and time consuming. Then, the
delivery team turned to Agile BI and Consensus.
With Consensus, the Balanced Insight Agile BI solution resulted in a clear defnition of the required
functionality. In just two weeks, operational managers had access to the data and insights necessary to
optimize collection activities and boost cash fow. The benefts of Agile BI and Consensus also included
reduced delivery cycle times and lower project costs.
Traditional Approach
10 people
8 weeks
3,600 hours
With Consensus
4 people
2 weeks
350 hours
vs
vs
People
Duration
Hours
Revenue Operations BI Solution
$218k
$21m
Project Costs
Reserves Removed
from Balance Sheet
Fast – 1/2 the time
Customers go live 50-75% faster than with traditional approaches
Engaging – IT-Business Collaboration -- Early and Often
With user involvement in the development process, adoption rates soar and
projected benefts get realized.
vs vs
Traditional Approach
25
$250k - $500k
With Consensus
48
$100k
# of Projects
Project Cost
Productive – 1/2 the cost
Business intelligence delivery productivity doubles,
while reducing budgets 50-75%
14
Next Steps: Making the Move to Agile BI
Organizations often want to know how to facilitate the move to Agile BI. Specifcally, what’s the frst step to
take? In our experience, a small-scale BI delivery project aligned to meet a discrete business need is the best
place to start.
When bringing Agile BI delivery to your organization, consider the following checklist:
• Assemble a team of developers and business representatives to work together
• Decide whether business representatives on the team should be business stakeholders or technical
leads/liaisons with a strong handle on business needs
• Identify and prioritize appropriate user stories or requirements to address during an initial project
• Select specifc functionality for a two- or three-week Sprint
• Assess various Agile BI delivery tools that can integrate with your existing data warehouse and
BI environment
• Move into development and prepare to iterate.
Certainly organizations new to Agile BI should recognize how it’s something of a way of life once you get
started. Many organizations never view traditional development the same way again. We believe this is all
the more reason to move forward with Agile BI.
The Industry on Balanced Insight Consensus
“Consensus is the “missing ingredient” for our Agile Data Warehousing(TM) methodology. Consensus gets
business users thinking and agreeing about warehouse requirements and designs instead of trading endless
emails. With Consensus we arrive at approved prototypes quickly, bypassing lengthy committee reviews
and getting us a green light to start development months earlier than before.”
–Ralph Hughes, Chief Systems Architect, Ceregenics, and lead author of Agile Data Warehousing.
“Consensus delivers an unbelievable value to the clients by really addressing data warehouse agility and
bridging the business/IT gap. Companies that take advantage of Consensus’ features will develop more
accurate scope based on the ability to assist business people in articulating their needs and then rapidly
deliver a mock up of the BI solution.”
– Wayne Eckerson, Director of Research, The Data Warehouse Institute (TDWI)
R
R
R
R
R
R
15
“Balanced Insight Consensus has had a profound impact on how ICC delivers business intelligence solutions.
We view Consensus as a true diferentiator for ICC and been able to drive down implementation timelines
for BI solutions.”
– Don Jackson, Director of Business Intelligence, Information Control Corporation
“I was blown away on how fast Balanced Insight Consensus translated my business requirements into the
proper data model. Literally this occurred with a push of a button.”
– Julie Shannon, Independent BI Consultant
About Balanced Insight
Balanced Insight is revolutionizing how IT and business users experience business intelligence delivery.
Delivering Agile BI capabilities “out of the box,” Balanced Insight Consensus® enables IT to provide
unprecedented value to business users, empowering them to engage in BI development like never
before. The results? Consensus enhances speed of deployment, collaboration and productivity
throughout the development process, increases end-user satisfaction, reduces costs, and complements
existing BI technologies. Balanced Insight was founded in 2008 by technologist, innovator and author
Tom Hammergren.
Balanced Insight Inc.
8170 Corporate Park Drive
Suite 200
Cincinnati, Ohio 45242
513.322.1646
www.balancedinsight.com
doc_551036253.pdf