Description
Business Intelligence enables organizations to make well informed business decisions and gain competitive advantage.
Business Intelligence:
Effective Decision Making
Current Status
Mountains of Data
What do I
do???
How do I increase
sales????
How do I make my
product better???
Business Users
Mountains of Data
? From Operational Systems
? ERP (Enterprise Resource Planning)
– Sales/Order
– Inventory
? Customer Relationship Management
(CRM)
? Web Sites
– Orders
– Click-stream
? …
Mountains of Data
? Organizations have lots of data
? Data is not in a form that is useful to
decision-makers
– Not easy to review
– Not informative nor insightful
Today’s Information Flow
? Business in 90?s invested in transactional
systems:
– Supply Chain Management (SCM)
– Customer Relationship Management (CRM)
– Enterprise Resource Planning (ERP)
– Manufacturing Resource Planning (MRP)
– Finance (budget, forecasting and reporting)
Proliferation of Data
MRP SCM CRM Finance
Operations Sales Finance
Procure-
ment
Silos of data by functional area
Transaction
Layer
Reporting
Layer
Data from Disparate Sources
Region: A Region: B Div 2 Div 1
Sales Sales Sales
Sales
Silos of data within large organizations
Transaction
Layer
Reporting
Layer
Business Intelligence
? Business is now investing in Business
Intelligence
? Business Intelligence is about making
effective business decisions
What is BI?
The process by which an organization
manages large amounts of data, extracting
pertinent information, and turning that
information into knowledge upon which
actions can be taken.
What is BI?
Business intelligence (BI) is a broad category
of application programs and technologies for
gathering, storing, analyzing, and providing
access to data to help enterprise users make
better business decisions.
BI
? Involves PEOPLE and Technology
? Involves using a rational approach to
management
? Involves a continuous cycle of measurement,
adjustment & re-measurement
The BI Cycle
BI
start
Reasons for BI
? BI enables organizations to make well
informed business decisions and gain
competitive advantage.
? BI enables organizations to use information to
quickly and constantly respond to changes.
Benefits of BI
? Improved performance based upon timely
and accurate information
? Elimination of guesswork
? Expedited decision making
? Early visibility of changes:
– Customer buying patterns
– Supply chain activity
– Financial arrangements
Benefits of BI
? “Single Version of the truth”
? Accurate, timely data available to all levels of
the organization
To Note:
Although we call it Business Intelligence, the
concepts and techniques are applicable to
almost any organization including those in
health care, biotech, education,
government …
BI Activities
BI applications include the activities of:
• decision support,
• query and reporting,
• online analytical processing (OLAP),
• statistical analysis,
• forecasting, and
• data mining.
BI Users
? There are many different users who can
benefit from business intelligence
– Executives
– Business Decision Makers
– Information Workers
– Line Workers
– Analysts
BI Solutions-
How to make it happen
? Two main components:
– Data Consolidation and Storage
– Data Retrieval, Analysis and Presentation
BI Curriculum
? Multi-Dimensional Analysis
? Data Warehousing
? Data Mining
? Dimensional Modeling
? Data Visualization
The Problem
Mountains of Data
Business People
GAP
How do I increase
sales????
How do I make my
product better???
How do I retain
customers?
Bridging the Gap
? Need data storage structures to facilitate fast
analysis of huge volumes of data
? Need software to provide access to the data,
allow flexible manipulation, and provide
meaningful presentation
Data Storage Structures
? Multi-Dimensional Databases
?Cubes
Multi-Dimensional Databases
? Measures
– Any quantitative expression
– Some are designated as Key Performance Indicators (KPI)
– Appropriate to the business process.
? Dimensions
– How we describe the measures:
Product/Customer/Region/Time
– These are the “By?s
– “What were our Customer Sales by Product Line by
Region by Quarter for the past two years?”.
Logical Structure
Multi-Dimensional Databases (Cubes)
ODS
ODS ODS
Data Warehouse
Multi-Dimensional
Database (Cube)
* ODS = Operational Data Store
Relational
Database
Programs
Business
Intelligence
Programs
Multi-Dimensional Databases
Multi-Dimensional
Database (Cube)
Software Applications
Multi-Dimensional
Database
(Cube)
Business
Person
Business
Person
Business
Person
Reporting
Applications
Analytic
Applications
Score Cards
Dashboards
Analytics
? Reporting Applications
– Limited user interaction
– Fulfill a significant portion of an organization?s
information needs
? Analytic Applications
– Allow users to visualize and explore data
following their train of thought
– Extensive interactivity
Analytic Application
Summary
? Students learn to:
– Create multi-dimensional databases
– Create professional quality reports
– Use analytics to provide in-depth data analysis
Data Warehousing
Designing a Data Warehouse
Data Warehouse Topics
? Decision Support Systems
– history
? Requirements Gathering
– Where data located, owners, definition, how often
updated
? Data Analysis
– Determine for table structures
Data Warehouse
? ETL Processes &
Deliverables
– Cleaning & Conforming
? Valid, missing
? Address, gender
– Schemas
? Dimension Tables
? Fact Tables
Data Consolidation & Storage
MRP CRM SCM Finance
Transaction
Layer
Shared Data
Layer
Data Warehouse
Customers Sales Procurement Suppliers Operations Finance
Shared
Reporting
? Operations and financial information is shared
across the organization from same core data
Data Warehouses
ODS*
ODS ODS
Data Warehouse
Multi-Dimensional
Database (Cube)
*ODS = Operational Data Store
How is data consolidated?
? This is difficult!!!!!
– Data is often spread across multiple systems,
stored in different formats, and may even be
localized for different countries
Transforming Data
? Data must be transformed for consistency and
meaning
– Transformations may be as simple as copying columns or
may be incredibly complex
– Common transformations include:
? Hard-coded changes („T? to 1)
? Looking up values in a table (mapping a customer number
across disparate systems)
? Inserting dummy records and mapping them to unknowns
(inserting an „Unknown? customer)
Cleansing Data
? Data must be cleansed to be meaningful
– All companies have “bad” data in their systems
– Data may be missing
– Data may be inconsistent
– Data may be wrong
Data Warehouses
? ETL (extract, transform and load) processes
are needed to create data warehouses
– This is an arduous and technical process that can
account for a large percentage of a BI project
cost!!!!
Data Mining
Data Mining
? The process of identifying patterns in data
? Goes beyond simple querying of the database
? Goes beyond multi-dimensional database
queries as well
Data Mining
? Data Mining works for problems like:
– Develop a general profile for credit card customers
…
– Differentiate individuals who are poor credit risks
…
– Determine what characteristics differentiate male
& female investors.
Data Mining vs. Data Query
? Use data query if you already almost know
what you are looking for.
? Use data mining to find regularities in data
that are not obvious.
Data Mining Applications
? Fraud detection
? Targeted Marketing
? Risk Management
? Business Analysis
Origins of Data Mining
? Mathematics
– Statistics
– Numerical Analysis
? Artificial Intelligence/Machine Learning
? Computer Science
– Data Storage and Manipulation
How does Data Mining work?
? Uses induction-based learning:
The process of forming general concept
definitions by observing specific examples of
concepts to be learned.
How does Data Mining work?
What-Cha-Ma-Call-Its
NOT What-Cha-Ma-Call-Its
How does Data Mining work?
Which of these are What-Cha-Ma-Call-Its?
Data Mining Process
List of Customers:
-some bicycle buyers
-some not
Data Mining
Software
Model
List of Prospective Buyers
Model
List of Likely Buyers
Overview of Mining Strategies
Data Mining
Strategies
Supervised
Learning
Market
Basket
Analysis
Unsupervised
Clustering
Note: This representation is over-simplified and data mining strategies
are continually being invented.
More on our Curriculum
Skills
? Written communication
? Problem Solving
– Analytical
– Troubleshooting
? Software
– Microsoft SQL Server Management Studio
– SQL Server BI Development Studio
– SQL Server Reporting Services
– Pro Clarity
Delivery Methods
? Online: Distance Education, reaches wider
market
? Telecourse: tremendous effort to create, but
once created easy to deliver
– Televised, DVDs, online for homework, exams
? Hybrid: Meet once a week, the rest online
? On campus: evenings only
Delivery Methods
? Use of Camtasia for
– Software demonstrations
– PowerPoint lectures
? Pod casting
Certificates
? Business Intelligence Analyst (5 classes)
– Multi-dimensional analysis, data warehousing, data
mining, statistics, general business
– 2 quarters full-time/ 3 quarters part-time
? Business Intelligence Developer (4 additional
classes)
– Dimensional modeling, data visualization, multi-
dimensional II, data warehousing II (more
programming with SQL Server)
? Web site: www.bcc.ctc.edu/bi
Certificates
? Relational Database Analyst (6 classes)
– SA & D, programming, reporting, spreadsheets, db
theory
– 2 quarters full-time/ 3 quarters part-time
? Relational Database Developer (3 additional
classes)
– Programming, SQL, group processes
? Web site: www.bcc.ctc.edu/bi
Jobs
? Business Analyst
? Data Analyst
? Functional Analyst
? Marketing Analyst
Jobs
? Report Developer
? Data Modeler
? ETL Developer
? Data Architect
? Data Warehouse Designer
? Data Warehouse Developer
? Data Warehouse Administrator
? Database Administrator
Jobs
? Business Intelligence Consultant
? Business Intelligence Developer
? Business Intelligence Analyst
? Business Intelligence Project Team Member
Jobs
? One of the fastest growing segments of IT
? Less likely to be outsourced
? May exist in business units rather than IT
? Knowledge/understanding of the organization
is key
doc_998911412.ppt
Business Intelligence enables organizations to make well informed business decisions and gain competitive advantage.
Business Intelligence:
Effective Decision Making
Current Status
Mountains of Data
What do I
do???
How do I increase
sales????
How do I make my
product better???
Business Users
Mountains of Data
? From Operational Systems
? ERP (Enterprise Resource Planning)
– Sales/Order
– Inventory
? Customer Relationship Management
(CRM)
? Web Sites
– Orders
– Click-stream
? …
Mountains of Data
? Organizations have lots of data
? Data is not in a form that is useful to
decision-makers
– Not easy to review
– Not informative nor insightful
Today’s Information Flow
? Business in 90?s invested in transactional
systems:
– Supply Chain Management (SCM)
– Customer Relationship Management (CRM)
– Enterprise Resource Planning (ERP)
– Manufacturing Resource Planning (MRP)
– Finance (budget, forecasting and reporting)
Proliferation of Data
MRP SCM CRM Finance
Operations Sales Finance
Procure-
ment
Silos of data by functional area
Transaction
Layer
Reporting
Layer
Data from Disparate Sources
Region: A Region: B Div 2 Div 1
Sales Sales Sales
Sales
Silos of data within large organizations
Transaction
Layer
Reporting
Layer
Business Intelligence
? Business is now investing in Business
Intelligence
? Business Intelligence is about making
effective business decisions
What is BI?
The process by which an organization
manages large amounts of data, extracting
pertinent information, and turning that
information into knowledge upon which
actions can be taken.
What is BI?
Business intelligence (BI) is a broad category
of application programs and technologies for
gathering, storing, analyzing, and providing
access to data to help enterprise users make
better business decisions.
BI
? Involves PEOPLE and Technology
? Involves using a rational approach to
management
? Involves a continuous cycle of measurement,
adjustment & re-measurement
The BI Cycle
BI
start
Reasons for BI
? BI enables organizations to make well
informed business decisions and gain
competitive advantage.
? BI enables organizations to use information to
quickly and constantly respond to changes.
Benefits of BI
? Improved performance based upon timely
and accurate information
? Elimination of guesswork
? Expedited decision making
? Early visibility of changes:
– Customer buying patterns
– Supply chain activity
– Financial arrangements
Benefits of BI
? “Single Version of the truth”
? Accurate, timely data available to all levels of
the organization
To Note:
Although we call it Business Intelligence, the
concepts and techniques are applicable to
almost any organization including those in
health care, biotech, education,
government …
BI Activities
BI applications include the activities of:
• decision support,
• query and reporting,
• online analytical processing (OLAP),
• statistical analysis,
• forecasting, and
• data mining.
BI Users
? There are many different users who can
benefit from business intelligence
– Executives
– Business Decision Makers
– Information Workers
– Line Workers
– Analysts
BI Solutions-
How to make it happen
? Two main components:
– Data Consolidation and Storage
– Data Retrieval, Analysis and Presentation
BI Curriculum
? Multi-Dimensional Analysis
? Data Warehousing
? Data Mining
? Dimensional Modeling
? Data Visualization
The Problem
Mountains of Data
Business People
GAP
How do I increase
sales????
How do I make my
product better???
How do I retain
customers?
Bridging the Gap
? Need data storage structures to facilitate fast
analysis of huge volumes of data
? Need software to provide access to the data,
allow flexible manipulation, and provide
meaningful presentation
Data Storage Structures
? Multi-Dimensional Databases
?Cubes
Multi-Dimensional Databases
? Measures
– Any quantitative expression
– Some are designated as Key Performance Indicators (KPI)
– Appropriate to the business process.
? Dimensions
– How we describe the measures:
Product/Customer/Region/Time
– These are the “By?s
– “What were our Customer Sales by Product Line by
Region by Quarter for the past two years?”.
Logical Structure
Multi-Dimensional Databases (Cubes)
ODS
ODS ODS
Data Warehouse
Multi-Dimensional
Database (Cube)
* ODS = Operational Data Store
Relational
Database
Programs
Business
Intelligence
Programs
Multi-Dimensional Databases
Multi-Dimensional
Database (Cube)
Software Applications
Multi-Dimensional
Database
(Cube)
Business
Person
Business
Person
Business
Person
Reporting
Applications
Analytic
Applications
Score Cards
Dashboards
Analytics
? Reporting Applications
– Limited user interaction
– Fulfill a significant portion of an organization?s
information needs
? Analytic Applications
– Allow users to visualize and explore data
following their train of thought
– Extensive interactivity
Analytic Application
Summary
? Students learn to:
– Create multi-dimensional databases
– Create professional quality reports
– Use analytics to provide in-depth data analysis
Data Warehousing
Designing a Data Warehouse
Data Warehouse Topics
? Decision Support Systems
– history
? Requirements Gathering
– Where data located, owners, definition, how often
updated
? Data Analysis
– Determine for table structures
Data Warehouse
? ETL Processes &
Deliverables
– Cleaning & Conforming
? Valid, missing
? Address, gender
– Schemas
? Dimension Tables
? Fact Tables
Data Consolidation & Storage
MRP CRM SCM Finance
Transaction
Layer
Shared Data
Layer
Data Warehouse
Customers Sales Procurement Suppliers Operations Finance
Shared
Reporting
? Operations and financial information is shared
across the organization from same core data
Data Warehouses
ODS*
ODS ODS
Data Warehouse
Multi-Dimensional
Database (Cube)
*ODS = Operational Data Store
How is data consolidated?
? This is difficult!!!!!
– Data is often spread across multiple systems,
stored in different formats, and may even be
localized for different countries
Transforming Data
? Data must be transformed for consistency and
meaning
– Transformations may be as simple as copying columns or
may be incredibly complex
– Common transformations include:
? Hard-coded changes („T? to 1)
? Looking up values in a table (mapping a customer number
across disparate systems)
? Inserting dummy records and mapping them to unknowns
(inserting an „Unknown? customer)
Cleansing Data
? Data must be cleansed to be meaningful
– All companies have “bad” data in their systems
– Data may be missing
– Data may be inconsistent
– Data may be wrong
Data Warehouses
? ETL (extract, transform and load) processes
are needed to create data warehouses
– This is an arduous and technical process that can
account for a large percentage of a BI project
cost!!!!
Data Mining
Data Mining
? The process of identifying patterns in data
? Goes beyond simple querying of the database
? Goes beyond multi-dimensional database
queries as well
Data Mining
? Data Mining works for problems like:
– Develop a general profile for credit card customers
…
– Differentiate individuals who are poor credit risks
…
– Determine what characteristics differentiate male
& female investors.
Data Mining vs. Data Query
? Use data query if you already almost know
what you are looking for.
? Use data mining to find regularities in data
that are not obvious.
Data Mining Applications
? Fraud detection
? Targeted Marketing
? Risk Management
? Business Analysis
Origins of Data Mining
? Mathematics
– Statistics
– Numerical Analysis
? Artificial Intelligence/Machine Learning
? Computer Science
– Data Storage and Manipulation
How does Data Mining work?
? Uses induction-based learning:
The process of forming general concept
definitions by observing specific examples of
concepts to be learned.
How does Data Mining work?
What-Cha-Ma-Call-Its
NOT What-Cha-Ma-Call-Its
How does Data Mining work?
Which of these are What-Cha-Ma-Call-Its?
Data Mining Process
List of Customers:
-some bicycle buyers
-some not
Data Mining
Software
Model
List of Prospective Buyers
Model
List of Likely Buyers
Overview of Mining Strategies
Data Mining
Strategies
Supervised
Learning
Market
Basket
Analysis
Unsupervised
Clustering
Note: This representation is over-simplified and data mining strategies
are continually being invented.
More on our Curriculum
Skills
? Written communication
? Problem Solving
– Analytical
– Troubleshooting
? Software
– Microsoft SQL Server Management Studio
– SQL Server BI Development Studio
– SQL Server Reporting Services
– Pro Clarity
Delivery Methods
? Online: Distance Education, reaches wider
market
? Telecourse: tremendous effort to create, but
once created easy to deliver
– Televised, DVDs, online for homework, exams
? Hybrid: Meet once a week, the rest online
? On campus: evenings only
Delivery Methods
? Use of Camtasia for
– Software demonstrations
– PowerPoint lectures
? Pod casting
Certificates
? Business Intelligence Analyst (5 classes)
– Multi-dimensional analysis, data warehousing, data
mining, statistics, general business
– 2 quarters full-time/ 3 quarters part-time
? Business Intelligence Developer (4 additional
classes)
– Dimensional modeling, data visualization, multi-
dimensional II, data warehousing II (more
programming with SQL Server)
? Web site: www.bcc.ctc.edu/bi
Certificates
? Relational Database Analyst (6 classes)
– SA & D, programming, reporting, spreadsheets, db
theory
– 2 quarters full-time/ 3 quarters part-time
? Relational Database Developer (3 additional
classes)
– Programming, SQL, group processes
? Web site: www.bcc.ctc.edu/bi
Jobs
? Business Analyst
? Data Analyst
? Functional Analyst
? Marketing Analyst
Jobs
? Report Developer
? Data Modeler
? ETL Developer
? Data Architect
? Data Warehouse Designer
? Data Warehouse Developer
? Data Warehouse Administrator
? Database Administrator
Jobs
? Business Intelligence Consultant
? Business Intelligence Developer
? Business Intelligence Analyst
? Business Intelligence Project Team Member
Jobs
? One of the fastest growing segments of IT
? Less likely to be outsourced
? May exist in business units rather than IT
? Knowledge/understanding of the organization
is key
doc_998911412.ppt