Description
The goals of the project were to extend the notion of interaction, to exploit domestic activities and familiar settings, and to make the user environment part of the visual experience
DELIVERABLE
IST-2000-28298
ICE-CREAM
Deliverable D22
ICE-CREAM-phr-0404-01/Janse
April 2004
The ICE-CREAM Project
Final Report
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 3 of 71
Project Number : IST-2000-28298
Project Title : ICE-CREAM
Deliverable Type : Report
Deliverable Number : D 22
Internal Project Number : ICE-CREAM-phr-0404-01/Janse
Contractual Delivery Date : 03 04
Actual Date of Delivery : 04 04
Title of Deliverable : The ICE-CREAM project: Final Report
Contributing workpackage : WP0
Nature of Deliverable : Report
Author(s) : Maddy Janse (Ed.), all partners
Abstract
The ICE-CREAM project investigates how to make compelling experiences for end-users based on
enabling technologies for interactive media, such as DVB-MHP, Internet and MPEG-4. The goals of the
ICE-CREAM project are to extend the notion of interaction, to exploit domestic activities and familiar
settings, and to make the user environment part of the visual experience. The ICE-CREAM project
develops applications in which live events (for example a soccer match) are enhanced on consumer
demand with content that is created in real-time, broadcast streams are enhanced with content
available from the Internet, and objects in the home are part of the presentation. The application
prototypes show different levels of interactivity for end-users. Business modelling frameworks are
developed to specify the interchangeable roles of stakeholders (i.e., creator, producer, and consumer)
to the service.
Keyword list
Immersive TV, Broadcast, Internet, DVB-MHP, MPEG-4, XML, SMIL, Interactive media, interactive services, user
interaction, interaction tools, application prototypes, business models.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 4 of 71
Blank page
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 5 of 71
Executive Summary
The ICE-CREAM project has been investigating how to make compelling experiences for end-
users based on enabling technologies for interactive media, such as DVB-MHP, Internet and
MPEG-4. The goals of the project were to extend the notion of interaction, to exploit domestic
activities and familiar settings, and to make the user environment part of the visual experience. To
this end, the project has developed applications in which live events (for example a football game)
are enhanced on consumer demand with content that is created in real-time, broadcast streams
are enhanced with content available from the Internet, and objects in the home are part of the
presentation. The application prototypes showed different levels of interactivity for the end-users.
Business modeling frameworks were developed to specify interchangeable roles of stakeholders
and to determine factors for assessing the business relevance of the applications. Public
demonstrations of these applications have been made at IFA 2003 and IBC 2003. These
demonstrations were highly appreciated by the visitors, e.g., Time Warner, BBC. The thematic
and technical differentiation of the three ICE-CREAM applications proved to be very successful in
demonstrating the technical and conceptual possibilities. They integrated different technologies
and demonstrated the potential for opening new avenues for services and platforms. Furthermore
they showed convincing evidence for migration of MPEG-4 and DVB-MHP, for Internet
interconnectivity and integration of 3D graphics technologies. The project adapted to trends in the
outside market and in the research world by explicitly working on applications that illuminated how
the addition of Internet can create new opportunities for applications and services.
User appreciation of applications is at the heart of the ICE-CREAM project and providing
consumers with interactivity in the applications is one of the driving forces. We operationalized
qualitative terms like interactivity, experience and appreciation in such a way that they could be
used to assess the applications. Different methodologies were used to obtain user feedback and
input. Users perceived our concepts as very promising and interesting. The results also showed
that there is still a long way to go with regard to development of methodologies for evaluation, the
design of assessment instruments, the exploitation of the possibilities for interaction with content
by users in their environments, at home or on the move, and the exploration of ambiance effects
in synchronization with events in the content.
Since content is a crucial and extremely valuable asset for any application, the project
investigated the possibilities for multiple usage of content by giving users different views on it and
by extending the interaction to the user’s physical environment and by distributing it over different
devices. The fact that media objects in a movie may also be served from the Internet, can provide
all kinds of opportunities for convergence or competition between Internet and broadcasting
services. Interaction with 3D virtual environment and objects in a broadcasting context is another
venue for new applications and businesses. The idea of using physical objects (such as lights
and toys) to connect the virtual world with the reality is an interesting concept for game-like
applications, but above all for creating a surround experience and affecting people’s perceptions.
The development of a personalizable MHP service with mobile extensibility may lead to a fruitful
exploitation of future 3G mobile services.
Three applications were developed in the project. Their characteristics can be described as
follows:
a). Live video enhanced with real-time events. Hot and cold contextual information is provided
during a football game. Highlights, video-clips and interactive 3D animations of goals and game
situations can be replayed on-demand. This application represents a familiar topic and context for
the user with regard to interaction and content consumption. The user interface is implemented
on a Trimedia set-top box, with a familiar remote control, trusted TV appearance, and live
broadcast. The application achieved unique propositions for users, such as:
• Enhancements with real-time events, contextual information and highlights available on
demand,
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 6 of 71
• Interactive 3D animations based on field measurements for replay, reconstruction and
analysis of the game,
• Changing viewing positions, from the player’s position, following the player and from behind
the player,
• Graceful aging of enhanced content presented in the user interface,
• Overviewing the game from above and measuring positions.
b). Video events enhanced with related content. Enhancements can be directly connected to
events in the video or accessed independently. This application represents a familiar topic and
context for the user, i.e., TV travel documentaries/programs. The interaction, however, is more
familiar to users of Internet and mobile devices as the additional information is linked to the video
content and can be ported for use on the move. The user interfaces were adapted to each device,
i.e., set-top box, connected PDA and 3G mobile phone. The application achieved unique
propositions:
• For users: access to content enhancements in relation to the video content and specific
scenes, e.g., attractions, what’s on agendas, quiz participation, ticket booking,
announcements, SMS messages, user preferences, tour lay-outs and suggestions.
• For providers: flexible and easy customization of content and services, content-centric
services, different means for publishing and presentation of content, integration of different
service platforms.
c). Video enhanced with ambiance events. Content is presented in a non-linear fashion to enable
multiple users to watch different facets of a story. Lights and toys are used as interaction devices.
This application is rather unfamiliar for the average TV viewer. Elements of this scenario,
however, are familiar for people who play computer games, visit theme parks and explore 3D
virtual reality environments. The application achieved unique propositions for users, such as:
• Ambiance effects with room lights to support the suspense of the scene,
• Different content elements displayed on different screens and enabling challenging and playful
situations,
• Different modes for interacting with the scenes, i.e., discovery, game, drama,
• Different interaction devices, interactive displays, toys,
• Role-play when multiple users are participating.
The results of the ICE-CREAM project are very promising for the strategic crossroad of the
converging worlds of TV and Internet. The project showed the importance of multidisciplinary co-
operation between different stakeholders as demonstrated by the composition of the consortium.
The advantages of integrating different technologies and the use of user-centered methodologies
for application development were demonstrated. It is recommended to implement the integrated
platform, to generalize the applications to other domains and topics, and to ensure the availability
of easy-to-use authoring and editing tools.
Flexibility and adaptability to changes in environment, in user demand and in user behaviour were
the driving factors for assessing business models and for analysis of the three ICE-CREAM
applications. Marketing factors, like product range, customer value, positioning, were identified
and compared for each application scenario.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 7 of 71
Table of content
1. INTRODUCTION...............................................................................................................9
2. ICE-CREAM APPLICATIONS .........................................................................................11
2.1. LIVE VIDEO ENHANCED WITH REAL-TIME EVENTS (FOOTBALL APPLICATION) ........................12
2.2. VIDEO ENHANCED WITH RELATED CONTENT (TRAVEL APPLICATION) ..................................13
2.3. VIDEO ENHANCED WITH AMBIANCE EVENTS (DEEPSEA APPLICATION).................................15
2.4. COMPARISON OF THE THREE APPLICATIONS......................................................................18
3. PLATFORM, TECHNOLOGIES AND TOOLS..............................................................19
3.1. BROADCAST ORIENTATION – CASE 1 ...............................................................................21
3.2. INTERNET ORIENTATION – CASE 2 ....................................................................................25
3.3. MOBILE ORIENTATION – EXTENSION OF CASE 1 AND CASE 2...............................................26
3.4. FUTURIST ORIENTATION – PC MEETS TV.........................................................................29
3.5. CLIENT PLATFORM...........................................................................................................30
3.6. COMPONENTS IN THE PRODUCTION CHAIN. ......................................................................30
3.7. TOOLS............................................................................................................................32
4. USER FEEDBACK...........................................................................................................38
4.1. EVALUATION OF THE FOOTBALL APPLICATION....................................................................38
4.2. EVALUATION OF THE TRAVEL APPLICATION .......................................................................42
4.3. EVALUATION OF THE DEEPSEA APPLICATION.....................................................................44
4.4. SUMMARY........................................................................................................................51
5. BUSINESS FRAMEWORK..............................................................................................52
5.1. LEVELS OF INTERACTIVITY................................................................................................52
5.2. PERSONALIZATION AND FAMILIARITY..................................................................................54
5.3. BUSINESS RULES AND IPR..............................................................................................58
5.4. EXTERNAL MARKET CONDITIONS AND CHANGE..................................................................60
5.5. RECOMMENDATIONS........................................................................................................61
6. CONCLUSIONS...............................................................................................................63
7. REFERENCES.................................................................................................................65
8. PROJECT CONSORTIUM.............................................................................................67
9. PROJECT DELIVERABLES...........................................................................................68
10. PUBLICATIONS, PRESENTATIONS AND DEMONSTRATIONS...........................69
10.1. PAPERS AND PRESENTATIONS .....................................................................................69
10.2. DEMONSTRATIONS AND EXHIBITIONS............................................................................71
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 8 of 71
Blank page
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 9 of 71
1. Introduction
ICE-CREAM – Interactive Consumption of Entertainment in Consumer Responsive, Engaging &
Active Media
The ICE-CREAM project was a European Commission sponsored two-year project lead by
Philips. The project consortium partners were Philips Research (in the Netherlands and in
France), Bitmanagement, de Pinxi, Euskaltel, Fraunhofer FOKUS, Imperial College London, NOB
cross media facilities, Symah Vision, and Tomorrow Focus.
The goals of the ICE-CREAM project were:
• To investigate how to make compelling experiences for end-users based on the possibilities of
integrating technologies for interactive media, such as, DVB-MHP, MPEG-4, and Internet
technologies.
• To extend the notion of interaction, to exploit domestic activities and familiar settings and to
make the user environment part of the visual experience.
The basic ingredients for the ICE-CREAM project were: platform and tool technology, business
model development, manipulation and enhancement of content, and interaction possibilities for
end-users. They reflect the challenges and directions that we see as crucial for the development
of applications and technologies based on the development of open standards and platforms
which elaborate on the integration of DVB-MHP, MPEG-4, Internet technology and 3D graphics.
Business models are an intrinsic component for the development of applications and
determinants for their success. The quality of the content, how the end-user can use it and
whether this can be done in a fun, easy and appropriate way, however, are the most essential
ingredients for end-user applications. Three applications were developed in the project. They were
chosen such that they covered a wide range of technical, business, social and conceptual
domains.
Converging technologies make it possible for new application and service scenarios to emerge.
Multimedia content is becoming a commodity and the roles of producer and consumer are
becoming interchangeable and dynamic. Consumers are becoming content creators who can
publish and share content, interact with it and present and store it on all kinds of devices.
Consumer electronics devices with a connection to the Internet offer users new ways of
interaction with content and communication with one another. These possibilities pose enormous
challenges for open standards and rights management, for interoperability between competing
businesses, and for defining appropriate business models. But above all, of identifying,
communicating and explaining what the benefits are for consumers, the end-users of these new
applications and services.
For example, Internet connectivity gives people unlimited possibilities for acquiring information and
enjoying all kinds of entertaining applications. But, this assumes that people are accomplished
‘do-it-yourselves’ and that they are always prepared and in the mood to ‘find, shop, design,
compose and create’ their own content and entertainment. This is not necessarily the case, as
we also know from other domains. For example, the same people that love to cook five star
dinners also enjoy the fast-food take-out and their decisions are based on zillions of different
reasons. Another major assumption made for future applications that are enabled by Internet
connectivity, such as, interactive television, networked gaming, on-line communities, multi-users
portals, is that they will enhance the shared media experience of people. This might be a valid
assumption from the technical point of view. From the behavioural point of view, however, it is
rather provocative. Questions such as: What does sharing media/content mean for people? To
what extent are people’s expectations and goals met? Does it hold for all content? Do people
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 10 of 71
really need to share content to increase their experience or is it only a technical requirement?
need to be considered. Furthermore, we have to take into account that activities like television
viewing, making phone calls, sending messages, watching photographs, going to the movies or
the ball game, are already shared experiences. In other words, human behaviour needs to be
taken into account to consider the possibilities of enriched and interactive applications.
Broadband connections make it possible for people to share all kinds of digital content. People are
very familiar to sharing tangible objects, consumables and experiences. They have well-
established social rules for these activities, which determine property ownership, initiation, turn
taking and so on. These rules are not so clear-cut for sharing digital content or for systems in
which people can have a video chat. As connectivity rises between devices in (different)
environments, the nature of social interaction between people changes. Systems will more and
more become enablers for social interaction and we need to start thinking about introducing
technology as a social mediator. These developments raise many questions with regard to
human behaviour and perception that application development has to take into account.
The ICE-CREAM project adopted a user-driven development approach. Application scenarios
were developed which comprised all ingredients, i.e., platform and tool technology, business
model development, content manipulation and enhancement, and interaction possibilities for end-
users. These scenarios functioned throughout the two-year project as the principal
communication mechanisms for maintaining focus and for ensuring that user objectives were at
the heart of the applications. The next chapter describes the applications that were developed in
the ICE-CREAM project. The applications will be described from the user perspective. Chapters
will follow on platform and tool development and implementation, user feedback and business
frameworks.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 11 of 71
2. ICE-CREAM applications
Established interactive TV services like EPG, shopping, and super teletext services often exist
independently from the current TV program. The video broadcast and the additional services just
happen to be presented on the same device, but are rarely integrated to achieve an enhanced
viewing experience. Enhanced broadcast often means a coexistence of various services, which,
for the viewer, only provides the benefit of the sum of the available services, instead of allowing
the services and the video broadcast to support each other and provide a benefit that is higher
than the sum of the individual benefits. To achieve this added value, all ICE-CREAM applications
were designed to integrate video content and additional services closely.
The ICE-CREAM applications explore the possibilities of using rich interactive content and
integrating different technologies. Four dimensions exemplify the relation between the applications
and the challenges that they address. They reflect the directions that we took as crucial for the
development of applications and open standards technology. The four dimensions are presented
in the diagram in Figure 1. The first dimension is determined by the technology for platform and
tools, i.e., the migration of DVB-MHP and MPEG-4, the integration of this platform with Internet
technology and the integration of 3D graphics to the platform. This is the left axis in the diagram of
Figure 1. The second dimension is determined by the interaction possibilities for the end-user.
This is the right axis in the diagram of Figure 1. The variables that are most important are: the
levels of interactivity that the user can have, from bi-directional remote control, to content editing,
to ambiance and emotion creation. The third dimension constitutes the development of the
business models. This is the top horizontal axis in the diagram of Figure 1. The hypothesis is that
for interactive media applications the business models will evolve from static to dynamic models.
The fourth dimension constitutes the possibilities for content enhancement and manipulation. This
is the bottom axis in the diagram. The applications addressed these dimensions, i.e., the
interaction requirements for the users and the potential of the platform, the use of rich interactive
content for real-time events, enhancement and personalization of content from Internet, the use of
non-fictional content for creating ambiance and experience. The arrows in the diagram reflect the
order of complexity, i.e., challenge and novelty for the applications. The project applications that
will be addressed are positioned along the diagonal. The focus for this positioning is the level of
interactivity and emotional involvement for the end-user, i.e., at the client side.
DVB-MHP
MPEG-4
Internet
3D Graphics
DVB-MHP
MPEG-4
Internet
DVB-MHP
MPEG-4
Enhancements
Dynamic
Edited content
Fictional content
Multi -layered
RC+
Messaging
Scene control
Emotion
experience
P
l
a
t
f
o
r
m
,
T
o
o
l
s
,
T
e
c
h
n
o
l
o
g
y
Content Manipulation & Enhancement
I
n
t
e
r
a
c
t
i
o
n
f
o
r
U
s
e
r
s
Business Models
Static
Dynamic
A
p
p
l
i
c
a
t
i
o
n
s
Figure 1: Dimensions for the selection of applications
Three applications were developed in the ICE-CREAM project. These applications can be
characterized by the properties of their contents and enhancements:
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 12 of 71
• Live video enhanced with real-time events
• Video enhanced with related content
• Video enhanced with ambiance events
2.1. Live video enhanced with real-time events (Football application)
In this application a live football match is broadcast and enhanced with commentary and
annotations of player actions and additional hot and cold statistics. Selected scenes can be
replayed in video clips or in 3D animated form. This application represents a familiar topic and
context for the user with regard to interaction and content consumption. The application shows
the possibilities and benefits of integrating DVB-MHP, Internet, MPEG-4 and 3D graphics and
animations. This integration is especially appropriate for enhancing the broadcast of live events,
which are characterized by existing action, and timing.
The major characteristics of this application are:
• Integration of Internet, MPEG-4 and MHP in TriMedia set-top box
• Delivery of real-time interactive multimedia event clips generated by tracking tools and
cameras on the field
• Presentation of video events on-demand
• Combination of background "cold" and timely "hot" contextual information during the football
game
• Playback of the event clips while watching the program
• Replay of interactive 3D animations of goals and game situations on-demand
User walkthrough
Users watch a football game on TV. The TV broadcast is enhanced with background information
about this game (cold statistics). This background information contains, for example, details about
the players, about the team and championship achievements. Furthermore, when a remarkable
event occurs in the game, the multimedia information about this event is sent to the set-top box –
‘hot statistics’. These events are, for example, a goal, an offside, a pass. The set-top box filters
this incoming information and displays it if the user demands it. A special icon that indicates the
presence of on-demand content is displayed on the TV screen when either background or real-
time event information is available. The user can decide to see this information or not.
Background information about the game (cold statistics) and the real-time events as particular
game scene replays (hot statistics) can be displayed. The information of the real-time events (hot
statistics) is encapsulated in a data-packet that is broadcast. For example, the commentator
describes an action and links it to a certain player. This information is put in the event data-packet.
Background information (cold statistics) can be accessed anytime during the game via the menu.
This means that the viewer can access information about, for example, all players in the game.
Real-time event information, i.e., video scenes of specific events, can be replayed during the
game. These event-packages pile up while the game goes on and become unavailable
(disappear) after some time, unless the viewer has marked them as ‘favourite’ on the set-top box.
Positions of the players are measured during the game by means of image processing using at
least one camera on the actual site of the football match. Users can interactively explore
significant scenes in a 3D environment. They can individually select a point from which to watch
the replay, view it from the position of the goal keeper, referee or other player, forward the scene
events step by step, measure distances, and do other things. Figure 2 shows events that are
added to the favourites and the playing of a replay. Figure 3 shows a 3D replay from different
points of view.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 13 of 71
Figure 2: Event added to favourites and
playing of replay
Figure 3: 3D replay –field view
2.2. Video enhanced with related content (Travel application)
In this application a travel video is enhanced with specific information about the various attractions
presented in the video and integrated with other travel related information, which is either part of
the broadcast or retrieved from the Internet. This application represents a familiar topic and
context for the user, i.e., TV travel documentaries/programs. The interaction, however, is more
familiar to users of Internet and mobile devices as the additional information is linked to the video
content and can be ported for use on the move. The user interfaces were adapted to each device,
i.e., set-top box, connected PDA and 3G mobile phone.
The major characteristics of the application are:
• For users: access to content enhancements in relation to the video content and specific
scenes, e.g., attractions, what’s on agendas, quiz participation, ticket booking,
announcements, SMS messages, user preferences, tour lay-outs and suggestions.
• For providers: flexible and easy customisation of content and services, content-centric
services, different means for publishing and presentation of content, integration of different
service platforms.
User walkthrough
Initially the users watch a travel video without any additional features on an MHP set-top box. As
soon as additional services that are related to the video content become available, a window
appears on the screen with the information. Figure 4 shows some of these screens. Users can
specify their preferences, start and stop the services explicitly or automatically. Table 1 shows the
services that users have available.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 14 of 71
Switching a service on
Setting user preferences
Quiz question
Chapter markers and video bookmarks
Figure 4: Travel application: services
Table 1: Services available to users of the Travel application
Service Description
Quiz While a timer is running, users can select answers with the
numeric keys on the remote control for multiple-choice
questions that are presented at specific moments in the
video. The correct answer is highlighted after time-out and
users receive a score if they had the correct answer.
Attraction information A menu shows additional information for attractions in
relation to the subjects in the video. Users can view,
bookmark and store this additional information by navigating
the menu with the colour keys of the remote control.
SMS and other
messages
Users can send SMS messages to a special phone number
to be displayed on the screen. Broadcasters can also attach
messages to the video, which will be displayed in the same
way as the SMS messages. These messages contain
relevant information about the video and its attractions.
City tours While the main video runs in a PIP (Picture in Picture)
window, predefined city tours are presented as short
teasers, i.e., concise overview of the tour theme. Users can
change between main screen and PIP. Users can select
and access these tours with the arrow keys on the remote
control, bookmark information about specific attractions and
create new city tours that will be added as Personal Tour to
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 15 of 71
the list of available tours. Book-marked information is also
sent to the mobile device.
Chapter markers and
video bookmarks
Each topic in the main video has a marker event. These
events or bookmarks are listed and displayed as chapter
names. They can be used to find and replay specific
sections of the video. User’s bookmarks are also listed as
markers.
Ticket booking Users can browse, select and book tickets for events that
occur in the attractions that are shown in the video. The
booking system queries and displays upcoming events. The
booking information is also transferred to the mobile device
to have it available during the actual travel.
Applications on mobile
device
Users can select cities of interest on their connected PDA
and access related services during travel from the city home
page, for example, sightseeing information, attractions,
upcoming events, ticket booking services, and personal
tours.
The location based services include, e.g., a “Where am I?”
function, which displays a map of the surroundings and
nearby attractions for a given street address, a display of the
route from the current position to a given destination.
2.3. Video enhanced with ambiance events (DeepSea application)
In this application a video is enhanced with fictional content, which uses 3D graphics and
animations. Video streams, 3D computer graphics, text, still pictures, soundtracks and
multilingual interfaces are combined. The interactive content is distributed to multiple devices
(lights, portable displays, robotic toys) to create ambience effects in the performance space – the
end user's environment.
The major characteristics of this application are:
• Distributed interfaces, which enable co-operative interaction with the movie among multiple
users.
• Synchronized lighting effects and robotic behaviours connecting the virtual 3D world to the
reality.
• Grabbing and storing of content elements for later reviewing by using a portable display.
• Providing users with a range of interaction possibilities, such as customisation of the journey,
interaction with content elements, and full control of navigation.
User Walkthrough
The fictional content of the application concerns a submarine adventure which has three different
interaction modes: the automated (drama) mode, the discovery mode and the game mode. The
content is composed of an undersea area with plants, some fifty different species of fish, a few
buoys, and four shipwrecks as archaeological sites, and a submarine as exploration vessel.
Users may discover these objects and interact with them. The boundary of the virtual space is
indicated with rocks. If the submarine hits the boundary, it bumps on the rocks and stagnates,
while a caption appears on the large display saying “NO WAY”. The surface of the water and the
bottom of the sea are also the boundaries. The presence of the submarine is linked to the
presence of a toy submarine. If the toy submarine does not exist or has been moved away, the
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 16 of 71
submarine in the virtual 3D space disappears and a cursor frame is presented instead. The 3D
scene is captured from right behind the submarine and the camera angle follows the direction of
the submarine. So the user can always see either the backside of the submarine or a cursor
frame from a first person point of view. When the user changes the direction of the submarine,
the camera angle changes accordingly.
Table 2 gives an overview of the interaction modes and describes what the user will see and can
do in these modes.
Table 2: Interaction modes in the DeepSea application
Interaction mode What the user can see and do
Automated mode (no control by the user).
This is the default mode in which the user
watches the show without any interaction
with the content. It automatically starts
when the user does not make a choice
among different modes.
This mode shows a panoramic view of
the 3D undersea space. The submarine
moves slowly in the space following an
automated route and the scenes look
smooth. This mode lasts about 30
seconds. It is followed by the game
mode, which commences automatically.
Game mode (limited navigation control).
This mode allows the user to move the
submarine in only two directions, left and
right, to navigate around obstacles, for
example, mines. The submarine drives
ahead in a fixed speed and the user
cannot change the overall direction or
speed of the submarine.
The goal for the user in this mode is to
avoid obstacles by manoeuvring the
submarine to the left or right. The
feedback on failure consists of light
effects that accompany the explosion
when a mine is hit, or of stagnation and
bumping of the submarine when it hits an
obstacle. This mode lasts about 30
seconds and ends with loud alarm
sounds and flashing red lights, after
which the submarine is out of the user
control and floats to the surface through a
narrow tunnel. The user receives a score,
which correlates to the performance of
avoiding the mines.
Discovery mode (full navigation control).
This mode allows the user to freely
navigate in the virtual space by controlling
the direction and speed of the submarine.
The user can move the submarine
forward, left, right, up and down at
different speeds, but not backward.
The goal of this mode is to wander
around at a preferred speed with the
freedom of direction. The user can try to
‘catch fish’ by moving the cursor frame or
submarine towards any of the fish in the
scene. When close enough, the fish is
caught, the picture of which is presented
on a secondary display (iPronto
interaction device) and stored for later
review. This activity is solely based on
user initiative. The user can also enter
shipwrecks. This track last about 4
minutes and ends in the same way as the
game mode.
The user starts the DeepSea application (Figure 5) by pushing a start button on the touch screen
of the interaction device (iPronto). On the display panel appears a submarine sailing on a sea and
then diving into the sea, while the background music starts. When the submarine arrives at the
bottom of the sea, the viewer sees a small TV set in the scenery, which turns on and shows a text
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 17 of 71
that cues the viewer to choose a language (English, Dutch or French). The user selects a
language on the iPronto by touching a button on its screen. Then, the small TV displays the three
modes and suggests the user to select one by pushing the appropriate button on the iPronto
graphical user interface. If the user does not choose a mode by giving a response on the iPronto
within a certain time, the automated mode (the default mode) starts. All three modes have a time
limit. The automated mode lasts 30 seconds, the limited-interaction mode 30 seconds, and the
full-interaction mode 4 minutes. After the time limit has been reached, the user looses control over
the content object and the submarine float to the surface through a tunnel. The graphical user
interface on the iPronto allows the user to switch the presentation from one mode to another
during any of the modes, and the new mode starts from its initial position. The user can also
customize the interface language and restart the entire application anytime after opening.
Figure 5: DeepSea application: walkthrough
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 18 of 71
2.4. Comparison of the three applications
From the end-user point of view, the use of content and the interaction possibilities are the most
interesting characteristics. These characteristics are summarized in Table 3. Table 3 also
illustrates the span of possibilities covered by the three application scenarios.
Table 3: Comparison of the three ICE-CREAM applications
Characteristics Football
application
scenario
Travel application
scenario
DeepSea
application
scenario
External devices for
interaction
Remote control
Remote control
Connected PDA,
3G mobile phone
Remote control
Interactive display
Toy, Light
Intended group size Single user Single user Group
Creation of content Live Pre-generated Pre-generated
Theme Based on fact Based on fact Based on fiction
Video content Existing Existing Custom made
Synchronization
between video and
additional content
< 1 sec + 1 sec N.A.
Video streams Multiple streams Single stream Single stream
Content
presentation
2D and 3D 2D 3D
The three applications will be worked out in the next sections.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 19 of 71
3. Platform, Technologies and Tools
This chapter describes the terminal software architecture of the systems that were developed for
the applications of the ICE-CREAM project.
The overall purpose is to enhance a broadcast program with interactivity in a personalized
manner. The Internet is used to convey the personal data. The trigg&link paradigm is used to
present the enhancements to the user [1], hence introducing the interactivity. Figure 6 depicts the
overall scenario for the platforms.
Control (XML)
Content
(enhancements)
User
Data base
enhancement
Data base
enhancement
MPEG - 4/SMIL
docs
Content
Provider
Content
Provider
Internet
Main video (MPEG - 2, MPEG - 4)
Triggers
Alerts
Broadcaster
Figure 6: The general scenario context.
Three distinct systems are linked to gain the required functionality: The “Uplink system” (Server
side), the “Transport system” (Network) and the “End-user system” (Client side). The “Server
side” is shown at the left side of the picture. The “Network” is represented by the arrows, and the
“Client side” is represented by the User-box. The main video is produced by a content provider
and transmitted by a broadcaster in the usual way. The broadcast stream is DVB compliant. It is
based on MPEG-2 transport streams and supports DSM-CC private sections. The receiver is an
MHP set-top box.
The focus of the platform for the sports application lies on the production and distribution of the
live events and instant replays. Those live events and replays can be distributed via the Internet
and/or via the DVB broadcast. An abstract overview of this environment is depicted in Figure 7.
The outgoing and bi-directional arrows to the right are linked to an ICE-CREAM terminal via a
direct connection to the Internet (broadband interaction channel) and terrestrial, cable or satellite
receiver (DVB tuner).
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 20 of 71
Production
of the
Annotations and
Video-clips
Enhancements
Video-on-
Demand
Distribution
AV + MHP
Production
DVB Broadcast
Trigger
distribution
Management of
Enhancements
Production Distribution
Internet
DVB
Figure 7: Production – Distribution chain
The same content provider can produce the enhancement application and its enhancement data,
or it can be produced by another, or by several (independent) content providers. We distinguish
between two cases. In the first the enhancement and its interactivity is provided and handled
completely by the broadcaster. In the second case the broadcaster provides the main video, but a
third party offers the enhancement service over the Internet.
Typically, in the first case the broadcaster creates a multiplexed stream containing the
enhancement application (an Xlet) and the various content items that are of general use to the
enhancement. The broadcaster may use the Internet as communication channel for the
personalized interactivity. A typical application example of this case is the Soccer Application with
(3D) MPEG-4 replays.
In the second case, being Internet oriented, the enhancement application is typically using XML
documents. More precisely it is often a SMIL document [2]. The third party has knowledge about
the main video, so that the enhancement and interactivity provide a coherent experience to the
user. The client at the receiver side co-ordinates the integration of main video and interactivity,
which thus often is a SMIL player.
The next step is to enable mobile access to the Internet and broadcast content via e.g. a GPRS
connection. The possibilities of the mobile case are demonstrated in the Travel Application.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 21 of 71
3.1. Broadcast Orientation – Case 1
The first case of the scenario, i.e., the broadcast approach, is represented by the implementation
of the football application. In this application, a live soccer match is enhanced with instant replays,
coded in MPEG-4 format, including 3D graphics. The broadcaster creates the clips, publishes
them on a server on the Internet, and notifies the viewers about their availability through the
trigg&link paradigm. The viewer can replay the clips by downloading them over the Internet. The
clips are prepared and encoded in MPEG-4 format. The receiver contains an MPEG-4 (3D) player.
In this way the viewer can interact with the replay, ultimately choosing different viewing angles.
Figure 8 shows the set-up.
SetTop Box
Internet
DVB-T/C/S
Live Content
Production
MPEG-4
Video on
Demand Clip
Server
DSM-CC Object
Carousel
MPEG-4
Encoder
MPEG-4 Clips
Clips
MPEG-4 Clips
MPEG-2
Encoder &
Multiplexer
Xlet +
Hot &
Cold
Stats
Live Audio &
Video
Carousel +
Events
Events
TCP/IP
Figure 8: Live event with interactive replay.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 22 of 71
The football application is an MHP (Java) Application. This “Xlet” is broadcast in a DSM-CC Object
Carousel: all the Java classes and assets (like the text and pictures used in the cold statistics)
are repeatedly put “on the air” like teletext pages. This carousel is merged with the live audio and
video of the soccer match and broadcast. The MHP terminal (with MPEG-4 replay capabilities)
receives the broadcast and loads and executes the Xlet. When the Xlet is started, it starts
“monitoring” for events (received via Internet or DVB) and it allows the user to interact with it. The
events and clips are generated via “live content production” facilities. The clips are encoded into
MPEG-4 and stored on the server(s) and the corresponding events are multiplexed in the
broadcast and/or sent via the Internet (subscription-based). The live video stream is also analysed
in real-time by Symah Vision tools: The positions of the players are tracked by means of image
processing and the data is stored in XML format. Once the XML description of the position
updates has been retrieved, the 3D content can be reconstructed in an automated way in real-
time to generate the 3D MPEG-4 clips. Figure 9 shows an excerpt from the corresponding XML
code.
<BDD_TRANSFO Index="406" nbItem="404">
<TRANSFO Index="1" TC="32420" K="0" Cx="360" Cy="288" AR="0.9375" IsValid="1">
<H i0="35.90076" i1="21.58959" i2="-1523.892" i3="-4.081663" i4="6.805422" i5="2.290844"
i6="8.287122E-03" i7="-1.377915E-02" i8="1" />
<Hinv i0="2.056601E-02" i1="-1.779707E-03" i2="31.34445" i3="1.233495E-02" i4="0.145979"
i5="18.46273" i6="-4.678071E-07" i7="2.026215E-03" i8="1" />
</TRANSFO>
<TRANSFO Index="2" TC="32422" K="0" Cx="360" Cy="288" AR="0.9375" IsValid="1">
<H i0="35.90015" i1="21.59887" i2="-1524.545" i3="-4.080273" i4="6.80006" i5="2.727768"
i6="8.290958E-03" i7="-1.377944E-02" i8="1" />
<Hinv i0="2.057967E-02" i1="-1.780217E-03" i2="31.3795" i3="1.234871E-02" i4="0.146094"
i5="18.42765" i6="-4.668669E-07" i7="2.027854E-03" i8="1" />
</TRANSFO>
Figure 9: XML positioning information for 3D replay
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 23 of 71
For broadcasting MPEG-4 clips of a 3D replay, the 3D MPEG-4 creation tool (Bitmanagement)
has been extended with a Video generating function. (This is useful for set-top boxes that are not
capable of rendering the 3D graphics themselves). After loading an interactive 3D or 2D MPEG-4
scene and adjusting data such as the position from which to view the scene on the generated
video the user presses a hot key. This hot key switches the MPEG-4 player into video capture
mode. In video capture mode, the MPEG-4 player stops rendering the scene and presents the
user with a list of available CoDecs (see Figure 10). The user can select a CoDec, for example
"XviD MPEG-4 Codec". There is an Option button that allows for adjusting CoDec specific
parameters like compression quality and bandwidth (see Figure 11).
Figure 10: Video capture mode
After the user clicks on "Ok", the MPEG-4 player begins with rendering the scene. Each frame is
captured from the video memory and sent to the selected CoDec for compression. In this phase
rendering takes place with maximum speed and with a constant time increment between the
frames. Thus the resulting video always has the correct frame rate, even if the available
computing power is not enough to render and compress the scene in real-time.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 24 of 71
Figure 11: Adjusting compression quality
For enabling online use on PC hardware the 3D replay content can be embedded in a HTML page
in the MPEG-4 format and put on a web server to be retrieved from the Internet (Figure 12).
Figure 12: Embedding the replay in a html web-page
For environments where bandwidth is very limited, e.g. on mobile devices, a 2D version of the
soccer goal replay can be made. The bandwidth constraints for retrieval of the whole animated
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 25 of 71
scene amount to a few kilobytes (Figure 13).
Figure 13: 2D replay output for mobile devices
3.2. Internet orientation – case 2
In the second case of the scenario we show a prototype where SMIL is used to integrate the video
broadcast stream with additional, personalized interactivity offered by a service provider on the
Internet. The interactivity is offered in the form of a SMIL document [2]. Each SMIL document is
tailored to the preference profile of the user who is requesting the enhancement service, much
like HTML documents are adapted to settings or parameters in a query that is requesting the
document. In this way personalization is realized.
In the prototype we use the broadcast of a classical concert as the main video. Enhancements
are offered through the trigg&link paradigm, which include additional information about the
composer, the conductor (e.g., his homepage), the concert hall (e.g., providing calendar and
ticket ordering), etc. Personalization appears in the form of offering calendar information that is
local to the geographic place from where the viewer is watching. This implementation shows the
feasibility of using SMIL for this purpose and of integrating the SMIL player with the MHP platform.
Figure 14 depicts the architecture.
Personalized part
Generic part
MHP
SMIL
doc
Timing Layout
Xlet: SMIL interpreter
SMIL profile
MHP APIs
Figure 14: Integration of SMIL presentation with MHP.
The SMIL interpreter parses the SMIL document and takes care of the control of such matters as
timing and layout. Control of the MHP box is realized through an Xlet. Since the enhancements are
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 26 of 71
typically shown as an overlay on top of the main video, a new user interface problem has been
introduced. It will be hard to design the overlay in such a way that the obscuring of the main video
does not annoy the user. A straightforward solution would be the presentation of the
enhancements on an additional, separate screen. Of course, the user premises must include
such an additional, personal screen. In particular when multiple users are watching the main video
and each of them is interested in another piece of enhancement content, the use of additional
screens becomes paramount.
The above architecture of the SMIL document, Figure 14, lends itself to such a distributed
extension in a fairly easy way. In case the additional screen is present, another SMIL document is
loaded to the user. It is also conceivable that a single document is created that declares both
situations, in which case the interpreter distinguishes between the forms of presentation. The
layout part of the new SMIL document is modified to declare the layout of content items over
multiple devices. Secondly, the SMIL interpreter is (logically) separated from the MHP player and
takes control over both the MHP player and the additional screens. In this way, synchronized
presentation and interaction is enabled over a distributed configuration of devices. Figure 15
depicts this extension, which is implemented in the current prototype.
Personalized part
Generic part
SMIL
doc
Timing Layout
Xlet: SMIL interpreter
SMIL profile
Distributed
Interface
MHP Additional screen Additional screen
Figure 15: Using SMIL for distributed presentation.
3.3. Mobile orientation – extension of case 1 and case 2
The mobile orientation gives an extra dimension to the two cases described in the scenario. End-
users are now no longer required to consume the interactive content at home; they can have it
anywhere, be it in limited form due to mobile terminal and bandwidth restrictions. The Travel
Application in which video content is enhanced with related content shows these benefits.
The target terminal for the travel application is a standard MHP set-top box with an Internet
connection, which is used for event booking and additional information retrieval facilities, and
additional local storage to provide PVR-like functionality. The PDA is connected to the Internet
using a GPRS connection. Figure 16 shows the production chain of the travel application.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 27 of 71
Figure 16: Production chain of the Travel application
The internal structure of the application that is running on the MHP set-top box is based on a core
module to which travel application specific modules (event booking, city tours) and re-useable
modules with modifiable layouts (e.g., quiz, SMS) are added. The application itself is multiplexed
with the video stream, some parts of the context and the layout descriptions for the re-useable
modules into a broadcast transport stream. The editing tool creates an XML file, containing all
timing information needed for the trigg&link functionality of the attraction information and for other
elements that are synchronized to the video (e.g., chapter bookmarks). This information is then
used by the multiplexer to insert the appropriate stream events into the transport stream. Content
that is not inserted into the video stream is retrieved via the Internet connection. This concerns
mostly image content that will only be used by a small percentage of the users. The Internet
connection will also be used for interactive services (e.g., connection to the event booking
system). If possible, all content (broadcast and that retrieved via Internet) will be stored locally on
the set-top box. Content provision is central to this implementation. Figure 17 depicts the provision
of content and services by different providers.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 28 of 71
MHP
Set-top Box
M
o
b
i
l
e
P
D
A
Service
Provider
Content
Provider
Broadcaster
SMS
On-line
services
Broadcast
additional
information
Services
Additional
content
Information
On demand
Personalization
Figure 17: Content providing in the travel application
The content used in the travel application is retrieved from various sources. Parts of the content is
created using the tools that were developed within the project, other parts are distilled from
existing sources prior to runtime and some parts are retrieved at run time. The layout or ‘look’ of
the travel application on the set-top box is neither coded in the application nor defined by content.
The layout is defined in separate layout definition files, which are generated with a task specific
layout editor and combined with a specific travel video by the application editor. The following
sections describe the flow of information between the components that make up the environment
of the travel application.
Attraction database for MHP application: The attraction information that accompanies the video
presentation constitutes the core content of the MHP city guide. It is located in a CityGuide
database, which contains approximately 6000 attraction descriptions from a large number of
cities. There are approximately 200 points of interest available per city. All descriptions of points of
interest contain basic facts, textual descriptions, photos links and map links. The information is
retrieved as an XML file containing all attractions for one or more cities. A complete set for one or
more cities is downloaded and used by the travel application editor. The travel editor keeps track
of those attractions that are referenced in the video and the city guide and extracts only those
attractions that are actually needed by the application. The resulting XML file can then be
broadcast to the set-top box.
CityTour information for MHP application: CityTours are retrieved from the attraction server as
XML files, containing a tour name, a tour title, a teaser text and a list of item ids, which are
references to the attractions in the attraction database. The appropriate city tour information for a
city is added to the broadcast stream at application edit-stage.
SMS message for MHP application: SMS messages are sent to a specific telephone number and
relayed to the message server. Conceptually, messages will then be inserted as stream events
into the broadcast stream. In the current implementation, the application retrieves SMS messages
from the SMS server using HTTP based communication on the back channel.
Event booking for MHP application: The broadcast stream contains triggers for venues that are
shown in the video. The payload of such a trigger is a reference to the event database. On
encountering such a trigger, the application uses the back channel to retrieve the current event
information for that specific venue from the event database server as XML data. If the user select
on of these events, the booking data is sent back to the event database server using an HTTP
based protocol.
Attractions and event bookmarks from MHP application to PDA application: If the user
bookmarks an attraction or buys a ticket in the MHP travel application, this information is sent to
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 29 of 71
the city guide server. The information sent to the server contains the ID of the event or the
attraction and a user ID. The user specific information is then stored in the city guide server and
used to build a user specific ‘My CityGuide’ web page, that can be accessed with the connected
PDA.
CityTour to PDA: All applications available on the connected PDA are accessed using HTML
pages and forms provided by the CityGuide server, which in turn accesses the CityGuide
database and the event/ticketing server to provide PDA optimised pages for the mobile device.
3.4. Futurist orientation – PC meets TV
The DeepSea application demonstrates a future environment, where broadcast and Internet are
integrated. The 3D scene descriptions of the virtual environments are broadcast to all receivers,
using the advantages of digital broadcast. For the server interactivity and future multi-user (“multi-
player”) aspects, a broadband Internet connection is required.
The platform for the DeepSea application in which fictional video is enhanced with ambiance
events is centred on a Linux PC platform. The PC is equipped with video and 3D computer
graphics capabilities. Linux was chosen for reasons of independence and for future alignment
with the developments for set-top boxes and embedded electronics. The platform is furthermore
composed of peripherals, i.e., external robot, light control, and sound control. The input devices
include a standard remote control, a game pad, an advanced remote control, The application
demonstrator was implemented on proprietary run-time software, name argoKernel™ (De Pinxi).
This kernel is based on openGL for graphics, and raw or MPEG video streams. It integrates multi-
layer video streams, computer graphics, and special I/O.
The DeepSea application was developed from the point of view of an audiovisual production
house. This implied that the development tools had to be suitable for media authors. The
objective was to create a broadcast fiction program and not to develop specific software.
The DeepSea fiction is based on a graphic database in LightWave format. The LightWave
database was first translated in VRML through an exporter and then the VRML was translated in
MPEG-4 with MPEG-4 tools. Since the standard VRML exporter does not support multi-texturing,
this was done manually. Textures were converted in PNG format. The animations were made
with a frame-by-frame position (backing) and not with interpolation (heavier database). A part of
the scenario scripting was implemented with JavaScript. The I/O connection was ported with
Java, which enabled the communication with "network" devices like the i-Pronto but also "serial"
devices like a game pad or a joystick, using Jini. The DeepSea platform uses the midi standard
for the control of sound and ambient lighting.
The presentation and interaction devices included in the application are a plasma display panel
(PDP) as the primary display, an iPronto as a secondary display, as well as speakers, lamps, and
a GamePad controller. The display panel, speakers, and lamps are positioned as elements in an
ordinary living room. The display panel is hung on a wall. The speakers and lamps stand in the
corners of the living room. A GamePad controller and iPronto are placed in front of the end users.
The iPronto is a small portable computer developed by Philips, with a 6-inch touch screen and
wireless network connection. It can be used as a universal remote control for home appliances,
such as, audio and video systems, lighting systems, and security systems. It also has other
applications such as Internet browser and Electronic Program Guide. In this project, we
developed a generic presentation server for the iPronto, in order to present content elements such
as sounds, images and web pages from the local storage, the local network and the Internet. In
the DeepSea application, this presentation server is used to construct the interface for content
navigation and customisation, and to present and store the content elements for later review. We
also implemented a robotic submarine as the physical counterpart of the submarine in the virtual
user-sea space. The user may direct the virtual submarine by tilting the toy and speed submarine
up by squeezing the toy. When the virtual submarine hits mines or obstacles, the toy vibrates to
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 30 of 71
give the user a tangible feedback.
3.5. Client platform
The client platform is more than “just” a terminal on which an MHP+ application is running. Here
“MHP+” means, that the application will not only use the MHP API, but also uses the functionality
as defined in the ICE-CREAM sub-system. Figure 18 depicts the underlying client architecture.
Figure 18: Underlying client architecture
3.6. Components in the production chain.
Figure 19 provides a detailed component view of the production-distribution platform. These
components are:
Annotation & Video-clip Production
During the live broadcast of the sports application several add-ons are created. The different add-
ons created by the Production Tools (WP7) are:
• "Video-clips": video-clips of live actions or video-clips of 3D-simulations.
• "The action annotations": descriptions of the actions of the players with additional background
information for the active players.
• "3D Scenes": 3D visualisations are sent to the users STB in combination with the action
annotations.
Video-clip-on-Demand distribution
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 31 of 71
The video-clips generated by the (automated) production tools are encoded to MPEG-4 Video and
stored on a central file-system. When the stored clips are to be published, a video server can
stream them over IP or they can be inserted in the DVB stream. Before they can be inserted into
DVB they need to be packaged and embedded in PES packets or MPEG-2 Sections. The actual
clip insertion is done in the DVB distribution block.
Management of Video-clips
The video-clips are continuously created and not all video-clips are suited for broadcast. The
director of the program can control the actually publishing of the clips with the Video-clip Manager.
Directly after the clips are created they appear in the GUI of the front-end of the Video-clip
Manager. When the director decides the video-clip has enough added value, he can publish the
video-clip through the user interface of the Video-clip Manager.
The backend of the video-clip manager controls the distribution over DVB or IP and creates the
triggers to be sent to the users.
Trigger distribution
The video-clip manager creates a “single message” that lists the video-clips currently available to
the user and passes this message to the Trigger distribution component. The “XML multicaster”
sends the list of video-clips, the action annotations and the 2D tracking data over IP to all
subscribed set-top boxes and to the “DVB event embedder”. This component stores all
processed messages together with a timestamp to a “log-file”. This enables the re-sending off all
messages during a re-broadcasts of the complete program.
AV & MHP production
Audio & Video is encoded "live” or is streamed from disk when a program is being re-broadcasted
during demonstrations and user-trials. The MHP application is authored before the actual
broadcast and will not change during the broadcast or future re-broadcasts. The application is
added to the DVB broadcast in a DSM-CC object carousel and will start automatically when the
user tunes in to the program. The MHP application will then contact the multicast server (IP) or
listen to the dynamic events or triggers in the received DVB broadcast.
DVB distribution
In the simplest form a live encoded or pre-created transport stream is streamed over ASI and
modulated to a QAM signal that can be distributed over the cable network of Euskaltel in Spain or
the internal cable network of the NOB in the Netherlands (CAI). With the ICE-CREAM
DVBStreamer a dynamically created MHP application can be added to an existing Audio/Video
transport stream. During the streaming of this new AV/MHP combination, it is possible to
dynamically insert triggers, events and embed a subset of the on-demand video-clips.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 32 of 71
Production Tools
AV
VOD
To Clip inserter
XML Multicaster = XML
Server
Video Clip Manager =
“Replay Manager”, enables final
control for director
Replay Builder
(in-out EDL generator)
3D-Rendering
Clips
MPEG4
Encoder
DVB Receiver
.mp4
files
“new videoclip created”
.mp4 files
“Publish”
Updated list with Videoclips (XML)
AV
AVI
AVI
Multicasted to
subscribed boxes
over IP
Video clips
on demand
served over
IP
DVB Signal / Source
+ MPEG2 Encoded AV
+ Static Xlets in DSMCC
+ Embedded Timeline
Event Embedder
QAM
QAM Modulator
XML
QAM
AV
3D-Generation
2D-Tracking
2D Tracking Data (XML)
Game Action
Annotation Tool
Annotation Events (XML)
3-rd party cold statistics (XML)
QAM
AV
Logfile (XML) =
To enable tests /
rebroadcasts
without
clock
“NPT” feedback
GUI
GUI
DB
Clip Embedder
Clip
inserter
TS
TS
.mp4 files
VOD Server
TS
ASI
Dsm/cc
builder
Signal
“Stream
Started”
Log, to enable
simulating the replay
builder
“streaming started”
MHP classes
MHP classes
HTTP
XML
“Publish”
Files
DVB Receiver
DVB
From Clip Embedder
File
System
File
System
Triggers
Clip Manager
Figure 19: Detailed view of the components of the production – delivery platform
3.7. Tools
The tools in ICE-CREAM have been constructed to facilitate a broad range of aspects of the
sports and the infotainment applications. They have been designed for the development and
adaptation of content for the production, manipulation and customisation of sophisticated
multimedia applications based on ISO-standards. Although some limitations are still present due
to the innovative nature of the applications and the prototypical stage of the implementation, the
tools have shown to reduce the complexity of the production for the ICE-CREAM project.
The production and authoring tools have been used mainly for the sports application, where a live
soccer match is enhanced with instant replays, coded in MPEG-4 format, including 3D graphics.
The events and clips are generated via “live content production” facilities. The clips are encoded
into MPEG-4 and stored on the server(s) and the corresponding events are multiplexed in the
broadcast and/or sent via the Internet (subscription-based).
The main travel application tool is intended for authoring ‘trigg&link’ presentations. The travel
application contains three services that are using the ‘trigg&link’ concept on the client side. These
three services are the CityGuide information, the quiz and the SMS message display. There are
further services within the travel application, for example ticket booking, but these are primarily
front ends to existing interactive services and require no specific authoring beyond the original
implementation.
Production tool
The production tool has to deliver all kind of interesting information concerning the sport event.
The “Ice Cream spectator” should then be able to access new on demand features like replays,
3D scenes or player’s information. One or many operators log the events that occur during the
match (e.g. goals, changes, etc.). All those information feed a database that also catch the
thumbnail of the action and put a bookmark on the video server. Other operators create the 3D
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 33 of 71
scenes of the action that fill the same database. All those inputs have to be synchronized and
connected to the video flow. An “Ice Cream Director” has a full view of this database and decides
either to reveal it to end-user or not.
The functionalities implemented give the possibilities to see cold statistics, hot statistics and
analysis (replays, 3D scenes). All the cold statistics will be delivered as XML files, whereas the
hot statistics have to be delivered live. A distributed architecture, where a software module
materializes each functionality, was used. The following modules were developed:
• Synchronizer – gives the tempo to all the elements of the system
• Logger – annotates the video events
• Thumbnail generator- snaps pictures of the video flow in association with an event
• Video server – Play-while record video server
• Director application – Chooses the events that will be delivered to the user
• Main database – contains all the logged events and pictures taken by the thumbnail generator
Authoring tool
The authoring tool (IAT) allows the manipulation of dynamic content. The positioning information of
the players is automatically imported and a 3D MPEG-4 file is generated. The semantic
information for the production team or the producer can be encapsulated in particular views and
presented in a XML/X3D like format to the user. These views show only interesting information,
e.g. omitting syntactical data and help to make the customisation as easy as possible. Within IAT
the relevant data for the application can be customized, e.g. change of player names and history
information, of advertising banners by drag and drop or change of player clothing. The result can
be exported as fully interactive 3D MPEG-4 file for web applications, as MPEG-4 video stream for
broadcasting and STB replay or as 2D MPEG-4 file for hand held devices or services with limited
display capability. As the concept of IAT is generic the tool can be used for other applications not
being related to the sports and soccer domain only. Figure 20 shows the concept of the IAT
enabling a production team of a TV broadcast to customize the 3D goal replays and produce
output for different channels of spectators.
Figure 20: Dynamic customisation of scene information with ICE-CREAM authoring tool.
The static 3D content is constructed in 3DSMax and exported to MPEG-4 via an ICE-CREAM
3DSMax
Construction
Tool
MPEG-4
Exporter and
Encoder 3D Static
Geometry
Static
User Interface
Dynamic
Scene Info
ICE-CREAM
Authoring Tool
Position
Converter
Plugin
3D
Scene
View
Textual
Info
Views
MPEG-4
Interactive
3D
Scene
MPEG-4
Interactive
2D
Scene
MPEG-4
Static
Video
Scene
Composition
XML Position
Information
?
DVD
Hand-held
Device
Computer
ICE-CREAM
Producer
Set-top
Box
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 34 of 71
enhanced plug-in (ICE-CREAM Encoder). This work results in a static file holding 3D geometry,
e.g. of the stadium in the soccer application. Within the 3D viewing window of IAT the static
geometry and the user interface data is composed. The XML position information generated from
the ICE-CREAM producer described in the chapter above is fed in to the IAT via a position
converter plug-in, which generates the dynamic scene info file holding information that can be
changed on the fly in IAT by the user.
Within IAT the user can now customise textual information represented in a 3D XML notation in
the left window and see changes updated in a 3D view on the right. After customisation is done
the user can export the scene to MPEG-4 video, a 2D version or the fully interactive 3D version of
the application.
Editing tools
In the travel application, there is a separation between content, application and layout for those
elements that are re-useable in the context of other MHP applications. These re-useable elements
include one-line message display, information selected by direct colour key access and multiple-
choice quiz presentations.
The separation of content and layout has also been imparted on the tools, which have been
produced. Alayout editor has been implemented that allows the definition of a specific layout of
any of the re-useable elements, independent of the actual application. In addition, an application
editor has been implemented, that fulfils the task of adding the various kinds of annotations
(attraction information, quiz questions, city tours, predefined messages, chapter names) to an
existing travel video to create content for a specific instance of the travel application to be
presented on the MHP device.
Both editors have been designed to perform their designated tasks as directly as possible and to
incorporate knowledge about their specific purpose. For example, since the layout editor ‘knows’
what a quiz is, it directly allows the previewing of a ‘correct’ or ‘selected’ answer, as opposed to a
‘turn visible/invisible’ on a generic graphic object.
Since the editors are only rarely used (normally the application editor for a travel programme
would be used for about an hour per week, while the layout editor might be used once a month),
care have been taken to make the re-acquaintance with the tools as simple as possible, by using
task related names and options instead of using generic editing tools.
Layout Editor
An important aspect of an interactive television application is its graphical appearance. Authors
want to develop applications that present their content to users in a way that is easy and intuitive
to view and understand, while respecting the communicative need of the underlying audio-visual
programme (i.e. by not distracting from or obscuring part of the programme) and the corporate
style of the broadcaster and content provider.
The layout editor is designed to facilitate authors in quickly generating, modifying and previewing
layouts for their interactive television application. As well as traditional layout editor functionality, it
is extended to support the specific needs of the author for TV application. In recognising that those
applications can be categorized into different types, and each type shares common generic
components and is constrained by common layout rules, the layout editor supports a set of
layout templates, each of which supporting the generation of the layout for a particular type of
application. For the travel application developed in the ICE-CREAM project, three distinct
application types have been identified:
• A message display
• An interactive quiz
• A supplementary information display
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 35 of 71
The layout editor has been implemented to support these three application types through specific
templates. It is designed to be extendable with additional templates according to authors needs.
Each template defines the layout objects (textual or graphical), which exist in the layout, and can
have rules constraining them within the layout. Each template is supplied with a default layout so
that authors can get started immediately.
Objects in the editor can be manipulated and changed quickly through standard Windows actions
such as drag-and-drop or selections from menus. All functions are also supplied with keyboard
shortcuts. Authors can change the position, size and depth of objects in the layout as well as alter
their textual style or textual/graphical content. Pixel-precise modifications are also possible. To
support the specific requirements of designing for a TV environment, the editor allows layouts to
be developed for any of the standard TV resolutions (PAL, NTSC and their wide-screen versions)
and displays optionally a ‘Safe Area’ graphic to indicate if layout objects are safely within the
bounds of the television display.
Authors are also provided with a number of options for previewing their layout before implementing
it within their application. To facilitate previewing, layout objects whose content will be provided
dynamically by the application (e.g. SMS messages) can be supplied with ‘sample’ content.
Additionally an optional background graphic can be added to the layout to test colour schemes or
to represent a typical scene from the audio-visual programme.
Each template defines sets of exclusive objects and, while it does not support temporal
characteristics, the editor allows authors to toggle between the exclusive objects through the
keyboard and hence imitate the expected user interaction in the application. For example, the quiz
layout has exclusive objects for the quiz answers, the indication of a selected answer, and the
indication of a correct answer. The quiz template provides the author with a keyboard-based
means to ‘select’ an answer and to preview the ‘correct’ answer.
Layouts are saved to a SMIL-based XML document format, supporting document interchange and
providing authors with the additional option of previewing layouts in a standard SMIL player. The
layouts produced by the layout editor can be imported into the travel application editor and so
applied to the interactive travel magazine.
By supporting the specific characteristics of interactive TV applications and their requirements of
authoring, the layout editor reduces the authoring effort required to produce suitable layouts for
use in future interactive TV applications. Figures 21 and 22 show views of the layout editor.
Figure 21: The default SMS layout
with Safe Area and sample text entry
Figure22: The default quiz layout, with
preview of selected & correct answer
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 36 of 71
Application Editor
The purpose of the application editor is to create a specific instance of the travel application for a
specific video. The starting point for the application editor is a travel video. This video is annotated
with attraction information, quiz questions, chapter names, short messages and, internally, city
tours.
The main visual feature of the application editor is the video area. This area shows the current
travel video and is also used for editing short messages, quiz questions and quiz answers as well
as for viewing the layout styles associated with the video (Figure 23 and 24).
Figure 23: Application editor with
video area
Figure 24: Editor with info and quiz
overlay on video
A timeline display for the different types of attachments is located at the top of the screen,
together with the various menu options. Under the video area, a standard media player play bar is
located. The user can either play the video using the normal play functions, or go to a specific
position in the video by clicking on the timeline display. If additional information would be presented
for that part of the video, the information is displayed as an overlay in the video area, using the
currently selected layout style.
The content of the quiz (questions, answer alternatives, indication of the correct answer,
countdown timer) can be edited directly on-screen. The same applies for the content of short
messages. They have been designed for the development and adaptation of content for the
production, manipulation and customisation of sophisticated multimedia applications based on
ISO-standards. Although some limitations are still present due to the innovative nature of the
applications and the prototypical stage of the implementation, the tools have shown to reduce the
complexity of the production for the ICE-CREAM project.
Various menus are available, for example:
File menu - Beside the usual features like exiting the application, generating an empty annotation,
choosing a video file and allowing the loading and saving of annotation files, the menu also allows
the selection of the layout style used in the application (which had previously been created using
the layout editor) and an export function that allows an easy transfer of the created application
instance to the set-top box for testing and previewing on the actual target platform.
Select City menu - This menu allow the selection of a specific city from a list of available cities.
The selected city not only determines the attractions available in the Insert Attraction Info menu,
but also the city tour that is added on export to the set-top box and the menu language for the set-
top box application.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 37 of 71
Insert Attraction Info - The content of this menu is based on the currently selected city and the
attraction information available for that city. The basic structure of this menu, however, is always a
three level menu. The first level denotes the main categories (for example hotels, culture, nightlife,
restaurants, shopping), the second level denotes categories specific to the main category (for
hotels these include luxury hotels, design hotels and budget hotels, for sightseeing they include
city and surroundings, for shopping the list includes books, shoes, clothing and furniture). At the
third menu level are the actual attractions located.
Three other fixed menus are mainly used for the insertion and deletion of annotations. They are
mainly provided as a convenience shortcut.
So far, the editing tools have only been used in-house, so no independent evaluation, which could
have lead to a lifecycle model based development, has been done. The layout editing tool,
however, has been given to our designer, who was not involved in the original implementation
process, for evaluation. Feedback from that designer has been used to enhance the tool.
Specifically, the capability of having ‘layered’ access to the layout and the ability of selecting
elements by layers has been added in response to this.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 38 of 71
4. User Feedback
Applications are the carriers in the project. They were chosen such that they covered a wide
range of technical, business, social and conceptual domains.
A major challenge that confronts application developers in this context is to evaluate and elicit the
feedback of users in a robust and reliable way and to make claims such as ‘providing users with a
compelling experience’ operational and measurable. In this report we present the evaluation of the
three ICE-CREAM project applications with end-users. The major challenges that will be
addressed are the formalization of the interaction possibilities for the end-users, the development
of a measurement instrument that captures the effect of distributed content presentation and
different levels of interactivity on the user experience.
4.1. Evaluation of the Football application
The goals for the evaluation of the Football application were:
• To assess the appreciation of end-users and to find weak and strong points of the application
and feedback for improvements
• To find out how the service provided by the application is economically valued by end-users
• To assess the usability of the Football application in terms of ease of use, efficiency of the
user interface, speed of response, ease of learning,
Method
Setting
The evaluation was conducted in the Euskaltelium centre. The Euskaltelium is a free user space
belonging to Euskaltel and located in Bilbao city centre. It exhibits the latest advancements in
computers and communications technology to the general public. People can freely enter the
building and use the exhibited electronic devices and systems. Euskaltel regularly uses the centre
for product evaluations. A living room was created to foster a home environment for the users.
They could freely walk in and out of this living room and use the application at leisure (Figure 25).
Figure 25: Evaluation facilities
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 39 of 71
Participants
Subjects were passers-by. They were visitors that freely and spontaneously came to test room in
the Euskaltelium. They where not compensated economically for their participation. A user-
screening questionnaire was used to determine their profiles afterwards. Table 4 shows the
profiles of the users who participated in the evaluation. 83 people participated in total. Children
below the age of 14 were not included in the formal part of the evaluation. Although a lot of children
enthusiastically explored the system.
Table 4: Participant profiles
Age Percentag
e
15-24 34 %
25-34 34 %
35-44 11 %
45-54 10 %
+55 12 %
% At home
usage
PC TV DVD/VCR
Every day 65% 83% 40%
Sometimes 21% 13% 51%
Never; nearly
never
14% 4% 10%
Participants were allowed to participate in groups or alone. 60% of the subjects participated alone
and 40% participated in a group of friends. 60% of the participants were male and 40% was
female. The ability to handle electronic devices was investigated by asking the subjects about
their skills in PC, TV, DVD and VHS usage at home. The distribution is shown in Table 4.
Procedure
Participants chose spontaneously the tasks they wanted to carry out, tried to find their way to
accomplish them and eventually discovered whether it was possible or not to fully achieve them.
A facilitator was taking notes on users behaviour and comments while exploring the system. The
participants were invited to have an interview with the facilitators.
The procedure consisted of 3 phases: User instructions, Application test and Interview. Each
phase comprises the following steps:
1. User
instructions
Participants receive upon entering the room a brief explanation about
the system’s functionality and how to use it from the facilitator.
2. Application
test
Participants use the system at their own pace with no predefined list of
tasks.
The facilitator records information on user behaviour, gestures and
verbal comments. Relevant sentences in conversations are also
recorded in case more than one participant does the exploration.
Participants can ask questions about the use of the system. The
facilitator records any relevant question about difficult tasks or even
tasks that cannot be done with the system’s present functionality.
Participants decide to finish the test on their own initiative.
3. Interview When participants have finished using the system the facilitator invites
them to participate in an interview and fill in a questionnaire.
Measurement Instruments
The following instruments were used for the assessments:
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 40 of 71
• Observation of users while exploring the system
• Structured interview focussing on the user
• Questionnaire addressing usability and appreciation aspects
The structured interview aimed at obtaining the overall impression of people of the product. This
interview was conducted before administering the questionnaire and after people had explored the
system. People were asked about their opinion on, for example, strongest assets of the product,
potential improvements, satisfaction with the product, interest in the football match, what they
would pay for it, and estimate the duration of their exploration of the system.
The questionnaire comprised a list of questions which people had to score by means of a Likert
scale (1-4), i.e. 1 = totally agree, 4 = totally disagree. The questions assessed the usability of the
product, appropriateness of the user interface and the quality of the user experience, including
control, responsiveness and enjoyment.
Results and discussion
The main results will be summarized here. Deliverable D23 of the ICE-CREAM project provides
the detailed results.
The results from the structured interview from all participants showed a high and keen interest in
the application. 93% of the interviewees would recommend the system to their friends and
families. They explored the system between 15 to 25 minutes. The average assessment of the
overall system was 7.36 on a score of 1 – 10 (10 being highest).
Receiving replays of the most interesting actions in the game (the so-called hot statistics) and
having the possibility to watch them when they wanted was appreciated by 31% of the
interviewees. The creation of 3D replays for the most interesting actions was liked the most by
29% of the interviewees. They also liked the possibility, enabled by the 3D tool, to select different
points of view for the actions. This was the top feature for 21% of the participants. Some of the
participants commented that they really requested watching replays from different points of view,
which is currently available in a passive way in some sport channels. Such comments imply that
people are rather comfortable with such enhancements. Having various possibilities for watching
a match was in the top-ranking list for 22% of the interviewees. Other features that the
interviewees listed as high with regard to their liking were: the possibility of having the point of view
of a player (10%), statistics about the league also called cold statistics (7%), the system and the
innovation idea (6%), the possibility of recording replays in the hard disk of the STB (5%), the easy
of use (1%) and the quality of the image (1%). 30% of the interviewees said that they liked every
tool of the application and that they couldn’t pose any weaknesses. The 20% of the participants
who didn’t like football assured that they would have enjoyed the application much more when it
would have been about another sport or event. The main weaknesses detected by the users
concerned the menu, which could disturb the view of the match, the content of the cold statistics
and some didn’t think the 3D tool would be useful while watching the match.
In short, we can conclude, based on the interview data, that this application opens new ways for
enjoying the viewing of a match and that it addresses dormant wishes of users.
The questionnaire provided more detailed results. Table 5 gives an overview of these results as
percentages of number of participants (n-83).
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 41 of 71
Table 5: Distribution of user assessment scores in % of participants
Score Distribution
Item 1 *) 2 3 4 Average
The system offered a high image quality 43 53 4 0 1,60
The Systems has worked correctly without errors 42 49 7 2 1.70
The system has a good Quickness of response 30 58 11 1 1,83
The system is easy to use and to learn 25 53 17 5 2,01
The user feels in control of the application all the
time
23 54 18 5 2,05
The system is clear and has understandable
information and user menu
41 40 13 6 1,84
The interactive content does not obscures the
content of the show itself
43 46 14 6 1,93
The menu enables you to go back easily in case of
error
30 66 4 0 1,73
The menu does not disturb the view of the match 25 42 16 17 2,12
The remote control is appropriate to interact with the
application
34 54 11 1 1,80
I felt comfortable during the trial 49 51 0 0 1,51
I had a good time doing the trial and I enjoyed it 42 55 1 2 1,64
I would probably recommend this product to my
family and friends
34 59 6 1 1,75
*) 1 = fully agree; 2 = agree; 3 = disagree; 4 = fully disagree
When asked about their intention to buy this product or recommend it to their family or friends, if it
would be in the market, people gave more affirmative responses than negative ones, confirming
that they liked the application. Table 6 gives an overview of these responses and the way they
change with change in price. These responses were elicited by explaining that, the football match
actually costs 6 Euro through Euskaltel’s digital TV system and asking whether they would pay 18
Euro, and if they wouldn’t pay 18 Euro, whether they would pay 12 Euro, and so on.
Table 6: Buying intention in relation to price (in percentage of responses)
Price quotation For sure Probably yes Probably not Not at all
No price quote 16 57 22 5
18 Euro 4 14 36 46
12 Euro 0 12 47 41
6 Euro 23 37 20 20
The appreciation for the system differed for the different participants profiles. The youngest people
liked the application the most. 96% of under 24 years old would buy the service, while this
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 42 of 71
percentage decreased to 62% for older than 24 years. 58% of the females who, in most cases
didn’t like to watch football, would like the application in a different environment (like tennis or other
sports) and 85% of them would recommend the application to their family and friends.
Recommendations
The main recommendations from the end users were quite subjective, like that the application
should be implemented for other sports, or that they would change the information of the cold
statistics, some would like to watch at the same time the replays and the current streaming, while
others would prefer just to watch to the replays. No significant recommendations were given
concerning modifying the implementation of the application, its structure or its user interface.
There were, however, a lot of personal points of view of how to improve the presentation on the
screen. These recommendations support the positive impression regarding the acceptance of
this type of new interactive content by the end users.
4.2. Evaluation of the Travel Application
As a complementary application to the MHP-based travel guide, a mobile travel guide service,
running on a connected PDA device was implemented. The objectives of this implementation
were:
• To create a travel-centric mobile service, offering all features that would be needed during a
city trip
• To integrate city-based information and content, i.e. point of interest localisation, sightseeing
information, movies programme, events, mobile ticketing, location-based services like
mapping and routing into a consistent user interface.
The goal for the user evaluation of the mobile travel application was
• To investigate which behaviours users would show with location based content and services
while travelling in a city, i.e., mapping and routing functions and guided city tours.
Method
The mobile travel guide application was installed on a connected PDA device, containing a GPRS
SIM-card. The connectivity of the device was assured by pre-configuring the required profile on
the device, so that all needed dial-in connections for accessing information were done
automatically.
Functionality of the application:
a) Point of Interest/ Travel Information:
i) Travel recommendations 200 points of interest (POI) with information about Hotels,
Restaurants, Nightlife, Culture, Shopping, Sightseeing.
ii) Area-wide, address-based POI-information on categories like Sightseeing, Museums,
Theatres, Movie theatres, Doctors, Pharmacies, Hospitals, and Police stations.
b) Events and Tickets:
i) List of concerts, theatre- and opera, other leisure events with timetables and addresses.
ii) Ticket booking through an online box office provider using the mobile device.
c) Location Based Services- maps and routes:
i) Users can receive a map on their PDA of their current position and environment.
ii) Users receive a step-by-step instruction for finding their way between two points.
iii) Guided city tours linking several points of interest to a tour through the city.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 43 of 71
Participants
Research experts from the ICE-CREAM project with partial knowledge about the developed
service and no knowledge about the mobile tool itself. They did have no or almost no experience
with using mobile data Internet services like WAP or iMode. The mobile travel guide was new to
them in behaviour and user experience. They were, however, interested in mobile services when
they become easier to use and more bandwidth is available (e.g. under UMTS).
Procedure
An on site evaluation was conducted. Participants could use the application in a real travel
situation. The task of the participants was to use the application for a guided tour in an unfamiliar
city, i.e., Hamburg. Their behaviour and performance of different groups of users was recorded.
The guided tours consisted of 7 different points of interest (POI). The participants worked in three
groups on the assignments. They had to find their way with the aid of the mobile Travel guide.
Their presence at the pre-defined POI was controlled. Two groups, with 5 people each and one
group with 4 people participated. One group-member functioned as logger, and recorded the
activities of the group. The tour lasted approximately 2 hours. The comparative composition of the
groups was:
• Group 1 used the mobile travel guide on a connected PDA, following Guided Tour 1;
• Group 2 used the mobile travel guide on a connected PDA, following Guided Tour 2;
• Group 3 used an ordinary street map, following also Guided Tour 2.
Measurement Instruments
The following assessment instruments were used:
• Observations and notes from the loggers assigned to the groups
• Questionnaire addressing usability and appreciation aspects.
Results and discussion
The ease of navigation was rated lower for the use of the mobile travel guide than for the use of an
ordinary map, average scores of 2,2 and 3.2 respectively (scale of 1-5, 5 being the highest value).
The logging data showed that the complex menu structure of the mobile city guide and the many
features that were integrated in one user interface hampered the on-site usage and prevented fast
and easy access to relevant information. Users had trouble recognizing relevant features such as
additional information on points of interest. They rated the system at 2.6 (1- 5 scale) for being
informative. Table 7 shows the average scores people gave to the travel guide features (scale of
1-5, 5 the highest value).
On average, all features of the mobile travel guide scored around 3. Observation data showed that
people in general liked the features, but that they didn’t like the implementation. For example,
manual localisation was too imprecise and unhandy, because street names and house numbers
could not easily be found. A dynamic localisation over GPS or Cell-Information was desired
instead. The route information was implemented with an ordinary routing plan, optimised for cars
and didn’t provide information on, for example, one-way streets or pedestrian zones. This should
be a strict requirement for further versions of the application, i.e., routing information optimised for
pedestrians. About 70% of the participants lost their way 1-4 times, while 16% never lost their
way and 14% lost their way many more times. The overall usability of the device with regard to
handiness and simplicity was rated as 3.3 and 2.9, respectively (scale 1-5, 5 highest value).
Choosing the travel guide as a replacement for a conventional travel guide (book and map) was
rated as 2.4. People especially liked to have a conventional map together with the mobile travel
guide tool to have the most appropriate orientation during a city walk. The display size on the PDA
is too limited to provide people with an overview and orientation compared to a normal map. The
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 44 of 71
mobile travel guide was perceived as an extra-tool for additional information and detailed
navigation.
Table 7: Average evaluation scores of Travel guide features
Feature Score
Maps 3,4
Route descriptions 2,9
“Where am I” 3,7
Guide tour 3,0
Recommandations/ travel guide
inform.
3,2
Other information 3,4
Concerning future improvements and enhancements, people rated the possibility for
personalization of the travel guide before the trip with 4.2 (a score of 5 is ‘very high’) and the
possibility to book hotels, tickets and trips with 3.6.
Recommendations
The main recommendations from the end users and conclusions concerned missing pedestrian
options in the guide, adequate street plans, pictures and overview maps, detailed input on, for
example, house numbers, alerts and recommendations along the route for sight-seeing. Clearly
available tourist and routing information data made for car drivers is quite unsuitable for
pedestrians in city centres. When people got lost while using the mobile travel guide they would
like to have information about changes in street names and cross roads, direction to take, starting
points, consistency between information and actual situation, landmarks, and most important fast
access to this information
In summary, there is a need for dynamic and optimised solutions for a mobile travel guide for
pedestrians, like tourists walking in a city. Mapping information needs to be optimised and to be
used in a combination of ordinary maps and computer-generated maps, or to be solved by
increasing size and flexibility of the display in mobile devices. Ease of use and handiness remain
critical factors. In addition the performance of the mobile tools need to be improved. The mobile
travel guide tools integrate several features into one consistent application. This requires an
integrated graphical user interface for navigational reasons and sufficient bandwidth. The users
were experiencing the speed of a GPRS connection during the evaluation tour, which was
definitely too slow. Better integration of additional sight information in the basic features of the
mobile tool, such as maps and routes, is needed for further development of the mobile travel
guide application. This requires the dynamic insertion of editorial content in maps and routing
information.
4.3. Evaluation of the DeepSea application
The goals for the evaluation of the DeepSea application were to assess the effects on the end
user experience of
• Different levels of content interactivity,
• Presenting distributed content on multiple devices, and
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 45 of 71
• The participation of a second user.
A major challenge is the formalization of the interaction possibilities for the end users and the
development of a measurement instrument that captures the effects.
Method
Setting
The evaluation was conducted in one of the rooms of the Philips Homelab. Figures 26 and 27
show the design of the setting and the actual working experimental setting. The HomeLab
provided the experiment with a wireless network, control systems, and user research facilities
such as hidden cameras and recording systems.
Figure 26: Conceptual setting Figure 27: Experimental setting
The features of the DeepSea application can be characterized from three different dimensions:
the interface devices that are being supported and integrated as part of the performance space,
the different modes of interaction that are supported by the structure of the content, and the
possibility for co-operation between multiple persons. Table 8 summarizes the dimensions of the
characteristics of the DeepSea application.
Table 8. Characteristic dimensions of DeepSea
Characteristics Description
Interaction devices
Multiple input/output devices:
• Primary display: a plasma display panel (42 inch);
• Secondary display: iPronto;
• Lights;
• Robotic toy: a toy submarine;
• GamePad controller;
Distributed allocation of the content to different output display
devices.
Content Interactivity
Multiple use of the same content at different level of interaction:
• Automated mode (no control by the user);
• Discovery mode (full navigation control);
• Game mode (limited navigation control).
Multi-user support Home / family setting
Interaction devices
The presentation and interaction devices included in the evaluation are a plasma display panel
(PDP) as the primary display, an iPronto as a secondary display, as well as speakers, lamps, and
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 46 of 71
a GamePad controller. The display panel, speakers, and lamps are positioned as elements in an
ordinary living room. The display panel is hung on a wall. The speakers and lamps stand in the
corners of the living room. A GamePad controller and iPronto are placed in front of the end users.
Display & Speakers: A 42-inch display panel was used to present the 3D movie. The speakers
were placed near the display panel. Special speakers were driven by a sound sampler in order to
carry realistic and compelling sounds.
Lamps: A total of four lamps were used to present the lighting effects. Two white lamps were
standing in the corners of the test room; an orange and a red light were attached under the
plasma display panel. The lighting effect is one of the distinguishing features of the application
scenario. The lighting effects presented in the content are related to the physical lights in the living
room. For example, when the submarine hit the mines in the virtual undersea space, the mines
will blow off. When this happens, the lamps in the user’s environment take part in the effect of the
explosion and glare-up as red, orange and white lights. When the submarine dives deeper into the
sea, the light gets darker; when the submarine floats out of the surface, the light becomes bright
again.
GamePad controller: A GamePad controller was used for the navigation control of the
submarine. The user could control the direction (left, right, up and down) with the direction buttons
in the left hand. With the right hand, the user can hold the accelerator button to drive the
submarine ahead, and release it to stop the submarine. By holding and releasing the accelerator
button, the user can actually control the speed of the submarine. Both hands were needed for this
navigation.
iPronto: The iPronto is a small portable computer developed by Philips, with a 6-inch touch
screen and wireless network connection. It can be used as a universal remote control to control
home appliances, such as, audio and video systems, lighting systems, and security systems. In
the DeepSea application, this presentation server is used to construct the interface for content
navigation and customisation, and to present and store the content elements for later review.
Participants
Eighteen students (9?, 9?) from various backgrounds participated in Experiment 1, eight Dutch,
four Ukrainians, two Belarussians, and one Chinese, French and Russian each. Nine subjects
were students of the postgraduate master’s Program in User System Interaction at the Eindhoven
University of Technology. Nine students were working on their internship at Philips Research. 50%
of the students had a background in the behavioural sciences and 50% had a background in the
engineering or natural sciences. The average age was 26 years, ranging from 23 to 30 years.
Subjects watched one to two hours TV per day. Most of them had experience with playing
computer games and about 50% frequently played games (once or twice a month). They were
rather unfamiliar with virtual reality applications.
Measurement instruments
The user’s viewing experience is a multidimensional construct, of which appeal is a major aspect.
Different dimensions that contribute to the appeal of media content are identified in the literature
[3], [4]. These dimensions are, amongst others, intrinsic and extrinsic motivation, engagement,
attention, curiosity, enjoyability, situational factors and presence. The presence dimension
accounts for the belief of being located in a world other than the physical one. Factors that
underlie presence are [5], for example, the extent and fidelity of the sensory information, the
mapping between a user’s actions and the perceptible spatio-temporal effects of those actions,
content factors, and user characteristics. These dimensions overlap. Currently there is not a
single measurement instrument that captures all these dimensions and provides clear descriptors
for data coding and interpretation. Multiple measurement instruments, which combined subjective
and objective measures, were used:
• Two questionnaires: a) Appeal questionnaire and b) Presence questionnaire
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 47 of 71
• Structured interview conducted after the participants had finished the experiment
The appeal and the enjoyment dimensions were addressed by the Appeal questionnaire. The
presence dimension was addressed by the Presence questionnaire.
The Appeal questionnaire was derived from a questionnaire that is being developed in Philips
Research for Dutch native speakers [6]. This questionnaire aims to measure the degree of
‘appeal’ that media content or consumer electronic products have for end-users. Eight
dimensions of this questionnaire were translated and used. Table 9 shows the structure of the
questionnaire. A Liker scale (1-7) was used.
Table 9: Structure Appeal Questionnaire
Dimension Description
Overall
Enjoyment
Eleven items, which focused on enjoying the presentation, impression
about the presentation, time perception, and duration.
Presence Six items, which focused on movement in a virtual environment and
content induced aspects like feelings of turbulence and unrest.
Attention Seven items, which focused on distractions induced by unrelated events.
Challenge Five items, which focused on challenge, interest, and excitement induced
by the presentation.
Motivation Four items, which focused on continuing and repeating of the
presentation.
Future Use Four items, which focused on recommending to others and ownership of
the presentation.
Situation Four items, which focused on comfort and ambiance.
Curiosity Two items, which focused on novelty.
The Presence Questionnaire was derived from the Independent Television Commission Sense Of
Presence Inventory (ITC-SOPI questionnaire) by adapting to the needs of the DeepSea
application. Four dimensions were used. Table 10 gives an overview of its structure.
Table 10: Structure Presence Questionnaire
Dimension Description
Spatial Presence Nine items, which focused on actively involvement, awareness,
and movement in the virtual area.
Negative effect Three items, which focused on negative effects and strains.
Engagement Two items, which focused on sensations of returning from a
journey and losing track of time.
Ecological validity One item, which addressed naturalness of the displayed
environment.
Experimental design
The DeepSea content provided three levels of content interactivity: 1) automated mode or non-
interaction mode, 2) game mode or limited-interaction mode, and 3) discovery mode or full-
interaction mode.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 48 of 71
The DeepSea application provided three levels of interaction devices and distributed output
display: 1) the plain setting, 2) the lighting effect setting and 3) the secondary display setting.
Table 5 gives an overview of the different experimental conditions. Since the system didn’t provide
for all the possibilities (marked as void in Table 11), we had to compromise on the experimental
set-up.
Table 11: Conditions of content interactivity and distributed output display
Conditions Plain Lighting Effect
Secondary
display
Non-Interaction mode A1 void void
Limited-Interaction
mode
B1 B2 void
Full-Interaction mode C1 void C3
The cells, A1 and C1 were compared to study the effect of the content interactivity. B1 and B2
were compared to examine the effect of the lighting effect. C1 and C3 were compared to examine
the effect of the secondary display. The order of the presentations to the subjects was
counterbalanced.
Procedure
All subjects received instructions on each mode and device, they practiced the content navigation
in the full-interaction mode for about 3 minutes, and they watched or played all the three modes.
After each mode, they were asked to fill in the Appeal questionnaire and the Presence
questionnaire. When the three sessions were finished, a semi-structured interview was
conducted. The subjects were divided into two groups. The first group was a plain setting group
(A1). The effect of the presence and participation of a second person was evaluated in a case
study. 3 pairs of subjects participated in the case study.
Results
The effect of content interactivity and the availability of different distributed interaction devices
were measured by the Appeal and the Presence questionnaires. Due to the implementation
variations only 3 comparisons could be made:
• Non-interaction mode vs. full-interaction mode (conditions A1 and C1 in Table 12)
• Limited-interaction mode without and with lighting effect (conditions B1 and B2)
• Full-interaction mode without and with a secondary display (conditions C1 and C3)
The mean scores on a 7-point scale for the Appeal and the Presence questionnaires for these
three comparisons are presented in Table 12 (* significant Wilcoxon Signed Ranks p<.05).
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 49 of 71
Table 12: Mean scores for the Appeal and Presence questionnaires for 6 conditions and
mean errors for the duration estimation.
Condition
Distributed
devices
Plain Lighting Secondary
display
Questionnaire
Dimensions
Content
interactivity
A1 B1 C1 B2 C3
Enjoyment 4.62 4.53 4.32 5.03 4.52
Attention 4.10 4.96 4.84 4.67 4.63
Challenge 3.71 4.49 3.87 4.64 4.22
Future Use 4.08 4.13 4.31 4.65 4.15
Situation 4.97 5.03 5.00 4.75 4.70
Appeal
Curiosity 4.94 5.11 4.89 5.35 5.40
Spatial
presence
2.64* 3.85 4.10* 4.16 4.23
Negative
effect
2.22 2.59 2.56 2.26 2.44
Engagement 2.44* 3.56 4.17* 3.67 4.06
Presence
Ecological
Validity
2.56 2.67 3.22 3.56 4.11
Effect of full-interaction mode: The possibility to interact with the content enhanced the level of
perceived presence and engagement of the user, but it did not effect the overall enjoyment, the
perceived naturalness and challenges. This result is supported by the interview data, which
revealed that most participants were familiar with this type of content.
Effect of lighting: The lighting effect increased the enjoyment and was perceived as an
entertaining element.
Effect of secondary display: The secondary display device provided more challenge, increased
the spatial presence and enhanced the engagement for the participants.
The dimensions that constitute the Appeal questionnaire relate to the content and the script of the
presentation, while the dimensions in the Presence questionnaire relate to the person’s
experience during the presentation. The condition A1, i.e., non-interaction or automated mode and
no light effects or secondary display, can be considered as a baseline. If adding more interactivity
(B1 and C1), light effects (B2) and displays (C3) doesn’t significantly effect the dimensions on the
Appeal questionnaire, then we may also conclude that i) there is a ceiling effect with respect to the
appreciation of content and script of the show which is not influenced by adding more interactivity
and show effects and ii) that the variation in subject’s responses fades out whatever effect there
might have been. The dimensions that constitute the Presence questionnaire relate to the
experience of the user during the show. Adding more interactivity and other show effects like lights
and a secondary display effect all the dimensions of the Presence questionnaire. They increase in
relation to the baseline condition A1. Especially the scores for ecological validity are interesting
(question: ‘the displayed world seemed natural’). The perception of spatial presence and
engagement was effected most. As for the Appeal questionnaire, the variation in subject’s
responses was very high.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 50 of 71
All in all, adding more interactivity and show effects appears to affect the dimensions of the Appeal
questionnaire less than the dimensions of the Perception questionnaire.
The structured interviews that were conducted after the evaluations re-enforced the observations
and the results from the questionnaires. Controlling the submarine in all modes was reported as
the biggest problem. That is, the synchronization of the feedback on user actions was inadequate.
This effect might have over-ruled all other conditional effects. The Full-interaction mode was the
most favourite of the interaction modes, i.e. 5.8 on average on a 1-7 point Liker scale, compared
to 4.6 on average for the Limited-interaction mode and 4.2 for the Non-interaction mode. In the
Non-interaction mode, subjects were presented with enhanced content with which they couldn’t
interact. In this passive condition, the quality of the content and the relationships between all the
effects become more critical for the user’s appreciation. In the limited-interaction mode people
had to perform a task that some perceived as challenging and exciting and that rewarded their
actions with surprising sound and light effects. However, the task was monotonous and boring
after a while. The limited control was hampering the users; they desired free navigation and a
well-defined task. The full-interaction mode had the largest potential among the three modes
according to the participants. They wanted to have more interaction with the content objects or
other players as enemy or friend to build their own story out of the experience. The lighting effect
intrigued the participants, although they found it monotonous and became bored with it rather
quickly. They wanted to have more and varying effects, for instance, a blue ambient wavy effect to
simulate, for example, the sea environment. The role and function of the secondary display was
not fully integrated in the script of the presentation and became rather distracting. Paying attention
to two screens at the same time is not easy, for dividing attention appropriate cues are needed.
The multi-user setting, which was evaluated as a case study, was appreciated more than the
single user setting. They were more attentive and engaged, conducted discussions about the
content. In short, they engaged much more in exploratory behaviour.
In short, the evaluation of the DeepSea application provided a lot of valuable feedback for follow-up
design processes. All in all, the participants perceived the concepts as very promising and
interesting. The evaluations also revealed that we still have a long way to go with regard to the
development of methodologies for evaluation, the design of the assessment instruments, the
exploitation of the possibilities for interaction with content by users in the home environment, and
the exploitation of ambiance effects in synchronization with events in the content.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 51 of 71
4.4. Summary
This chapter documents the results of the user evaluations of the three application designed and
implemented in the ICE-CREAM project, i.e., I) Football application - live video enhanced with real-
time events, ii) Travel application – video enhanced with related content, and iii) DeepSea
application – video enhanced with ambiance events.
New evaluation methodologies were investigated. Methodologies were adapted to the beta status
of the applications. In all cases, state of the art methodologies were used. Context of use was an
important factor for the design of the evaluations. The major findings are:
Football application: 93% of the interviewees would recommend the product to their friends an
family, 31% of all the interviewees appreciated the feature to receive the replays of the most
interesting actions, and 29% of the interviewees liked the creation of 3D replays for the most
interesting actions the most. The users advised to integrate the application for other sports, to
improve the view of the match while the menu appears on the screen, or the replays are
displayed, and to improve the quality of the pictures in the 3D graphics.
Travel application: The feedback concerned optimisation of the application, i.e., dynamic
localisation and delivery of routing information for pedestrians is needed, mapping information
should include a combination of ordinary maps and computer generated maps, and the basic
features of the mobile tool, such as additional information, sight information, maps and routes
should be better integrated. The performance was too slow. Users were experiencing the speed
of a GPRS connection during the evaluation tour which is definitely too slow. The conclusion is
therefore, that integrated graphical user interfaces on mobile devices should only be made
available with 3G bandwidth.
Deep Sea application: The participants perceived the concept as very promising and interesting.
The notion of presence was enhanced by the possibility to interact with the content. To improve
enjoyment, the design of script and content should be dedicated specifically to the home
environment. Distribution of the content presentation on different displays enhanced the
environment of the user with ambiance effects. Synchronization of these effects on different
devices with script and content is critical.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 52 of 71
5. Business Framework
A business model (BM) is defined as a systematic way to describe the economic architecture of a
business. It consists of at least the following components:
• Stakeholders - actors or agents involved in a system
• Roles and interactions – type of activities a stakeholder can partake in
• Requirements and expectations – desired features and capabilities of a system
• Obligations – responsibilities of the stakeholders
• Policies – constraints on the use and operation of the system
• Transactions – the realisation of income
This definition has its origins in studies carried out on the EC NexTV project [7]. Key underlying
considerations are the level of interactivity in the service offering, the Intellectual Property Rights
implications, as well as emergent revenue streams that could be had from the ICE-CREAM
service scenarios.
The analysis framework used for the ICE-CREAM service scenarios is based on core aspects
like interactivity and personalisation, business rules like intellectual property rights (IPR) and
external market conditions. Uncertainty about technology choices and the shape of the market are
underlying characteristics for the development of dynamic business models. Business models
that consider uncertainty have, for example, incorporated concepts like community affiliation and
collaboration, barter (‘free’ goods in exchange for information or forced viewing of advertisements
for marketing purposes), personalised courses, third-party content to attract consumers, free
Internet content or new technology to deliver goods or enhance user experiences. These models
were at the core for the analyses of the ICE-CREAM application scenarios.
5.1. Levels of interactivity
Interactivity is at the core of all ICE-CREAM service scenarios. Three different levels of
interactivity and potential service offerings were derived from literature studies (Van Dijk & De
Vos, Green [8]). Table 13 provides a general overview of these levels of interactivity. In Table 14
the revenue potential for the ICE-CREAM application scenarios is detailed for one level of
interactivity, i.e., level III (Exchange and Communication) with special attention for additional T-
commerce and off-line services. These different revenue-generators are then categorised in order
to select the ones that have a high probability of success. This analysis is based on current
success stories for digital TV and predicted future successes. Especially web-hosting, web-cast,
shared revenue, pay-per-play and betting revenues are the categories, which show potential. All
scenarios share the following revenue streams: Retail Revenue, Premium Content/Service
Revenue, Pay-per-View Revenue, Subscription Revenue, and Shared Revenue (from co-creation)
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 53 of 71
Table 13: Levels of interactivity in relation to service offerings
I. Choice from menus and transactions Video on Demand (VOD)
Customisation
Camera angle (replay)
More item display, plots or storylines
Additional channels
E-Commerce
II. Producing Information Participation in programmes (directed)
Reaction/commentary to programmes
Contribution to programmes/channels (not directed)
Production of own programmes/channels
III. Exchange/Communication Communication about/in parallel to TV programmes
(viewer/user communities)
Table 14: Revenue opportunities for ICE-CREAM scenarios
Scenario Revenue Potential Revenue Stream
Football
application
Place bets- Prior to game and during game
Watch web-cast of post-match interviews for a fee
Build user communities - provide match
commentary off-line, vote for best player/goal, etc.
Charge users to read premium content (hot and
cold statistics; 3D graphics replays; different
viewpoints)
Sell merchandise on & off-line e.g. f/ball kit, books
Charge subscription fees for web-site
Betting Revenue
Web-Cast Revenue
Retail Revenue
Subscription Revenue
Pay-per-View (of video clips)
Shared Revenue (from co-
creation)
Travel
application
Provide e-ticketing, e-booking, etc.
Charge for premium service to `arrange a
personalised package holiday'
Encourage Users to upload pictures, give tips,
etc.
Sell travel books, paraphernalia on & off-line
Charge for Web-hosting services to travel
agencies, hotels, car-rental cos., etc.
Charge subscription fees for w/site
Retail Revenue
Premium Service Revenue
Web-Hosting Revenue
Shared Revenue (from co-
creation)
Subscription Revenue
Pay-per-view Revenue (of video
clips)
DeepSea
application
Charge
- For sales of interactive game off-line
- To play interactive game
- For "storybooks" composed by users
- For merchandise based on characters
- For even more genre choices
- For subscription fees for w/site
Users who adapt functionality can charge for it
Retail Revenue
Pay-per-Play Revenue (tiered
fees for different levels of game-
play)
Shared Revenue (from co-
creation)
Subscription Revenue
Premium Content Revenue
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 54 of 71
Predictions with regard to survival of revenue streams based on surveys for interactive TV are
based on early success stories. For example, Screen Digest (www.screendigest.com, June
2002) predicts UK consumer spending on interactive TV games will more than double from £50.7
millions in 2002 to £132 millions in 2006. The total European interactive TV gaming market will be
worth close to €470 millions in 2006. Betting, available on Sky Active, generates half of BSkyB’s 9-
month interactive revenue (£136.3 millions), with the rest divided between, amongst others,
Internet Advertising, E-Commerce, Teletext services. (Source: Credit Lyonnais Securities, 2002)
[9]. In the Pace Report [10] news ranked number one, in both the US and the UK, as the type of
program most watched by the public. But in terms of what they were willing to pay for, movies
were listed first in both the US (75%) and the UK (71%). News came second in the UK (57%) and
last for the US. Music, education and sports were almost similarly favoured by both the US and
UK respondents. In the US, overall, only 27% were interested in paying for games. However, this
changes with different age segments: 41% of the 18-34 year old Americans surveyed, would pay
for video games.
In light of this, the Football application scenario and the DeepSea application scenario should be
well positioned for revenue generation. The Travel application scenario should also do quite well
as long as the information content does not overwhelm the entertainment features. Adding the
functionality of pay-per-view documentaries or movies (related to travel destinations) to this
application should enhance this scenario’s viability enormously. In Europe, reality shows like
France’s Loft Story and Holland’s Big Brother are a big hit. The Travel application scenario, for
instance, could be made more attractive and commercially lucrative by adapting this popular idea
of a reality show to allow audiences to watch a family on a week or two-week vacation in a
destination of choice (destination voted on by end users). Sports content is widely favoured. It is,
However, price-insensitive and time-sensitive, i.e., the commercial value of a sports event
declines rapidly if it is not viewed in real time or soon after a match is played. This point was taken
in the Football application by distinguishing between ‘cold’ and ‘hot’ statistics for the content
enhancements and by a graceful aging mechanism in the user interface.
5.2. Personalization and familiarity
Despite the lack of published data, the most successful current business models are based on
technologies that are familiar and in use by consumers. These models are based on charging for
premium content, betting and gaming, micro-payments and revenue-sharing with
telecommunication companies for traffic generation (European Interactive Television Report,
2002: www.vandusseldorp.com).
Personalisation is an important component of the ICE-CREAM scenarios. From the user
perspective, it can be considered as a generalised form of interactivity. Personalisation has an
impact on business models, and particularly its trade-off with IPR issues needs to be considered.
Furthermore, personalisation is not just a technical and economic issue, but also a legal and a
social one. It is inevitable that a successful interactive media business model would permit
specific modifications by consumers of content, which users would consider as “personalised
content”. The following scenarios can be envisioned:
• Modification by users of various objects in IP-protected content. Examples would include the
following:
o Embellishment of user interface, e.g. new skins designs
o More attractive backgrounds, in software and video
o Remix of audio
• Modification of storylines in linear/non-linear formats. Examples include:
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 55 of 71
o Extra scenes are created by users to embellish the storyline
o Entire “alternative” beginnings or endings are formulated by users
o New hypertext/hyperlink information is added by users that generally enhances the
user experience
• The “Linux Model”. In this model, a community of freelance developers makes independent
and original contributions that are submitted to a central authority, which then distributes the
content to users. This is particularly relevant in a P2P based business model that continually
allows for the recycling of content.
These scenarios open up very complex legal issues. Given the impossibility of trying to police
every instance of such personalised content, the best option for the commercial content
producers appears to be to embrace rather than oppose such personalised content from
consumers, i.e., a limited form of personalisation should be permitted, such as point (1) above,
but others such as (2) and (3) should be permissible only with fair monetary compensation.
A balance between access to information and rights protection (e.g. the payment for that access)
needs to be achieved. If end users feel that every little click requires payment (micro-payment),
they may become disenchanted and stop using the service or product, thus decreasing the value
of that intellectual property. On the other hand, if rights holders do not pursue infringements of
their rights, the value of that right becomes nullified. Alternate views, like Dyson (1995) [11] and
Barlow (1994) [12] suggest that companies should give up on the hope of profiting from
intellectual property on the net (digital media), and instead choose alternative forms of profit.
Paying for translations, which are usually part of the free service offering in websites, for instance,
may turn away end users. For example, Barter or Swap business models offer these services for
free to all end users in exchange for profile information. Another option is to offer end users who
are already paying a subscription fee for the digital service, some of the other personalisation
features at different starter package levels.
In a risk-neutral setting the idea is to provide as complete an interactive experience as possible for
target segments in areas that end users are already familiar with. The off-line social context is
replicated as a solo or group activity and enhanced through the enabling technology. That is, end
users are offered what they already enjoy doing in a particular market-space. Up to date and
anticipatory knowledge on end user habits, tastes, motivations, and so on is crucial. This
proposition is risk-neutral, if insights are based on information gathered about the end user. It is
off the mark to wonder what level of interactivity end users would desire, as they are unaware of
these distinctions. What is important to them is the ‘authentic simulation’ of the overall experience
they already have, off-line, or the completely new experiences/applications (still based on end
user information) that they can have.
A risk-neutral revenue model would initially seek revenue from domains that are already
successful today. In parallel, all revenue streams should be explored as the end users’ tastes,
preferences and behaviour may change over time. At present, betting revenue and Pay-per-play to
play games are the most likely revenue generators among the various potential revenue
categories identified. Incorporating games where possible in all scenarios should be considered.
In addition, revenue sharing with end users as a result of co-creation (e.g. adapting the
functionality of the scenarios and in the process creating new transactions) is a new revenue
category that may succeed. Co-creation and revenue-share will harness end users’ creativity in a
positive, mutually beneficial manner instead of stifling it or encouraging hacking or negative
creativity. Finally, facilitation of t-commerce could help the build up of communities, trigger the
desire to discover new scenarios and as a by-product the creation of new revenue streams.
The Football application scenario is built around the familiarity of people with watching and going
to a sports game. According to Ovum (2001) [13], most of the revenue in 2001 was generated by
premium sports content, and the rest by premium movies. The excitement generated by the 2002
Football World Cup recently speaks for itself. For instance, in the UK, there were big TVs set-up in
offices, schools allowed pupils to watch matches in the school hall and the celebrations after
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 56 of 71
each game won (also in Japan, Korea, Brazil, Ireland, etc), is not equalled by any other sport.
Other than movies, sports have been the most successful content area for interactive TV
applications. It was identified by a little more than 50% of the respondents in the UK and US, as
programming they were likely to pay for (Pace Report, 2001), [10]. In the UK, BSkyB claims that
over 40% of viewers access interactive features at least once during any specific match on which
they are available (Unna, 2002) [14]. With the ability to respond to triggers for additional
information on players, watch replays again and the added functionality of video-chat, the Football
application should expect to have some success in the marketplace. This is because it recreates
what end users actually do and want to see while they watch matches. In addition, a business
proposition for the Football application should consider the potential advantages of end user
behaviour prior to and after matches, which could contribute other sources of revenue. With a
digital rights management model, end users could be paying for downloading of replays,
personalisation through skins and other enhancements. The potential of retail/ t-commerce
revenue, betting revenue and premium telephony revenue, through interactive voting could be
added as possibilities.
The Travel application scenario takes the explosive growth of the use of information and
communication technologies in the travel and tourism industry into account. Travel content
became very popular as people travel more widely and usually for short one-week breaks. End
users can walk into physical travel agencies, watch travel programmes on TV or surf the web to
gather information and make bookings/reservations. Interactive holiday shopping offered by e.g.
Sky Travel Active in the UK, provides an extra channel for End users who can then book holidays
from the comfort of their armchair Allowing ticket reservation and ticket purchase during the
program is an idea worth considering. The Travel application scenario is focusing on city travel
(e.g. extended weekends) which is related to a more fixed type of package(s) that may be used in
association to, e.g., specific sporting events, opera in the park etc. This application scenario can
be easily enhanced with capabilities to purchase flight tickets, customised holiday packages and
other related services. In addition quizzes, virtual chatting with people who have first-hand
information about a travel destination via e.g. SMS messages, on and off-line competitions, and a
myriad of other possibilities like e.g. interactive holiday shopping (e.g. souvenirs etc) are possible.
The DeepSea application scenario is a novel scenario in that it allows end users to `compose
stories’ from the many options given, while playing games, if they choose to do so. This form of
entertainment is seen as one of the most interesting usage models for interactive TV (Unna,
2002) [14]. Such offerings with various levels of interactivity are currently not in the market and
there are no data to show that end users will have a positive response to this type of scenario. It
offers wide revenue generation opportunities from different prices for game-playing at different
levels, to charging for story down-loads, soundtracks and to buy the interactive game, etc.
Incorporating t-commerce and off-line communication will increase this potential. Although t-
commerce is not profitable now, Ovum (2001) [13] predicts that it will be the greatest business-to-
consumer revenue generator by 2006 followed by premium sports content.
Consumers select applications and services based on perceived usefulness and/or perceived
playfulness (Moon & Kim, 2001). What appears to be clear is that consumer priority is primarily
for self-fulfilment. A demographic breakdown shows that the ICE-CREAM scenarios hold value
propositions for different age segments. Although teenagers are a growing segment of the ‘buyer’
market, they and children also influence the purchasing choices of their adult parents (Table 15).
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 57 of 71
Table 15: ICE-CREAM scenarios in relation to demographic breakdown and interactivity
attractiveness
User Group Football application Travel application DeepSea application
Children Element of fun –
football
Animation, competition
Element of fun,
competition, – story,
adventure, sounds,
animation
Teenagers Entertainment (football)
Novelty – 3D camera
angles
Entertainment-video
clips, quizzes, SMS
Entertainment (chance
element) – game,
communication
25-34
Year-olds
Need to be entertained;
Socialize
Information (guide,
booking system),
entertainment-video
clips, quizzes
Entertaining and
educational for kids
Over 40’s Meaningful interaction-
replay, stats. for more
information; betting
Information (guide,
booking system),
Entertainment-
video-clips
Entertaining and
educational for kids
Table 16 shows some of the opportunities for ‘customised consumption’, offered by the ICE-
CREAM scenarios.
Table 16: Opportunities for customisation in the ICE-CREAM scenarios.
Football application Travel application Deep Sea application
Personal choice Obtain statistics,
replay, camera angles,
present different points
of view in 3D animation
Play video-clips Navigate through
obstacles
Contributing to
programmes
Opinions, tips- via
SMS
Influence story
composition, user-
creation
Games Could use statistical
data to create a `fun’
maths game
Quiz Play game and/or watch
show
e-Communication SMS
Some features of the three scenarios provide customisation opportunities and satisfy or exceed
current demand. However, we note that the children’s segment has not been specifically
addressed by either the football application (except for 3D replays, which would be very appealing
to young and old fans) or the travel application, in terms of interactive-readiness and should
therefore incorporate more fun elements. For example, an animated tourist guides or games (e.g.
Treasure Hunt) to be played in the featured cities. As influencers of buying decisions, children’s
impact cannot be under-estimated. The football application could incorporate games and
opportunities for viewers to participate off-line (e.g. football coaching clinics for kids and teens).
The DeepSea application could try and capture the parent group in other ways e.g. provide them
the opportunity to contribute ideas for the show, to write/buy storybooks, toys or other
merchandise related to the show. Both the football application and the DeepSea application
should be a social (family) viewing experience. The travel application does not need close social
contact, particularly when it is ported on to a mobile device. Here, it works better as an individual
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 58 of 71
experience (due to screen-size, font size, etc). This is an area that the travel application could
improve upon, i.e. to appeal to the whole family. The travel application’s generic services (quiz,
message display and information display) are useable in variant forms in other contexts like
sports shows, infotainment and other programmes. For example, other than SMS, the short
message display can also be used for news flashes, stock information and so on. This
represents a valuable revenue-generating opportunity for the travel application.
The football application appears to offer less customising opportunities than the other scenarios,
but its value lies in the real-time nature of its content. It merely requires enhancements like
information provision, and the ability to change camera angles to make the core experience even
more pleasurable for the consumer, and to generate more revenue opportunities for the producer.
Exclusive content, 3D replay animation in picture in picture is the key value proposition offered by
the football application to its consumer marketplace. The Football and Travel applications can
benefit substantially from the inclusion of some kind of interactive gaming. In relation to time
shifting, the users of the Travel and DeepSea applications would benefit from DVR support built-
in, as the value of these applications does not decline with time. This is in stark contrast to football
application, where real-time viewing is key to its value. However, 3D replays in football application
are very suitable for storage and would appeal to die-hard fans, which would appreciate the ability
for repeated viewing. This would also be another revenue-generating opportunity.
5.3. Business rules and IPR
IPR is a key variable in the development of viable business models. Important is to overcome
uncertainties by a market architecture that reflects and supports the successive use of content
and underpins the cumulative nature of innovation. A market architecture that can account for
nested rights can also support multiple slices of this kind. Rights and obligations are the
foundations of IP law, and every piece of an intellectual creation embodies layers of rights. Each
layer of rights is associated with a “role”, and can be licensed, sold or assigned to others with
preset conditions attached. This applies to both tangible and intangible goods. This thinking is at
the heart of the idea of “sliced property rights” (www.iprsystems.com). This recycling of
copyrighted material opens up a Pandora’s Box of legal, social and commercial issues, resulting
in a shift from the traditional “creation waterfall” model to the “creation life-cycle” model (Figure
28).
Create
Use
Trade
Waterfall models
Producer owns, consumer pays
Create
Use Trade
Life-cycle models
Blurred roles producer & consumer
Figure 28: The traditional “Creation Waterfall” model of the production-consumption
chain and the “lifecycle” model of the knowledge economy.
In developing novel business models for the ICE-CREAM project, the particular form of the
Producer-Consumer relationships has to be classified for each application scenario and the
relevant IPR models identified. From this, business models may be extrapolated. Such a
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 59 of 71
classification is shown in Table 17
Table 17: IPR and Producer-Consumer models for ICE-CREAM scenarios
ICE-CREAM Scenario Producer-Consumer Model IPR Model(s)
Football application Creation waterfall & Creation
life cycle
Monolithic (old-style)
Travel application Creation life-cycle Mostly sliced
DeepSea application Market Mesh Combination of sliced, nested or
monolithic
The Football application scenario is a linear creation waterfall model: a broadcaster purchases the
rights to a (football) game, and makes money on the sale of viewing rights to the consumer. The
sale would involve either sponsorship from other companies (traditional broadcasting, B2B), or
pay-per-view (interactive TV, B2C). Clearly, this IPR model is old-style monolithic, since the rights
to broadcast the game will be retained by the broadcaster alone. Furthermore, this particular case
has a business model that is unique for the three ICE-CREAM scenarios: it is bounded by time,
since the value of viewing the game will diminish if it is not enjoyed in real time. IPR issues do not
hamper the interactive chat option that is an extra value-added, since normal chat-room talk is
neither classified nor copyrighted. The 3D and event replays are the value-added parts introduced
by other parties. These fall under the purview of the creation life-cycle model. The Travel scenario
involves enhanced video with non-fiction content. The non-fiction video part represents the Core
Content, and the “enhancements” are the value-added part that could be introduced by other
producers or consumers. Clearly, this falls under the purview of the creation life-cycle model of
Producers and Consumers – as more and more value is added to the content, the IPR becomes
more and more finely sliced on each iteration through the production-consumption loop. The
DeepSea application scenario is the most general service scenario considered. With enhanced
video, in the fiction category, we have a situation where any or all of the IPR models could co-
exist. The Producers and Consumers may frequently swap roles, with each possibility realising, in
principle, a different fine-grained business model. The IPR issues are particularly complex, since
each implementation could involve aspects of sliced, nested or monolithic IPR in dynamic
proportions.
Football application: This application will benefit from the preponderance of existing and emerging
mobile devices: it may be envisaged that the sports fans will now be able to e.g. view the match
itself on a laptop, or obtain on-line information on PDA or a 3G telephone set. Compatibility with
the open mobile alliance (OMA) standard will serve to abstract the underlying hardware and
software combination and allow the fans to open a viewing channel with the football broadcast or
multicast. From the point of view of consumers, mobile agents could be instructed to initiate
automatic payment for a pay-per-view transaction that allows the sports fan to see the match on
his/her chosen device; the presence of agent technology would allow the user to select the set of
matches they wish to see days or weeks ahead of the actual event itself. A slight variation on the
above would involve the non-real time viewing of a sports match, i.e., ‘time-shifting’. The greater
the time shift, the less value can realistically be attached to the content: ultimately, within a few
days, the content would become an archive item with commodity pricing and free advert. Another
extension of this would be the case where one person is able to supply an informed commentary,
and perhaps would like to compare a given match with clips from previous performances.
Personal ‘web logs’ of such discussions are already now something of a sensation on the
Internet, and ‘blogging’ is in many ways a natural progression of an informed user’s chat session.
Travel application: This case is tailor-made to the client-server model of networked computing. An
enhanced travel booking application is easily possible with a centralised web server, hosting a
page with all the necessary audio-visual material or with links to such material. Such a system
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 60 of 71
would in fact be possible even with simple technologies such as HTML. It will truly benefit from
exploiting technologies such as web services and agents to deliver a state of the art service. For
example, an enhanced travel application could be implemented using agent technology at the
consumer end and web services at the producer end. In this case, financial agent and holiday
agent interact with the user’s diary agent to ensure end-users’ service satisfaction. If a second
person is involved in the holiday, the personal holiday agents of the two users would firstly contact
each other, to ensure that there are no diary clashes, for instance, and then act in unison in going
through the above steps.
Deepsea application: This scenario is also a classic case of a client-server model of networking.
It would also benefit from web services, but in addition, the possibility of modifying the fiction
content would enable it to also benefit from P2P networking. In this case a user’s personal
entertainment agent would facilitate the selection of these choices. The appropriate set of web
services at the content providers’ end would ensure payment trust and the non-compromise of
digital rights.
5.4. External market conditions and change
Based on extensive literature reviews (ICE-CREAM project Deliverable D19), Business Models
that could account for changing conditions were classified in three categories. These categories
are: a) Successful or Sustainable business models; b) Strategic business models; and c)
Generic (component-based) business models. They were analysed with regard to their
possibilities to cope with dynamic features in the market and their relevance for the ICE-CREAM
application scenarios. Table 18 gives a summarized overview of this analysis. The generic
business models are not included in the table as there is a magnitude of divergence between
them and fundamental concepts are not yet clarified.
Table 18: Analyses of Business Models in relation to market dynamics and application
scenarios.
Business model categories Dynamic features of the
business models
Relevance of ICE-CREAM
application scenarios
Successful / sustainable business models
Four different types of change
models
• Realisation models –
maximise returns from
exiting operation logic.
• Renewal models –
consistent and conscious
revitalizing of product,
service platforms, brands,
cost structures and
technology bases.
• Extension models –
stretching operation by
including new markets,
value chain functions, and
product and service lines.
• Journey models –
deliberate and purposeful
moving to a new operating
model with no return to the
old model.
Most companies adapt to
change by changing the
organisational structure (83 %)
or the mindset (13%).
Most common and fastest
change techniques are Partner,
Anchor & Extend, Converge and
Plug&Play.
Exploitation of:
• First mover advantage
• Innovative pricing and
purchasing through non-
restrictive DRM (for
example iTunes).
• Customers as co-workers:
they grow the business due
to ‘listen, learn and enable’
approach or are partners in
an auction model. (for
example eBay)
Pro-active and flexible
approach, which protects rights-
holders and benefits end-
users.
Expand application
implementation gradually over
geographical regions.
Evolve and add new features
over time.
Strive to exceed stakeholders’
expectations
Leverage capabilities, e.g., third
party licensing of 3D-replay
technology, authoring tools for
service development.
Port application to other
consumer devices
Introduce micro-payments for
services
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 61 of 71
Strategic business models
Two different classes:
• Invention of new business
concepts–
• Achieving customer
bonding
Exploitation of:
• Profit boosters, like for
example competitor
lockout.
• Portfolio breadth, operating
agility or lower break-even
features.
• Three distinct strategic
options: best product, total
customer solutions, and
system lock-in options
(Delta model)
Create a unique value
proposition.
Create a portfolio with a broad
offering of services and
applications.
Continually update and grow
core competences and
customer types
Create bonding between
customer and service, e.g., total
customer solutions.
From this analysis a dynamic business model for ICE-CREAM applications can be derived that
should comprise the following elements:
• An overarching mission (goals, vision, value proposition) for the business,
• A total customer solution (TCS) as in the Delta Model (Hax & Wilde II, 1999, 2002) [15], with
understanding the customer as its focus,
• Core components of a business model (adapted from Afuah & Tucci, 2003 [16]; Kruger et al.,
2003), [17]
• Change Models (Linder & Cantrell, 2000) [18] that focus appropriate strategies for different
phases of a product/service cycle (including a disruptive change phase).
5.5. Recommendations
What is becoming clear is that firms make deliberate and focussed changes to achieve their
goals. Timing of product/service launch, good marketing and non-restrictive DRM are essential
success factors. In addition, successful e-Business models seem to use extension and/or
renewal models in order to affect change. Consumer-centric or democratic business models (like
that of eBay) which `listen, adapt to and empower’ consumers are the way forward. Further, a
firm that offers abundant types of services/products rather than some core ones is in danger of
diluting its brand value.
Finally, recommendations and propositions based on best practices resulting from the business
model analyses can be summarized as follows:
• Change business models from being only B2C to B2B/intermediaries and C2C.
o B2C and B2B appear to be the way forward. B2C operators like Amazon have
successfully shifted business models to become a B2B and C2C marketplace, thereby
reducing costs and enhancing revenue opportunities.
o Industries like the real estate and airline industries are moving from being information
monopolies to becoming consumer (-driven) democracies
• DRM is fine as long as it is not too restrictive.
o Plausible alternatives to DRM exist – e.g. Creative Commons (cc), which allows rights-
holders to allow some usage of their IP for free.
• Use Open Source. It is most favourable for most consumers (best example here is the
meteoric rise of Linux).
o Couple this with solid industry standards to make a compelling mix benefiting both
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 62 of 71
producers and consumers.
• Aim at interoperability and a horizontal marketplace for the future..
• Explore strategies or models used by other industries like the music and airlines industry.
o Alliances of major companies in each industry, new intermediaries in the value chain (both
dis- and re- inter-mediation exist) and innovative pricing models are all changing business
models in the increasingly Web-based economy.
o Successful business models are subject to almost immediate imitation as evidenced by
copycat models to Apple’s iTunes.
• Create adaptable pricing models. They appear to be a profitable strategy as can be seen by
priceline.com, and supermarkets matching prices of similar offering from rivals.
• Police and continuously listen, learn and adapt to consumers behaviour and wishes.
o Consumer focussed organisations like eBay have a very profitable business model.
• With regard to application development, work directly with consumers (B2C) and become a
new intermediary in the value chain, i.e., a B2B player move
o By allowing its customers (end-users) to make modifications and behave as co-creators
(and rights-holders), selling modified versions to other consumers or to the aggregator; a
C2C model can be included as well. This would create a robust, growing marketplace,
stimulating creativity and collaboration while enhancing revenue opportunities.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 63 of 71
6. Conclusions
The major results from the ICE-CREAM project can be summarized as follows:
• Development and evaluation with users of three applications, Football application, Travel
application and the Deep Sea application.
• Formulation of a business model framework based on change drivers, digital rights
management aspects and adaptation factors.
• The development of an MHP-based platform that is extended with network services, MPEG-4
audio/video streaming and 3D graphics support.
• Prototype, which uses SMIL to integrate content from the Internet and to interact with the
content on a distributed presentation system.
• Development and adaptation of tools for production, manipulation and customisation of
content.
The three applications that were developed in the ICE-CREAM project showed promising results
for the strategic crossroad of TV and Internet convergence. They integrated broadcast and
Internet content and were developed on an MHP platform that was extended with network
services, MPEG-4 A/V streaming & 3D graphics support. Encouraging feedback was obtained in
user evaluations with regard to functionality and appreciation of the applications. This feedback is
encouraging enough to strongly recommend implementation of the integrated platform and to
generalize the applications to other domains and topics.
At the beginning of the project we set out to define a few factors that we considered as crucial
requirements for the perceived user benefits of interactive experiences. With regard to
involvement and appeal these major perceived user benefits of the applications are:
• Control over the content. Users decide whether they want the enhancements or not. In
addition, the DeepSea application gives people the possibility to decide on the path to take and
on controlling the level of participation and involvement.
• Participating in the program content. This is accomplished in the Travel application by the
involvement of the user in composing their personal ‘city tours’ and in the Football application
by the different views that are possible. Users can put themselves in the shoes of their
favourite football player. The DeepSea application entices users to get involved with the
content by playing a game.
• Ability to produce something useful and fun. With all applications users can compose and play
if-then variations. They can convert the applications into a game, conduct different types of
analysis and be part of a team.
• Feedback in the user interface. The user interfaces for the Football and Travel applications
were kept as close as possible to the well-known and trusted TV interface. The feedback is
simply the content that they receive. The user interface of the DeepSea application is still very
challenging and requires many adjustments. Many of these were identified in the user
evaluations. We still have a long way to go to understand the possibilities and the
impossibilities of using different devices and displays. Especially the integration with content
requires a different approach.
• Reliability of the application. Since the applications are clearly positioned as TV, people
perceive them as reliable as good old TV.
• Different and worthwhile to have and to view. This requires user trials with a larger population,
over an extended period of time and with different content domains. The applications provided
users with functionalities, which support familiar activities in their intimate environment, which
they could perceive as natural extensions of their habits.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 64 of 71
The focus of the ICE-CREAM project was on the exploitation and experimentation with emerging
standards, e.g., MPEG-4, DVB-MHP, and Internet. The project identified the need within DVB for
dual video support and within MHP the need to control the additional video feed in a similar
manner as the main feed is controlled. User requirements were submitted to DVB-MHP for more
powerful composition support like, MPEG-4 BIFS and SMIL. The project demonstrated how crucial
the future in-home networking capabilities of MHP are for distributed presentation of content The
ICE-CREAM authoring tool is one of the first tools capable of reading, customizing and converging
X3D files. The project showed how SMIL could be used for distributed presentation in
TV/Broadcast applications based on MHP. The opportunities of the employment of standards like
MPEG-4, VRML and X3D versus proprietary technology were explored. These opportunities
comprise, for example, the opening up of new markets, the entering of existing markets, diversity
of markets, different possibilities for licensing of content and technology, various possibilities for
SME’s to be part of the standard value chain, and no need to deliver end to end solutions as with
proprietary technology.
An operational demonstrator was build for the different applications. In this demonstrator,
broadcast programs are enhanced with interactivity in a personalized manner, the Internet is used
to convey the personal data and the trigg&link paradigm is used for the presentation of the
enhancements to the users. Authoring and production tools were further developed, integrated
and/or adapted to achieve the end-to-end system that could be demonstrated in the football
application. The major focus was on the production and customisation of MPEG-4 content, i.e.,
especially for 3D and interactivity components. The availability of easy-to-use authoring and
editing tools is a must for easy and fast development of applications and services, for updating
and timely provision of relevant and personalized information.
Content adaptation aspects of interactive TV and especially the personalization of video broadcast
through mixing in internet based content, such as XHTML pages or MPEG-4 video streams, was a
main focus. A prototype was build to demonstrate presentation and interaction on a distributed set
of displays, where shared content and personal content is presented in a corresponding manner.
Flexibility and adaptability to changes in environment, in user demand and in user behaviour were
the driving factors for assessing business models and for analyses of the three ICE-CREAM
applications. Marketing factors, like product range, customer value, positioning, were identified
and compared for each application scenario.
The project applications were demonstrated at several conferences and exhibitions. Especially
successful were the demonstrations of the applications at the IFA2003 and the IBC2003, which
generated lots of positive feedback (e.g., from BBC, Turner Entertainment). The demonstrations
showed a level of interactivity, which went far beyond other MHP demonstrations that have been
showed at IBC and IFA. A user trial was conducted at the Euskaltelium in Bilbao. This is a free
user space in the centre of Bilbao, which receives around a thousand people a day.
A number of deliverables or parts thereof were used and will be used to write publications in
academic journals, present papers and posters at international conferences.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 65 of 71
7. References
[1] W.R.Th. ten Kate, A.F. Schoenmakers, N. de Jong, H. Charman and P. Matthews,
Trigg&Link - A new dimension in television program making, 2nd EcMast, Milan, May
1997, in Lecture notes in computer science 1242, S. Fdida and M. Morganti (eds.), Springer
(1997) 51-65.
[2] W3C, Synchronized Multimedia Integration Language (SMIL 2.0), W3C Recommendation,
http://www.w3.org/TR/smil20/, Aug. 2001.
[3] Schickenberg, B. & Hoonhout, H.C.M. (2003), Towards an ‘Enriched media content Appeal
questionnaire’. Philips Research Report (to be published) based on B. Schickenberg’s
Graduation thesis, Faculty of Psychology, Maastricht University, Maastricht (NL)
[4] Malone, T. W. & Lepper, M.R. (1987), Making learning fun: A taxonomy of intrinsic
motivations for learning. In: R.E. Snow and M.J. Farr (Eds.)< Aptitude, learning and
instruction. Hillsdale: N.J.Erlbaum.
[5] Hoonhout, M.C.M. & Stienstra, M. (2003). Exploring enjoyability: which factors in a consumer
device make the user smile? In: D. de Waard, K.A. brokhuis, S.M. Sommer, and W.B.
verwey (2003), Human factors in the age of Virtual reality. Maastricht (NL): shaker
Publishing.
[6] Freeman, J., Lessiter, J., & Ijsselsteijn, W., (1999) Immersive TV. From electric source,
https://www3.bps.org.uk/publications/thepsychologist%5Cimtv.pdf.
[7] NexTV Deliverable, (2000), Preliminary Business Framework and choice of applications-
WP1_D1, EU-IST project
[8] Van Dijk, J., de Vos, Loes (2001), Searching for the Holy Grail- Images of Interactive
Television, New Media & Society, Vol. 3, Issue 4, Dec. 2001. Sage Publications, London.
[9] Credit Lyonnais Securities Report, BSkyB, July 2002
[10] The Pace Report (2001), Consumer attitudes towards digital television: the UK and the
US, Pace Micro Technology plc.
[11] Dyson, E. (1995), Intellectual Value, www.wired.com/wired/archive/3.07/, July 1995.
[12] Barlow, J. P., (1994), The Economy of Ideas, http://www.wired.com/wired/archive/2.03/
[13] Dransfeld, H. (2001), Determining Which of the Existing Business Models for iTV has
proved the most profitable; IIR Fifth Annual iTV Event, 27-28 Nov 2001. (OVUM)
[14] Unna, H., (2002) Digital Interactive TV: loved or abandoned; IPA, January 2002
[15] Hax & Wilde II (1999). The Delta Model: Adaptive Management for a Changing World.
Sloan Management Review. vol. 40, no. 2, Winter 1999. pp.11-28.
[16] Afuah, A., C., Tucci (2001). Internet Business Models and Strategies. Boston: McGraw
Hill.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 66 of 71
[17] Kruger, C. et al. (2003). Business Model Formulation within the Online News Market: The
Core + Complement Business Model Framework. 16
th
Bled eCommerce Conference (June
9-11, 2003). Bled. Slovenia.
[18] Linder, J. & Cantrell, S. (2000). Carved in Water: Changing Business Models Fluidly.
Working Paper from the Accenture Institute for Strategic Change.
[19] ICE-CREAM Deliverable D16&D18 "Interactive TV applications: prototypes, methodology,
user interfaces and interaction devices",
http://www.extra.research.philips.com/euprojects/icecream/internal/deliverables.htm, Nov.
2003
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 67 of 71
8. Project Consortium
ICE-CREAM consortium partners and contact persons.
Philips Research (Project Co-ordinator)
Prof. Holstlaan 4
5656 AA Eindhoven, The Netherlands
Maddy D. Janse
[email protected]
De Pinxi
Avenue Huart Hamoir 54
1030 Brussels, Belgium
Philippe Chiwy
[email protected]
NOB Cross media facilities
P.O.Box 10
Media Park
Sumatralaan 45
1200 JB Hilversum, The Netherlands
Ruud Leyendekker
[email protected]
Imperial College London
Electrical and Electronic Engineering Department
Exhibition road
SW7 2AZ London, United Kingdom
Javier Barria
[email protected]
Philips Digital Networks France – MP4Net
51 Rue Carnot – B.P. 301
92156 Suresnes Cedex, France
Raoul Mallart
[email protected]
Tomorrow Focus AG
Alsterufer 1
20354 Hamburg, Germany
Oliver von Wersch
[email protected]
Fraunhofer FOKUS
Kaiserin-Augusta-Allee 31
10589 Berlin, Germany
Angela Scheller
[email protected]
Bitmanagement Software GmbH
Weiherfeld 22
82335 Berg, Germany
Peter Schickel
[email protected]
Symah Vision
27 Rue Jean Bleuzen
92170 Vanves, France
Olivier Gafsou
[email protected]
Euskaltel S.A.
Parque Tecnologico, Edif.809
48160 DERIO (Bizkaia), Spain
Leire Aginako
[email protected]
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 68 of 71
9. Project Deliverables
D1 Project summary Feb 2002
D2 Project handbook Feb 2002
D3, D4,
D5
Applications: End users scenario’s, Storyboards and Specifications Jul. 2002
D6 Platform requirements specification Aug. 2002
D7 Evaluation results of interactive design concepts and applications
scenarios
Nov. 2002
D8 Tools Oct. 2002
D9 Report on Business Framework Nov. 2002
D10 Specification of terminal software architecture – An XML/SMIL based
presentation sub-system
Feb. 2003
D10, D13 Specification of terminal software architecture – The structure for the
end user trial and test platforms
Apr. 2003
D11, D12 Prototypes of applications, user interfaces and end user tools Apr. 2003
D14 Tools prototype – Accompanying information to software deliverable Apr. 2003
D15 Evaluation results of prototype May 2003
D16, D18 Interactive TV applications: prototypes, methodology, user interfaces
and interaction devices.
Nov. 2003
D17, D20 Operational Demonstrator Nov. 2003
D19 Business framework evolution, IPR aspects and recommendations Nov. 2003
D21 Tools Dec. 2003
D23 Evaluation results of application on integrated platform in home
environment
Feb. 2003
D22 Final Report Jun. 2004
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 69 of 71
10. Publications, presentations and
demonstrations
10.1. Papers and presentations
Overview of papers and presentations listed by first author and their affiliation.
Author Title of paper
A. Scheller
FhG FOKUS
Mehr als Fernsehen. IFA 2003: TWF Science and Technology Forum: Preview of
tomorrow’s media world, August 2003, Berlin.
B. Neville
(ICSTM)
A Computational Framework for Social Agents in Agent Mediated E-Commerce. 4
th
International Workshop of "Engineering Societies in the Agents World", (ESAW),
London, UK, 29-31 Oct. 2003.
D. Targett
ICSTM
Towards a general taxonomy on business models, Journal of Strategic Information
Systems, to be submitted for publication.
D. Targett
ICSTM
A meta-framework for the evaluation of new consumer technologies, to be submitted for
publication.
F. Alavi
ICSTM
Trading Digital Goods I: Econometric Analysis, IEEE Internet Computing, to be
submitted for publication
F. Alavi
ICSTM
Trading Digital Goods II: Game-Theoretic Analysis, IEEE Internet Computing, to be
submitted for publication.
J.Barria
ICSTM
Business Framework: Producers, Consumers, Content. FP6 IST Networking Sessions,
Milan, Italy, October 3-5, 2003,
Jun Hu
Philips Research
Enabling Distributed Interfaces for Immersive Storytelling. Poster: Interaction Design and
Children August 2002 Conference, Eindhoven University of Technology, 2002
Jun Hu
Philips Research
An adaptive architecture for presenting interactive media onto distributed interfaces. In
the Proceedings of the 21st IASTED International Multi-Conference on Applied
Informatics (AI 2003), P899-904, Innsbruck, Austria, 2003. NL-MS-22.374, Oct 10,
2002.
Jun Hu
Philips Research
An Agent-based Architecture for Distributed Interfaces and Timed Media in a Storytelling
Application. In the Proceedings of the 2nd International Joint Conference on Autonomous
Agents and Multi-agent Systems, Melbourne, Australia, 2003, P1012-1013
Jun Hu
Philips Research
StoryML: Enabling Distributed Interfaces for Interactive Media. The Twelfth International
World Wide Web Conference, P135. Budapest, Hungary, 05-20-2003
Jun Hu
Philips Research
Rapid Prototyping for Interactive robots. 8
th
Conference on Intelligent autonomous
systems (IAS-8), Amsterdam, The Nether-lands. To appear 2004
L. Nixon
FhG FOKUS
Addressing the 'what' and 'how' of integrating Internet content with audio-visual material.
IEEE International Symposium on Consumer Electronics ISCE02, Erfurt, Sept 24-26,
2002
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 70 of 71
L. Nixon
FhG FOKUS
Integrating multimedia components into a Semantic Web. ERCIM News, Special:
Semantic Web. Number 51, October 2002, Page 38/39
L. Nixon
FhG FOKUS
A Semantic Web based System for realizing Multimedia Presentations from
heterogeneous XML based sources. XML-Technologien für das Semantic Web (XSW)
October 2003, Berlin.
L. Nixon
FhG FOKUS
Building semantic interoperability into a content integration application, The Eighth IEEE
Symposium on Computers and Communications (ISCC) July 2003, Antalya, Turkey
Maddy Janse
Philips Research
Presentation of ICE-CREAM and NexTV projects to course participants of the Media
Academie, Hilversum, March 7, 2002
Maddy Janse
Philips Research
NexTV: From Technology to Experience, EURO-China 2002, Beijing, April 16-20, 2002.
NL-MS 21.945
Maddy Janse
Philips Research
What makes media compelling, EBU Multimedia Forum, Geneva, May 16-17, 2003. NL-
MS 21.965
Maddy Janse
Philips Research
What makes media content compelling- year 2010. Workshop Danish Broadcast,
Copenhagen, Sept 3-4, 2002.
Maddy Janse
Philips Research
Ambient Intelligence: Interaction Technologies for the Home of the Future. Human
Interface 2002 Conference, Ede, NL, Oct 1, 2002.
Maddy Janse
Philips Research
Interactive services for next generation users. Workshop: The different flavours of
interactive media, at the IST 2002 Information Society Technology Conference,
Copenhagen, Nov 4-6, 2002.
Maddy Janse
Philips Research
Workshop: ‘Converging technologies: new horizons for users and businesses’, FP6 IST
Networking Session, Milan, Italy, October 3-5, 2003
Maddy Janse
Philips Research
Converging technologies: new horizons for users and businesses, Workshop, IST2003,
Milan, October 2-5, 2003.
Maddy Janse
Philips Research
Content Management and User Navigation. Panel, IBC 2003 conference. Digital life
styles, Amsterdam, September 11-15, 2003
Maddy Janse
Philips Research
Interactive Applications: Special Devices. Workshop, IST E3 Concertation meeting,
Brussels, March 13-14, 2003
Maddy Janse
Philips Research
Converging Technologies: New Horizons for Users and Businesses. ENTER conference -
International Federation for IT and Travel & Tourism. Cairo Egypt, January 26-28, 2004
Maddy Janse
Philips Research
Internet applications: How Natural and Indispensable Can they Become. Philips
InterWebT Conference, Eindhoven, March 3 2004
Mark van Doorn
Philips Research
Ambient intelligence by mass customisation in ambient narratives. Journal of Persuasive
Computing, Submitted for publication 2003.
P. Pleven (SV) ICE-CREAM: Production. Workshop, IST E3 Concertation meeting, Brussels, March 13-
14, 2003.
P. Schickel
BITmanagement
Business opportunities with standard technology in diverse markets. FP6 IST
Networking Session, Milan, Italy, October 3-5, 2003
P. Schickel
BITmanagement
Front-end solutions for portals of next generation. 8
th
International Conference on 3D
Web Technology. St. Malo, France. March 12, 2003.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 71 of 71
Warner ten Kate,
Philips Research
Internet: Technology to Host Intelligence. Chapter in book: Emile Aarts and Stefano
Marzano, Ambient Intelligence, July 2002, NL-MS 22.178
10.2. Demonstrations and Exhibitions
Event or exhibition Nature of demonstration
Net-at-home, Cannes,
14-16 Nov 2002.
Two application concepts were demonstrated in the ICE-CREAM booth:
Interactive travel application and live football (FhG FOKUS, Tomorrow Focus,
NOB). A poster presentation was part of the exhibition (Philips)
IBC2002, Amsterdam,
13-17 Sept 2002
Symah Vision presented an early version of the platform for Live Production
Tools at the IBC in September 2002.
IFA 2003 Berlin, 29 Aug
– 3 Sep’03
Exhibition: ICE-CREAM booth within IST village. Demonstration of: Interactive
travel application with travel guide service and personalisable mobile application
and 3D version of the football application
IBC 2003 Amsterdam,
11-16 Sep 2003
Exhibition: ICE-CREAM booth within IST village. Demonstration of:
Interactive travel application with travel guide service, personalisable mobile
application. & Corresponding authoring tool.
Integrated football application with MPEG-4 on a MHP set-top box and 3D
animations and replays based on live data,
DeepSea application with 3D graphic objects and ambience effects.
CeBit 2003, Hanover, 12-
19 Mar., 2003
Exhibition: Demonstration of 2D and 3D MPEG-4 prototype.
Film and video meets 3D
conference, Hanover,
April 2003.
Conference and Exhibition. Organized by the government of Lower Saxony the
exhibition showed possible future trends in the area of integrated media.
Demonstration of 2D and 3D MPEG-4 prototype.
CAT.PRO, Stuttgart, 07-
11Oct 2003
Conference and Exhibition: Demonstration of 3D MPEG-4 soccer application
and tools.
TRIAL, Nov-Dec 2003 Trial of soccer application with end users in the Euskaltelium building in Bilbao
(Spain)
MFG Medien
entwicklung. Baden-
Württemberg Public
Workshop, 2-3 May
2003
Presentation of ICE-CREAM mobile travel guide scenario and prototype
application.
Philips InterWebT
Conference, Eindhoven
March 3, 2004
Demonstration of the ICE-CREAM football application
doc_584090804.pdf
The goals of the project were to extend the notion of interaction, to exploit domestic activities and familiar settings, and to make the user environment part of the visual experience
DELIVERABLE
IST-2000-28298
ICE-CREAM
Deliverable D22
ICE-CREAM-phr-0404-01/Janse
April 2004
The ICE-CREAM Project
Final Report
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 3 of 71
Project Number : IST-2000-28298
Project Title : ICE-CREAM
Deliverable Type : Report
Deliverable Number : D 22
Internal Project Number : ICE-CREAM-phr-0404-01/Janse
Contractual Delivery Date : 03 04
Actual Date of Delivery : 04 04
Title of Deliverable : The ICE-CREAM project: Final Report
Contributing workpackage : WP0
Nature of Deliverable : Report
Author(s) : Maddy Janse (Ed.), all partners
Abstract
The ICE-CREAM project investigates how to make compelling experiences for end-users based on
enabling technologies for interactive media, such as DVB-MHP, Internet and MPEG-4. The goals of the
ICE-CREAM project are to extend the notion of interaction, to exploit domestic activities and familiar
settings, and to make the user environment part of the visual experience. The ICE-CREAM project
develops applications in which live events (for example a soccer match) are enhanced on consumer
demand with content that is created in real-time, broadcast streams are enhanced with content
available from the Internet, and objects in the home are part of the presentation. The application
prototypes show different levels of interactivity for end-users. Business modelling frameworks are
developed to specify the interchangeable roles of stakeholders (i.e., creator, producer, and consumer)
to the service.
Keyword list
Immersive TV, Broadcast, Internet, DVB-MHP, MPEG-4, XML, SMIL, Interactive media, interactive services, user
interaction, interaction tools, application prototypes, business models.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 4 of 71
Blank page
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 5 of 71
Executive Summary
The ICE-CREAM project has been investigating how to make compelling experiences for end-
users based on enabling technologies for interactive media, such as DVB-MHP, Internet and
MPEG-4. The goals of the project were to extend the notion of interaction, to exploit domestic
activities and familiar settings, and to make the user environment part of the visual experience. To
this end, the project has developed applications in which live events (for example a football game)
are enhanced on consumer demand with content that is created in real-time, broadcast streams
are enhanced with content available from the Internet, and objects in the home are part of the
presentation. The application prototypes showed different levels of interactivity for the end-users.
Business modeling frameworks were developed to specify interchangeable roles of stakeholders
and to determine factors for assessing the business relevance of the applications. Public
demonstrations of these applications have been made at IFA 2003 and IBC 2003. These
demonstrations were highly appreciated by the visitors, e.g., Time Warner, BBC. The thematic
and technical differentiation of the three ICE-CREAM applications proved to be very successful in
demonstrating the technical and conceptual possibilities. They integrated different technologies
and demonstrated the potential for opening new avenues for services and platforms. Furthermore
they showed convincing evidence for migration of MPEG-4 and DVB-MHP, for Internet
interconnectivity and integration of 3D graphics technologies. The project adapted to trends in the
outside market and in the research world by explicitly working on applications that illuminated how
the addition of Internet can create new opportunities for applications and services.
User appreciation of applications is at the heart of the ICE-CREAM project and providing
consumers with interactivity in the applications is one of the driving forces. We operationalized
qualitative terms like interactivity, experience and appreciation in such a way that they could be
used to assess the applications. Different methodologies were used to obtain user feedback and
input. Users perceived our concepts as very promising and interesting. The results also showed
that there is still a long way to go with regard to development of methodologies for evaluation, the
design of assessment instruments, the exploitation of the possibilities for interaction with content
by users in their environments, at home or on the move, and the exploration of ambiance effects
in synchronization with events in the content.
Since content is a crucial and extremely valuable asset for any application, the project
investigated the possibilities for multiple usage of content by giving users different views on it and
by extending the interaction to the user’s physical environment and by distributing it over different
devices. The fact that media objects in a movie may also be served from the Internet, can provide
all kinds of opportunities for convergence or competition between Internet and broadcasting
services. Interaction with 3D virtual environment and objects in a broadcasting context is another
venue for new applications and businesses. The idea of using physical objects (such as lights
and toys) to connect the virtual world with the reality is an interesting concept for game-like
applications, but above all for creating a surround experience and affecting people’s perceptions.
The development of a personalizable MHP service with mobile extensibility may lead to a fruitful
exploitation of future 3G mobile services.
Three applications were developed in the project. Their characteristics can be described as
follows:
a). Live video enhanced with real-time events. Hot and cold contextual information is provided
during a football game. Highlights, video-clips and interactive 3D animations of goals and game
situations can be replayed on-demand. This application represents a familiar topic and context for
the user with regard to interaction and content consumption. The user interface is implemented
on a Trimedia set-top box, with a familiar remote control, trusted TV appearance, and live
broadcast. The application achieved unique propositions for users, such as:
• Enhancements with real-time events, contextual information and highlights available on
demand,
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 6 of 71
• Interactive 3D animations based on field measurements for replay, reconstruction and
analysis of the game,
• Changing viewing positions, from the player’s position, following the player and from behind
the player,
• Graceful aging of enhanced content presented in the user interface,
• Overviewing the game from above and measuring positions.
b). Video events enhanced with related content. Enhancements can be directly connected to
events in the video or accessed independently. This application represents a familiar topic and
context for the user, i.e., TV travel documentaries/programs. The interaction, however, is more
familiar to users of Internet and mobile devices as the additional information is linked to the video
content and can be ported for use on the move. The user interfaces were adapted to each device,
i.e., set-top box, connected PDA and 3G mobile phone. The application achieved unique
propositions:
• For users: access to content enhancements in relation to the video content and specific
scenes, e.g., attractions, what’s on agendas, quiz participation, ticket booking,
announcements, SMS messages, user preferences, tour lay-outs and suggestions.
• For providers: flexible and easy customization of content and services, content-centric
services, different means for publishing and presentation of content, integration of different
service platforms.
c). Video enhanced with ambiance events. Content is presented in a non-linear fashion to enable
multiple users to watch different facets of a story. Lights and toys are used as interaction devices.
This application is rather unfamiliar for the average TV viewer. Elements of this scenario,
however, are familiar for people who play computer games, visit theme parks and explore 3D
virtual reality environments. The application achieved unique propositions for users, such as:
• Ambiance effects with room lights to support the suspense of the scene,
• Different content elements displayed on different screens and enabling challenging and playful
situations,
• Different modes for interacting with the scenes, i.e., discovery, game, drama,
• Different interaction devices, interactive displays, toys,
• Role-play when multiple users are participating.
The results of the ICE-CREAM project are very promising for the strategic crossroad of the
converging worlds of TV and Internet. The project showed the importance of multidisciplinary co-
operation between different stakeholders as demonstrated by the composition of the consortium.
The advantages of integrating different technologies and the use of user-centered methodologies
for application development were demonstrated. It is recommended to implement the integrated
platform, to generalize the applications to other domains and topics, and to ensure the availability
of easy-to-use authoring and editing tools.
Flexibility and adaptability to changes in environment, in user demand and in user behaviour were
the driving factors for assessing business models and for analysis of the three ICE-CREAM
applications. Marketing factors, like product range, customer value, positioning, were identified
and compared for each application scenario.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 7 of 71
Table of content
1. INTRODUCTION...............................................................................................................9
2. ICE-CREAM APPLICATIONS .........................................................................................11
2.1. LIVE VIDEO ENHANCED WITH REAL-TIME EVENTS (FOOTBALL APPLICATION) ........................12
2.2. VIDEO ENHANCED WITH RELATED CONTENT (TRAVEL APPLICATION) ..................................13
2.3. VIDEO ENHANCED WITH AMBIANCE EVENTS (DEEPSEA APPLICATION).................................15
2.4. COMPARISON OF THE THREE APPLICATIONS......................................................................18
3. PLATFORM, TECHNOLOGIES AND TOOLS..............................................................19
3.1. BROADCAST ORIENTATION – CASE 1 ...............................................................................21
3.2. INTERNET ORIENTATION – CASE 2 ....................................................................................25
3.3. MOBILE ORIENTATION – EXTENSION OF CASE 1 AND CASE 2...............................................26
3.4. FUTURIST ORIENTATION – PC MEETS TV.........................................................................29
3.5. CLIENT PLATFORM...........................................................................................................30
3.6. COMPONENTS IN THE PRODUCTION CHAIN. ......................................................................30
3.7. TOOLS............................................................................................................................32
4. USER FEEDBACK...........................................................................................................38
4.1. EVALUATION OF THE FOOTBALL APPLICATION....................................................................38
4.2. EVALUATION OF THE TRAVEL APPLICATION .......................................................................42
4.3. EVALUATION OF THE DEEPSEA APPLICATION.....................................................................44
4.4. SUMMARY........................................................................................................................51
5. BUSINESS FRAMEWORK..............................................................................................52
5.1. LEVELS OF INTERACTIVITY................................................................................................52
5.2. PERSONALIZATION AND FAMILIARITY..................................................................................54
5.3. BUSINESS RULES AND IPR..............................................................................................58
5.4. EXTERNAL MARKET CONDITIONS AND CHANGE..................................................................60
5.5. RECOMMENDATIONS........................................................................................................61
6. CONCLUSIONS...............................................................................................................63
7. REFERENCES.................................................................................................................65
8. PROJECT CONSORTIUM.............................................................................................67
9. PROJECT DELIVERABLES...........................................................................................68
10. PUBLICATIONS, PRESENTATIONS AND DEMONSTRATIONS...........................69
10.1. PAPERS AND PRESENTATIONS .....................................................................................69
10.2. DEMONSTRATIONS AND EXHIBITIONS............................................................................71
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 8 of 71
Blank page
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 9 of 71
1. Introduction
ICE-CREAM – Interactive Consumption of Entertainment in Consumer Responsive, Engaging &
Active Media
The ICE-CREAM project was a European Commission sponsored two-year project lead by
Philips. The project consortium partners were Philips Research (in the Netherlands and in
France), Bitmanagement, de Pinxi, Euskaltel, Fraunhofer FOKUS, Imperial College London, NOB
cross media facilities, Symah Vision, and Tomorrow Focus.
The goals of the ICE-CREAM project were:
• To investigate how to make compelling experiences for end-users based on the possibilities of
integrating technologies for interactive media, such as, DVB-MHP, MPEG-4, and Internet
technologies.
• To extend the notion of interaction, to exploit domestic activities and familiar settings and to
make the user environment part of the visual experience.
The basic ingredients for the ICE-CREAM project were: platform and tool technology, business
model development, manipulation and enhancement of content, and interaction possibilities for
end-users. They reflect the challenges and directions that we see as crucial for the development
of applications and technologies based on the development of open standards and platforms
which elaborate on the integration of DVB-MHP, MPEG-4, Internet technology and 3D graphics.
Business models are an intrinsic component for the development of applications and
determinants for their success. The quality of the content, how the end-user can use it and
whether this can be done in a fun, easy and appropriate way, however, are the most essential
ingredients for end-user applications. Three applications were developed in the project. They were
chosen such that they covered a wide range of technical, business, social and conceptual
domains.
Converging technologies make it possible for new application and service scenarios to emerge.
Multimedia content is becoming a commodity and the roles of producer and consumer are
becoming interchangeable and dynamic. Consumers are becoming content creators who can
publish and share content, interact with it and present and store it on all kinds of devices.
Consumer electronics devices with a connection to the Internet offer users new ways of
interaction with content and communication with one another. These possibilities pose enormous
challenges for open standards and rights management, for interoperability between competing
businesses, and for defining appropriate business models. But above all, of identifying,
communicating and explaining what the benefits are for consumers, the end-users of these new
applications and services.
For example, Internet connectivity gives people unlimited possibilities for acquiring information and
enjoying all kinds of entertaining applications. But, this assumes that people are accomplished
‘do-it-yourselves’ and that they are always prepared and in the mood to ‘find, shop, design,
compose and create’ their own content and entertainment. This is not necessarily the case, as
we also know from other domains. For example, the same people that love to cook five star
dinners also enjoy the fast-food take-out and their decisions are based on zillions of different
reasons. Another major assumption made for future applications that are enabled by Internet
connectivity, such as, interactive television, networked gaming, on-line communities, multi-users
portals, is that they will enhance the shared media experience of people. This might be a valid
assumption from the technical point of view. From the behavioural point of view, however, it is
rather provocative. Questions such as: What does sharing media/content mean for people? To
what extent are people’s expectations and goals met? Does it hold for all content? Do people
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 10 of 71
really need to share content to increase their experience or is it only a technical requirement?
need to be considered. Furthermore, we have to take into account that activities like television
viewing, making phone calls, sending messages, watching photographs, going to the movies or
the ball game, are already shared experiences. In other words, human behaviour needs to be
taken into account to consider the possibilities of enriched and interactive applications.
Broadband connections make it possible for people to share all kinds of digital content. People are
very familiar to sharing tangible objects, consumables and experiences. They have well-
established social rules for these activities, which determine property ownership, initiation, turn
taking and so on. These rules are not so clear-cut for sharing digital content or for systems in
which people can have a video chat. As connectivity rises between devices in (different)
environments, the nature of social interaction between people changes. Systems will more and
more become enablers for social interaction and we need to start thinking about introducing
technology as a social mediator. These developments raise many questions with regard to
human behaviour and perception that application development has to take into account.
The ICE-CREAM project adopted a user-driven development approach. Application scenarios
were developed which comprised all ingredients, i.e., platform and tool technology, business
model development, content manipulation and enhancement, and interaction possibilities for end-
users. These scenarios functioned throughout the two-year project as the principal
communication mechanisms for maintaining focus and for ensuring that user objectives were at
the heart of the applications. The next chapter describes the applications that were developed in
the ICE-CREAM project. The applications will be described from the user perspective. Chapters
will follow on platform and tool development and implementation, user feedback and business
frameworks.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 11 of 71
2. ICE-CREAM applications
Established interactive TV services like EPG, shopping, and super teletext services often exist
independently from the current TV program. The video broadcast and the additional services just
happen to be presented on the same device, but are rarely integrated to achieve an enhanced
viewing experience. Enhanced broadcast often means a coexistence of various services, which,
for the viewer, only provides the benefit of the sum of the available services, instead of allowing
the services and the video broadcast to support each other and provide a benefit that is higher
than the sum of the individual benefits. To achieve this added value, all ICE-CREAM applications
were designed to integrate video content and additional services closely.
The ICE-CREAM applications explore the possibilities of using rich interactive content and
integrating different technologies. Four dimensions exemplify the relation between the applications
and the challenges that they address. They reflect the directions that we took as crucial for the
development of applications and open standards technology. The four dimensions are presented
in the diagram in Figure 1. The first dimension is determined by the technology for platform and
tools, i.e., the migration of DVB-MHP and MPEG-4, the integration of this platform with Internet
technology and the integration of 3D graphics to the platform. This is the left axis in the diagram of
Figure 1. The second dimension is determined by the interaction possibilities for the end-user.
This is the right axis in the diagram of Figure 1. The variables that are most important are: the
levels of interactivity that the user can have, from bi-directional remote control, to content editing,
to ambiance and emotion creation. The third dimension constitutes the development of the
business models. This is the top horizontal axis in the diagram of Figure 1. The hypothesis is that
for interactive media applications the business models will evolve from static to dynamic models.
The fourth dimension constitutes the possibilities for content enhancement and manipulation. This
is the bottom axis in the diagram. The applications addressed these dimensions, i.e., the
interaction requirements for the users and the potential of the platform, the use of rich interactive
content for real-time events, enhancement and personalization of content from Internet, the use of
non-fictional content for creating ambiance and experience. The arrows in the diagram reflect the
order of complexity, i.e., challenge and novelty for the applications. The project applications that
will be addressed are positioned along the diagonal. The focus for this positioning is the level of
interactivity and emotional involvement for the end-user, i.e., at the client side.
DVB-MHP
MPEG-4
Internet
3D Graphics
DVB-MHP
MPEG-4
Internet
DVB-MHP
MPEG-4
Enhancements
Dynamic
Edited content
Fictional content
Multi -layered
RC+
Messaging
Scene control
Emotion
experience
P
l
a
t
f
o
r
m
,
T
o
o
l
s
,
T
e
c
h
n
o
l
o
g
y
Content Manipulation & Enhancement
I
n
t
e
r
a
c
t
i
o
n
f
o
r
U
s
e
r
s
Business Models
Static
Dynamic
A
p
p
l
i
c
a
t
i
o
n
s
Figure 1: Dimensions for the selection of applications
Three applications were developed in the ICE-CREAM project. These applications can be
characterized by the properties of their contents and enhancements:
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 12 of 71
• Live video enhanced with real-time events
• Video enhanced with related content
• Video enhanced with ambiance events
2.1. Live video enhanced with real-time events (Football application)
In this application a live football match is broadcast and enhanced with commentary and
annotations of player actions and additional hot and cold statistics. Selected scenes can be
replayed in video clips or in 3D animated form. This application represents a familiar topic and
context for the user with regard to interaction and content consumption. The application shows
the possibilities and benefits of integrating DVB-MHP, Internet, MPEG-4 and 3D graphics and
animations. This integration is especially appropriate for enhancing the broadcast of live events,
which are characterized by existing action, and timing.
The major characteristics of this application are:
• Integration of Internet, MPEG-4 and MHP in TriMedia set-top box
• Delivery of real-time interactive multimedia event clips generated by tracking tools and
cameras on the field
• Presentation of video events on-demand
• Combination of background "cold" and timely "hot" contextual information during the football
game
• Playback of the event clips while watching the program
• Replay of interactive 3D animations of goals and game situations on-demand
User walkthrough
Users watch a football game on TV. The TV broadcast is enhanced with background information
about this game (cold statistics). This background information contains, for example, details about
the players, about the team and championship achievements. Furthermore, when a remarkable
event occurs in the game, the multimedia information about this event is sent to the set-top box –
‘hot statistics’. These events are, for example, a goal, an offside, a pass. The set-top box filters
this incoming information and displays it if the user demands it. A special icon that indicates the
presence of on-demand content is displayed on the TV screen when either background or real-
time event information is available. The user can decide to see this information or not.
Background information about the game (cold statistics) and the real-time events as particular
game scene replays (hot statistics) can be displayed. The information of the real-time events (hot
statistics) is encapsulated in a data-packet that is broadcast. For example, the commentator
describes an action and links it to a certain player. This information is put in the event data-packet.
Background information (cold statistics) can be accessed anytime during the game via the menu.
This means that the viewer can access information about, for example, all players in the game.
Real-time event information, i.e., video scenes of specific events, can be replayed during the
game. These event-packages pile up while the game goes on and become unavailable
(disappear) after some time, unless the viewer has marked them as ‘favourite’ on the set-top box.
Positions of the players are measured during the game by means of image processing using at
least one camera on the actual site of the football match. Users can interactively explore
significant scenes in a 3D environment. They can individually select a point from which to watch
the replay, view it from the position of the goal keeper, referee or other player, forward the scene
events step by step, measure distances, and do other things. Figure 2 shows events that are
added to the favourites and the playing of a replay. Figure 3 shows a 3D replay from different
points of view.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 13 of 71
Figure 2: Event added to favourites and
playing of replay
Figure 3: 3D replay –field view
2.2. Video enhanced with related content (Travel application)
In this application a travel video is enhanced with specific information about the various attractions
presented in the video and integrated with other travel related information, which is either part of
the broadcast or retrieved from the Internet. This application represents a familiar topic and
context for the user, i.e., TV travel documentaries/programs. The interaction, however, is more
familiar to users of Internet and mobile devices as the additional information is linked to the video
content and can be ported for use on the move. The user interfaces were adapted to each device,
i.e., set-top box, connected PDA and 3G mobile phone.
The major characteristics of the application are:
• For users: access to content enhancements in relation to the video content and specific
scenes, e.g., attractions, what’s on agendas, quiz participation, ticket booking,
announcements, SMS messages, user preferences, tour lay-outs and suggestions.
• For providers: flexible and easy customisation of content and services, content-centric
services, different means for publishing and presentation of content, integration of different
service platforms.
User walkthrough
Initially the users watch a travel video without any additional features on an MHP set-top box. As
soon as additional services that are related to the video content become available, a window
appears on the screen with the information. Figure 4 shows some of these screens. Users can
specify their preferences, start and stop the services explicitly or automatically. Table 1 shows the
services that users have available.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 14 of 71
Switching a service on
Setting user preferences
Quiz question
Chapter markers and video bookmarks
Figure 4: Travel application: services
Table 1: Services available to users of the Travel application
Service Description
Quiz While a timer is running, users can select answers with the
numeric keys on the remote control for multiple-choice
questions that are presented at specific moments in the
video. The correct answer is highlighted after time-out and
users receive a score if they had the correct answer.
Attraction information A menu shows additional information for attractions in
relation to the subjects in the video. Users can view,
bookmark and store this additional information by navigating
the menu with the colour keys of the remote control.
SMS and other
messages
Users can send SMS messages to a special phone number
to be displayed on the screen. Broadcasters can also attach
messages to the video, which will be displayed in the same
way as the SMS messages. These messages contain
relevant information about the video and its attractions.
City tours While the main video runs in a PIP (Picture in Picture)
window, predefined city tours are presented as short
teasers, i.e., concise overview of the tour theme. Users can
change between main screen and PIP. Users can select
and access these tours with the arrow keys on the remote
control, bookmark information about specific attractions and
create new city tours that will be added as Personal Tour to
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 15 of 71
the list of available tours. Book-marked information is also
sent to the mobile device.
Chapter markers and
video bookmarks
Each topic in the main video has a marker event. These
events or bookmarks are listed and displayed as chapter
names. They can be used to find and replay specific
sections of the video. User’s bookmarks are also listed as
markers.
Ticket booking Users can browse, select and book tickets for events that
occur in the attractions that are shown in the video. The
booking system queries and displays upcoming events. The
booking information is also transferred to the mobile device
to have it available during the actual travel.
Applications on mobile
device
Users can select cities of interest on their connected PDA
and access related services during travel from the city home
page, for example, sightseeing information, attractions,
upcoming events, ticket booking services, and personal
tours.
The location based services include, e.g., a “Where am I?”
function, which displays a map of the surroundings and
nearby attractions for a given street address, a display of the
route from the current position to a given destination.
2.3. Video enhanced with ambiance events (DeepSea application)
In this application a video is enhanced with fictional content, which uses 3D graphics and
animations. Video streams, 3D computer graphics, text, still pictures, soundtracks and
multilingual interfaces are combined. The interactive content is distributed to multiple devices
(lights, portable displays, robotic toys) to create ambience effects in the performance space – the
end user's environment.
The major characteristics of this application are:
• Distributed interfaces, which enable co-operative interaction with the movie among multiple
users.
• Synchronized lighting effects and robotic behaviours connecting the virtual 3D world to the
reality.
• Grabbing and storing of content elements for later reviewing by using a portable display.
• Providing users with a range of interaction possibilities, such as customisation of the journey,
interaction with content elements, and full control of navigation.
User Walkthrough
The fictional content of the application concerns a submarine adventure which has three different
interaction modes: the automated (drama) mode, the discovery mode and the game mode. The
content is composed of an undersea area with plants, some fifty different species of fish, a few
buoys, and four shipwrecks as archaeological sites, and a submarine as exploration vessel.
Users may discover these objects and interact with them. The boundary of the virtual space is
indicated with rocks. If the submarine hits the boundary, it bumps on the rocks and stagnates,
while a caption appears on the large display saying “NO WAY”. The surface of the water and the
bottom of the sea are also the boundaries. The presence of the submarine is linked to the
presence of a toy submarine. If the toy submarine does not exist or has been moved away, the
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 16 of 71
submarine in the virtual 3D space disappears and a cursor frame is presented instead. The 3D
scene is captured from right behind the submarine and the camera angle follows the direction of
the submarine. So the user can always see either the backside of the submarine or a cursor
frame from a first person point of view. When the user changes the direction of the submarine,
the camera angle changes accordingly.
Table 2 gives an overview of the interaction modes and describes what the user will see and can
do in these modes.
Table 2: Interaction modes in the DeepSea application
Interaction mode What the user can see and do
Automated mode (no control by the user).
This is the default mode in which the user
watches the show without any interaction
with the content. It automatically starts
when the user does not make a choice
among different modes.
This mode shows a panoramic view of
the 3D undersea space. The submarine
moves slowly in the space following an
automated route and the scenes look
smooth. This mode lasts about 30
seconds. It is followed by the game
mode, which commences automatically.
Game mode (limited navigation control).
This mode allows the user to move the
submarine in only two directions, left and
right, to navigate around obstacles, for
example, mines. The submarine drives
ahead in a fixed speed and the user
cannot change the overall direction or
speed of the submarine.
The goal for the user in this mode is to
avoid obstacles by manoeuvring the
submarine to the left or right. The
feedback on failure consists of light
effects that accompany the explosion
when a mine is hit, or of stagnation and
bumping of the submarine when it hits an
obstacle. This mode lasts about 30
seconds and ends with loud alarm
sounds and flashing red lights, after
which the submarine is out of the user
control and floats to the surface through a
narrow tunnel. The user receives a score,
which correlates to the performance of
avoiding the mines.
Discovery mode (full navigation control).
This mode allows the user to freely
navigate in the virtual space by controlling
the direction and speed of the submarine.
The user can move the submarine
forward, left, right, up and down at
different speeds, but not backward.
The goal of this mode is to wander
around at a preferred speed with the
freedom of direction. The user can try to
‘catch fish’ by moving the cursor frame or
submarine towards any of the fish in the
scene. When close enough, the fish is
caught, the picture of which is presented
on a secondary display (iPronto
interaction device) and stored for later
review. This activity is solely based on
user initiative. The user can also enter
shipwrecks. This track last about 4
minutes and ends in the same way as the
game mode.
The user starts the DeepSea application (Figure 5) by pushing a start button on the touch screen
of the interaction device (iPronto). On the display panel appears a submarine sailing on a sea and
then diving into the sea, while the background music starts. When the submarine arrives at the
bottom of the sea, the viewer sees a small TV set in the scenery, which turns on and shows a text
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 17 of 71
that cues the viewer to choose a language (English, Dutch or French). The user selects a
language on the iPronto by touching a button on its screen. Then, the small TV displays the three
modes and suggests the user to select one by pushing the appropriate button on the iPronto
graphical user interface. If the user does not choose a mode by giving a response on the iPronto
within a certain time, the automated mode (the default mode) starts. All three modes have a time
limit. The automated mode lasts 30 seconds, the limited-interaction mode 30 seconds, and the
full-interaction mode 4 minutes. After the time limit has been reached, the user looses control over
the content object and the submarine float to the surface through a tunnel. The graphical user
interface on the iPronto allows the user to switch the presentation from one mode to another
during any of the modes, and the new mode starts from its initial position. The user can also
customize the interface language and restart the entire application anytime after opening.
Figure 5: DeepSea application: walkthrough
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 18 of 71
2.4. Comparison of the three applications
From the end-user point of view, the use of content and the interaction possibilities are the most
interesting characteristics. These characteristics are summarized in Table 3. Table 3 also
illustrates the span of possibilities covered by the three application scenarios.
Table 3: Comparison of the three ICE-CREAM applications
Characteristics Football
application
scenario
Travel application
scenario
DeepSea
application
scenario
External devices for
interaction
Remote control
Remote control
Connected PDA,
3G mobile phone
Remote control
Interactive display
Toy, Light
Intended group size Single user Single user Group
Creation of content Live Pre-generated Pre-generated
Theme Based on fact Based on fact Based on fiction
Video content Existing Existing Custom made
Synchronization
between video and
additional content
< 1 sec + 1 sec N.A.
Video streams Multiple streams Single stream Single stream
Content
presentation
2D and 3D 2D 3D
The three applications will be worked out in the next sections.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 19 of 71
3. Platform, Technologies and Tools
This chapter describes the terminal software architecture of the systems that were developed for
the applications of the ICE-CREAM project.
The overall purpose is to enhance a broadcast program with interactivity in a personalized
manner. The Internet is used to convey the personal data. The trigg&link paradigm is used to
present the enhancements to the user [1], hence introducing the interactivity. Figure 6 depicts the
overall scenario for the platforms.
Control (XML)
Content
(enhancements)
User
Data base
enhancement
Data base
enhancement
MPEG - 4/SMIL
docs
Content
Provider
Content
Provider
Internet
Main video (MPEG - 2, MPEG - 4)
Triggers
Alerts
Broadcaster
Figure 6: The general scenario context.
Three distinct systems are linked to gain the required functionality: The “Uplink system” (Server
side), the “Transport system” (Network) and the “End-user system” (Client side). The “Server
side” is shown at the left side of the picture. The “Network” is represented by the arrows, and the
“Client side” is represented by the User-box. The main video is produced by a content provider
and transmitted by a broadcaster in the usual way. The broadcast stream is DVB compliant. It is
based on MPEG-2 transport streams and supports DSM-CC private sections. The receiver is an
MHP set-top box.
The focus of the platform for the sports application lies on the production and distribution of the
live events and instant replays. Those live events and replays can be distributed via the Internet
and/or via the DVB broadcast. An abstract overview of this environment is depicted in Figure 7.
The outgoing and bi-directional arrows to the right are linked to an ICE-CREAM terminal via a
direct connection to the Internet (broadband interaction channel) and terrestrial, cable or satellite
receiver (DVB tuner).
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 20 of 71
Production
of the
Annotations and
Video-clips
Enhancements
Video-on-
Demand
Distribution
AV + MHP
Production
DVB Broadcast
Trigger
distribution
Management of
Enhancements
Production Distribution
Internet
DVB
Figure 7: Production – Distribution chain
The same content provider can produce the enhancement application and its enhancement data,
or it can be produced by another, or by several (independent) content providers. We distinguish
between two cases. In the first the enhancement and its interactivity is provided and handled
completely by the broadcaster. In the second case the broadcaster provides the main video, but a
third party offers the enhancement service over the Internet.
Typically, in the first case the broadcaster creates a multiplexed stream containing the
enhancement application (an Xlet) and the various content items that are of general use to the
enhancement. The broadcaster may use the Internet as communication channel for the
personalized interactivity. A typical application example of this case is the Soccer Application with
(3D) MPEG-4 replays.
In the second case, being Internet oriented, the enhancement application is typically using XML
documents. More precisely it is often a SMIL document [2]. The third party has knowledge about
the main video, so that the enhancement and interactivity provide a coherent experience to the
user. The client at the receiver side co-ordinates the integration of main video and interactivity,
which thus often is a SMIL player.
The next step is to enable mobile access to the Internet and broadcast content via e.g. a GPRS
connection. The possibilities of the mobile case are demonstrated in the Travel Application.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 21 of 71
3.1. Broadcast Orientation – Case 1
The first case of the scenario, i.e., the broadcast approach, is represented by the implementation
of the football application. In this application, a live soccer match is enhanced with instant replays,
coded in MPEG-4 format, including 3D graphics. The broadcaster creates the clips, publishes
them on a server on the Internet, and notifies the viewers about their availability through the
trigg&link paradigm. The viewer can replay the clips by downloading them over the Internet. The
clips are prepared and encoded in MPEG-4 format. The receiver contains an MPEG-4 (3D) player.
In this way the viewer can interact with the replay, ultimately choosing different viewing angles.
Figure 8 shows the set-up.
SetTop Box
Internet
DVB-T/C/S
Live Content
Production
MPEG-4
Video on
Demand Clip
Server
DSM-CC Object
Carousel
MPEG-4
Encoder
MPEG-4 Clips
Clips
MPEG-4 Clips
MPEG-2
Encoder &
Multiplexer
Xlet +
Hot &
Cold
Stats
Live Audio &
Video
Carousel +
Events
Events
TCP/IP
Figure 8: Live event with interactive replay.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 22 of 71
The football application is an MHP (Java) Application. This “Xlet” is broadcast in a DSM-CC Object
Carousel: all the Java classes and assets (like the text and pictures used in the cold statistics)
are repeatedly put “on the air” like teletext pages. This carousel is merged with the live audio and
video of the soccer match and broadcast. The MHP terminal (with MPEG-4 replay capabilities)
receives the broadcast and loads and executes the Xlet. When the Xlet is started, it starts
“monitoring” for events (received via Internet or DVB) and it allows the user to interact with it. The
events and clips are generated via “live content production” facilities. The clips are encoded into
MPEG-4 and stored on the server(s) and the corresponding events are multiplexed in the
broadcast and/or sent via the Internet (subscription-based). The live video stream is also analysed
in real-time by Symah Vision tools: The positions of the players are tracked by means of image
processing and the data is stored in XML format. Once the XML description of the position
updates has been retrieved, the 3D content can be reconstructed in an automated way in real-
time to generate the 3D MPEG-4 clips. Figure 9 shows an excerpt from the corresponding XML
code.
<BDD_TRANSFO Index="406" nbItem="404">
<TRANSFO Index="1" TC="32420" K="0" Cx="360" Cy="288" AR="0.9375" IsValid="1">
<H i0="35.90076" i1="21.58959" i2="-1523.892" i3="-4.081663" i4="6.805422" i5="2.290844"
i6="8.287122E-03" i7="-1.377915E-02" i8="1" />
<Hinv i0="2.056601E-02" i1="-1.779707E-03" i2="31.34445" i3="1.233495E-02" i4="0.145979"
i5="18.46273" i6="-4.678071E-07" i7="2.026215E-03" i8="1" />
</TRANSFO>
<TRANSFO Index="2" TC="32422" K="0" Cx="360" Cy="288" AR="0.9375" IsValid="1">
<H i0="35.90015" i1="21.59887" i2="-1524.545" i3="-4.080273" i4="6.80006" i5="2.727768"
i6="8.290958E-03" i7="-1.377944E-02" i8="1" />
<Hinv i0="2.057967E-02" i1="-1.780217E-03" i2="31.3795" i3="1.234871E-02" i4="0.146094"
i5="18.42765" i6="-4.668669E-07" i7="2.027854E-03" i8="1" />
</TRANSFO>
Figure 9: XML positioning information for 3D replay
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 23 of 71
For broadcasting MPEG-4 clips of a 3D replay, the 3D MPEG-4 creation tool (Bitmanagement)
has been extended with a Video generating function. (This is useful for set-top boxes that are not
capable of rendering the 3D graphics themselves). After loading an interactive 3D or 2D MPEG-4
scene and adjusting data such as the position from which to view the scene on the generated
video the user presses a hot key. This hot key switches the MPEG-4 player into video capture
mode. In video capture mode, the MPEG-4 player stops rendering the scene and presents the
user with a list of available CoDecs (see Figure 10). The user can select a CoDec, for example
"XviD MPEG-4 Codec". There is an Option button that allows for adjusting CoDec specific
parameters like compression quality and bandwidth (see Figure 11).
Figure 10: Video capture mode
After the user clicks on "Ok", the MPEG-4 player begins with rendering the scene. Each frame is
captured from the video memory and sent to the selected CoDec for compression. In this phase
rendering takes place with maximum speed and with a constant time increment between the
frames. Thus the resulting video always has the correct frame rate, even if the available
computing power is not enough to render and compress the scene in real-time.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 24 of 71
Figure 11: Adjusting compression quality
For enabling online use on PC hardware the 3D replay content can be embedded in a HTML page
in the MPEG-4 format and put on a web server to be retrieved from the Internet (Figure 12).
Figure 12: Embedding the replay in a html web-page
For environments where bandwidth is very limited, e.g. on mobile devices, a 2D version of the
soccer goal replay can be made. The bandwidth constraints for retrieval of the whole animated
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 25 of 71
scene amount to a few kilobytes (Figure 13).
Figure 13: 2D replay output for mobile devices
3.2. Internet orientation – case 2
In the second case of the scenario we show a prototype where SMIL is used to integrate the video
broadcast stream with additional, personalized interactivity offered by a service provider on the
Internet. The interactivity is offered in the form of a SMIL document [2]. Each SMIL document is
tailored to the preference profile of the user who is requesting the enhancement service, much
like HTML documents are adapted to settings or parameters in a query that is requesting the
document. In this way personalization is realized.
In the prototype we use the broadcast of a classical concert as the main video. Enhancements
are offered through the trigg&link paradigm, which include additional information about the
composer, the conductor (e.g., his homepage), the concert hall (e.g., providing calendar and
ticket ordering), etc. Personalization appears in the form of offering calendar information that is
local to the geographic place from where the viewer is watching. This implementation shows the
feasibility of using SMIL for this purpose and of integrating the SMIL player with the MHP platform.
Figure 14 depicts the architecture.
Personalized part
Generic part
MHP
SMIL
doc
Timing Layout
Xlet: SMIL interpreter
SMIL profile
MHP APIs
Figure 14: Integration of SMIL presentation with MHP.
The SMIL interpreter parses the SMIL document and takes care of the control of such matters as
timing and layout. Control of the MHP box is realized through an Xlet. Since the enhancements are
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 26 of 71
typically shown as an overlay on top of the main video, a new user interface problem has been
introduced. It will be hard to design the overlay in such a way that the obscuring of the main video
does not annoy the user. A straightforward solution would be the presentation of the
enhancements on an additional, separate screen. Of course, the user premises must include
such an additional, personal screen. In particular when multiple users are watching the main video
and each of them is interested in another piece of enhancement content, the use of additional
screens becomes paramount.
The above architecture of the SMIL document, Figure 14, lends itself to such a distributed
extension in a fairly easy way. In case the additional screen is present, another SMIL document is
loaded to the user. It is also conceivable that a single document is created that declares both
situations, in which case the interpreter distinguishes between the forms of presentation. The
layout part of the new SMIL document is modified to declare the layout of content items over
multiple devices. Secondly, the SMIL interpreter is (logically) separated from the MHP player and
takes control over both the MHP player and the additional screens. In this way, synchronized
presentation and interaction is enabled over a distributed configuration of devices. Figure 15
depicts this extension, which is implemented in the current prototype.
Personalized part
Generic part
SMIL
doc
Timing Layout
Xlet: SMIL interpreter
SMIL profile
Distributed
Interface
MHP Additional screen Additional screen
Figure 15: Using SMIL for distributed presentation.
3.3. Mobile orientation – extension of case 1 and case 2
The mobile orientation gives an extra dimension to the two cases described in the scenario. End-
users are now no longer required to consume the interactive content at home; they can have it
anywhere, be it in limited form due to mobile terminal and bandwidth restrictions. The Travel
Application in which video content is enhanced with related content shows these benefits.
The target terminal for the travel application is a standard MHP set-top box with an Internet
connection, which is used for event booking and additional information retrieval facilities, and
additional local storage to provide PVR-like functionality. The PDA is connected to the Internet
using a GPRS connection. Figure 16 shows the production chain of the travel application.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 27 of 71
Figure 16: Production chain of the Travel application
The internal structure of the application that is running on the MHP set-top box is based on a core
module to which travel application specific modules (event booking, city tours) and re-useable
modules with modifiable layouts (e.g., quiz, SMS) are added. The application itself is multiplexed
with the video stream, some parts of the context and the layout descriptions for the re-useable
modules into a broadcast transport stream. The editing tool creates an XML file, containing all
timing information needed for the trigg&link functionality of the attraction information and for other
elements that are synchronized to the video (e.g., chapter bookmarks). This information is then
used by the multiplexer to insert the appropriate stream events into the transport stream. Content
that is not inserted into the video stream is retrieved via the Internet connection. This concerns
mostly image content that will only be used by a small percentage of the users. The Internet
connection will also be used for interactive services (e.g., connection to the event booking
system). If possible, all content (broadcast and that retrieved via Internet) will be stored locally on
the set-top box. Content provision is central to this implementation. Figure 17 depicts the provision
of content and services by different providers.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 28 of 71
MHP
Set-top Box
M
o
b
i
l
e
P
D
A
Service
Provider
Content
Provider
Broadcaster
SMS
On-line
services
Broadcast
additional
information
Services
Additional
content
Information
On demand
Personalization
Figure 17: Content providing in the travel application
The content used in the travel application is retrieved from various sources. Parts of the content is
created using the tools that were developed within the project, other parts are distilled from
existing sources prior to runtime and some parts are retrieved at run time. The layout or ‘look’ of
the travel application on the set-top box is neither coded in the application nor defined by content.
The layout is defined in separate layout definition files, which are generated with a task specific
layout editor and combined with a specific travel video by the application editor. The following
sections describe the flow of information between the components that make up the environment
of the travel application.
Attraction database for MHP application: The attraction information that accompanies the video
presentation constitutes the core content of the MHP city guide. It is located in a CityGuide
database, which contains approximately 6000 attraction descriptions from a large number of
cities. There are approximately 200 points of interest available per city. All descriptions of points of
interest contain basic facts, textual descriptions, photos links and map links. The information is
retrieved as an XML file containing all attractions for one or more cities. A complete set for one or
more cities is downloaded and used by the travel application editor. The travel editor keeps track
of those attractions that are referenced in the video and the city guide and extracts only those
attractions that are actually needed by the application. The resulting XML file can then be
broadcast to the set-top box.
CityTour information for MHP application: CityTours are retrieved from the attraction server as
XML files, containing a tour name, a tour title, a teaser text and a list of item ids, which are
references to the attractions in the attraction database. The appropriate city tour information for a
city is added to the broadcast stream at application edit-stage.
SMS message for MHP application: SMS messages are sent to a specific telephone number and
relayed to the message server. Conceptually, messages will then be inserted as stream events
into the broadcast stream. In the current implementation, the application retrieves SMS messages
from the SMS server using HTTP based communication on the back channel.
Event booking for MHP application: The broadcast stream contains triggers for venues that are
shown in the video. The payload of such a trigger is a reference to the event database. On
encountering such a trigger, the application uses the back channel to retrieve the current event
information for that specific venue from the event database server as XML data. If the user select
on of these events, the booking data is sent back to the event database server using an HTTP
based protocol.
Attractions and event bookmarks from MHP application to PDA application: If the user
bookmarks an attraction or buys a ticket in the MHP travel application, this information is sent to
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 29 of 71
the city guide server. The information sent to the server contains the ID of the event or the
attraction and a user ID. The user specific information is then stored in the city guide server and
used to build a user specific ‘My CityGuide’ web page, that can be accessed with the connected
PDA.
CityTour to PDA: All applications available on the connected PDA are accessed using HTML
pages and forms provided by the CityGuide server, which in turn accesses the CityGuide
database and the event/ticketing server to provide PDA optimised pages for the mobile device.
3.4. Futurist orientation – PC meets TV
The DeepSea application demonstrates a future environment, where broadcast and Internet are
integrated. The 3D scene descriptions of the virtual environments are broadcast to all receivers,
using the advantages of digital broadcast. For the server interactivity and future multi-user (“multi-
player”) aspects, a broadband Internet connection is required.
The platform for the DeepSea application in which fictional video is enhanced with ambiance
events is centred on a Linux PC platform. The PC is equipped with video and 3D computer
graphics capabilities. Linux was chosen for reasons of independence and for future alignment
with the developments for set-top boxes and embedded electronics. The platform is furthermore
composed of peripherals, i.e., external robot, light control, and sound control. The input devices
include a standard remote control, a game pad, an advanced remote control, The application
demonstrator was implemented on proprietary run-time software, name argoKernel™ (De Pinxi).
This kernel is based on openGL for graphics, and raw or MPEG video streams. It integrates multi-
layer video streams, computer graphics, and special I/O.
The DeepSea application was developed from the point of view of an audiovisual production
house. This implied that the development tools had to be suitable for media authors. The
objective was to create a broadcast fiction program and not to develop specific software.
The DeepSea fiction is based on a graphic database in LightWave format. The LightWave
database was first translated in VRML through an exporter and then the VRML was translated in
MPEG-4 with MPEG-4 tools. Since the standard VRML exporter does not support multi-texturing,
this was done manually. Textures were converted in PNG format. The animations were made
with a frame-by-frame position (backing) and not with interpolation (heavier database). A part of
the scenario scripting was implemented with JavaScript. The I/O connection was ported with
Java, which enabled the communication with "network" devices like the i-Pronto but also "serial"
devices like a game pad or a joystick, using Jini. The DeepSea platform uses the midi standard
for the control of sound and ambient lighting.
The presentation and interaction devices included in the application are a plasma display panel
(PDP) as the primary display, an iPronto as a secondary display, as well as speakers, lamps, and
a GamePad controller. The display panel, speakers, and lamps are positioned as elements in an
ordinary living room. The display panel is hung on a wall. The speakers and lamps stand in the
corners of the living room. A GamePad controller and iPronto are placed in front of the end users.
The iPronto is a small portable computer developed by Philips, with a 6-inch touch screen and
wireless network connection. It can be used as a universal remote control for home appliances,
such as, audio and video systems, lighting systems, and security systems. It also has other
applications such as Internet browser and Electronic Program Guide. In this project, we
developed a generic presentation server for the iPronto, in order to present content elements such
as sounds, images and web pages from the local storage, the local network and the Internet. In
the DeepSea application, this presentation server is used to construct the interface for content
navigation and customisation, and to present and store the content elements for later review. We
also implemented a robotic submarine as the physical counterpart of the submarine in the virtual
user-sea space. The user may direct the virtual submarine by tilting the toy and speed submarine
up by squeezing the toy. When the virtual submarine hits mines or obstacles, the toy vibrates to
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 30 of 71
give the user a tangible feedback.
3.5. Client platform
The client platform is more than “just” a terminal on which an MHP+ application is running. Here
“MHP+” means, that the application will not only use the MHP API, but also uses the functionality
as defined in the ICE-CREAM sub-system. Figure 18 depicts the underlying client architecture.
Figure 18: Underlying client architecture
3.6. Components in the production chain.
Figure 19 provides a detailed component view of the production-distribution platform. These
components are:
Annotation & Video-clip Production
During the live broadcast of the sports application several add-ons are created. The different add-
ons created by the Production Tools (WP7) are:
• "Video-clips": video-clips of live actions or video-clips of 3D-simulations.
• "The action annotations": descriptions of the actions of the players with additional background
information for the active players.
• "3D Scenes": 3D visualisations are sent to the users STB in combination with the action
annotations.
Video-clip-on-Demand distribution
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 31 of 71
The video-clips generated by the (automated) production tools are encoded to MPEG-4 Video and
stored on a central file-system. When the stored clips are to be published, a video server can
stream them over IP or they can be inserted in the DVB stream. Before they can be inserted into
DVB they need to be packaged and embedded in PES packets or MPEG-2 Sections. The actual
clip insertion is done in the DVB distribution block.
Management of Video-clips
The video-clips are continuously created and not all video-clips are suited for broadcast. The
director of the program can control the actually publishing of the clips with the Video-clip Manager.
Directly after the clips are created they appear in the GUI of the front-end of the Video-clip
Manager. When the director decides the video-clip has enough added value, he can publish the
video-clip through the user interface of the Video-clip Manager.
The backend of the video-clip manager controls the distribution over DVB or IP and creates the
triggers to be sent to the users.
Trigger distribution
The video-clip manager creates a “single message” that lists the video-clips currently available to
the user and passes this message to the Trigger distribution component. The “XML multicaster”
sends the list of video-clips, the action annotations and the 2D tracking data over IP to all
subscribed set-top boxes and to the “DVB event embedder”. This component stores all
processed messages together with a timestamp to a “log-file”. This enables the re-sending off all
messages during a re-broadcasts of the complete program.
AV & MHP production
Audio & Video is encoded "live” or is streamed from disk when a program is being re-broadcasted
during demonstrations and user-trials. The MHP application is authored before the actual
broadcast and will not change during the broadcast or future re-broadcasts. The application is
added to the DVB broadcast in a DSM-CC object carousel and will start automatically when the
user tunes in to the program. The MHP application will then contact the multicast server (IP) or
listen to the dynamic events or triggers in the received DVB broadcast.
DVB distribution
In the simplest form a live encoded or pre-created transport stream is streamed over ASI and
modulated to a QAM signal that can be distributed over the cable network of Euskaltel in Spain or
the internal cable network of the NOB in the Netherlands (CAI). With the ICE-CREAM
DVBStreamer a dynamically created MHP application can be added to an existing Audio/Video
transport stream. During the streaming of this new AV/MHP combination, it is possible to
dynamically insert triggers, events and embed a subset of the on-demand video-clips.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 32 of 71
Production Tools
AV
VOD
To Clip inserter
XML Multicaster = XML
Server
Video Clip Manager =
“Replay Manager”, enables final
control for director
Replay Builder
(in-out EDL generator)
3D-Rendering
Clips
MPEG4
Encoder
DVB Receiver
.mp4
files
“new videoclip created”
.mp4 files
“Publish”
Updated list with Videoclips (XML)
AV
AVI
AVI
Multicasted to
subscribed boxes
over IP
Video clips
on demand
served over
IP
DVB Signal / Source
+ MPEG2 Encoded AV
+ Static Xlets in DSMCC
+ Embedded Timeline
Event Embedder
QAM
QAM Modulator
XML
QAM
AV
3D-Generation
2D-Tracking
2D Tracking Data (XML)
Game Action
Annotation Tool
Annotation Events (XML)
3-rd party cold statistics (XML)
QAM
AV
Logfile (XML) =
To enable tests /
rebroadcasts
without
clock
“NPT” feedback
GUI
GUI
DB
Clip Embedder
Clip
inserter
TS
TS
.mp4 files
VOD Server
TS
ASI
Dsm/cc
builder
Signal
“Stream
Started”
Log, to enable
simulating the replay
builder
“streaming started”
MHP classes
MHP classes
HTTP
XML
“Publish”
Files
DVB Receiver
DVB
From Clip Embedder
File
System
File
System
Triggers
Clip Manager
Figure 19: Detailed view of the components of the production – delivery platform
3.7. Tools
The tools in ICE-CREAM have been constructed to facilitate a broad range of aspects of the
sports and the infotainment applications. They have been designed for the development and
adaptation of content for the production, manipulation and customisation of sophisticated
multimedia applications based on ISO-standards. Although some limitations are still present due
to the innovative nature of the applications and the prototypical stage of the implementation, the
tools have shown to reduce the complexity of the production for the ICE-CREAM project.
The production and authoring tools have been used mainly for the sports application, where a live
soccer match is enhanced with instant replays, coded in MPEG-4 format, including 3D graphics.
The events and clips are generated via “live content production” facilities. The clips are encoded
into MPEG-4 and stored on the server(s) and the corresponding events are multiplexed in the
broadcast and/or sent via the Internet (subscription-based).
The main travel application tool is intended for authoring ‘trigg&link’ presentations. The travel
application contains three services that are using the ‘trigg&link’ concept on the client side. These
three services are the CityGuide information, the quiz and the SMS message display. There are
further services within the travel application, for example ticket booking, but these are primarily
front ends to existing interactive services and require no specific authoring beyond the original
implementation.
Production tool
The production tool has to deliver all kind of interesting information concerning the sport event.
The “Ice Cream spectator” should then be able to access new on demand features like replays,
3D scenes or player’s information. One or many operators log the events that occur during the
match (e.g. goals, changes, etc.). All those information feed a database that also catch the
thumbnail of the action and put a bookmark on the video server. Other operators create the 3D
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 33 of 71
scenes of the action that fill the same database. All those inputs have to be synchronized and
connected to the video flow. An “Ice Cream Director” has a full view of this database and decides
either to reveal it to end-user or not.
The functionalities implemented give the possibilities to see cold statistics, hot statistics and
analysis (replays, 3D scenes). All the cold statistics will be delivered as XML files, whereas the
hot statistics have to be delivered live. A distributed architecture, where a software module
materializes each functionality, was used. The following modules were developed:
• Synchronizer – gives the tempo to all the elements of the system
• Logger – annotates the video events
• Thumbnail generator- snaps pictures of the video flow in association with an event
• Video server – Play-while record video server
• Director application – Chooses the events that will be delivered to the user
• Main database – contains all the logged events and pictures taken by the thumbnail generator
Authoring tool
The authoring tool (IAT) allows the manipulation of dynamic content. The positioning information of
the players is automatically imported and a 3D MPEG-4 file is generated. The semantic
information for the production team or the producer can be encapsulated in particular views and
presented in a XML/X3D like format to the user. These views show only interesting information,
e.g. omitting syntactical data and help to make the customisation as easy as possible. Within IAT
the relevant data for the application can be customized, e.g. change of player names and history
information, of advertising banners by drag and drop or change of player clothing. The result can
be exported as fully interactive 3D MPEG-4 file for web applications, as MPEG-4 video stream for
broadcasting and STB replay or as 2D MPEG-4 file for hand held devices or services with limited
display capability. As the concept of IAT is generic the tool can be used for other applications not
being related to the sports and soccer domain only. Figure 20 shows the concept of the IAT
enabling a production team of a TV broadcast to customize the 3D goal replays and produce
output for different channels of spectators.
Figure 20: Dynamic customisation of scene information with ICE-CREAM authoring tool.
The static 3D content is constructed in 3DSMax and exported to MPEG-4 via an ICE-CREAM
3DSMax
Construction
Tool
MPEG-4
Exporter and
Encoder 3D Static
Geometry
Static
User Interface
Dynamic
Scene Info
ICE-CREAM
Authoring Tool
Position
Converter
Plugin
3D
Scene
View
Textual
Info
Views
MPEG-4
Interactive
3D
Scene
MPEG-4
Interactive
2D
Scene
MPEG-4
Static
Video
Scene
Composition
XML Position
Information
?
DVD
Hand-held
Device
Computer
ICE-CREAM
Producer
Set-top
Box
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 34 of 71
enhanced plug-in (ICE-CREAM Encoder). This work results in a static file holding 3D geometry,
e.g. of the stadium in the soccer application. Within the 3D viewing window of IAT the static
geometry and the user interface data is composed. The XML position information generated from
the ICE-CREAM producer described in the chapter above is fed in to the IAT via a position
converter plug-in, which generates the dynamic scene info file holding information that can be
changed on the fly in IAT by the user.
Within IAT the user can now customise textual information represented in a 3D XML notation in
the left window and see changes updated in a 3D view on the right. After customisation is done
the user can export the scene to MPEG-4 video, a 2D version or the fully interactive 3D version of
the application.
Editing tools
In the travel application, there is a separation between content, application and layout for those
elements that are re-useable in the context of other MHP applications. These re-useable elements
include one-line message display, information selected by direct colour key access and multiple-
choice quiz presentations.
The separation of content and layout has also been imparted on the tools, which have been
produced. Alayout editor has been implemented that allows the definition of a specific layout of
any of the re-useable elements, independent of the actual application. In addition, an application
editor has been implemented, that fulfils the task of adding the various kinds of annotations
(attraction information, quiz questions, city tours, predefined messages, chapter names) to an
existing travel video to create content for a specific instance of the travel application to be
presented on the MHP device.
Both editors have been designed to perform their designated tasks as directly as possible and to
incorporate knowledge about their specific purpose. For example, since the layout editor ‘knows’
what a quiz is, it directly allows the previewing of a ‘correct’ or ‘selected’ answer, as opposed to a
‘turn visible/invisible’ on a generic graphic object.
Since the editors are only rarely used (normally the application editor for a travel programme
would be used for about an hour per week, while the layout editor might be used once a month),
care have been taken to make the re-acquaintance with the tools as simple as possible, by using
task related names and options instead of using generic editing tools.
Layout Editor
An important aspect of an interactive television application is its graphical appearance. Authors
want to develop applications that present their content to users in a way that is easy and intuitive
to view and understand, while respecting the communicative need of the underlying audio-visual
programme (i.e. by not distracting from or obscuring part of the programme) and the corporate
style of the broadcaster and content provider.
The layout editor is designed to facilitate authors in quickly generating, modifying and previewing
layouts for their interactive television application. As well as traditional layout editor functionality, it
is extended to support the specific needs of the author for TV application. In recognising that those
applications can be categorized into different types, and each type shares common generic
components and is constrained by common layout rules, the layout editor supports a set of
layout templates, each of which supporting the generation of the layout for a particular type of
application. For the travel application developed in the ICE-CREAM project, three distinct
application types have been identified:
• A message display
• An interactive quiz
• A supplementary information display
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 35 of 71
The layout editor has been implemented to support these three application types through specific
templates. It is designed to be extendable with additional templates according to authors needs.
Each template defines the layout objects (textual or graphical), which exist in the layout, and can
have rules constraining them within the layout. Each template is supplied with a default layout so
that authors can get started immediately.
Objects in the editor can be manipulated and changed quickly through standard Windows actions
such as drag-and-drop or selections from menus. All functions are also supplied with keyboard
shortcuts. Authors can change the position, size and depth of objects in the layout as well as alter
their textual style or textual/graphical content. Pixel-precise modifications are also possible. To
support the specific requirements of designing for a TV environment, the editor allows layouts to
be developed for any of the standard TV resolutions (PAL, NTSC and their wide-screen versions)
and displays optionally a ‘Safe Area’ graphic to indicate if layout objects are safely within the
bounds of the television display.
Authors are also provided with a number of options for previewing their layout before implementing
it within their application. To facilitate previewing, layout objects whose content will be provided
dynamically by the application (e.g. SMS messages) can be supplied with ‘sample’ content.
Additionally an optional background graphic can be added to the layout to test colour schemes or
to represent a typical scene from the audio-visual programme.
Each template defines sets of exclusive objects and, while it does not support temporal
characteristics, the editor allows authors to toggle between the exclusive objects through the
keyboard and hence imitate the expected user interaction in the application. For example, the quiz
layout has exclusive objects for the quiz answers, the indication of a selected answer, and the
indication of a correct answer. The quiz template provides the author with a keyboard-based
means to ‘select’ an answer and to preview the ‘correct’ answer.
Layouts are saved to a SMIL-based XML document format, supporting document interchange and
providing authors with the additional option of previewing layouts in a standard SMIL player. The
layouts produced by the layout editor can be imported into the travel application editor and so
applied to the interactive travel magazine.
By supporting the specific characteristics of interactive TV applications and their requirements of
authoring, the layout editor reduces the authoring effort required to produce suitable layouts for
use in future interactive TV applications. Figures 21 and 22 show views of the layout editor.
Figure 21: The default SMS layout
with Safe Area and sample text entry
Figure22: The default quiz layout, with
preview of selected & correct answer
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 36 of 71
Application Editor
The purpose of the application editor is to create a specific instance of the travel application for a
specific video. The starting point for the application editor is a travel video. This video is annotated
with attraction information, quiz questions, chapter names, short messages and, internally, city
tours.
The main visual feature of the application editor is the video area. This area shows the current
travel video and is also used for editing short messages, quiz questions and quiz answers as well
as for viewing the layout styles associated with the video (Figure 23 and 24).
Figure 23: Application editor with
video area
Figure 24: Editor with info and quiz
overlay on video
A timeline display for the different types of attachments is located at the top of the screen,
together with the various menu options. Under the video area, a standard media player play bar is
located. The user can either play the video using the normal play functions, or go to a specific
position in the video by clicking on the timeline display. If additional information would be presented
for that part of the video, the information is displayed as an overlay in the video area, using the
currently selected layout style.
The content of the quiz (questions, answer alternatives, indication of the correct answer,
countdown timer) can be edited directly on-screen. The same applies for the content of short
messages. They have been designed for the development and adaptation of content for the
production, manipulation and customisation of sophisticated multimedia applications based on
ISO-standards. Although some limitations are still present due to the innovative nature of the
applications and the prototypical stage of the implementation, the tools have shown to reduce the
complexity of the production for the ICE-CREAM project.
Various menus are available, for example:
File menu - Beside the usual features like exiting the application, generating an empty annotation,
choosing a video file and allowing the loading and saving of annotation files, the menu also allows
the selection of the layout style used in the application (which had previously been created using
the layout editor) and an export function that allows an easy transfer of the created application
instance to the set-top box for testing and previewing on the actual target platform.
Select City menu - This menu allow the selection of a specific city from a list of available cities.
The selected city not only determines the attractions available in the Insert Attraction Info menu,
but also the city tour that is added on export to the set-top box and the menu language for the set-
top box application.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 37 of 71
Insert Attraction Info - The content of this menu is based on the currently selected city and the
attraction information available for that city. The basic structure of this menu, however, is always a
three level menu. The first level denotes the main categories (for example hotels, culture, nightlife,
restaurants, shopping), the second level denotes categories specific to the main category (for
hotels these include luxury hotels, design hotels and budget hotels, for sightseeing they include
city and surroundings, for shopping the list includes books, shoes, clothing and furniture). At the
third menu level are the actual attractions located.
Three other fixed menus are mainly used for the insertion and deletion of annotations. They are
mainly provided as a convenience shortcut.
So far, the editing tools have only been used in-house, so no independent evaluation, which could
have lead to a lifecycle model based development, has been done. The layout editing tool,
however, has been given to our designer, who was not involved in the original implementation
process, for evaluation. Feedback from that designer has been used to enhance the tool.
Specifically, the capability of having ‘layered’ access to the layout and the ability of selecting
elements by layers has been added in response to this.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 38 of 71
4. User Feedback
Applications are the carriers in the project. They were chosen such that they covered a wide
range of technical, business, social and conceptual domains.
A major challenge that confronts application developers in this context is to evaluate and elicit the
feedback of users in a robust and reliable way and to make claims such as ‘providing users with a
compelling experience’ operational and measurable. In this report we present the evaluation of the
three ICE-CREAM project applications with end-users. The major challenges that will be
addressed are the formalization of the interaction possibilities for the end-users, the development
of a measurement instrument that captures the effect of distributed content presentation and
different levels of interactivity on the user experience.
4.1. Evaluation of the Football application
The goals for the evaluation of the Football application were:
• To assess the appreciation of end-users and to find weak and strong points of the application
and feedback for improvements
• To find out how the service provided by the application is economically valued by end-users
• To assess the usability of the Football application in terms of ease of use, efficiency of the
user interface, speed of response, ease of learning,
Method
Setting
The evaluation was conducted in the Euskaltelium centre. The Euskaltelium is a free user space
belonging to Euskaltel and located in Bilbao city centre. It exhibits the latest advancements in
computers and communications technology to the general public. People can freely enter the
building and use the exhibited electronic devices and systems. Euskaltel regularly uses the centre
for product evaluations. A living room was created to foster a home environment for the users.
They could freely walk in and out of this living room and use the application at leisure (Figure 25).
Figure 25: Evaluation facilities
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 39 of 71
Participants
Subjects were passers-by. They were visitors that freely and spontaneously came to test room in
the Euskaltelium. They where not compensated economically for their participation. A user-
screening questionnaire was used to determine their profiles afterwards. Table 4 shows the
profiles of the users who participated in the evaluation. 83 people participated in total. Children
below the age of 14 were not included in the formal part of the evaluation. Although a lot of children
enthusiastically explored the system.
Table 4: Participant profiles
Age Percentag
e
15-24 34 %
25-34 34 %
35-44 11 %
45-54 10 %
+55 12 %
% At home
usage
PC TV DVD/VCR
Every day 65% 83% 40%
Sometimes 21% 13% 51%
Never; nearly
never
14% 4% 10%
Participants were allowed to participate in groups or alone. 60% of the subjects participated alone
and 40% participated in a group of friends. 60% of the participants were male and 40% was
female. The ability to handle electronic devices was investigated by asking the subjects about
their skills in PC, TV, DVD and VHS usage at home. The distribution is shown in Table 4.
Procedure
Participants chose spontaneously the tasks they wanted to carry out, tried to find their way to
accomplish them and eventually discovered whether it was possible or not to fully achieve them.
A facilitator was taking notes on users behaviour and comments while exploring the system. The
participants were invited to have an interview with the facilitators.
The procedure consisted of 3 phases: User instructions, Application test and Interview. Each
phase comprises the following steps:
1. User
instructions
Participants receive upon entering the room a brief explanation about
the system’s functionality and how to use it from the facilitator.
2. Application
test
Participants use the system at their own pace with no predefined list of
tasks.
The facilitator records information on user behaviour, gestures and
verbal comments. Relevant sentences in conversations are also
recorded in case more than one participant does the exploration.
Participants can ask questions about the use of the system. The
facilitator records any relevant question about difficult tasks or even
tasks that cannot be done with the system’s present functionality.
Participants decide to finish the test on their own initiative.
3. Interview When participants have finished using the system the facilitator invites
them to participate in an interview and fill in a questionnaire.
Measurement Instruments
The following instruments were used for the assessments:
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 40 of 71
• Observation of users while exploring the system
• Structured interview focussing on the user
• Questionnaire addressing usability and appreciation aspects
The structured interview aimed at obtaining the overall impression of people of the product. This
interview was conducted before administering the questionnaire and after people had explored the
system. People were asked about their opinion on, for example, strongest assets of the product,
potential improvements, satisfaction with the product, interest in the football match, what they
would pay for it, and estimate the duration of their exploration of the system.
The questionnaire comprised a list of questions which people had to score by means of a Likert
scale (1-4), i.e. 1 = totally agree, 4 = totally disagree. The questions assessed the usability of the
product, appropriateness of the user interface and the quality of the user experience, including
control, responsiveness and enjoyment.
Results and discussion
The main results will be summarized here. Deliverable D23 of the ICE-CREAM project provides
the detailed results.
The results from the structured interview from all participants showed a high and keen interest in
the application. 93% of the interviewees would recommend the system to their friends and
families. They explored the system between 15 to 25 minutes. The average assessment of the
overall system was 7.36 on a score of 1 – 10 (10 being highest).
Receiving replays of the most interesting actions in the game (the so-called hot statistics) and
having the possibility to watch them when they wanted was appreciated by 31% of the
interviewees. The creation of 3D replays for the most interesting actions was liked the most by
29% of the interviewees. They also liked the possibility, enabled by the 3D tool, to select different
points of view for the actions. This was the top feature for 21% of the participants. Some of the
participants commented that they really requested watching replays from different points of view,
which is currently available in a passive way in some sport channels. Such comments imply that
people are rather comfortable with such enhancements. Having various possibilities for watching
a match was in the top-ranking list for 22% of the interviewees. Other features that the
interviewees listed as high with regard to their liking were: the possibility of having the point of view
of a player (10%), statistics about the league also called cold statistics (7%), the system and the
innovation idea (6%), the possibility of recording replays in the hard disk of the STB (5%), the easy
of use (1%) and the quality of the image (1%). 30% of the interviewees said that they liked every
tool of the application and that they couldn’t pose any weaknesses. The 20% of the participants
who didn’t like football assured that they would have enjoyed the application much more when it
would have been about another sport or event. The main weaknesses detected by the users
concerned the menu, which could disturb the view of the match, the content of the cold statistics
and some didn’t think the 3D tool would be useful while watching the match.
In short, we can conclude, based on the interview data, that this application opens new ways for
enjoying the viewing of a match and that it addresses dormant wishes of users.
The questionnaire provided more detailed results. Table 5 gives an overview of these results as
percentages of number of participants (n-83).
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 41 of 71
Table 5: Distribution of user assessment scores in % of participants
Score Distribution
Item 1 *) 2 3 4 Average
The system offered a high image quality 43 53 4 0 1,60
The Systems has worked correctly without errors 42 49 7 2 1.70
The system has a good Quickness of response 30 58 11 1 1,83
The system is easy to use and to learn 25 53 17 5 2,01
The user feels in control of the application all the
time
23 54 18 5 2,05
The system is clear and has understandable
information and user menu
41 40 13 6 1,84
The interactive content does not obscures the
content of the show itself
43 46 14 6 1,93
The menu enables you to go back easily in case of
error
30 66 4 0 1,73
The menu does not disturb the view of the match 25 42 16 17 2,12
The remote control is appropriate to interact with the
application
34 54 11 1 1,80
I felt comfortable during the trial 49 51 0 0 1,51
I had a good time doing the trial and I enjoyed it 42 55 1 2 1,64
I would probably recommend this product to my
family and friends
34 59 6 1 1,75
*) 1 = fully agree; 2 = agree; 3 = disagree; 4 = fully disagree
When asked about their intention to buy this product or recommend it to their family or friends, if it
would be in the market, people gave more affirmative responses than negative ones, confirming
that they liked the application. Table 6 gives an overview of these responses and the way they
change with change in price. These responses were elicited by explaining that, the football match
actually costs 6 Euro through Euskaltel’s digital TV system and asking whether they would pay 18
Euro, and if they wouldn’t pay 18 Euro, whether they would pay 12 Euro, and so on.
Table 6: Buying intention in relation to price (in percentage of responses)
Price quotation For sure Probably yes Probably not Not at all
No price quote 16 57 22 5
18 Euro 4 14 36 46
12 Euro 0 12 47 41
6 Euro 23 37 20 20
The appreciation for the system differed for the different participants profiles. The youngest people
liked the application the most. 96% of under 24 years old would buy the service, while this
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 42 of 71
percentage decreased to 62% for older than 24 years. 58% of the females who, in most cases
didn’t like to watch football, would like the application in a different environment (like tennis or other
sports) and 85% of them would recommend the application to their family and friends.
Recommendations
The main recommendations from the end users were quite subjective, like that the application
should be implemented for other sports, or that they would change the information of the cold
statistics, some would like to watch at the same time the replays and the current streaming, while
others would prefer just to watch to the replays. No significant recommendations were given
concerning modifying the implementation of the application, its structure or its user interface.
There were, however, a lot of personal points of view of how to improve the presentation on the
screen. These recommendations support the positive impression regarding the acceptance of
this type of new interactive content by the end users.
4.2. Evaluation of the Travel Application
As a complementary application to the MHP-based travel guide, a mobile travel guide service,
running on a connected PDA device was implemented. The objectives of this implementation
were:
• To create a travel-centric mobile service, offering all features that would be needed during a
city trip
• To integrate city-based information and content, i.e. point of interest localisation, sightseeing
information, movies programme, events, mobile ticketing, location-based services like
mapping and routing into a consistent user interface.
The goal for the user evaluation of the mobile travel application was
• To investigate which behaviours users would show with location based content and services
while travelling in a city, i.e., mapping and routing functions and guided city tours.
Method
The mobile travel guide application was installed on a connected PDA device, containing a GPRS
SIM-card. The connectivity of the device was assured by pre-configuring the required profile on
the device, so that all needed dial-in connections for accessing information were done
automatically.
Functionality of the application:
a) Point of Interest/ Travel Information:
i) Travel recommendations 200 points of interest (POI) with information about Hotels,
Restaurants, Nightlife, Culture, Shopping, Sightseeing.
ii) Area-wide, address-based POI-information on categories like Sightseeing, Museums,
Theatres, Movie theatres, Doctors, Pharmacies, Hospitals, and Police stations.
b) Events and Tickets:
i) List of concerts, theatre- and opera, other leisure events with timetables and addresses.
ii) Ticket booking through an online box office provider using the mobile device.
c) Location Based Services- maps and routes:
i) Users can receive a map on their PDA of their current position and environment.
ii) Users receive a step-by-step instruction for finding their way between two points.
iii) Guided city tours linking several points of interest to a tour through the city.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 43 of 71
Participants
Research experts from the ICE-CREAM project with partial knowledge about the developed
service and no knowledge about the mobile tool itself. They did have no or almost no experience
with using mobile data Internet services like WAP or iMode. The mobile travel guide was new to
them in behaviour and user experience. They were, however, interested in mobile services when
they become easier to use and more bandwidth is available (e.g. under UMTS).
Procedure
An on site evaluation was conducted. Participants could use the application in a real travel
situation. The task of the participants was to use the application for a guided tour in an unfamiliar
city, i.e., Hamburg. Their behaviour and performance of different groups of users was recorded.
The guided tours consisted of 7 different points of interest (POI). The participants worked in three
groups on the assignments. They had to find their way with the aid of the mobile Travel guide.
Their presence at the pre-defined POI was controlled. Two groups, with 5 people each and one
group with 4 people participated. One group-member functioned as logger, and recorded the
activities of the group. The tour lasted approximately 2 hours. The comparative composition of the
groups was:
• Group 1 used the mobile travel guide on a connected PDA, following Guided Tour 1;
• Group 2 used the mobile travel guide on a connected PDA, following Guided Tour 2;
• Group 3 used an ordinary street map, following also Guided Tour 2.
Measurement Instruments
The following assessment instruments were used:
• Observations and notes from the loggers assigned to the groups
• Questionnaire addressing usability and appreciation aspects.
Results and discussion
The ease of navigation was rated lower for the use of the mobile travel guide than for the use of an
ordinary map, average scores of 2,2 and 3.2 respectively (scale of 1-5, 5 being the highest value).
The logging data showed that the complex menu structure of the mobile city guide and the many
features that were integrated in one user interface hampered the on-site usage and prevented fast
and easy access to relevant information. Users had trouble recognizing relevant features such as
additional information on points of interest. They rated the system at 2.6 (1- 5 scale) for being
informative. Table 7 shows the average scores people gave to the travel guide features (scale of
1-5, 5 the highest value).
On average, all features of the mobile travel guide scored around 3. Observation data showed that
people in general liked the features, but that they didn’t like the implementation. For example,
manual localisation was too imprecise and unhandy, because street names and house numbers
could not easily be found. A dynamic localisation over GPS or Cell-Information was desired
instead. The route information was implemented with an ordinary routing plan, optimised for cars
and didn’t provide information on, for example, one-way streets or pedestrian zones. This should
be a strict requirement for further versions of the application, i.e., routing information optimised for
pedestrians. About 70% of the participants lost their way 1-4 times, while 16% never lost their
way and 14% lost their way many more times. The overall usability of the device with regard to
handiness and simplicity was rated as 3.3 and 2.9, respectively (scale 1-5, 5 highest value).
Choosing the travel guide as a replacement for a conventional travel guide (book and map) was
rated as 2.4. People especially liked to have a conventional map together with the mobile travel
guide tool to have the most appropriate orientation during a city walk. The display size on the PDA
is too limited to provide people with an overview and orientation compared to a normal map. The
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 44 of 71
mobile travel guide was perceived as an extra-tool for additional information and detailed
navigation.
Table 7: Average evaluation scores of Travel guide features
Feature Score
Maps 3,4
Route descriptions 2,9
“Where am I” 3,7
Guide tour 3,0
Recommandations/ travel guide
inform.
3,2
Other information 3,4
Concerning future improvements and enhancements, people rated the possibility for
personalization of the travel guide before the trip with 4.2 (a score of 5 is ‘very high’) and the
possibility to book hotels, tickets and trips with 3.6.
Recommendations
The main recommendations from the end users and conclusions concerned missing pedestrian
options in the guide, adequate street plans, pictures and overview maps, detailed input on, for
example, house numbers, alerts and recommendations along the route for sight-seeing. Clearly
available tourist and routing information data made for car drivers is quite unsuitable for
pedestrians in city centres. When people got lost while using the mobile travel guide they would
like to have information about changes in street names and cross roads, direction to take, starting
points, consistency between information and actual situation, landmarks, and most important fast
access to this information
In summary, there is a need for dynamic and optimised solutions for a mobile travel guide for
pedestrians, like tourists walking in a city. Mapping information needs to be optimised and to be
used in a combination of ordinary maps and computer-generated maps, or to be solved by
increasing size and flexibility of the display in mobile devices. Ease of use and handiness remain
critical factors. In addition the performance of the mobile tools need to be improved. The mobile
travel guide tools integrate several features into one consistent application. This requires an
integrated graphical user interface for navigational reasons and sufficient bandwidth. The users
were experiencing the speed of a GPRS connection during the evaluation tour, which was
definitely too slow. Better integration of additional sight information in the basic features of the
mobile tool, such as maps and routes, is needed for further development of the mobile travel
guide application. This requires the dynamic insertion of editorial content in maps and routing
information.
4.3. Evaluation of the DeepSea application
The goals for the evaluation of the DeepSea application were to assess the effects on the end
user experience of
• Different levels of content interactivity,
• Presenting distributed content on multiple devices, and
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 45 of 71
• The participation of a second user.
A major challenge is the formalization of the interaction possibilities for the end users and the
development of a measurement instrument that captures the effects.
Method
Setting
The evaluation was conducted in one of the rooms of the Philips Homelab. Figures 26 and 27
show the design of the setting and the actual working experimental setting. The HomeLab
provided the experiment with a wireless network, control systems, and user research facilities
such as hidden cameras and recording systems.
Figure 26: Conceptual setting Figure 27: Experimental setting
The features of the DeepSea application can be characterized from three different dimensions:
the interface devices that are being supported and integrated as part of the performance space,
the different modes of interaction that are supported by the structure of the content, and the
possibility for co-operation between multiple persons. Table 8 summarizes the dimensions of the
characteristics of the DeepSea application.
Table 8. Characteristic dimensions of DeepSea
Characteristics Description
Interaction devices
Multiple input/output devices:
• Primary display: a plasma display panel (42 inch);
• Secondary display: iPronto;
• Lights;
• Robotic toy: a toy submarine;
• GamePad controller;
Distributed allocation of the content to different output display
devices.
Content Interactivity
Multiple use of the same content at different level of interaction:
• Automated mode (no control by the user);
• Discovery mode (full navigation control);
• Game mode (limited navigation control).
Multi-user support Home / family setting
Interaction devices
The presentation and interaction devices included in the evaluation are a plasma display panel
(PDP) as the primary display, an iPronto as a secondary display, as well as speakers, lamps, and
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 46 of 71
a GamePad controller. The display panel, speakers, and lamps are positioned as elements in an
ordinary living room. The display panel is hung on a wall. The speakers and lamps stand in the
corners of the living room. A GamePad controller and iPronto are placed in front of the end users.
Display & Speakers: A 42-inch display panel was used to present the 3D movie. The speakers
were placed near the display panel. Special speakers were driven by a sound sampler in order to
carry realistic and compelling sounds.
Lamps: A total of four lamps were used to present the lighting effects. Two white lamps were
standing in the corners of the test room; an orange and a red light were attached under the
plasma display panel. The lighting effect is one of the distinguishing features of the application
scenario. The lighting effects presented in the content are related to the physical lights in the living
room. For example, when the submarine hit the mines in the virtual undersea space, the mines
will blow off. When this happens, the lamps in the user’s environment take part in the effect of the
explosion and glare-up as red, orange and white lights. When the submarine dives deeper into the
sea, the light gets darker; when the submarine floats out of the surface, the light becomes bright
again.
GamePad controller: A GamePad controller was used for the navigation control of the
submarine. The user could control the direction (left, right, up and down) with the direction buttons
in the left hand. With the right hand, the user can hold the accelerator button to drive the
submarine ahead, and release it to stop the submarine. By holding and releasing the accelerator
button, the user can actually control the speed of the submarine. Both hands were needed for this
navigation.
iPronto: The iPronto is a small portable computer developed by Philips, with a 6-inch touch
screen and wireless network connection. It can be used as a universal remote control to control
home appliances, such as, audio and video systems, lighting systems, and security systems. In
the DeepSea application, this presentation server is used to construct the interface for content
navigation and customisation, and to present and store the content elements for later review.
Participants
Eighteen students (9?, 9?) from various backgrounds participated in Experiment 1, eight Dutch,
four Ukrainians, two Belarussians, and one Chinese, French and Russian each. Nine subjects
were students of the postgraduate master’s Program in User System Interaction at the Eindhoven
University of Technology. Nine students were working on their internship at Philips Research. 50%
of the students had a background in the behavioural sciences and 50% had a background in the
engineering or natural sciences. The average age was 26 years, ranging from 23 to 30 years.
Subjects watched one to two hours TV per day. Most of them had experience with playing
computer games and about 50% frequently played games (once or twice a month). They were
rather unfamiliar with virtual reality applications.
Measurement instruments
The user’s viewing experience is a multidimensional construct, of which appeal is a major aspect.
Different dimensions that contribute to the appeal of media content are identified in the literature
[3], [4]. These dimensions are, amongst others, intrinsic and extrinsic motivation, engagement,
attention, curiosity, enjoyability, situational factors and presence. The presence dimension
accounts for the belief of being located in a world other than the physical one. Factors that
underlie presence are [5], for example, the extent and fidelity of the sensory information, the
mapping between a user’s actions and the perceptible spatio-temporal effects of those actions,
content factors, and user characteristics. These dimensions overlap. Currently there is not a
single measurement instrument that captures all these dimensions and provides clear descriptors
for data coding and interpretation. Multiple measurement instruments, which combined subjective
and objective measures, were used:
• Two questionnaires: a) Appeal questionnaire and b) Presence questionnaire
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 47 of 71
• Structured interview conducted after the participants had finished the experiment
The appeal and the enjoyment dimensions were addressed by the Appeal questionnaire. The
presence dimension was addressed by the Presence questionnaire.
The Appeal questionnaire was derived from a questionnaire that is being developed in Philips
Research for Dutch native speakers [6]. This questionnaire aims to measure the degree of
‘appeal’ that media content or consumer electronic products have for end-users. Eight
dimensions of this questionnaire were translated and used. Table 9 shows the structure of the
questionnaire. A Liker scale (1-7) was used.
Table 9: Structure Appeal Questionnaire
Dimension Description
Overall
Enjoyment
Eleven items, which focused on enjoying the presentation, impression
about the presentation, time perception, and duration.
Presence Six items, which focused on movement in a virtual environment and
content induced aspects like feelings of turbulence and unrest.
Attention Seven items, which focused on distractions induced by unrelated events.
Challenge Five items, which focused on challenge, interest, and excitement induced
by the presentation.
Motivation Four items, which focused on continuing and repeating of the
presentation.
Future Use Four items, which focused on recommending to others and ownership of
the presentation.
Situation Four items, which focused on comfort and ambiance.
Curiosity Two items, which focused on novelty.
The Presence Questionnaire was derived from the Independent Television Commission Sense Of
Presence Inventory (ITC-SOPI questionnaire) by adapting to the needs of the DeepSea
application. Four dimensions were used. Table 10 gives an overview of its structure.
Table 10: Structure Presence Questionnaire
Dimension Description
Spatial Presence Nine items, which focused on actively involvement, awareness,
and movement in the virtual area.
Negative effect Three items, which focused on negative effects and strains.
Engagement Two items, which focused on sensations of returning from a
journey and losing track of time.
Ecological validity One item, which addressed naturalness of the displayed
environment.
Experimental design
The DeepSea content provided three levels of content interactivity: 1) automated mode or non-
interaction mode, 2) game mode or limited-interaction mode, and 3) discovery mode or full-
interaction mode.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 48 of 71
The DeepSea application provided three levels of interaction devices and distributed output
display: 1) the plain setting, 2) the lighting effect setting and 3) the secondary display setting.
Table 5 gives an overview of the different experimental conditions. Since the system didn’t provide
for all the possibilities (marked as void in Table 11), we had to compromise on the experimental
set-up.
Table 11: Conditions of content interactivity and distributed output display
Conditions Plain Lighting Effect
Secondary
display
Non-Interaction mode A1 void void
Limited-Interaction
mode
B1 B2 void
Full-Interaction mode C1 void C3
The cells, A1 and C1 were compared to study the effect of the content interactivity. B1 and B2
were compared to examine the effect of the lighting effect. C1 and C3 were compared to examine
the effect of the secondary display. The order of the presentations to the subjects was
counterbalanced.
Procedure
All subjects received instructions on each mode and device, they practiced the content navigation
in the full-interaction mode for about 3 minutes, and they watched or played all the three modes.
After each mode, they were asked to fill in the Appeal questionnaire and the Presence
questionnaire. When the three sessions were finished, a semi-structured interview was
conducted. The subjects were divided into two groups. The first group was a plain setting group
(A1). The effect of the presence and participation of a second person was evaluated in a case
study. 3 pairs of subjects participated in the case study.
Results
The effect of content interactivity and the availability of different distributed interaction devices
were measured by the Appeal and the Presence questionnaires. Due to the implementation
variations only 3 comparisons could be made:
• Non-interaction mode vs. full-interaction mode (conditions A1 and C1 in Table 12)
• Limited-interaction mode without and with lighting effect (conditions B1 and B2)
• Full-interaction mode without and with a secondary display (conditions C1 and C3)
The mean scores on a 7-point scale for the Appeal and the Presence questionnaires for these
three comparisons are presented in Table 12 (* significant Wilcoxon Signed Ranks p<.05).
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 49 of 71
Table 12: Mean scores for the Appeal and Presence questionnaires for 6 conditions and
mean errors for the duration estimation.
Condition
Distributed
devices
Plain Lighting Secondary
display
Questionnaire
Dimensions
Content
interactivity
A1 B1 C1 B2 C3
Enjoyment 4.62 4.53 4.32 5.03 4.52
Attention 4.10 4.96 4.84 4.67 4.63
Challenge 3.71 4.49 3.87 4.64 4.22
Future Use 4.08 4.13 4.31 4.65 4.15
Situation 4.97 5.03 5.00 4.75 4.70
Appeal
Curiosity 4.94 5.11 4.89 5.35 5.40
Spatial
presence
2.64* 3.85 4.10* 4.16 4.23
Negative
effect
2.22 2.59 2.56 2.26 2.44
Engagement 2.44* 3.56 4.17* 3.67 4.06
Presence
Ecological
Validity
2.56 2.67 3.22 3.56 4.11
Effect of full-interaction mode: The possibility to interact with the content enhanced the level of
perceived presence and engagement of the user, but it did not effect the overall enjoyment, the
perceived naturalness and challenges. This result is supported by the interview data, which
revealed that most participants were familiar with this type of content.
Effect of lighting: The lighting effect increased the enjoyment and was perceived as an
entertaining element.
Effect of secondary display: The secondary display device provided more challenge, increased
the spatial presence and enhanced the engagement for the participants.
The dimensions that constitute the Appeal questionnaire relate to the content and the script of the
presentation, while the dimensions in the Presence questionnaire relate to the person’s
experience during the presentation. The condition A1, i.e., non-interaction or automated mode and
no light effects or secondary display, can be considered as a baseline. If adding more interactivity
(B1 and C1), light effects (B2) and displays (C3) doesn’t significantly effect the dimensions on the
Appeal questionnaire, then we may also conclude that i) there is a ceiling effect with respect to the
appreciation of content and script of the show which is not influenced by adding more interactivity
and show effects and ii) that the variation in subject’s responses fades out whatever effect there
might have been. The dimensions that constitute the Presence questionnaire relate to the
experience of the user during the show. Adding more interactivity and other show effects like lights
and a secondary display effect all the dimensions of the Presence questionnaire. They increase in
relation to the baseline condition A1. Especially the scores for ecological validity are interesting
(question: ‘the displayed world seemed natural’). The perception of spatial presence and
engagement was effected most. As for the Appeal questionnaire, the variation in subject’s
responses was very high.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 50 of 71
All in all, adding more interactivity and show effects appears to affect the dimensions of the Appeal
questionnaire less than the dimensions of the Perception questionnaire.
The structured interviews that were conducted after the evaluations re-enforced the observations
and the results from the questionnaires. Controlling the submarine in all modes was reported as
the biggest problem. That is, the synchronization of the feedback on user actions was inadequate.
This effect might have over-ruled all other conditional effects. The Full-interaction mode was the
most favourite of the interaction modes, i.e. 5.8 on average on a 1-7 point Liker scale, compared
to 4.6 on average for the Limited-interaction mode and 4.2 for the Non-interaction mode. In the
Non-interaction mode, subjects were presented with enhanced content with which they couldn’t
interact. In this passive condition, the quality of the content and the relationships between all the
effects become more critical for the user’s appreciation. In the limited-interaction mode people
had to perform a task that some perceived as challenging and exciting and that rewarded their
actions with surprising sound and light effects. However, the task was monotonous and boring
after a while. The limited control was hampering the users; they desired free navigation and a
well-defined task. The full-interaction mode had the largest potential among the three modes
according to the participants. They wanted to have more interaction with the content objects or
other players as enemy or friend to build their own story out of the experience. The lighting effect
intrigued the participants, although they found it monotonous and became bored with it rather
quickly. They wanted to have more and varying effects, for instance, a blue ambient wavy effect to
simulate, for example, the sea environment. The role and function of the secondary display was
not fully integrated in the script of the presentation and became rather distracting. Paying attention
to two screens at the same time is not easy, for dividing attention appropriate cues are needed.
The multi-user setting, which was evaluated as a case study, was appreciated more than the
single user setting. They were more attentive and engaged, conducted discussions about the
content. In short, they engaged much more in exploratory behaviour.
In short, the evaluation of the DeepSea application provided a lot of valuable feedback for follow-up
design processes. All in all, the participants perceived the concepts as very promising and
interesting. The evaluations also revealed that we still have a long way to go with regard to the
development of methodologies for evaluation, the design of the assessment instruments, the
exploitation of the possibilities for interaction with content by users in the home environment, and
the exploitation of ambiance effects in synchronization with events in the content.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 51 of 71
4.4. Summary
This chapter documents the results of the user evaluations of the three application designed and
implemented in the ICE-CREAM project, i.e., I) Football application - live video enhanced with real-
time events, ii) Travel application – video enhanced with related content, and iii) DeepSea
application – video enhanced with ambiance events.
New evaluation methodologies were investigated. Methodologies were adapted to the beta status
of the applications. In all cases, state of the art methodologies were used. Context of use was an
important factor for the design of the evaluations. The major findings are:
Football application: 93% of the interviewees would recommend the product to their friends an
family, 31% of all the interviewees appreciated the feature to receive the replays of the most
interesting actions, and 29% of the interviewees liked the creation of 3D replays for the most
interesting actions the most. The users advised to integrate the application for other sports, to
improve the view of the match while the menu appears on the screen, or the replays are
displayed, and to improve the quality of the pictures in the 3D graphics.
Travel application: The feedback concerned optimisation of the application, i.e., dynamic
localisation and delivery of routing information for pedestrians is needed, mapping information
should include a combination of ordinary maps and computer generated maps, and the basic
features of the mobile tool, such as additional information, sight information, maps and routes
should be better integrated. The performance was too slow. Users were experiencing the speed
of a GPRS connection during the evaluation tour which is definitely too slow. The conclusion is
therefore, that integrated graphical user interfaces on mobile devices should only be made
available with 3G bandwidth.
Deep Sea application: The participants perceived the concept as very promising and interesting.
The notion of presence was enhanced by the possibility to interact with the content. To improve
enjoyment, the design of script and content should be dedicated specifically to the home
environment. Distribution of the content presentation on different displays enhanced the
environment of the user with ambiance effects. Synchronization of these effects on different
devices with script and content is critical.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 52 of 71
5. Business Framework
A business model (BM) is defined as a systematic way to describe the economic architecture of a
business. It consists of at least the following components:
• Stakeholders - actors or agents involved in a system
• Roles and interactions – type of activities a stakeholder can partake in
• Requirements and expectations – desired features and capabilities of a system
• Obligations – responsibilities of the stakeholders
• Policies – constraints on the use and operation of the system
• Transactions – the realisation of income
This definition has its origins in studies carried out on the EC NexTV project [7]. Key underlying
considerations are the level of interactivity in the service offering, the Intellectual Property Rights
implications, as well as emergent revenue streams that could be had from the ICE-CREAM
service scenarios.
The analysis framework used for the ICE-CREAM service scenarios is based on core aspects
like interactivity and personalisation, business rules like intellectual property rights (IPR) and
external market conditions. Uncertainty about technology choices and the shape of the market are
underlying characteristics for the development of dynamic business models. Business models
that consider uncertainty have, for example, incorporated concepts like community affiliation and
collaboration, barter (‘free’ goods in exchange for information or forced viewing of advertisements
for marketing purposes), personalised courses, third-party content to attract consumers, free
Internet content or new technology to deliver goods or enhance user experiences. These models
were at the core for the analyses of the ICE-CREAM application scenarios.
5.1. Levels of interactivity
Interactivity is at the core of all ICE-CREAM service scenarios. Three different levels of
interactivity and potential service offerings were derived from literature studies (Van Dijk & De
Vos, Green [8]). Table 13 provides a general overview of these levels of interactivity. In Table 14
the revenue potential for the ICE-CREAM application scenarios is detailed for one level of
interactivity, i.e., level III (Exchange and Communication) with special attention for additional T-
commerce and off-line services. These different revenue-generators are then categorised in order
to select the ones that have a high probability of success. This analysis is based on current
success stories for digital TV and predicted future successes. Especially web-hosting, web-cast,
shared revenue, pay-per-play and betting revenues are the categories, which show potential. All
scenarios share the following revenue streams: Retail Revenue, Premium Content/Service
Revenue, Pay-per-View Revenue, Subscription Revenue, and Shared Revenue (from co-creation)
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 53 of 71
Table 13: Levels of interactivity in relation to service offerings
I. Choice from menus and transactions Video on Demand (VOD)
Customisation
Camera angle (replay)
More item display, plots or storylines
Additional channels
E-Commerce
II. Producing Information Participation in programmes (directed)
Reaction/commentary to programmes
Contribution to programmes/channels (not directed)
Production of own programmes/channels
III. Exchange/Communication Communication about/in parallel to TV programmes
(viewer/user communities)
Table 14: Revenue opportunities for ICE-CREAM scenarios
Scenario Revenue Potential Revenue Stream
Football
application
Place bets- Prior to game and during game
Watch web-cast of post-match interviews for a fee
Build user communities - provide match
commentary off-line, vote for best player/goal, etc.
Charge users to read premium content (hot and
cold statistics; 3D graphics replays; different
viewpoints)
Sell merchandise on & off-line e.g. f/ball kit, books
Charge subscription fees for web-site
Betting Revenue
Web-Cast Revenue
Retail Revenue
Subscription Revenue
Pay-per-View (of video clips)
Shared Revenue (from co-
creation)
Travel
application
Provide e-ticketing, e-booking, etc.
Charge for premium service to `arrange a
personalised package holiday'
Encourage Users to upload pictures, give tips,
etc.
Sell travel books, paraphernalia on & off-line
Charge for Web-hosting services to travel
agencies, hotels, car-rental cos., etc.
Charge subscription fees for w/site
Retail Revenue
Premium Service Revenue
Web-Hosting Revenue
Shared Revenue (from co-
creation)
Subscription Revenue
Pay-per-view Revenue (of video
clips)
DeepSea
application
Charge
- For sales of interactive game off-line
- To play interactive game
- For "storybooks" composed by users
- For merchandise based on characters
- For even more genre choices
- For subscription fees for w/site
Users who adapt functionality can charge for it
Retail Revenue
Pay-per-Play Revenue (tiered
fees for different levels of game-
play)
Shared Revenue (from co-
creation)
Subscription Revenue
Premium Content Revenue
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 54 of 71
Predictions with regard to survival of revenue streams based on surveys for interactive TV are
based on early success stories. For example, Screen Digest (www.screendigest.com, June
2002) predicts UK consumer spending on interactive TV games will more than double from £50.7
millions in 2002 to £132 millions in 2006. The total European interactive TV gaming market will be
worth close to €470 millions in 2006. Betting, available on Sky Active, generates half of BSkyB’s 9-
month interactive revenue (£136.3 millions), with the rest divided between, amongst others,
Internet Advertising, E-Commerce, Teletext services. (Source: Credit Lyonnais Securities, 2002)
[9]. In the Pace Report [10] news ranked number one, in both the US and the UK, as the type of
program most watched by the public. But in terms of what they were willing to pay for, movies
were listed first in both the US (75%) and the UK (71%). News came second in the UK (57%) and
last for the US. Music, education and sports were almost similarly favoured by both the US and
UK respondents. In the US, overall, only 27% were interested in paying for games. However, this
changes with different age segments: 41% of the 18-34 year old Americans surveyed, would pay
for video games.
In light of this, the Football application scenario and the DeepSea application scenario should be
well positioned for revenue generation. The Travel application scenario should also do quite well
as long as the information content does not overwhelm the entertainment features. Adding the
functionality of pay-per-view documentaries or movies (related to travel destinations) to this
application should enhance this scenario’s viability enormously. In Europe, reality shows like
France’s Loft Story and Holland’s Big Brother are a big hit. The Travel application scenario, for
instance, could be made more attractive and commercially lucrative by adapting this popular idea
of a reality show to allow audiences to watch a family on a week or two-week vacation in a
destination of choice (destination voted on by end users). Sports content is widely favoured. It is,
However, price-insensitive and time-sensitive, i.e., the commercial value of a sports event
declines rapidly if it is not viewed in real time or soon after a match is played. This point was taken
in the Football application by distinguishing between ‘cold’ and ‘hot’ statistics for the content
enhancements and by a graceful aging mechanism in the user interface.
5.2. Personalization and familiarity
Despite the lack of published data, the most successful current business models are based on
technologies that are familiar and in use by consumers. These models are based on charging for
premium content, betting and gaming, micro-payments and revenue-sharing with
telecommunication companies for traffic generation (European Interactive Television Report,
2002: www.vandusseldorp.com).
Personalisation is an important component of the ICE-CREAM scenarios. From the user
perspective, it can be considered as a generalised form of interactivity. Personalisation has an
impact on business models, and particularly its trade-off with IPR issues needs to be considered.
Furthermore, personalisation is not just a technical and economic issue, but also a legal and a
social one. It is inevitable that a successful interactive media business model would permit
specific modifications by consumers of content, which users would consider as “personalised
content”. The following scenarios can be envisioned:
• Modification by users of various objects in IP-protected content. Examples would include the
following:
o Embellishment of user interface, e.g. new skins designs
o More attractive backgrounds, in software and video
o Remix of audio
• Modification of storylines in linear/non-linear formats. Examples include:
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 55 of 71
o Extra scenes are created by users to embellish the storyline
o Entire “alternative” beginnings or endings are formulated by users
o New hypertext/hyperlink information is added by users that generally enhances the
user experience
• The “Linux Model”. In this model, a community of freelance developers makes independent
and original contributions that are submitted to a central authority, which then distributes the
content to users. This is particularly relevant in a P2P based business model that continually
allows for the recycling of content.
These scenarios open up very complex legal issues. Given the impossibility of trying to police
every instance of such personalised content, the best option for the commercial content
producers appears to be to embrace rather than oppose such personalised content from
consumers, i.e., a limited form of personalisation should be permitted, such as point (1) above,
but others such as (2) and (3) should be permissible only with fair monetary compensation.
A balance between access to information and rights protection (e.g. the payment for that access)
needs to be achieved. If end users feel that every little click requires payment (micro-payment),
they may become disenchanted and stop using the service or product, thus decreasing the value
of that intellectual property. On the other hand, if rights holders do not pursue infringements of
their rights, the value of that right becomes nullified. Alternate views, like Dyson (1995) [11] and
Barlow (1994) [12] suggest that companies should give up on the hope of profiting from
intellectual property on the net (digital media), and instead choose alternative forms of profit.
Paying for translations, which are usually part of the free service offering in websites, for instance,
may turn away end users. For example, Barter or Swap business models offer these services for
free to all end users in exchange for profile information. Another option is to offer end users who
are already paying a subscription fee for the digital service, some of the other personalisation
features at different starter package levels.
In a risk-neutral setting the idea is to provide as complete an interactive experience as possible for
target segments in areas that end users are already familiar with. The off-line social context is
replicated as a solo or group activity and enhanced through the enabling technology. That is, end
users are offered what they already enjoy doing in a particular market-space. Up to date and
anticipatory knowledge on end user habits, tastes, motivations, and so on is crucial. This
proposition is risk-neutral, if insights are based on information gathered about the end user. It is
off the mark to wonder what level of interactivity end users would desire, as they are unaware of
these distinctions. What is important to them is the ‘authentic simulation’ of the overall experience
they already have, off-line, or the completely new experiences/applications (still based on end
user information) that they can have.
A risk-neutral revenue model would initially seek revenue from domains that are already
successful today. In parallel, all revenue streams should be explored as the end users’ tastes,
preferences and behaviour may change over time. At present, betting revenue and Pay-per-play to
play games are the most likely revenue generators among the various potential revenue
categories identified. Incorporating games where possible in all scenarios should be considered.
In addition, revenue sharing with end users as a result of co-creation (e.g. adapting the
functionality of the scenarios and in the process creating new transactions) is a new revenue
category that may succeed. Co-creation and revenue-share will harness end users’ creativity in a
positive, mutually beneficial manner instead of stifling it or encouraging hacking or negative
creativity. Finally, facilitation of t-commerce could help the build up of communities, trigger the
desire to discover new scenarios and as a by-product the creation of new revenue streams.
The Football application scenario is built around the familiarity of people with watching and going
to a sports game. According to Ovum (2001) [13], most of the revenue in 2001 was generated by
premium sports content, and the rest by premium movies. The excitement generated by the 2002
Football World Cup recently speaks for itself. For instance, in the UK, there were big TVs set-up in
offices, schools allowed pupils to watch matches in the school hall and the celebrations after
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 56 of 71
each game won (also in Japan, Korea, Brazil, Ireland, etc), is not equalled by any other sport.
Other than movies, sports have been the most successful content area for interactive TV
applications. It was identified by a little more than 50% of the respondents in the UK and US, as
programming they were likely to pay for (Pace Report, 2001), [10]. In the UK, BSkyB claims that
over 40% of viewers access interactive features at least once during any specific match on which
they are available (Unna, 2002) [14]. With the ability to respond to triggers for additional
information on players, watch replays again and the added functionality of video-chat, the Football
application should expect to have some success in the marketplace. This is because it recreates
what end users actually do and want to see while they watch matches. In addition, a business
proposition for the Football application should consider the potential advantages of end user
behaviour prior to and after matches, which could contribute other sources of revenue. With a
digital rights management model, end users could be paying for downloading of replays,
personalisation through skins and other enhancements. The potential of retail/ t-commerce
revenue, betting revenue and premium telephony revenue, through interactive voting could be
added as possibilities.
The Travel application scenario takes the explosive growth of the use of information and
communication technologies in the travel and tourism industry into account. Travel content
became very popular as people travel more widely and usually for short one-week breaks. End
users can walk into physical travel agencies, watch travel programmes on TV or surf the web to
gather information and make bookings/reservations. Interactive holiday shopping offered by e.g.
Sky Travel Active in the UK, provides an extra channel for End users who can then book holidays
from the comfort of their armchair Allowing ticket reservation and ticket purchase during the
program is an idea worth considering. The Travel application scenario is focusing on city travel
(e.g. extended weekends) which is related to a more fixed type of package(s) that may be used in
association to, e.g., specific sporting events, opera in the park etc. This application scenario can
be easily enhanced with capabilities to purchase flight tickets, customised holiday packages and
other related services. In addition quizzes, virtual chatting with people who have first-hand
information about a travel destination via e.g. SMS messages, on and off-line competitions, and a
myriad of other possibilities like e.g. interactive holiday shopping (e.g. souvenirs etc) are possible.
The DeepSea application scenario is a novel scenario in that it allows end users to `compose
stories’ from the many options given, while playing games, if they choose to do so. This form of
entertainment is seen as one of the most interesting usage models for interactive TV (Unna,
2002) [14]. Such offerings with various levels of interactivity are currently not in the market and
there are no data to show that end users will have a positive response to this type of scenario. It
offers wide revenue generation opportunities from different prices for game-playing at different
levels, to charging for story down-loads, soundtracks and to buy the interactive game, etc.
Incorporating t-commerce and off-line communication will increase this potential. Although t-
commerce is not profitable now, Ovum (2001) [13] predicts that it will be the greatest business-to-
consumer revenue generator by 2006 followed by premium sports content.
Consumers select applications and services based on perceived usefulness and/or perceived
playfulness (Moon & Kim, 2001). What appears to be clear is that consumer priority is primarily
for self-fulfilment. A demographic breakdown shows that the ICE-CREAM scenarios hold value
propositions for different age segments. Although teenagers are a growing segment of the ‘buyer’
market, they and children also influence the purchasing choices of their adult parents (Table 15).
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 57 of 71
Table 15: ICE-CREAM scenarios in relation to demographic breakdown and interactivity
attractiveness
User Group Football application Travel application DeepSea application
Children Element of fun –
football
Animation, competition
Element of fun,
competition, – story,
adventure, sounds,
animation
Teenagers Entertainment (football)
Novelty – 3D camera
angles
Entertainment-video
clips, quizzes, SMS
Entertainment (chance
element) – game,
communication
25-34
Year-olds
Need to be entertained;
Socialize
Information (guide,
booking system),
entertainment-video
clips, quizzes
Entertaining and
educational for kids
Over 40’s Meaningful interaction-
replay, stats. for more
information; betting
Information (guide,
booking system),
Entertainment-
video-clips
Entertaining and
educational for kids
Table 16 shows some of the opportunities for ‘customised consumption’, offered by the ICE-
CREAM scenarios.
Table 16: Opportunities for customisation in the ICE-CREAM scenarios.
Football application Travel application Deep Sea application
Personal choice Obtain statistics,
replay, camera angles,
present different points
of view in 3D animation
Play video-clips Navigate through
obstacles
Contributing to
programmes
Opinions, tips- via
SMS
Influence story
composition, user-
creation
Games Could use statistical
data to create a `fun’
maths game
Quiz Play game and/or watch
show
e-Communication SMS
Some features of the three scenarios provide customisation opportunities and satisfy or exceed
current demand. However, we note that the children’s segment has not been specifically
addressed by either the football application (except for 3D replays, which would be very appealing
to young and old fans) or the travel application, in terms of interactive-readiness and should
therefore incorporate more fun elements. For example, an animated tourist guides or games (e.g.
Treasure Hunt) to be played in the featured cities. As influencers of buying decisions, children’s
impact cannot be under-estimated. The football application could incorporate games and
opportunities for viewers to participate off-line (e.g. football coaching clinics for kids and teens).
The DeepSea application could try and capture the parent group in other ways e.g. provide them
the opportunity to contribute ideas for the show, to write/buy storybooks, toys or other
merchandise related to the show. Both the football application and the DeepSea application
should be a social (family) viewing experience. The travel application does not need close social
contact, particularly when it is ported on to a mobile device. Here, it works better as an individual
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 58 of 71
experience (due to screen-size, font size, etc). This is an area that the travel application could
improve upon, i.e. to appeal to the whole family. The travel application’s generic services (quiz,
message display and information display) are useable in variant forms in other contexts like
sports shows, infotainment and other programmes. For example, other than SMS, the short
message display can also be used for news flashes, stock information and so on. This
represents a valuable revenue-generating opportunity for the travel application.
The football application appears to offer less customising opportunities than the other scenarios,
but its value lies in the real-time nature of its content. It merely requires enhancements like
information provision, and the ability to change camera angles to make the core experience even
more pleasurable for the consumer, and to generate more revenue opportunities for the producer.
Exclusive content, 3D replay animation in picture in picture is the key value proposition offered by
the football application to its consumer marketplace. The Football and Travel applications can
benefit substantially from the inclusion of some kind of interactive gaming. In relation to time
shifting, the users of the Travel and DeepSea applications would benefit from DVR support built-
in, as the value of these applications does not decline with time. This is in stark contrast to football
application, where real-time viewing is key to its value. However, 3D replays in football application
are very suitable for storage and would appeal to die-hard fans, which would appreciate the ability
for repeated viewing. This would also be another revenue-generating opportunity.
5.3. Business rules and IPR
IPR is a key variable in the development of viable business models. Important is to overcome
uncertainties by a market architecture that reflects and supports the successive use of content
and underpins the cumulative nature of innovation. A market architecture that can account for
nested rights can also support multiple slices of this kind. Rights and obligations are the
foundations of IP law, and every piece of an intellectual creation embodies layers of rights. Each
layer of rights is associated with a “role”, and can be licensed, sold or assigned to others with
preset conditions attached. This applies to both tangible and intangible goods. This thinking is at
the heart of the idea of “sliced property rights” (www.iprsystems.com). This recycling of
copyrighted material opens up a Pandora’s Box of legal, social and commercial issues, resulting
in a shift from the traditional “creation waterfall” model to the “creation life-cycle” model (Figure
28).
Create
Use
Trade
Waterfall models
Producer owns, consumer pays
Create
Use Trade
Life-cycle models
Blurred roles producer & consumer
Figure 28: The traditional “Creation Waterfall” model of the production-consumption
chain and the “lifecycle” model of the knowledge economy.
In developing novel business models for the ICE-CREAM project, the particular form of the
Producer-Consumer relationships has to be classified for each application scenario and the
relevant IPR models identified. From this, business models may be extrapolated. Such a
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 59 of 71
classification is shown in Table 17
Table 17: IPR and Producer-Consumer models for ICE-CREAM scenarios
ICE-CREAM Scenario Producer-Consumer Model IPR Model(s)
Football application Creation waterfall & Creation
life cycle
Monolithic (old-style)
Travel application Creation life-cycle Mostly sliced
DeepSea application Market Mesh Combination of sliced, nested or
monolithic
The Football application scenario is a linear creation waterfall model: a broadcaster purchases the
rights to a (football) game, and makes money on the sale of viewing rights to the consumer. The
sale would involve either sponsorship from other companies (traditional broadcasting, B2B), or
pay-per-view (interactive TV, B2C). Clearly, this IPR model is old-style monolithic, since the rights
to broadcast the game will be retained by the broadcaster alone. Furthermore, this particular case
has a business model that is unique for the three ICE-CREAM scenarios: it is bounded by time,
since the value of viewing the game will diminish if it is not enjoyed in real time. IPR issues do not
hamper the interactive chat option that is an extra value-added, since normal chat-room talk is
neither classified nor copyrighted. The 3D and event replays are the value-added parts introduced
by other parties. These fall under the purview of the creation life-cycle model. The Travel scenario
involves enhanced video with non-fiction content. The non-fiction video part represents the Core
Content, and the “enhancements” are the value-added part that could be introduced by other
producers or consumers. Clearly, this falls under the purview of the creation life-cycle model of
Producers and Consumers – as more and more value is added to the content, the IPR becomes
more and more finely sliced on each iteration through the production-consumption loop. The
DeepSea application scenario is the most general service scenario considered. With enhanced
video, in the fiction category, we have a situation where any or all of the IPR models could co-
exist. The Producers and Consumers may frequently swap roles, with each possibility realising, in
principle, a different fine-grained business model. The IPR issues are particularly complex, since
each implementation could involve aspects of sliced, nested or monolithic IPR in dynamic
proportions.
Football application: This application will benefit from the preponderance of existing and emerging
mobile devices: it may be envisaged that the sports fans will now be able to e.g. view the match
itself on a laptop, or obtain on-line information on PDA or a 3G telephone set. Compatibility with
the open mobile alliance (OMA) standard will serve to abstract the underlying hardware and
software combination and allow the fans to open a viewing channel with the football broadcast or
multicast. From the point of view of consumers, mobile agents could be instructed to initiate
automatic payment for a pay-per-view transaction that allows the sports fan to see the match on
his/her chosen device; the presence of agent technology would allow the user to select the set of
matches they wish to see days or weeks ahead of the actual event itself. A slight variation on the
above would involve the non-real time viewing of a sports match, i.e., ‘time-shifting’. The greater
the time shift, the less value can realistically be attached to the content: ultimately, within a few
days, the content would become an archive item with commodity pricing and free advert. Another
extension of this would be the case where one person is able to supply an informed commentary,
and perhaps would like to compare a given match with clips from previous performances.
Personal ‘web logs’ of such discussions are already now something of a sensation on the
Internet, and ‘blogging’ is in many ways a natural progression of an informed user’s chat session.
Travel application: This case is tailor-made to the client-server model of networked computing. An
enhanced travel booking application is easily possible with a centralised web server, hosting a
page with all the necessary audio-visual material or with links to such material. Such a system
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 60 of 71
would in fact be possible even with simple technologies such as HTML. It will truly benefit from
exploiting technologies such as web services and agents to deliver a state of the art service. For
example, an enhanced travel application could be implemented using agent technology at the
consumer end and web services at the producer end. In this case, financial agent and holiday
agent interact with the user’s diary agent to ensure end-users’ service satisfaction. If a second
person is involved in the holiday, the personal holiday agents of the two users would firstly contact
each other, to ensure that there are no diary clashes, for instance, and then act in unison in going
through the above steps.
Deepsea application: This scenario is also a classic case of a client-server model of networking.
It would also benefit from web services, but in addition, the possibility of modifying the fiction
content would enable it to also benefit from P2P networking. In this case a user’s personal
entertainment agent would facilitate the selection of these choices. The appropriate set of web
services at the content providers’ end would ensure payment trust and the non-compromise of
digital rights.
5.4. External market conditions and change
Based on extensive literature reviews (ICE-CREAM project Deliverable D19), Business Models
that could account for changing conditions were classified in three categories. These categories
are: a) Successful or Sustainable business models; b) Strategic business models; and c)
Generic (component-based) business models. They were analysed with regard to their
possibilities to cope with dynamic features in the market and their relevance for the ICE-CREAM
application scenarios. Table 18 gives a summarized overview of this analysis. The generic
business models are not included in the table as there is a magnitude of divergence between
them and fundamental concepts are not yet clarified.
Table 18: Analyses of Business Models in relation to market dynamics and application
scenarios.
Business model categories Dynamic features of the
business models
Relevance of ICE-CREAM
application scenarios
Successful / sustainable business models
Four different types of change
models
• Realisation models –
maximise returns from
exiting operation logic.
• Renewal models –
consistent and conscious
revitalizing of product,
service platforms, brands,
cost structures and
technology bases.
• Extension models –
stretching operation by
including new markets,
value chain functions, and
product and service lines.
• Journey models –
deliberate and purposeful
moving to a new operating
model with no return to the
old model.
Most companies adapt to
change by changing the
organisational structure (83 %)
or the mindset (13%).
Most common and fastest
change techniques are Partner,
Anchor & Extend, Converge and
Plug&Play.
Exploitation of:
• First mover advantage
• Innovative pricing and
purchasing through non-
restrictive DRM (for
example iTunes).
• Customers as co-workers:
they grow the business due
to ‘listen, learn and enable’
approach or are partners in
an auction model. (for
example eBay)
Pro-active and flexible
approach, which protects rights-
holders and benefits end-
users.
Expand application
implementation gradually over
geographical regions.
Evolve and add new features
over time.
Strive to exceed stakeholders’
expectations
Leverage capabilities, e.g., third
party licensing of 3D-replay
technology, authoring tools for
service development.
Port application to other
consumer devices
Introduce micro-payments for
services
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 61 of 71
Strategic business models
Two different classes:
• Invention of new business
concepts–
• Achieving customer
bonding
Exploitation of:
• Profit boosters, like for
example competitor
lockout.
• Portfolio breadth, operating
agility or lower break-even
features.
• Three distinct strategic
options: best product, total
customer solutions, and
system lock-in options
(Delta model)
Create a unique value
proposition.
Create a portfolio with a broad
offering of services and
applications.
Continually update and grow
core competences and
customer types
Create bonding between
customer and service, e.g., total
customer solutions.
From this analysis a dynamic business model for ICE-CREAM applications can be derived that
should comprise the following elements:
• An overarching mission (goals, vision, value proposition) for the business,
• A total customer solution (TCS) as in the Delta Model (Hax & Wilde II, 1999, 2002) [15], with
understanding the customer as its focus,
• Core components of a business model (adapted from Afuah & Tucci, 2003 [16]; Kruger et al.,
2003), [17]
• Change Models (Linder & Cantrell, 2000) [18] that focus appropriate strategies for different
phases of a product/service cycle (including a disruptive change phase).
5.5. Recommendations
What is becoming clear is that firms make deliberate and focussed changes to achieve their
goals. Timing of product/service launch, good marketing and non-restrictive DRM are essential
success factors. In addition, successful e-Business models seem to use extension and/or
renewal models in order to affect change. Consumer-centric or democratic business models (like
that of eBay) which `listen, adapt to and empower’ consumers are the way forward. Further, a
firm that offers abundant types of services/products rather than some core ones is in danger of
diluting its brand value.
Finally, recommendations and propositions based on best practices resulting from the business
model analyses can be summarized as follows:
• Change business models from being only B2C to B2B/intermediaries and C2C.
o B2C and B2B appear to be the way forward. B2C operators like Amazon have
successfully shifted business models to become a B2B and C2C marketplace, thereby
reducing costs and enhancing revenue opportunities.
o Industries like the real estate and airline industries are moving from being information
monopolies to becoming consumer (-driven) democracies
• DRM is fine as long as it is not too restrictive.
o Plausible alternatives to DRM exist – e.g. Creative Commons (cc), which allows rights-
holders to allow some usage of their IP for free.
• Use Open Source. It is most favourable for most consumers (best example here is the
meteoric rise of Linux).
o Couple this with solid industry standards to make a compelling mix benefiting both
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 62 of 71
producers and consumers.
• Aim at interoperability and a horizontal marketplace for the future..
• Explore strategies or models used by other industries like the music and airlines industry.
o Alliances of major companies in each industry, new intermediaries in the value chain (both
dis- and re- inter-mediation exist) and innovative pricing models are all changing business
models in the increasingly Web-based economy.
o Successful business models are subject to almost immediate imitation as evidenced by
copycat models to Apple’s iTunes.
• Create adaptable pricing models. They appear to be a profitable strategy as can be seen by
priceline.com, and supermarkets matching prices of similar offering from rivals.
• Police and continuously listen, learn and adapt to consumers behaviour and wishes.
o Consumer focussed organisations like eBay have a very profitable business model.
• With regard to application development, work directly with consumers (B2C) and become a
new intermediary in the value chain, i.e., a B2B player move
o By allowing its customers (end-users) to make modifications and behave as co-creators
(and rights-holders), selling modified versions to other consumers or to the aggregator; a
C2C model can be included as well. This would create a robust, growing marketplace,
stimulating creativity and collaboration while enhancing revenue opportunities.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 63 of 71
6. Conclusions
The major results from the ICE-CREAM project can be summarized as follows:
• Development and evaluation with users of three applications, Football application, Travel
application and the Deep Sea application.
• Formulation of a business model framework based on change drivers, digital rights
management aspects and adaptation factors.
• The development of an MHP-based platform that is extended with network services, MPEG-4
audio/video streaming and 3D graphics support.
• Prototype, which uses SMIL to integrate content from the Internet and to interact with the
content on a distributed presentation system.
• Development and adaptation of tools for production, manipulation and customisation of
content.
The three applications that were developed in the ICE-CREAM project showed promising results
for the strategic crossroad of TV and Internet convergence. They integrated broadcast and
Internet content and were developed on an MHP platform that was extended with network
services, MPEG-4 A/V streaming & 3D graphics support. Encouraging feedback was obtained in
user evaluations with regard to functionality and appreciation of the applications. This feedback is
encouraging enough to strongly recommend implementation of the integrated platform and to
generalize the applications to other domains and topics.
At the beginning of the project we set out to define a few factors that we considered as crucial
requirements for the perceived user benefits of interactive experiences. With regard to
involvement and appeal these major perceived user benefits of the applications are:
• Control over the content. Users decide whether they want the enhancements or not. In
addition, the DeepSea application gives people the possibility to decide on the path to take and
on controlling the level of participation and involvement.
• Participating in the program content. This is accomplished in the Travel application by the
involvement of the user in composing their personal ‘city tours’ and in the Football application
by the different views that are possible. Users can put themselves in the shoes of their
favourite football player. The DeepSea application entices users to get involved with the
content by playing a game.
• Ability to produce something useful and fun. With all applications users can compose and play
if-then variations. They can convert the applications into a game, conduct different types of
analysis and be part of a team.
• Feedback in the user interface. The user interfaces for the Football and Travel applications
were kept as close as possible to the well-known and trusted TV interface. The feedback is
simply the content that they receive. The user interface of the DeepSea application is still very
challenging and requires many adjustments. Many of these were identified in the user
evaluations. We still have a long way to go to understand the possibilities and the
impossibilities of using different devices and displays. Especially the integration with content
requires a different approach.
• Reliability of the application. Since the applications are clearly positioned as TV, people
perceive them as reliable as good old TV.
• Different and worthwhile to have and to view. This requires user trials with a larger population,
over an extended period of time and with different content domains. The applications provided
users with functionalities, which support familiar activities in their intimate environment, which
they could perceive as natural extensions of their habits.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 64 of 71
The focus of the ICE-CREAM project was on the exploitation and experimentation with emerging
standards, e.g., MPEG-4, DVB-MHP, and Internet. The project identified the need within DVB for
dual video support and within MHP the need to control the additional video feed in a similar
manner as the main feed is controlled. User requirements were submitted to DVB-MHP for more
powerful composition support like, MPEG-4 BIFS and SMIL. The project demonstrated how crucial
the future in-home networking capabilities of MHP are for distributed presentation of content The
ICE-CREAM authoring tool is one of the first tools capable of reading, customizing and converging
X3D files. The project showed how SMIL could be used for distributed presentation in
TV/Broadcast applications based on MHP. The opportunities of the employment of standards like
MPEG-4, VRML and X3D versus proprietary technology were explored. These opportunities
comprise, for example, the opening up of new markets, the entering of existing markets, diversity
of markets, different possibilities for licensing of content and technology, various possibilities for
SME’s to be part of the standard value chain, and no need to deliver end to end solutions as with
proprietary technology.
An operational demonstrator was build for the different applications. In this demonstrator,
broadcast programs are enhanced with interactivity in a personalized manner, the Internet is used
to convey the personal data and the trigg&link paradigm is used for the presentation of the
enhancements to the users. Authoring and production tools were further developed, integrated
and/or adapted to achieve the end-to-end system that could be demonstrated in the football
application. The major focus was on the production and customisation of MPEG-4 content, i.e.,
especially for 3D and interactivity components. The availability of easy-to-use authoring and
editing tools is a must for easy and fast development of applications and services, for updating
and timely provision of relevant and personalized information.
Content adaptation aspects of interactive TV and especially the personalization of video broadcast
through mixing in internet based content, such as XHTML pages or MPEG-4 video streams, was a
main focus. A prototype was build to demonstrate presentation and interaction on a distributed set
of displays, where shared content and personal content is presented in a corresponding manner.
Flexibility and adaptability to changes in environment, in user demand and in user behaviour were
the driving factors for assessing business models and for analyses of the three ICE-CREAM
applications. Marketing factors, like product range, customer value, positioning, were identified
and compared for each application scenario.
The project applications were demonstrated at several conferences and exhibitions. Especially
successful were the demonstrations of the applications at the IFA2003 and the IBC2003, which
generated lots of positive feedback (e.g., from BBC, Turner Entertainment). The demonstrations
showed a level of interactivity, which went far beyond other MHP demonstrations that have been
showed at IBC and IFA. A user trial was conducted at the Euskaltelium in Bilbao. This is a free
user space in the centre of Bilbao, which receives around a thousand people a day.
A number of deliverables or parts thereof were used and will be used to write publications in
academic journals, present papers and posters at international conferences.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 65 of 71
7. References
[1] W.R.Th. ten Kate, A.F. Schoenmakers, N. de Jong, H. Charman and P. Matthews,
Trigg&Link - A new dimension in television program making, 2nd EcMast, Milan, May
1997, in Lecture notes in computer science 1242, S. Fdida and M. Morganti (eds.), Springer
(1997) 51-65.
[2] W3C, Synchronized Multimedia Integration Language (SMIL 2.0), W3C Recommendation,
http://www.w3.org/TR/smil20/, Aug. 2001.
[3] Schickenberg, B. & Hoonhout, H.C.M. (2003), Towards an ‘Enriched media content Appeal
questionnaire’. Philips Research Report (to be published) based on B. Schickenberg’s
Graduation thesis, Faculty of Psychology, Maastricht University, Maastricht (NL)
[4] Malone, T. W. & Lepper, M.R. (1987), Making learning fun: A taxonomy of intrinsic
motivations for learning. In: R.E. Snow and M.J. Farr (Eds.)< Aptitude, learning and
instruction. Hillsdale: N.J.Erlbaum.
[5] Hoonhout, M.C.M. & Stienstra, M. (2003). Exploring enjoyability: which factors in a consumer
device make the user smile? In: D. de Waard, K.A. brokhuis, S.M. Sommer, and W.B.
verwey (2003), Human factors in the age of Virtual reality. Maastricht (NL): shaker
Publishing.
[6] Freeman, J., Lessiter, J., & Ijsselsteijn, W., (1999) Immersive TV. From electric source,
https://www3.bps.org.uk/publications/thepsychologist%5Cimtv.pdf.
[7] NexTV Deliverable, (2000), Preliminary Business Framework and choice of applications-
WP1_D1, EU-IST project
[8] Van Dijk, J., de Vos, Loes (2001), Searching for the Holy Grail- Images of Interactive
Television, New Media & Society, Vol. 3, Issue 4, Dec. 2001. Sage Publications, London.
[9] Credit Lyonnais Securities Report, BSkyB, July 2002
[10] The Pace Report (2001), Consumer attitudes towards digital television: the UK and the
US, Pace Micro Technology plc.
[11] Dyson, E. (1995), Intellectual Value, www.wired.com/wired/archive/3.07/, July 1995.
[12] Barlow, J. P., (1994), The Economy of Ideas, http://www.wired.com/wired/archive/2.03/
[13] Dransfeld, H. (2001), Determining Which of the Existing Business Models for iTV has
proved the most profitable; IIR Fifth Annual iTV Event, 27-28 Nov 2001. (OVUM)
[14] Unna, H., (2002) Digital Interactive TV: loved or abandoned; IPA, January 2002
[15] Hax & Wilde II (1999). The Delta Model: Adaptive Management for a Changing World.
Sloan Management Review. vol. 40, no. 2, Winter 1999. pp.11-28.
[16] Afuah, A., C., Tucci (2001). Internet Business Models and Strategies. Boston: McGraw
Hill.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 66 of 71
[17] Kruger, C. et al. (2003). Business Model Formulation within the Online News Market: The
Core + Complement Business Model Framework. 16
th
Bled eCommerce Conference (June
9-11, 2003). Bled. Slovenia.
[18] Linder, J. & Cantrell, S. (2000). Carved in Water: Changing Business Models Fluidly.
Working Paper from the Accenture Institute for Strategic Change.
[19] ICE-CREAM Deliverable D16&D18 "Interactive TV applications: prototypes, methodology,
user interfaces and interaction devices",
http://www.extra.research.philips.com/euprojects/icecream/internal/deliverables.htm, Nov.
2003
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 67 of 71
8. Project Consortium
ICE-CREAM consortium partners and contact persons.
Philips Research (Project Co-ordinator)
Prof. Holstlaan 4
5656 AA Eindhoven, The Netherlands
Maddy D. Janse
[email protected]
De Pinxi
Avenue Huart Hamoir 54
1030 Brussels, Belgium
Philippe Chiwy
[email protected]
NOB Cross media facilities
P.O.Box 10
Media Park
Sumatralaan 45
1200 JB Hilversum, The Netherlands
Ruud Leyendekker
[email protected]
Imperial College London
Electrical and Electronic Engineering Department
Exhibition road
SW7 2AZ London, United Kingdom
Javier Barria
[email protected]
Philips Digital Networks France – MP4Net
51 Rue Carnot – B.P. 301
92156 Suresnes Cedex, France
Raoul Mallart
[email protected]
Tomorrow Focus AG
Alsterufer 1
20354 Hamburg, Germany
Oliver von Wersch
[email protected]
Fraunhofer FOKUS
Kaiserin-Augusta-Allee 31
10589 Berlin, Germany
Angela Scheller
[email protected]
Bitmanagement Software GmbH
Weiherfeld 22
82335 Berg, Germany
Peter Schickel
[email protected]
Symah Vision
27 Rue Jean Bleuzen
92170 Vanves, France
Olivier Gafsou
[email protected]
Euskaltel S.A.
Parque Tecnologico, Edif.809
48160 DERIO (Bizkaia), Spain
Leire Aginako
[email protected]
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 68 of 71
9. Project Deliverables
D1 Project summary Feb 2002
D2 Project handbook Feb 2002
D3, D4,
D5
Applications: End users scenario’s, Storyboards and Specifications Jul. 2002
D6 Platform requirements specification Aug. 2002
D7 Evaluation results of interactive design concepts and applications
scenarios
Nov. 2002
D8 Tools Oct. 2002
D9 Report on Business Framework Nov. 2002
D10 Specification of terminal software architecture – An XML/SMIL based
presentation sub-system
Feb. 2003
D10, D13 Specification of terminal software architecture – The structure for the
end user trial and test platforms
Apr. 2003
D11, D12 Prototypes of applications, user interfaces and end user tools Apr. 2003
D14 Tools prototype – Accompanying information to software deliverable Apr. 2003
D15 Evaluation results of prototype May 2003
D16, D18 Interactive TV applications: prototypes, methodology, user interfaces
and interaction devices.
Nov. 2003
D17, D20 Operational Demonstrator Nov. 2003
D19 Business framework evolution, IPR aspects and recommendations Nov. 2003
D21 Tools Dec. 2003
D23 Evaluation results of application on integrated platform in home
environment
Feb. 2003
D22 Final Report Jun. 2004
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 69 of 71
10. Publications, presentations and
demonstrations
10.1. Papers and presentations
Overview of papers and presentations listed by first author and their affiliation.
Author Title of paper
A. Scheller
FhG FOKUS
Mehr als Fernsehen. IFA 2003: TWF Science and Technology Forum: Preview of
tomorrow’s media world, August 2003, Berlin.
B. Neville
(ICSTM)
A Computational Framework for Social Agents in Agent Mediated E-Commerce. 4
th
International Workshop of "Engineering Societies in the Agents World", (ESAW),
London, UK, 29-31 Oct. 2003.
D. Targett
ICSTM
Towards a general taxonomy on business models, Journal of Strategic Information
Systems, to be submitted for publication.
D. Targett
ICSTM
A meta-framework for the evaluation of new consumer technologies, to be submitted for
publication.
F. Alavi
ICSTM
Trading Digital Goods I: Econometric Analysis, IEEE Internet Computing, to be
submitted for publication
F. Alavi
ICSTM
Trading Digital Goods II: Game-Theoretic Analysis, IEEE Internet Computing, to be
submitted for publication.
J.Barria
ICSTM
Business Framework: Producers, Consumers, Content. FP6 IST Networking Sessions,
Milan, Italy, October 3-5, 2003,
Jun Hu
Philips Research
Enabling Distributed Interfaces for Immersive Storytelling. Poster: Interaction Design and
Children August 2002 Conference, Eindhoven University of Technology, 2002
Jun Hu
Philips Research
An adaptive architecture for presenting interactive media onto distributed interfaces. In
the Proceedings of the 21st IASTED International Multi-Conference on Applied
Informatics (AI 2003), P899-904, Innsbruck, Austria, 2003. NL-MS-22.374, Oct 10,
2002.
Jun Hu
Philips Research
An Agent-based Architecture for Distributed Interfaces and Timed Media in a Storytelling
Application. In the Proceedings of the 2nd International Joint Conference on Autonomous
Agents and Multi-agent Systems, Melbourne, Australia, 2003, P1012-1013
Jun Hu
Philips Research
StoryML: Enabling Distributed Interfaces for Interactive Media. The Twelfth International
World Wide Web Conference, P135. Budapest, Hungary, 05-20-2003
Jun Hu
Philips Research
Rapid Prototyping for Interactive robots. 8
th
Conference on Intelligent autonomous
systems (IAS-8), Amsterdam, The Nether-lands. To appear 2004
L. Nixon
FhG FOKUS
Addressing the 'what' and 'how' of integrating Internet content with audio-visual material.
IEEE International Symposium on Consumer Electronics ISCE02, Erfurt, Sept 24-26,
2002
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 70 of 71
L. Nixon
FhG FOKUS
Integrating multimedia components into a Semantic Web. ERCIM News, Special:
Semantic Web. Number 51, October 2002, Page 38/39
L. Nixon
FhG FOKUS
A Semantic Web based System for realizing Multimedia Presentations from
heterogeneous XML based sources. XML-Technologien für das Semantic Web (XSW)
October 2003, Berlin.
L. Nixon
FhG FOKUS
Building semantic interoperability into a content integration application, The Eighth IEEE
Symposium on Computers and Communications (ISCC) July 2003, Antalya, Turkey
Maddy Janse
Philips Research
Presentation of ICE-CREAM and NexTV projects to course participants of the Media
Academie, Hilversum, March 7, 2002
Maddy Janse
Philips Research
NexTV: From Technology to Experience, EURO-China 2002, Beijing, April 16-20, 2002.
NL-MS 21.945
Maddy Janse
Philips Research
What makes media compelling, EBU Multimedia Forum, Geneva, May 16-17, 2003. NL-
MS 21.965
Maddy Janse
Philips Research
What makes media content compelling- year 2010. Workshop Danish Broadcast,
Copenhagen, Sept 3-4, 2002.
Maddy Janse
Philips Research
Ambient Intelligence: Interaction Technologies for the Home of the Future. Human
Interface 2002 Conference, Ede, NL, Oct 1, 2002.
Maddy Janse
Philips Research
Interactive services for next generation users. Workshop: The different flavours of
interactive media, at the IST 2002 Information Society Technology Conference,
Copenhagen, Nov 4-6, 2002.
Maddy Janse
Philips Research
Workshop: ‘Converging technologies: new horizons for users and businesses’, FP6 IST
Networking Session, Milan, Italy, October 3-5, 2003
Maddy Janse
Philips Research
Converging technologies: new horizons for users and businesses, Workshop, IST2003,
Milan, October 2-5, 2003.
Maddy Janse
Philips Research
Content Management and User Navigation. Panel, IBC 2003 conference. Digital life
styles, Amsterdam, September 11-15, 2003
Maddy Janse
Philips Research
Interactive Applications: Special Devices. Workshop, IST E3 Concertation meeting,
Brussels, March 13-14, 2003
Maddy Janse
Philips Research
Converging Technologies: New Horizons for Users and Businesses. ENTER conference -
International Federation for IT and Travel & Tourism. Cairo Egypt, January 26-28, 2004
Maddy Janse
Philips Research
Internet applications: How Natural and Indispensable Can they Become. Philips
InterWebT Conference, Eindhoven, March 3 2004
Mark van Doorn
Philips Research
Ambient intelligence by mass customisation in ambient narratives. Journal of Persuasive
Computing, Submitted for publication 2003.
P. Pleven (SV) ICE-CREAM: Production. Workshop, IST E3 Concertation meeting, Brussels, March 13-
14, 2003.
P. Schickel
BITmanagement
Business opportunities with standard technology in diverse markets. FP6 IST
Networking Session, Milan, Italy, October 3-5, 2003
P. Schickel
BITmanagement
Front-end solutions for portals of next generation. 8
th
International Conference on 3D
Web Technology. St. Malo, France. March 12, 2003.
IST-2000-28298: ICE-CREAM ice-cream-phr-0404-01-/janse
Deliverable D22: Final Report page: 71 of 71
Warner ten Kate,
Philips Research
Internet: Technology to Host Intelligence. Chapter in book: Emile Aarts and Stefano
Marzano, Ambient Intelligence, July 2002, NL-MS 22.178
10.2. Demonstrations and Exhibitions
Event or exhibition Nature of demonstration
Net-at-home, Cannes,
14-16 Nov 2002.
Two application concepts were demonstrated in the ICE-CREAM booth:
Interactive travel application and live football (FhG FOKUS, Tomorrow Focus,
NOB). A poster presentation was part of the exhibition (Philips)
IBC2002, Amsterdam,
13-17 Sept 2002
Symah Vision presented an early version of the platform for Live Production
Tools at the IBC in September 2002.
IFA 2003 Berlin, 29 Aug
– 3 Sep’03
Exhibition: ICE-CREAM booth within IST village. Demonstration of: Interactive
travel application with travel guide service and personalisable mobile application
and 3D version of the football application
IBC 2003 Amsterdam,
11-16 Sep 2003
Exhibition: ICE-CREAM booth within IST village. Demonstration of:
Interactive travel application with travel guide service, personalisable mobile
application. & Corresponding authoring tool.
Integrated football application with MPEG-4 on a MHP set-top box and 3D
animations and replays based on live data,
DeepSea application with 3D graphic objects and ambience effects.
CeBit 2003, Hanover, 12-
19 Mar., 2003
Exhibition: Demonstration of 2D and 3D MPEG-4 prototype.
Film and video meets 3D
conference, Hanover,
April 2003.
Conference and Exhibition. Organized by the government of Lower Saxony the
exhibition showed possible future trends in the area of integrated media.
Demonstration of 2D and 3D MPEG-4 prototype.
CAT.PRO, Stuttgart, 07-
11Oct 2003
Conference and Exhibition: Demonstration of 3D MPEG-4 soccer application
and tools.
TRIAL, Nov-Dec 2003 Trial of soccer application with end users in the Euskaltelium building in Bilbao
(Spain)
MFG Medien
entwicklung. Baden-
Württemberg Public
Workshop, 2-3 May
2003
Presentation of ICE-CREAM mobile travel guide scenario and prototype
application.
Philips InterWebT
Conference, Eindhoven
March 3, 2004
Demonstration of the ICE-CREAM football application
doc_584090804.pdf