Description
The Innovative Workplaces Project fits within such a ‘business support’ model in that it offered a range of in-depth support to ten diverse organisations in the East Midlands.
Research Summary
Unlocking Engagement:
A Review of the ‘Innovative Workplaces’ Initiative nitiative nitiative
Report prepared by
Lynette Harris, Alan Tuckman, Derek Watling and Bernadette Downes
Nottingham Business School
Nottingham Trent University
March 2011
For any further information on this study, or other aspects of the Acas
Research and Evaluation programme, please telephone 020 7210 3673
or email [email protected]
Acas research publications can be found at
www.acas.org.uk/researchpapers
Unlocking Engagement:
A Review of the ‘Innovative Workplaces’ Initiative
Report prepared by
Lynette Harris, Alan Tuckman, Derek Watling and Bernadette
Downes
Nottingham Business School
Nottingham Trent University
March 2011
Acknowledgements
Particular thanks go to Gill Dix and Jonathan Cooper of the Acas National
Research and Evaluation Section for their inputs and positive support throughout
the evaluation process. We also wish to thank, Noel Lambert and Kate Nowicki of
the Acas East Midlands office, Peter Totterdill and Rosemary Exton of UKWON,
Vanessa Fraser-Davis of New College, Nottingham and the Acas facilitators for
their many contributions to the evaluation process. Thank you also to Christopher
Hale from Ecorys for his work on the economic impact assessment of the project.
However, our special thanks must go to the Gatekeepers and their colleagues in
the participating organisations for their invaluable contribution to the project’s
evaluation and to the East Midlands Development Agency for providing the
funding which made the project possible in the first place.
2
Table of Contents
Acknowledgements...............................................................................2
Foreword from Ed Sweeney, Acas Chair................................................6
SUMMARY.................................................................................................7
SECTION ONE – INTRODUCTION............................................................14
1.1 The Project..................................................................................... 14
1.2 Project Aims and Objectives.............................................................. 15
1.3 The Context.................................................................................... 16
1.4 The Recruitment Process .................................................................. 16
1.5 The Participating Organisations ......................................................... 17
1.6 Programme Design and Delivery........................................................ 20
SECTION TWO – THE EVALUATION.........................................................24
2.1 The Evaluation Process..................................................................... 24
2.2 Outline Methodology and Data Sources............................................... 26
2.3 Reflections on the Evaluation Process ................................................. 27
SECTION THREE – EVALUATION FINDINGS............................................28
3.1 Changes and Benefits ...................................................................... 28
3.2 Approach to Management ................................................................. 41
3.3 Benefits to Individuals in their Jobs.................................................... 42
3.4 Unanticipated Outcomes................................................................... 43
3.5 Constraints on Progress.................................................................... 44
3.6 Levels of Support ............................................................................ 44
3.7 Learning Transfer ............................................................................ 46
3.8 The Personal Development of the Gatekeepers .................................... 48
3.9 Outcome Measures .......................................................................... 50
3.10 Probability of Change without the IWP .............................................. 52
3.11 Sustainability ................................................................................ 53
3.12 Evaluating the IWP Methodology...................................................... 53
3.13 Messages for the Future ................................................................. 61
3.14 Paying for Future Programmes......................................................... 64
SECTION FOUR – ECONOMIC IMPACT.....................................................66
4.1 Summary of the Economic Impact of the IWP...................................... 66
4.2 Availability and Take-up of Alternative Sources of Support .................... 66
4.3 Improvements made by Participating Organisations ............................. 66
4.4 Gross Additional Impacts of Improvements ......................................... 66
3
4.5 Leakage, Displacement, Substitution and Multiplier Effects .................... 68
4.6 Present Value of GVA Impacts........................................................... 69
4.7 Return on Investment ...................................................................... 70
SECTION FIVE – CONCLUSION ...............................................................72
5.1 Why the Messages from the IWP Evaluation are of Value....................... 72
5.2 The Project’s Impact on the Participant Organisations........................... 72
5.3 Developing Management and Leadership Skills .................................... 74
5.4 The Value of the Different Elements of the IWP.................................... 75
References .............................................................................................77
Appendix A: Case Studies of the IWP Participating Organisations..........78
1. Brush Electrical Machines (BEM) Ltd ............................................ 78
2. Caterpillar Logistics................................................................... 80
3. The Health Store ...................................................................... 82
4. Liquid Control .......................................................................... 84
5. NHS East Midlands Strategic Health Authority (SHA)...................... 86
6. Northampton College ................................................................ 88
7. Pendragon Contracts and National Fleet Solutions (Derby).............. 90
8. Thorpe Kilworth........................................................................ 92
Appendix B: Acas short courses provided to participants.......................94
Appendix C: Anderson’s Evaluation Model ..............................................96
Appendix D: Ecorys Economic Assessment Report .................................97
4
List of Tables
Table One: Evaluation Sources and Schedule 26
Table Two: Reported key organisational achievements as a result of IWP
participation
28
Table Three: The most important reported organisational change as a result
of the IWP
32
Table Four: Changes or planned changes to HR Policies and Procedures 36
Table Five: IWP’s contribution to the development of Management and
Leadership Skills
40
Table Six: Six examples of changes in the approach to management 42
Table Seven: Summary of Gatekeepers’ reported three best outcomes from
IWP participation
49
Table Eight: Gatekeepers’ perceptions of the usefulness of the different
elements of the programme
54
Table Nine: The extent to which the key achievements would have occurred
without each element of the programme
55
Table Ten: Gatekeepers’ proposals for future programmes 62
Table Eleven: Gross additional economic impacts 67
Table Twelve: Net additional economic impacts per annum 69
Table Thirteen: Present value of GVA benefits 70
Table Fourteen: Return on Investment 70
List of Figures
Figure One: Gatekeepers’ reported levels of organisational support 45
5
Foreword from Ed Sweeney, Acas Chair
Throughout the recent recession Acas has operated more than ever as an enabler
to economic growth. Part of this role has been to provide a variety of tailored
support for organisations; to help them introduce new forms of work organisation
and improve employment relations – and in so doing to bring about increased
competitiveness and productivity.
The Innovative Workplaces Project fits within such a ‘business support’ model in
that it offered a range of in-depth support to ten diverse organisations in the East
Midlands. Funded by the East Midland Development Agency (emda) and run in
partnership with the UK Work Organisation Network (UKWON), Acas has
successfully delivered an innovative workplace support project which, as this
report sets out, has delivered many benefits. At an overall level, there was a
positive ratio of benefits to costs for the economy. At a micro level, the benefits
for participating workplaces included improved communication, innovation in
work organisation, better management and leadership and increased staff
engagement.
The findings of the project support the view of David MacLeod and Nita Clarke in
their review of employee engagement
1
- that organisations which fully engage
with their employees benefit greatly through better innovation, productivity and
performance. However employee engagement was only part of the Innovative
Workplaces story. A key strength of this project has been its practical and
tailored approach to helping organisations tackle challenging workplace problems.
The Innovative Workplaces Project was conceived in a positive economic climate
but by the start of the initiative, the economy had entered a recession.
Undertaking such an innovative project during a financial crisis might not on the
face of it seem well-timed – but these are in fact the very times when
organisations can benefit most from such business support projects. The IWP
project design – with its combination of leadership and management training, in-
company support and networking across organisations – was a pilot. Its results
tell us that the more organisations which Acas can help through similar projects
in the future, the more robust those organisations will be to survive future
economic turbulence.
This report provides a valuable resource for future reference. It maps both the
structure and outcomes of the Innovative Workplaces Project; and provides a
unique insight into the factors contributing to workplace change from the
perspective of multiple stakeholders. On behalf of Acas I wish to thank colleagues
from the Nottingham Trent University for their work in evaluating the project.
1
Macleod, D. and Clarke, N. (2009) ‘Enhancing performance through employee
engagement’
6
SUMMARY
A review of the Acas led business development initiative,
‘Innovative workplaces - Developing organisations for the future’
The overall purpose of the Innovative Workplaces project (IWP), the subject of
this evaluation report, was to offer ‘in depth’ support to some ten diverse
organisations in the East Midlands region who were seeking to change workplace
practices and increase employee engagement as a means of improving
organisational performance. This project, funded by the East Midlands
Development Agency (emda) in 2009/10, was managed and delivered by Acas in
East Midlands. The United Kingdom Work Organisation Network (UKWON) was
the delivery partner on the project and the Nottingham Trent University Business
School (NTU) was responsible for its evaluation.
The IWP approach
The IWP was innovative in a number of respects. It set out to:
? stimulate organisational change,
? be workplace focused,
? provide customised organisational support,
? develop management and leadership skills through a practical, rather than
a theoretical approach,
? provide an integrated evaluation of the impact of this pilot initiative.
The IWP offered direct support to two nominated representatives (referred to as
gatekeepers) from each of the ten participating organisations through a
combination of different organisational support interventions provided from June
2009 to May 2010, although some of these were extended to September 2010.
The gatekeepers were the focal point of the project, not least because
management and leadership was a priority for emda. The underpinning rationale
was that a handful of key individuals could successfully be the catalyst for
sustainable organisational change. The gatekeepers were therefore nominated
and selected for their ability to get things done rather than simply their formal
job role.
The IWP began with the gatekeepers undertaking a three and half day course,
spread over three months, to develop their management and leadership skills,
devise their action plans and begin to establish the interactions necessary for
successful action learning.
An Acas facilitator (and a back up facilitator) was linked to each organisation and
they helped the gatekeepers draw up their action plans. They also provided
customised practical support for bringing these action plans to life back in the
participants’ workplaces. Additional support was provided where there was an
identified need, for example through Acas training.
7
Full day monthly participant network meetings facilitated by UKWON incorporated
action learning sets in which the participants were invited to both support and
challenge one another when devising and implementing their action plans.
The action plans were pivotal as they informed the projects within each
organisation and provided the basis for evaluating the IWP’s impact. The
gatekeepers presented their plans to their fellow gatekeepers in action learning
sets on the morning of 23
rd
September 2009. That afternoon the project was
formally launched at an event featuring national and local business figures able to
pass on the benefit of their experiences of change through engagement.
The organisational benefits subsequently achieved proved to be considerably
wider than the scope of the initial action plans for many of the participating
organisations.
Changes and Benefits
All the participating organisations reported that the IWP had led, not only to the
achievement of some of the workplace changes sought in their initial plans, but
also to improvements in the wider employee relations climate. For the majority
what they had sought from participating in the IWP was achieved to a large
extent and a range of different, but frequently related, organisational issues had
been addressed; these included improving levels of employee engagement,
morale, communications between management and employees or staff in
different functional areas, workforce flexibility and the implementation of change.
Respondents from the smaller organisations were more positive and more likely
to have a shared view within the organisation about the outcomes of the IWP and
its business benefits. In the SME business context the impact of what had been
achieved was, arguably, easier to identify and more visible to the workforce. In
contrast, the two public sector organisations appeared to experience the most
difficulty in clarifying the aims and scope of their action plans at the outset,
partly due to the presence of other related, and potentially overlapping,
organisational initiatives being underway at the same time as the IWP (for
example, a leadership development programme).
The following summary identifies the reported key outcomes of the project for
the eight organisations that completed the IWP and the participant’s perceptions
of its contribution in terms of achieving the changes they were seeking. The UK
was in recession for almost all of the IWP; an economic context that had a
bearing both on continuing participation and the progress of individual
organisational projects.
Communication and Engagement
Improved communication was identified by respondents from all the participating
organisations as the ‘single most important change’ resulting from the IWP by
December 2010. This was the view of the managers, employee representatives
and the gatekeepers who took part in the project’s final evaluation. In six of the
organisations improved communication was identified as leading directly to
increased levels of employee engagement. In each organisation, improvements in
communication and employee engagement stemmed from the adoption of
8
mechanisms for capturing direct inputs from the workforce and listening to
employees’ views. Mechanisms for improving employee voice ranged from the
establishment of a workplace employee forum, to a steering group with employee
representatives with reporting task groups and/or employee focus groups
facilitated by Acas.
The IWP had resulted in seven out of eight organisations putting in place
mechanisms to promote and capture new ideas. The smaller businesses found it
rather easier to provide instances where ideas from the workforce had been
proposed, explored and implemented. At five organisations, where Acas
facilitated focus groups had taken place, respondents reported that these had
resulted in increased levels of employee engagement and a greater willingness to
contribute ideas. There was a more mixed message from some other
respondents who identified that, whilst the IWP had encouraged the sharing of
ideas, they had concerns about longer term levels of organisational interest and
engagement with the ideas put forward by employees.
The organisational benefits associated with improved communication varied
depending on the issues facing each organisation. For example, participation in
the IWP had enabled one organisation to return to levels of productive, informal
communication that had characterised the business prior to its expansion and
move to a larger premises. At another participation had led to the achievement of
one of its main aims in joining the project; a 10 percentage point improvement in
the employee engagement score in its annual company employee survey.
Managers in half of the participating organisations reported that one outcome of
their involvement in the IWP was that issues formerly referred directly to them
were now being resolved at a lower level in the management chain or by
employees themselves. This was identified by respondents as saving
management time with consequent improvements in efficiency and productivity.
These benefits were particularly identified by participants in the smaller
businesses and were seen to be the result of increased employee involvement.
For example, one SME manager, who was a gatekeeper on the programme,
reported a 75 per cent reduction in the time he personally spent addressing
workplace disciplinary and grievance issues.
HR policies and procedures
Seven of the eight organisations reported the implementation of at least one new
or improved human resource policy or procedure and all had plans for future
improvements following their participation in the IWP. The most widely reported
were improvements to processes for informing and consulting with employees
and absence management.
Workplace climate
Identifying those factors which contribute to improved morale is complex.
Notwithstanding, the majority of respondents identified that workplace morale
had improved following participation in the IWP but it was not always possible to
identify whether or not this improvement could be attributed directly to the
impact of the project. There were also a number of organisations where events
had or were taking place, frequently due to the economic climate, which were felt
9
to be impacting negatively on workforce morale; these included a pay freeze and
redundancies.
Management and Leadership Skills
In terms of improving management and leadership skills, the final evaluation
revealed that the majority of respondents felt this had happened either partly or
to a large extent as a result of their organisation’s participation. The benefits
associated with improvements in management included improved trust levels
between employees and management. This was reported by the majority of
respondents across all the organisations evaluated although this was not
necessarily a shared view across all the respondents from the same organisation.
The reasons for this varied; for example, at one organisation the level of
management and leadership skills had not been identified as an issue to be
addressed and, at another, a dispute over pay had led to internal differences
between management and employees.
Asked whether they could have obtained the support provided by the IWP from
another source, respondents in managerial roles felt this might have been the
case. However, the survey data revealed that it was highly unlikely that such
support would have been taken up with the participants in the smaller businesses
most consistently reporting that this would not have happened and that the
reported changes would not have occurred without the IWP. Most significantly,
the IWP was held to have acted as the catalyst for organisational change by the
vast majority of respondents; a view shared by both the delivery partners and
the Acas facilitators.
Economic Impact
The economic impact assessment of the IWP reported an overall minimum return
on investment of £4 for every £1 of public sector expenditure and a measurable
positive economic impact of approximately £906,000 attributable to the IWP’s
expenditure. Participating organisations attributed proportions of improvements
in staff productivity and present and future employment growth directly to the
IWP. The economic impacts quoted can be thought of as cautious estimates, due
to the fact that it was not possible to fully measure all benefits – such as
participating organisations reporting that their difficulties (often related to the
recession) would have been considerably greater without the organisation’s
involvement in the IWP, but being unable to quantify such impacts.
Evaluating the IWP Methodology
As described, there were a number of components to the IWP approach. Whilst
the impact of the IWP overall stemmed from the ‘sum of its parts’, the
customised facilitation process was regarded as particularly useful in terms of its
contribution to the final outcomes; indeed the majority reported that they would
have liked continued support from an Acas facilitator. The evaluation revealed
that the facilitation process was increasingly valued as time progressed when it
became more apparent what it had actually contributed. This may explain why a
number of participants said that they wished they had made more use of their
facilitator before the facilitation process ended and that they would have done so
10
if they had realised earlier what this could offer. From the perspective of the
facilitators themselves, it was unanimously identified that having terms of
reference agreed with senior management in place at the outset would have
enabled more rapid progress against the action plans and reduced the time they
personally spent in developing an understanding of what some of the workplace
projects were aiming to achieve.
A feature of the IWP was that it had intentionally included a diverse range of
organisations. From the perspective of the participants this diversity had the
advantage of offering new perspectives. The network events and action learning
sets facilitated by UKWON were particularly valued as an opportunity to share
experiences and their individual project’s progress but gatekeepers also reported
that learning about the practices adopted by other participating organisations
tended to have limited relevance to their own situations because of the differing
nature of their businesses or sectors. One suggestion from gatekeepers at a
couple of the participating organisations was that having access to a mentor from
a similar industry or sector would be a valuable addition to the interventions
offered by the project.
The initiative achieved changes in a relatively short period of time, particularly in
terms of improving employee engagement through enhanced communication in
many of the participating organisations. The evaluation findings suggest that this
is most likely due to the mix of interventions that were offered which could be
adapted to suit the needs of individual organisations and their representatives.
This approach worked particularly well where there was a gatekeeper
representing the organisation who had the ability and sufficient influence in their
workplace to progress their project, particularly when there were obstacles to be
dealt with.
The programme was designed to be sufficiently flexible to accommodate the
needs of the different organisations, their emergent action plans and the
development needs of their gatekeepers. This meant that its interventions were
loosely structured but some gatekeepers reported that they would have
welcomed more delivered inputs which provided specific information that they
could take back to their organisation, for example, the presentations provided by
the speakers at the highly successful IWP launch event on employee
engagement.
Perceptions of the value of each element of the programme varied for each
organisation and the individual gatekeepers depending on what they were
seeking. The need to accommodate this diversity was recognised by UKWON and
had informed the design of the elements they provided. Their aim was to ‘ground
pre-existing knowledge in the task at hand’ for gatekeepers with prior knowledge
and experience and ‘to provide sufficient actionable knowledge on employee
involvement and participation’ for those with less knowledge and experience. The
evidence from the evaluation is that this was largely achieved. But the overall
message from the IWP is that it was essentially the combination of the different
elements that led to the final outcomes for both the participating organisations
and the gatekeepers. In practice, the participants took what they needed at
different times from the IWP.
11
Levels of support for the IWP
Senior management of the organisations signed the application forms for IWP to
register their support for the project but it was always intended that the
organisation gatekeepers and Acas facilitators would have to work together to
build support for the project within the organisations, including from senior
management. Levels of support for the IWP from senior management, and from
some employees where there was identified low morale, emerged as a particular
concern for over half the nominated gatekeepers in the initial interviews. But, the
final evaluation revealed a positive message about the actual levels of support for
the project from senior managers, middle managers and employees. In four
organisations it was reported that management support had grown as a result of
the inputs from the Acas facilitation process and as the benefits stemming from
the IWP became apparent. The degree of senior management support, and
ensuring their early engagement, were identified as critical factors in progressing
organisational projects and in achieving their aims, not only by the gatekeepers,
but also by the delivery partners and the Acas facilitators in their final reflections
on the IWP.
Sustainability
There was clear evidence in the final evaluation that respondents increasingly
viewed the workplace achievements resulting from their involvement in the IWP
as part of ‘on going’ programmes of change. However, a third of the
organisations were concerned that, without the IWP, the momentum would not
be sustained. This was particularly reported where there had been a loss of
individuals who had championed the project due to staff turnover.
Looking to the Future
Asked about paying for a similar programme to the IWP in the future, only the
larger organisations felt this might be a possibility but that this would depend on
factors such as the economic climate, their ability to pay and (for the private
sector companies) evidence that such an intervention would increase turnover.
Despite identifying the most positive outcomes from the IWP, the smaller
businesses reported that it was very unlikely that they would be able to pay for
such a programme. Notwithstanding, all the participants identified that if they
were to consider such a project in the future, it would be important to them that
it was provided by Acas on the grounds of its reputation and impartiality.
Overall, the evidence from the evaluation reveals that the Innovative Workplace
Project led to improvements in the participating organisations, predominantly in
the areas of workplace communications, employee involvement and engagement
as well as other dimensions of employee relations which impacted positively on
their organisation. The outcomes of the IWP lend support to the key conclusion
of MacLeod and Clarke’s 2009 review of employee engagement for the
Department of Business; that improving employee engagement can improve
business performance. To varying degrees, the IWP also met its objective of
developing leadership and management skills and was a vehicle for the
identification of skills gaps in several organisations. Furthermore, the economic
impact assessment analysis revealed that these changes, when aggregated,
12
generated a positive return for the organisations as a whole in difficult economic
times.
A key message from the evaluation of the IWP is that similar future initiatives,
customised to meet the needs of individual organisations, could offer valuable
support to employers who are seeking to engage their work forces and work
collaboratively in initiatives to improve efficiency and changes to working
practices.
13
SECTION ONE – INTRODUCTION
1.1 The Project
The purpose of this report is to provide an independent evaluation of an Acas
initiative ‘Innovative Workplaces - Developing Organisations for the Future’
(referred to hereafter as ‘IWP’) which took place during 2009 and 2010. The
overall aim of the initiative was to offer in-depth support to a number of
organisations in the East Midlands region who were seeking to change workplace
practices and increase employee engagement as a means of improving
organisational performance. The project was funded by the East Midlands
Development Agency (emda), and managed by Acas in the East Midlands region.
Acas also delivered the project programme in collaboration with the United
Kingdom Work Organisation Network (UKWON), its appointed delivery partner.
Additional strategic input was provided by the Acas National Research and
Evaluation Section (RES) and Acas’ Academic Adviser Professor John Purcell. The
East Midlands Employment Relations Forum (ERF) steering group acted in an
advisory capacity throughout the project.
Over a period of twelve months, the Innovative Workplace Project (IWP) provided
a number of different interventions to ten participating organisations. The IWP
began in April 2009 with the recruitment and selection of organisations seeking
to participate. The core elements of the IWP were: an initial three and half day
short course designed both to develop leadership and management skills and to
build trust and dialogue between participants; monthly meetings of the
participants; and facilitation by senior Acas advisers to provide customised
workplace support. A key requirement of the initiative was that there should be
an independent evaluation of its impact in terms of the experiences and
outcomes for the participating organisations and an identification of lessons
learnt for the purposes of wider dissemination. In this sense, the IWP was
regarded as a pilot. Nottingham Trent University was commissioned to lead the
evaluation. The independent research agency, Ecorys, was appointed towards the
end of the IWP to undertake an analysis of the economic impact of the project
which used, and complemented, the data collated by the Nottingham Trent
evaluation
1
.
The report sets out the key findings from the evaluation which culminated in final
review interviews focusing on the outcomes for the participating organisations
and individuals (designated as the ‘gatekeepers’). These final interviews took
place during November and December 2010, six months after the conclusion of
the main elements of the IWP.
The report is divided into five sections. Section One outlines the IWP’s aims and
objectives, the design and rationale for the different elements of the delivered
programme, the participating organisations and what they were hoping to gain
from the IWP. Section Two details the evaluation process. Section Three reports
on the outcomes for the participating organisations, the individual gatekeepers
and their evaluation of the IWP programme. Section Four provides a summary of
the economic impact of the IWP and Section Five identifies the main conclusions
1
A summary of the economic impact analysis is in section four, and the full Ecorys report
is in Appendix D.
14
that can be drawn from the project in terms of its impact and any wider
messages that can be drawn from its outcomes which could inform similar
interventions in the future.
To avoid confusion in the report’s terminology, the overall initiative is referred to
as the project or the IWP whereas the delivered elements provided by the project
are referred to as ‘the programme’; ‘individual projects’ refers to those initiatives
that took place within the participating organisations and their nominated
representatives on the programme are described as the ‘gatekeepers’. As this
was the descriptor used throughout the life of project, it has the merit of being
familiar to all those involved when it came to evaluation. Where ‘respondents’ are
referred to in the report this includes all the individuals, including the
gatekeepers, who were interviewed as part of the evaluation process in the
participating organisations.
1.2 Project Aims and Objectives
The IWP’s overall purpose was to enable a diverse ‘pilot’ group of organisations
from across the East Midlands to increase profitability and competitiveness, as
well as improving working lives, through the development of enhanced leadership
competencies, new forms of work organisation and increased employee
involvement and engagement.
The IWP builds on research evidence that employee involvement and
participative forms of work organisation contribute to organisational performance
and enhanced competitiveness (Purcell et al., 2003; Boxall and Purcell, 2007;
Ashton and Sung, 2002). Existing research, reviewed by Sisson (2005) in a
report for emda, indicates that the return on investment in skills development is
not fully realised unless firms develop working practices based on job enrichment
and employee initiative. Sisson argues that the term “workplace innovation”, part
of the project’s rationale, is commonly used to describe the introduction of
workplace practices that are new to the company itself and which have to be
reinvented within each organisational context. It is this definition of workplace
innovation that is reflected in the project’s title.
The initial aims and objectives developed by Acas and emda for the IWP were to:
? Facilitate sustainable organisational change and improve business
performance by focusing on the development of essential management
and leadership skills, better work organisation and employee involvement
and engagement;
? Capture a body of evidence by gathering the stories from each of the
intended ten participating organisations, to provide a guide to effective
organisational change drawn from practice;
? Share the lessons learned and the outcomes achieved in order to
encourage other organisations both in the East Midlands and nationally to
adopt similar interventions;
? Influence policy makers and stakeholders as to the importance of
management and leadership skills development, work organisation and
employee engagement for business productivity;
? Provide an example of how Acas, working in partnership, can improve
business productivity and working lives in a sub-national economy.
15
1.3 The Context
The initiative was conceived of prior to the published findings of the MacLeod
Review on Employee Engagement (MacLeod and Clarke, 2009). But MacLeod’s
findings together with the principles of the Acas Model Workplace (2005) and
UKWON’s international involvement in workplace innovation initiatives provided a
backdrop for the IWP’s design and the delivery of its different elements. By the
time recruitment for the IWP began in 2009, the UK economy was in a recession
which deepened during the life of the initiative; a factor which impacted to
varying degrees on the participating organisations and the individuals selected as
the gatekeepers on the IWP. For example, one company had to withdraw when it
went into receivership; another went into receivership with a subsequent
management buy out; at another, a gatekeeper was made redundant. This
uncertain and challenging environment needs to be taken into account in any
consideration of the initiative’s final outcomes. It was encouraging that, despite
the various pressures experienced by the participating organisations, there was a
continuing commitment to their action plans and to their involvement in the
project.
1.4 The Recruitment Process
In Spring 2009, the opportunity to participate in the IWP was widely advertised
through emda, Acas and UKWON with a series of open access familiarisation
sessions for organisations interested in learning more about the initiative.
Following a process of application, a number of organisations from across the
East Midlands region were invited by Acas to face-to-face discussions during May
and June 2009. The Project Manager was variously joined by Acas and UKWON
colleagues for the discussions which took place with a mix of senior managers,
potential gatekeepers and HR professionals from organisations interested in
participating. Organisations were asked in the discussions to set out their overall
objective in seeking to join the IWP; why they felt it would benefit them; and to
demonstrate their commitment to engaging and staying with the IWP from start
to finish. The latter was an especially important criteria in determining which
organisations would be invited to become involved. The outcome of these
discussions was that eleven organisations were recruited to participate in the
IWP. These represented considerable diversity in terms of their size, sector and
geographical location across the region.
Two employees were nominated as gatekeepers by each organisation to attend
the programme and act as the catalyst in taking their plans for action forward
with the support provided by Acas and UKWON as the delivery partners. The
suggested criteria for selecting the gatekeepers were that they should be
proactive individuals who ‘would get things done’. One company decided to
withdraw from the programme at the beginning of the initial short management
and leadership course. The ten remaining organisations, the size of the workforce
at the workplace involved in the IWP and what they were seeking from their
participation are described briefly below. Further details of those organisations
that completed the IWP can be found in the short case studies in Appendix A.
16
1.5 The Participating Organisations
Brush Electrical Machines Ltd (BEM), a manufacturer of generators for steam
and gas turbines based in Loughborough, Leicestershire with 800 employees at
the start of the project. Following a recent change of ownership, the company’s
aim in participating in the IWP was to improve morale, trust and employee
engagement. It was hoped that its participation would assist BEM to become a
more innovative work organisation and reduce resistance to change. At the time
of joining the IWP, the company’s particular concerns were that, whilst the
workforce was highly skilled, there was a need to develop a less functional, more
process driven perspective among the workforce to take the organisation
forward. The nominated gatekeepers were a Business Analyst and a Senior Unite
Trade Union representative. Participation in the IWP was first suggested by a full
time Unite Official and was the only instance where a gatekeeper was also a
workplace trade union representative.
Caterpillar Logistics, a wholly owned subsidiary of Caterpillar Inc. providing
third party warehousing and distribution and other services. At the start of the
project it had 87 employees at its Hinckley site where the project took place. The
Hinckley operation resulted from a takeover of a Caterpillar customer. The
Company’s key aim for the project was to specifically improve upon the level of
employee engagement, reported as 62% in its 2008 employee survey, to a target
of 72% or more through improvements to employee participation and
cooperation at all levels. At the time of applying for the IWP, issues of
communications at the site had been identified as being aggravated by the
presence of different cultures at the Hinckley site resulting from the takeover.
Several changes in employer were identified as contributing to a high degree of
workforce scepticism with regard to change, new ideas and initiatives. The
nominated gatekeepers were the Logistics Centre Manager and an HR Manager
who also had responsibilities across three other Caterpillar sites.
Donington Park Leisure (DPL), an events management company known for its
race circuit events for motor sports and ‘would be home’ of the British Formula
One Grand Prix with 33 employees based in Castle Donington, Derbyshire. Its
rationale for participating in the IWP was to support its aim of being at the
forefront of customer service through an investment in staff and ultimately to be
‘Investors in People’ accredited. The company was particularly seeking to
enhance customer service through improvements to employee engagement and
internal communications. Its nominated gatekeepers were the Sales Manager and
the HR Manager. Unfortunately DPL had to leave the project as it went into
administration in November 2009 following its unsuccessful bid to host the 2010
British Formula One Grand Prix.
Liquid Control, a builder and supplier of standard and custom built machines for
processing (metering, mixing and dispensing) single and multi-component
liquids/pastes based in Northamptonshire with a workforce of 20 employees. At
the time of applying to join the IWP, the company was looking at ways to
improve the way the business was managed. It had identified that productivity
and efficiency gains were needed to sustain the business and the IWP was seen
as a means of increasing workforce productivity and efficiency through greater
employee engagement. The nominated gatekeepers were the Engineering
Manager and the Service Manager.
17
MAHLE Powertrain, an engineering and consultancy firm developing
technologies and products for reducing fuel consumption which also offers
traditional engineering services for the automotive and engine industry. With 300
employees at its operations in Northampton where their project took place,
Mahle’s aim in participating in the IWP was to retain skilled staff in difficult
economic times through increased empowerment and employee voice and to
improve morale, trust and a sense of recognition among the workforce. Mahle
intended to establish a cross functional employee forum to improve two way
communications. Progress with their project was delayed due to short time
working and redundancies at the site as a result of the impact of the recession on
the business. The nominated gatekeeper was a Senior Principal Analysis
Engineer; there were initially two gatekeepers but one of the gatekeepers left the
IWP not long after it began due to redundancy. Concerns about commercial and
organisational sensitivities resulted in an executive decision by the company in
April 2010 that they would not continue to participate in the IWP.
Notwithstanding, their project continued internally and the organisation made a
contribution to the final evaluation of the IWP.
NHS East Midlands, the strategic health authority (SHA) for the region
providing leadership of the NHS to ensure that health systems operate effectively
and efficiently for a population of 4.5 million spread across Derbyshire,
Leicestershire, Lincolnshire, Northamptonshire, Nottinghamshire and Rutland.
Established in July 2006 with a role to relay and explain national policy, set
direction and support and develop all NHS Trust bodies, its workforce of 350
employees are mainly based at the SHA’s Headquarters in Sandiacre,
Nottinghamshire where the project took place. The SHA’s initial aim in
participating in the IWP was to maximise the benefits of electronic staff records
(ESR) by transferring ownership to individuals and providing high quality
information for employees and line managers. The intention was that, if this led
to a greater staff utilisation of the SHA’s Oracle Learning Manager (OLM), it would
support the implementation of the organisation’s Talent Management Framework
and NHS East Midlands’ commitment to “grow its own” leaders and managers for
the future. It was identified at an early stage in the project that communication
was a key issue and this became the critical focus. The nominated gatekeepers
for the IWP were a Business Manager in Public Health (who left during the IWP
for a period of maternity leave) and the Workstream Leader (Planning and
Projects) in the Information Management and Technology Department.
Northampton College, a provider of Further and Higher Education to around
12,000 learners split over four sites in Northampton which employs some 1,000
employees. The project took place across all four sites. The College’s specific aim
in participating in the IWP was to attain an improved Ofsted grade by 2013. They
saw the IWP as providing an opportunity to achieve the internal culture change
identified as necessary to improve the 2009 ranking of Satisfactory. There were
challenges facing the College in terms of its student success rates, developing
management and leadership capability with little history of centralised strategy
and corporate coherence leading to long established local practices. In addition,
the college had a number of recognised trade unions resulting in a range of
consultative processes. An additional challenge for the college, at the time of
joining the IWP, was that it was in the early stages of demolishing and rebuilding
its main campus without relocating its operations. The nominated gatekeepers
were the Manager for Professional Development and a Management
Lecturer/Team Co-ordinator.
18
Pendragon Contract Hire and Fleet Solutions is the commercial and
contracts leasing division of an automotive retail network for new and used
vehicles also trading as Stratstone, Evans Halshaw and Chatfields. This part of
the business, located in Derby, employed 98 staff at the start of the IWP. Due to
the economic climate and its impact on the industry, there had been a series of
restructures and redundancies within the company which had led to feelings of
uncertainty in terms of job security and a resultant loss of employee engagement
which impacted on overall business outcomes. The company’s aim in
participating in the IWP was to improve employee engagement primarily through
the establishment of a consultative forum to gain the views of the workforce.
Having completed its first employee survey shortly before applying to join the
IWP, it was seeking to increase the 50% response rate and the level of positive
responses. The IWP was seen as a means of improving employee relations and
moving the business forward by developing better people management. The two
gatekeepers nominated for the IWP were the Customer Services Director and the
HR Manager.
The Health Store, a customer owned cooperative health food wholesaler and
distribution centre for its members, with a production unit producing own label
dried goods based in Nottinghamshire which employs 102 employees. The
company’s aim in participating in the IWP was to gain a forward thinking and
trained workforce able to contribute to the future of the business through
improved employee engagement and improved communications between
management and employees. At the time the company joined the IWP it had
identified that morale had dropped since moving, in 2007, to its new, purpose-
built premises, and was seeking to improve this as a result of improved
communications between management and employees. The nominated
gatekeepers were the Operations Director and the Warehouse Manager. One of
the gatekeepers ceased his formal participation in the programme at the time of
the initial short course but remained very involved in the internal project within
the workplace.
Thorpe Kilworth, a designer, manufacturer and installer of furniture for
education, laboratory and health care establishments with 99 employees based in
Corby, Northamptonshire. At the time of joining the IWP, the company had
identified the need to be more responsive to changes in the market and the
needs of its clients. Its aim was to use their project as the means of addressing
these issues through increased workforce versatility and flexibility enabling them
to move individuals from one process to another in order to meet fluctuations in
demand. The aim was to improve the company’s competitive edge by maintaining
quality but reducing waste and inefficiencies. One particular focus was to improve
employee engagement in a workforce viewed as loyal and responsive but with
concerns about changing long established ways of working. Improving two way
communications so that the workforce understood and were committed to
changes in the business was seen as a key factor in increasing employee
engagement. The nominated gatekeepers for the IWP were the Organisational
Development Manager and the Production Manager. Thorpe Kilworth went into
administration in June 2010 but had completed the IWP and continued to be
included in the evaluation process.
19
1.6 Programme Design and Delivery
The Programme Design
Acas and UKWON’s stated principles for the design of the Innovative Workplaces
project were as follows:
a) The effectiveness of support for companies individually (through facilitation or
consultancy) is considerably enhanced by group-based learning and
knowledge exchange combined with peer-review of change proposals and
implementation processes.
b) Participative work organisation and employee involvement can lead to
convergence between improved performance and improved quality of working
life.
c) Change Entrepreneurs able to instigate dialogue, mobilise diverse actors and
work between formal organisational structures can play a key role in securing
effective and sustainable change. But to be effective in this role, individuals
need to be able to see themselves as entrepreneurial and to receive high-
level support even when they challenge established practice.
These design principles led to the ten participating organisations receiving three
strands of support:
1. An initial short course of three and half days delivered over three months
which was designed to enable participants to learn about good practice,
develop their leadership skills, evaluate their own organisations with
reference to the Acas Model Workplace and to formulate an action plan for
change.
2. Six full day participant network meetings consisting of half-day thematic
inputs and discussion on shared issues and problems and a half-day of action
learning sets in which each organisation presented its progress,
achievements, obstacles and dilemmas for peer review and discussion. There
were two additional half day meetings of the action learning sets.
3. Acas facilitation for a period of up to eleven days where an Acas Senior
Adviser gave practical in-company support and Acas advice and guidance.
This was supplemented by Acas short courses and further inputs from UKWON
where a particular need was identified.
The Acas Project Manager was responsible for the coordination of the different
elements and monitoring progress in the participating organisations. The latter
was undertaken through the Acas facilitators and UKWON, who between them
were in regular contact with all the participants. Taking the above core
objectives, and the design principles together, the different elements of the IWP
were each designed to achieve a range of objectives, as set out below:
Build Commitment
Building and maintaining the commitment of the organisations’ nominated
participants was an important strand. At the outset the two gatekeepers from
each organisation took part in the short, action-learning based programme. The
course, designed and extensively piloted in a previous project by UKWON, was
delivered over a period of weeks by New College, Nottingham and provided
optional Institute of Leadership and Management accreditation for the
20
participants. Each organisation devised and refined its action plan during this
stage of the programme. Plans were then presented by each gatekeeper and
subjected to scrutiny by their peers during the final half day of the course on the
morning of 23
rd
September 2009.
During the afternoon of 23
rd
September, the IWP was publically launched with an
inception event. A range of guests from the participant organisations, including
some senior managers and employee representatives as well as the gatekeepers,
the project managers and delivery partners and the Acas facilitators attended the
event. The key speakers for the event included David MacLeod, co-author of the
MacLeod Review on Employee Engagement (MacLeod and Clarke, 2009), Warren
Glover of Lindum Construction (an East Midlands employer from the Times Top
100) and John Purcell (Strategic Adviser to Acas with a specialism in the link
between employee involvement and organisational performance). The event, as
well as providing an opportunity to publicise the initiative, was also designed to
provide further stimulus to the project participants in preparation for their return
to their workplaces to implement their action plans. An end of project
dissemination event took place on 20 October 2010; this was an interactive
symposium on employee engagement with guest speakers and inputs from the
gatekeepers about the IWP’s impact on their organisations and their own
development.
Facilitate Change
Identifying and facilitating change, as fitted the individual organisations’
priorities, were core objectives. In their role as facilitators the Acas Senior
Advisers provided practical in-company advice and guidance in accordance with
the Facilitator’s Brief. In all cases this began with initial meetings to explore and
discuss the action plans. The Advisers used their skills and experience to
facilitate the process of turning ideas and aspirations into practical strategies.
This facilitation role was core to the relationship between the Senior Advisers and
the organisations. In many cases the initial advice was followed by diagnostic
workshops and focus groups, facilitated by the Senior Adviser. These provided
powerful information and gave a sound basis for the direction which the
individual projects then took. As well as initial advice and workshops, Acas also
gave advice on setting up consultative forums, on staff surveys, and on wider
policy development. The contact with the Acas Adviser provided a central core of
support which was supplemented by additional Acas training (see Appendix B).
As supplements to the core programme, UKWON provided additional guidance to
Northampton College, at a point where greater clarity was needed, and on work
organisation to Thorpe Kilworth. Acas made its open access training courses
available to all the organisations. They were developed and provided in
consultation with the participant organisations. Details of this provision and
attendance are provided in Appendix B.
In some cases the Acas Senior Adviser provided specific in-company bespoke
training to support individual projects. Pendragon received training on ‘Essential
Skills for Supervisors’ and ‘Training for Workplace Representatives’ and
Caterpillar and The Health Store each received bespoke training for their new
employee representatives.
21
Provide Peer Learning and Review
Gatekeepers took part in monthly network meetings designed to provide a
greater depth of understanding in relation to specific aspects of work organisation
and employee involvement. The network meetings aimed to facilitate the
exchange of knowledge and experience between participants. The content of
these meetings was, as far as possible, responsive to needs expressed in the
action learning sets and issues raised by the Senior Acas Advisers. UKWON
organised and facilitated the network meetings which were also attended by the
Acas Project Manager.
In the afternoons, following the network meetings, action learning sets facilitated
by UKWON enabled participants to reflect on progress and refine their action
plans based on the exchange of ideas between gatekeepers and peer review.
The programme structure provided a framework within which the gatekeepers
could reflect on the strengths and weaknesses of existing practices in their
organisations, learn from and crucially challenge each other, test ideas and
proposals in a safe and supportive environment, and share both problems and
achievements as their work progressed.
Evaluate the Project
Evaluation was a key element of the IWP and it was a requirement of the emda
funding that there was an evaluation of the IWP’s outcomes. In practice,
evaluation processes were woven into all aspects of the delivery, diagnostic and
learning processes with a view to identifying:
a) the impact of the IWP (including its economic impact) from multiple
perspectives within the organisation, to include specific benefits to
participants and their organisations and unforeseen outcomes;
b) the effectiveness of the development interventions; for example, the
workplace facilitation and the network forums from the perspective of the
participants;
c) information for dissemination about an innovative project based on
partnership working, particularly about any ‘lessons learnt’ for future
interventions, for organisations, delivery partners and other regions
considering similar projects.
Disseminate the Lessons Learnt
There was a focus throughout the IWP to capture both its outcomes and
experiences of the participants to:
? enable other organisations in the East Midlands and nationally to follow
suit;
? publicise the approach and achievements of the individual organisational
projects to policy makers, strategists and stakeholders regionally and
nationally;
? provide opportunity for the region, the organisations, participants and
those involved in the delivery to share their experiences of an innovative
project which champions a partnership approach.
22
? aside from a final written report, the main intended vehicles for the
dissemination process were:
o regionally the Acas Employment Relations Forums in the East
Midlands, a themed local event;
o internal Acas meetings, such as senior management team
gatherings;
o ad hoc meetings with policy makers, strategists and stakeholders;
o short filmed summaries including a project overview from the
managers and delivery partners, and short (3-4 minutes) filmed
accounts of the work and impact of the IWP in some, if not all, of
the participant organisations;
o short case studies of the participating organisations;
o the Acas website, particularly the East Midlands pages.
23
SECTION TWO – THE EVALUATION
2.1 The Evaluation Process
The evaluation element of the IWP was designed to reflect the different
approaches to assessing learning value contribution based on Anderson’s model
(2007) of value and evaluation set out in Appendix C. There was a particular
focus on evaluating
? the return on expectations in terms of the extent to which intended
organisational outcomes were realised
? the economic impact/return on investment through a range of
performance indicators
? the extent to which the wider aims of the intervention had been achieved,
? the efficiency and effectiveness of the learning and development process
and activities.
A ‘cornerstone’ of the monitoring/evaluation approach was obtaining a ‘pluralistic
perspective’ of each organisation’s individual project and its outcomes. As a
result interviews took place with a range of stakeholders at each participant
organisation in addition to the nominated gatekeepers at the beginning of the
project and six months after the programme had finished. These usually included
a senior manager and/or line manager, an HR manager and an employee
representative.
A multi-method research design was adopted to generate both qualitative and
quantitative data in order to evaluate the IWP’s overall impact against its four
overall aims. This required an evaluation of a number of specific outcomes from
different stakeholder perspectives with a particular focus on:
? the organisational changes from participating in the IWP, including any
unforeseen outcomes;
? the development of the individual gatekeepers;
? the extent to which skills and knowledge had been transferred from the
gatekeepers to others within the organisation;
? the extent of sharing learning and knowledge between the gatekeepers on
the programme;
? the effectiveness of the different development interventions provided by
the programme from the perspective of the participating gatekeepers;
? the lessons learnt from the IWP in terms of what worked well and less well
to help for the purposes of wider dissemination;
? the cost/benefits to the participating organisations and a set of questions
designed specifically to calculate the economic impact of the IWP.
The Evaluation Stages
The evaluation consisted of the following four stages.
Stage One
At the beginning of the programme, short postal questionnaires were completed
by each participating organisation (in most instances by an HR Manager) to gain
initial baseline data. Once the participating organisations had drawn up their
24
action plans, face-to-face interviews were conducted in October 2009 to identify
different organisational stakeholders’ expectations of the organisational benefits
to be gained from involvement in the IWP, and their perspectives on the
proposed changes in their organisation’s action plan. This data formed the basis
for the final evaluation of the IWP’s overall impact. In addition face-to-face or
telephone interviews were conducted with all the delivery partners to explore
their expectations for and perspectives on the IWP and its objectives.
Stage Two
The analysis throughout the IWP of any feedback data on the delivered elements
of the programme which included feedback forms from the initial short course,
participant network meetings (which included the half day action learning sets)
and any supplementary Acas short course attendance. It had been intended to
also gain information about the learning of participants and how this was being
shared and applied in the workplaces through the maintenance of learning logs
while they were undertaking the IWP. In practice, these learning logs were not
completed although a couple of gatekeepers had maintained a log of their
personal learning.
Stage Three
Telephone interviews at the end of the facilitation process (in all but one
organisation) with all the gatekeepers to obtain an evaluation of progress against
their action plans and their experiences of the different elements of the IWP.
These interviews took place in May 2010 and referred back to the participants’
initial plans and expectations. In addition, telephone interviews were conducted
with the Acas facilitator for each participant organisation. These interviews were
supplemented by reflective summary reports provided by each member of the
Acas facilitation team.
Stage Four
The final stage of the evaluation took place during the last three months of 2010,
approximately six months after the programme delivery had finished.
Questionnaires, completed by the participant organisations, revisited the areas
addressed in the base line survey undertaken at Stage One and included a
number of questions designed specifically to evaluate the economic impact of the
organisations participating in the IWP at an organisation and economy-wide level.
Final interviews were conducted in November/December 2010 with, wherever
possible, the same individuals at each organisation interviewed at Stage One.
End of project interviews were also conducted with the UKWON and Acas delivery
partners which included the Acas Project Manager. The key purpose of the final
evaluation stage was to identify the overall impact of the IWP for the participant
organisations and the individual gatekeepers. The data collated was compared to
the data provided at Stage One. The aim was to explore the extent of changes,
achieved or ongoing, as a result of the IWP.
A key challenge for the evaluation process was that there had been other
changes at a number of the participant organisations, including staffing, whilst
undertaking the IWP. Acknowledging that many outcomes/impacts would be
ongoing and realised in the future, the optimal time chosen to balance the
reporting deadline, the availability of data and allowing sufficient time for change
to take place was six months after the delivered elements of IWP had largely
finished.
25
2.2 Outline Methodology and Data Sources
Various methods were adopted to measure the outcomes of the IWP. Using the
range of instruments summarised in Table One, and supplementary sources of
data generated by the IWP (for example the Acas Facilitators’ reports) helped to
secure more robust evidence for the analysis of organisational impact as well as
provide opportunities for the triangulation of certain results. Qualitative and
quantitative, baseline and post-project data drawn from questionnaires were
inputted into a simple database for analysis. Interview data was transcribed and
coded for scrutiny using Nvivo software for qualitative analysis.
Table One: Evaluation Sources and Schedule
Evaluation Research
Instrument
Source N Date
Baseline Pre-project
Questionnaire
One competed by each
participating organisation
10 September 2009
Initial Interviews in
Participant Organisations
Gatekeepers, Manager, HR
Manager, Employee
Representative at each
participating organisation.
38 October 2009
Initial Interviews with
Project Manager,
Delivery Partners
Acas, UKWON, Course
Tutor (New College) and
emda
6 November 2009
Telephone Interviews
with Gatekeepers at the
end of project delivery.
Gatekeepers at each
participating organisation
13 April/May 2010
Facilitator Telephone
Interviews
Acas Senior Advisers acting
as facilitators
5 June 2010
Final interviews with
Project Manager and
Delivery Partners
Acas and UKWON 4 September/
October 2010
Final Post-project
Questionnaire
One completed by each
participating organisation
8 November 2010
Final Interview Schedule
Gatekeepers, Manager, HR
Manager, Employee
Representative at each
participating organisation
where possible
29 October/
November 2010
Supplementary Data Sources:
? Gatekeeper Feedback Questionnaires on ILM course
? Gatekeeper Feedback Questionnaires on Participant Network and Action
Learning Meetings
? Evaluator Observation at Acas Facilitators’ Meetings
? Facilitators’ End of Project Reports
? Organisational Action Plans
26
Table One also provides a summary of the sources of data that were used and
when these were obtained. One major challenge for the evaluators was that
during the period of the project there were organisational changes and events
(for example redundancies, maternity leave or staff leaving) which impacted on
those involved in the project in each organisation. The differences in the number
of respondents in the initial interviews and those who took part in the final
interviews is partly due to the reduction from ten to eight organisations but is
also due to these changes in personnel. For example the 21 gatekeepers
interviewed at Stage One of the evaluation had reduced to 13 by the final
evaluation interviews. The research instruments can be requested by interested
parties from Jonathan Cooper or Gill Dix in the Acas Research and Evaluation
team.
2.3 Reflections on the Evaluation Process
In terms of reflections on the evaluation process, there were a number of
learning points for future evaluations of similar projects.
? Earlier knowledge of the participating organisations and their individual
action plans to develop the evaluation team’s understanding of each
organisation’s context and issues. This would have been assisted by the
evaluation team being present when the organisations presented their
individual actions plans at the launch event and would have provided an
early opportunity to gain ‘the buy in’ of the participating organisations and
individuals into the evaluation process (Guba and Lincoln 1989).
? A comprehensive briefing on the rationale for and the nature of the
evaluation strategy for all the nominated gatekeepers at the outset of the
IWP would have assisted the evaluation process and helped to address
any concerns of individual gatekeepers about its purpose.
? The research design was based on evaluating participants’ learning logs
and observing the action learning elements of the programme. This was
not possible with the result that it was more difficult to report on the
gatekeeper learning that took place and how this was applied during their
participation in the IWP.
27
SECTION THREE – EVALUATION FINDINGS
3.1 Changes and Benefits
At the end of their initial short course, the participating companies presented
their action plans at the IWP launch event in September 2009. Whilst these plans
informed the projects within each organisation and provided the basis for
evaluating the IWP’s impact, the reality was that the organisational changes
reported as a result of participation in the IWP were wider than those sought in
the initial action plans. These changes are examined throughout this section and
the key achievements reported by the respondents at each organisation in the
final interviews and the post-project questionnaires, are summarised in Table
Two below.
Table Two: Reported key organisational achievements as a result of IWP
participation
Organisation Initial Action Plan Reported Achievements
Brush Electrical
Machines Ltd
(BEM)
? Improve two way
communication
? Enhance managerial
awareness of the employee
perspective
? Improve employee
awareness of management’s
perspective
? Establishment of a steering
committee and focus groups
? The introduction of a company
newsletter to assist
communications
? Better equipped to meet the
challenges of an increasingly
difficult economic climate
Caterpillar
Logistics
? Introduce measures to
enhance employee
engagement
? Increase the employee
engagement score in the
company employee survey
by 10 percentage points
? Improve communication
between different groups of
staff
? Establishment of an employee
forum
? Improved communication
between staff groups
? Changes to the application of
the absence policy
? Employee engagement score
improved by 10 percentage
points
Liquid Control ? Develop workforce flexibility
? Identify skills gaps and
employ apprentices to fill the
gaps left by employees due
to retire
? Obtain ISO 9001 by the end
of 2010
? Undertake a Stress
Questionnaire amongst
employees
? Workforce skills analysis
? Introduction of developmental
appraisals for all employees
? Workforce training which has
increased flexibility
? Recruitment of apprentice(s)
? Implementation of an
employee engagement survey
? The introduction of quarterly
company meetings
? The introduction of weekly
departmental meetings
28
Northampton
College
? Initial action plan - to
enhance Leadership and
Management capability
? Later action plan – to
address issues of employee
consultation, communication
and involvement
Outcomes from the IWP are still
evolving. It is reported that they
are likely to be:
? Enhanced employee
involvement
? Development of leadership
skills among managers at all
levels
? The introduction of joint
problem solving task groups
Pendragon
? Improve team member
engagement
? Encourage better team
participation and
departmental interaction
? Improve customer Service
? The establishment of a
employee forum
? Introduction of team building
events
? Improved employee
engagement
? Improved team member
(employee) communication
throughout the division
? Review and revision of
‘housekeeping’ policies and
practices
? Introduction of monthly team
leader meetings
? Re-introduction of a customer
service survey
Strategic Health
Authority
(NHS East
Midlands)
? Engage staff to maximise the
use of the Electronic Staff
Records System (ESR)
? Transfer ownership of
personal data to individuals
? Enable managers to better
maintain employee data
? Reduce levels of data
handling to enhance
administrative efficiency
? Improved facility for ‘employee
voice’
? Increased staff usage of the
ESR
? Increasing staff ownership of
personal development
? More accurate HR information
? Improved reliability,
productivity and efficiency in
the handling of personal data
The Health Store ? Increase employee
engagement
? Improve two way
communication
? Establish an employee forum
? Elect employee
representatives
? Encourage employee
suggestions for innovation
? Elected and trained employee
representatives
? Establishment of a joint
management and employee
forum (production and
warehouse areas)
? Employee representatives
attendance at monthly
management meetings
? Improved workplace
communication and morale
and employee engagement
? Significant decrease in the
number of disciplinary cases
? Improved working practices as a
result of employee suggestions
29
Thorpe Kilworth
? Improve the company’s
competitive edge
? Improve manufacturing
efficiency
? Enhance employee
engagement and
communication
? Facilitate challenge to long
held beliefs and working
practices
? The establishment of a cross-
functional working party
? The establishment of a staff
consultative forum
? The introduction of employee
representative training
? Enhanced problem solving
capability
? Re-organisation of the stores
department
? Introduction of elements of
‘lean manufacturing’
? Development of an employee
engagement survey
Sources: Organisational Action Plans, initial interviews, final interviews and post-
project questionnaires
Achievements of Action Plans
Identifying what the organisations were seeking to achieve from their IWP
workplace projects was a focus of the initial interviews; the extent of progress
against their action plans was explored in the end of programme telephone
interviews with the gatekeepers and revisited in the final interviews. As a result,
the extent to which the workplace action plans were achieved is taken as the
starting point for identifying organisational changes and their benefits as a result
of organisational participation in the IWP. In addition to the summary of key
achievements provided in Table Two, further details of the issues, action plans
and outcomes are provided in the short case studies of the participating
organisation in Appendix A.
In practice, the initial action plans underwent subsequent development and
modification with input from the allocated Acas facilitators. The reasons for these
changes varied; in several instances the facilitation process revealed a need for
greater clarity about what the organisation was seeking to achieve or it became
evident that the scope of a project needed to be adjusted. At the final evaluation,
gatekeepers and other respondents from the participating companies were asked
to identify to what extent their organisations had achieved their action plans
using the following ranking scale: a) Completely (100%), b) To a large extent
(75%), c) Partially (50%) and d) To some extent (25%).
Across the organisations the majority of respondents (69 per cent) reported that
their action plans had been achieved ‘to a large extent’. Other than one
gatekeeper at Thorpe Kilworth, no one identified that their action plans had been
‘completely achieved.’ One explanation for this is that the respondents regarded
the interventions in their action plans as elements of a programme of continuous
improvement. For example, whilst identifying very positive outcomes, Pendragon,
The Health Store and Liquid Control all reported that they were seeking further
improvements than originally envisaged in their action plans. The perception that
the workplace projects were ongoing organisational initiatives to strengthen the
business was more in evidence at the final evaluation stage than in the telephone
interviews conducted with gatekeepers six months earlier. Using the same
ranking scale, three gatekeepers had then reported their action plans as
‘completely achieved’, four ‘to a large extent’ and six as ‘partially achieved’. The
30
shift towards a longer term view of the project as a programme of continuing
improvement had taken place across all the organisations by December 2010.
The final evaluation revealed differences between the responses of the
gatekeepers and other respondents who were more likely to report that an action
plan’s aims had been achieved ‘partially’ or ‘to some extent’. Responses at The
Health Store, Liquid Control and the SHA were both the most positive and
consistent across interviewees from the same organisation. One likely
explanation for the variation in responses between gatekeepers and other
respondents is the gatekeepers’ greater knowledge of the action plans and the
final outcomes; another is that their generally more positive view of the final
outcomes was shaped by their direct involvement in and engagement with the
project.
The Most I mportant Organisational Change
Respondents in each organisation were asked to single out what they saw as the
most important organisational change(s) in the final interviews. Table Three
provides a summary of their responses.
31
Table Three: The most important reported organisational change as a
result of the IWP
? BEM - improved communication as a result of the reintroduction of a
newsletter with increased managerial awareness of the employee perspective as
well as more employee awareness of business priorities (a view of improved
communication not shared by the gatekeeper who had left the programme).
? Caterpillar - following the establishment of an employee forum, improved
communications and closer working between the two distinct staff groups which
had resulted from a merger in 2007, evidenced by improved employee
engagement scores in the 2010 company employee survey.
? Liquid Control – greater workforce flexibility and utilisation of staff
resources as a result of improved communication and a skills analysis undertaken
by Acas which had led to better knowledge of individual job roles, closer working
and targeted training for skills development.
? Northampton College - due to delays in identifying the key issues and
facilitation continuing up to the final evaluation, outcomes arising from the IWP
are still evolving. It is envisaged that the most important changes to be reported
are likely to be enhanced employee involvement through improved
communications assisted by the College’s programme for developing leadership
skills among college managers at all levels.
? Pendragon - following the establishment of an employee forum,
improvements in communications at divisional level which are supported by
improved results in the most recent employee engagement survey.
? NHS East Midlands Strategic Health Authority - improved
communication arising from Acas facilitated focus groups which enabled the
implementation and integration of an Electronic Staff Records (ESR) System; this
had resulted in individuals beginning to take ownership of their own personal
data which in turn has begun to facilitate a more efficient handling of individual
data within directorates.
? The Health Store – the introduction of a joint management and staff
forum and elected employee representatives was the catalyst for improved
communications, levels of trust and employee engagement across the Production
and Warehouse operations, evidenced by growing examples of employee
proposals for improving working methods to increase productivity.
? Thorpe Kilworth - improved workplace communication due to
establishing a cross functional working party and a staff consultative forum with
employee representatives; one outcome of these improvements being a re-
organisation of the stores which had been of benefit across the business.
Source: Final interviews with all respondents
32
At Mahle Powertrain the most important change reported as a result of their
involvement was a recognition that to retain and develop the creativity and
innovation of its workforce, communication needed to more effective as a means
of improving employee voice, morale and recognition. This had resulted in the
establishment of a steering group with task groups to address particular issues.
Their project was ongoing internally but again was viewed as a continuous
improvement process.
Communications and Engagement
All eight of the participating organisations reported improved communications
arising from their participation in the IWP with six organisations identifying this
as leading to improved levels of employee engagement. Improved workplace
communications was the one change identified by all the participants in the final
survey data and in the final interviews. As Table Three illustrates it was also
reported as the most important organisational changes that had taken place as a
consequence of the IWP. But the nature of the reported achievements in this area
varied considerably across the participating organisations as a result of the
different issues to be addressed, their size, sector and individual histories. A
further influencing factor was the degree of business turbulence that was
experienced during the life time of the project. For example, the anticipated
improvements in employee engagement were adversely impacted upon in two
organisations as a result of workplace redundancies and the participation of
another was suspended for three months due to the impact of the recession on
its business.
In several instances, a particular change in the organisation prior to the IWP had
created problems in communication. For example, at Caterpillar a change in
ownership in 2007 and a TUPE transfer of staff had led to different working
cultures among its staff group; at The Health Store an expansion and relocation
to a larger purpose built premises in 2007 had led to the loss of previous close
working relationships, a situation described in the following terms by the
gatekeeper:
“We followed all the management practices that they tell you to follow and we
got nothing except a big shed with a lot of miserable people in it who were doing
just enough.”
One benefit of improved levels of communication and employee engagement
identified by a number of respondents in managerial roles was a reduction in the
time they spent on matters that could be resolved without their direct
involvement; these were now being addressed at a lower level as illustrated by
the following responses:
“There is definitely better communication in the business now. I just don’t get all
the tittle tattle and everything in my office any more. Things are being sorted
outside of management really. Production is a really good example. They put
plans together, they propose the plans and then actions are taken. That area has
come on significantly…” Senior Manager - BEM
“Since the project got into its stride there have been six developments that have
been gained from the employee engagement process which could well have been
driven by a manager, supervisor or through Health and Safety. But they certainly
33
wouldn’t have occurred at the pace they have without the employees latching on
to it and driving it themselves.” Operations Director - Health Store
For others, improved communication was seen as reducing divisions and
improving collaborative working between different parts of the organisation
although such improvements were widely acknowledged to be work in progress,
for example:
“I think the way we operate and communicate has probably got better. We work
closer together rather than being segmented. … I don’t think communication is
great but we do work better across departments now.” Manager - Liquid Control
“The key achievement would definitely be that we’ve managed to get some cross
functional or cross department dialogue going on which was a massive issue for
us in the first instance. We always struggle to get the likes of Finance and
Customer Service to actually talk to one another. We are not 100% there,
absolutely not, but I think they work slightly more closely together so that’s been
a massive achievement.” Gatekeeper - Pendragon
Capturing New I deas
Respondents were asked whether new mechanisms had been put in place to
encourage, exchange and capture new ideas and new ways of doings things as a
result of participating in the IWP and to identify what these were. Seven of the
eight organisations identified that they had introduced some form of employee
involvement arrangement, providing examples of where this had occurred as a
result of the focus groups facilitated by Acas, or through an employee forum or a
steering committee with task groups involving staff and management. In three
organisations it was pointed out that there were existing processes for capturing
employees’ ideas, for example a suggestion card scheme at Caterpillar, a
recognition scheme at Pendragon as well as a relaunched scheme called CANI
(Constant and Never-ending Improvement), and a number of processes for
improvement at the SHA which included a ‘Sounding Board’ and a forum called
‘Connect’ set up to capture any new ideas.
At The Health Store six examples were reported of instances where employees’
proposals for operational improvements had been considered and implemented.
Whilst some of these were broad in their scope, for example an employee’s
proposals was in progress for reorganising the warehouse, others could be
regarded as small improvements. These included the disposal of waste material
and revising purchase order documentation to make it more effective. However,
their cumulative impact was reported as making a real difference to productivity.
They also illustrated increasing levels of employee engagement with change
processes and the ownership of the operations they were involved in. One
employee representative elected as a result of the IWP explained how employees’
ideas were being progressed in the following terms:
“They come to me if there are any issues or there is anything they want to put
forward ... then we have a meeting so I can put their ideas forward and we will
sit and discuss which is the best route to go down.”
It was pointed out that new ideas were also increasingly being communicated
directly to managers as a result of the improved informal communications, a
34
dimension identified earlier and one which was seen as a key achievement of the
IWP.
At Liquid Control it was reported that appraisals had begun to take place again
(as a result of the IWP) which provided employees with the opportunity to put
forward ideas about their jobs and that input from the workforce was now pro-
actively sought in machine design and development, for example:
“We‘re encouraging the guys on the shop floor to come forward and have their
input into how machines are designed and built in the first place - they know how
things go together.”
There was less clarity about the adoption of new ideas at Northampton College.
Here respondents felt that it was too early to tell whether the consultative groups
or the open forums that were taking place on different themes (for example, the
new build) would lead to the adoption of ideas put forward by the workforce.
Where focus groups had taken place as a result of Acas facilitation, respondents
at five organisations reported very positively on their impact in terms of
encouraging individuals to put forward ideas and the extent of employee
contribution. There was a more mixed message from four respondents at three of
these organisations about maintaining longer term levels of interest and
engagement with the ideas put forward by employees. The concern was that,
whilst the IWP had created the ‘buzz’ reported by both gatekeepers and the Acas
facilitators, its impact would not be sustained and there would be a return to the
status quo which had prevailed prior to the project, described by one respondent
as “sticking an idea in a post box somewhere and nothing really ever happens.”
Communication of Outcomes
As part of the exploration of workplace communication in the evaluation, all
respondents were asked in the final interviews how the outcomes from their
organisational projects had been communicated to the workforce. Whilst most
organisations had indicated that they had introduced new or improved
mechanisms for employee communication, when it came to the communication of
the outcomes, the methods adopted prior to the IWP were generally utilised to
communicate the outcomes, for example notice boards, e-mail and organisational
intranets. The actual reported knowledge of project outcomes was patchy in
three of the organisations among respondents other than the gatekeepers.
HR Policies and Procedures
In the final survey respondents were asked to report changes that had taken
place or were planned, in HR policies or procedures as a result of participating in
the IWP. These are examined in more detail in the economic assessment impact
undertaken by Ecorys, and presented in Appendix D but the final questionnaire
results are provided in Table Four.
35
Table Four: Changes or planned changes to HR Policies and Procedures
HR Policies and
Procedures
Number of organisations reporting
Types of HR
policy/procedure
New Policy Improved
Policy
Planned
Policy
Review
Discipline 2 1 3
Grievance 1 1 3
Redundancy 0 1 1
Informing and Consulting
Employees
1 4 1
Other (e.g. equality and
absence management)
1 3 1
Source: Post - project questionnaires
Table Four reveals that at least one change in an HR policy or procedure was
reported as having taken place by respondents at seven of the eight participating
organisations.
The main reported changes were explored in more depth in the final evaluation
interviews. These confirmed that the changes involved mostly new or improved
processes for informing and consulting with employees or changes in approaches
to managing absence. Some of the participant organisations were part of multi-
site operations and were subject to policies and procedures determined at
corporate level which meant that they could not make changes independently at
site level, as this respondent from Pendragon pointed out:
“our company policies, they’re set from above. So none of those are going to
change because that would affect our handbook … and our handbooks are
standard.”
Issues stemming from standardised company policies and procedures were
fundamental to the problems which had led to Caterpillar Logistics applying to
participate in the IWP. As a result of its involvement in IWP, increased employee
involvement and communication had led to a reinterpretation of a previously
rigidly applied company sickness absence policy.
What was evident in the reported changes to HR processes was that where two
way communications had improved through a range of mechanisms, specific
problems and concerns could be identified and addressed as illustrated by the
observation of this gatekeeper at The Health Store:
“There were things that we’ve brought in that probably (we) hadn’t got round to
before because we didn’t know what was happening.”
Similarly better communication had led to beneficial changes to certain
procedures or practices at other organisations, for example, to the appraisal
systems at Northampton College, the SHA and Mahle Powertrain. Whilst the
interview data indicated that improvements in communication had allowed long
running, yet suppressed, problems to be aired and resolved this was not
36
manifest, in all cases by formal changes to policies and procedures. In some
instances it emerged in more informal ways.
Further evidence from the final evaluation interviews suggested that where a
climate of greater openness resulted from an individual project, it had
encouraged more organisational proactivity in addressing issues identified by
employees. It should be noted, however, that due to the challenging economic
climate and consequent wider changing organisational context, respondents
sometimes found difficulties in identifying where there were changes in HR
policies and procedures that could be directly attributable to the IWP as opposed
to this wider changing organisational context.
Workplace Climate
At the outset and at the end of the IWP respondents were asked to describe
workplace morale at their organisations on the following scale:
a) Very poor; b) Poor; c) Good d) Very good
By December 2010, six months after the project had finished, eighteen
respondents (69 per cent) reported that workplace morale was either ‘good’ or
‘very good’. Although the majority felt morale was good, only three respondents
saw it as ‘very good’. Perceptions of workplace morale had improved significantly
since the initial interviews in Autumn 2009 when only five respondents had
described workplace morale as ‘good’ and none described it as ‘very good’.
Respondents were then asked to identify whether or not workplace morale had
improved as a result of participation in the IWP. A number of respondents found
this a difficult question to answer either because there were wider external
factors impacting on the organisation (for example, uncertainty about the future
of strategic health authorities in the light of the Government’s review of NHS
structures) or the presence of internal issues (such as a pay freeze or recent
redundancies) which were felt to be having an adverse effect on morale.
Notwithstanding such factors, the majority of interviewees identified that the
improvement in workplace morale was as a result of participation in the IWP; 58
per cent reported it had led to an improvement and 19 per cent that it had not,
with the remaining 23 per cent feeling unable to answer the question due to
other issues impacting on morale.
The most consistently positive perceptions of the impact of the IWP on morale
came from the smaller businesses, reinforcing the earlier suggestion that the
impact of the IWP was more immediately evident in these organisations. With the
exception of a totally shared view among respondents at The Health Store and at
Thorpe Kilworth that there had been an improvement, gatekeepers were
generally the most positive in their view of the IWP’s impact on morale, possibly
again as a result of their closer knowledge of the outcomes and their personal
commitment to the IWP.
Reflecting upon the impact that participation in the IWP had on morale within
their workplaces, six respondents in managerial roles identified that morale would
have been considerably worse without the IWP, a view illustrated by these
managers’ observations:
37
“I think if we hadn’t been doing the talking, if we hadn’t been doing these things
then it would have been worse.” BEM
“Taking into account that we’re coming out of recession and the year preceding
that everyone was thinking they were going to be made redundant its hard to
quantify but it would all have been much more of a challenge without this
project.” Health Store
Asked to give examples to illustrate improvements in workplace morale,
respondents tended to describe a more positive employee relations climate rather
than providing specific instances to support their view. Gatekeepers found it
easier to point to an internal change or action as a result of the IWP which had
contributed to improved morale and all but three were able to do so, for
example:
“Probably the team events we have done, I think that boosts morale because
there is something at the end of it. We’ve done various out of work team member
events and social activities and done some internally. I think that has definitely
had a knock on effect.” Pendragon
“People are putting themselves out to sort of help the company you know and
make better work practices ... we get a lot of continuous improvements
suggestions as well, a lot of ideas. We’re meant to get three ideas per person
per year and we’re now getting about six or so.” Caterpillar
“The guys on the benches were working more as a team, helping one another
and taking the trouble to go to the stores once we had reorganised it as opposed
to ‘Oh I can’t be bothered to go over there because I’ll never find it anyway’ … it
was a benefit.” Thorpe Kilworth
Levels of Trust
At the final evaluation respondents were asked to describe the present level of
trust between managers and employees on the same scale used to describe
workplace morale:
a) Very poor; b) Poor; c) Good d) Very good
The majority of respondents (54 per cent) described it is as ‘good’ or ‘very good’.
Responses were the most consistent at Thorpe Kilworth, the SHA and The Health
Store where all interviewees reported it to be ‘good’ and at Liquid Control where
trust levels were perceived to be ‘very good’. At BEM it was described as poor
due the internal issues already referred to; at Pendragon, respondents all felt
that levels of trust varied from good to poor but were agreed that there had been
improvements in some areas.
Improved levels of trust between managers and employees were reported at all
eight organisations since the IWP began but this was not necessarily a view
shared by all respondents at the same organisation. The exception to this was in
the smaller businesses (Liquid Control, The Health Store and Thorpe Kilworth)
who all felt that the IWP had contributed to improved levels of trust. The pay
dispute at BEM had led to an observed divergence in respondents’ views and only
one respondent (who reported an improvement) at the SHA felt able to answer
this question. There were different views among respondents at Pendragon
38
depending on whether the question was concerned with trust between employees
and their direct management or interdepartmental trust; the perception was that
levels of trust were higher within than across departments although across the
workplace it had been described as ‘quite good’ when the IWP began.
At Caterpillar trust levels were described as ‘good and improved’ by three
respondents but a fourth respondent did not feel it had changed and was still
‘poor’. Respondents at Northampton College reported trust levels as ‘different in
different areas’ and that ‘it all depended who you talked to’ but nonetheless
described it as either ‘good’ or ‘quite good’ but, despite identifying that trust had
improved in the past year, did not feel it was possible to single out what might
account for this.
Management and Leadership Development
A core aim of the project was the development of management and leadership
skills. Respondents were asked whether this had been an outcome of the IWP in
the post-project questionnaire and in the final interviews where it was also
explored from the perspective of whether it was felt there had been changes in
approaches to management as a result of the project. Both dimensions are
examined below.
Developing Management and Leadership Skills
A key objective of the IWP was to facilitate long term organisational change by
focusing on the development of organisational management and leadership skills
through the gatekeepers in the first instance. The intention was that gatekeepers
would transfer their learning from the programme to stimulate a wider workplace
enhancement of these skills. Respondents were asked to what extent they felt
the IWP had met its objective of facilitating organisational change by enhancing
management and leadership skills (not just of those in managerial roles) and to
identify whether this had occurred to:
a) To a large extent; b) Partly; c) Not at all; d) Don’t know
At the final evaluation the majority of respondents (61 per cent) said this had
happened either ‘partly’ or to ‘a large extent’ with seven respondents reporting
that this had happened to ‘a large extent’, ten that it had partly occurred, five felt
that it had not happened, five did not know with one respondent unable to
answer. Table Five provides reported examples of how the project contributed to
the development of management and leadership skills with the participating
organisations.
39
Table Five: IWP’s contribution to the development of Management and
Leadership Skills
? It made the management team of the operations department at least look
at themselves and say well yeah, maybe I’m not as good as I think I am and
I’d lump myself in there as well.
? Because as gatekeepers we report to the operations manager, we’ve
shown him what we’ve learnt and it’s pinpointed that the same mistakes have
been made by management over again.
? People that are involved in it feel more empowered to do things off their
own back and to do things without me saying ‘can you do this because I want
XYZ.’ Now they come to me and say do you think? I say ‘go for it’.
? If nothing else the training courses that were offered have enhanced at
least some of the department leaders in dealing with difficult issues …from a
purely selfish point of view, dealing with some of the conflict that used to lead
to me getting involved, if nothing else that will have helped them.
Source: All respondents in the final interviews
Not unexpectedly the majority of the gatekeepers (78 per cent) reported that the
IWP had ‘to a large extent’ or ‘partly’ achieved its objective of developing
management and leadership skills, but it was encouraging that 43 percent of
other respondents reported that the IWP had ‘to a large extent’ or ‘partly’
achieved this aim. Only at The Health Store did all the respondents report that
this had been achieved to a ‘large extent’. This may have been more evident
because the gatekeeper was in a senior managerial role which, by definition,
required such skills. At Liquid Control the majority view was that this had been
achieved to a ‘large extent’ but here one respondent reported not knowing
whether or not these skills had been developed, in common with a respondent at
Caterpillar.
The same question had been explored with the gatekeepers during the telephone
interviews six months earlier; by the final evaluation the number of gatekeepers
who thought the IWP had met this objective ‘to a large extent’ had increased
from three to seven, an increase which may be due to further developmental
interventions taking place in the intervening period, predominantly through
attendance at Acas short courses by gatekeepers and other attendees from the
participating organisations.
The question proved to be particularly challenging for some gatekeepers who,
whilst reporting that they had personally developed as a result of the IWP, found
it more difficult to identify whether they had developed management and
leadership skills which had facilitated organisational change. The benefits they
reported from being participants on the IWP and the value they obtained from its
different elements are discussed later in the evaluation of the different elements
of the programme. All of them, however, had experienced the initial development
programme ‘Managing the Workplace of Tomorrow’ accredited by the Institute of
Leadership and Management (ILM) at NVQ level 3. As previously mentioned this
offered the option of completing the requirements for the qualification, 12 out of
the original 20 attendees chose to do so and were all successful.
40
3.2 Approach to Management
The final evaluation found that 16 of the 26 respondents in the participating
organisations thought the approach to management had changed as a result of
the IWP. Eight thought there had been no real changes and two felt unable to
answer. Compared to other areas of evaluation there was rather greater
consistency among respondents within the same organisation where it was
perceived that changes had taken place, for example at Liquid Control, The
Health Store and Northampton College. This was also the case where it was felt
changes had not occurred, for example at Pendragon where it was reported ‘that
nothing much had changed but there wasn’t really a problem to start with.’
The types of changes reported were dominated by better communication, more
delegation and a shift towards a more consultative management style. Whilst not
a complete list, Table Six provides the type of changes respondents commonly
identified had taken place in the approach to management as a result of IWP
participation.
41
Table Six: Six examples of changes in the approach to management
? The management style is less authoritarian with more explanation and
information about the reasons for asking for something to be done. There is less
of ‘Well I’m the manager, this (is) how we do things ... managers are more
approachable.
? There is greater delegation due to better knowledge of the skills set - the
work is better spread.
? Managers have seen the benefits of one to one dialogue and employee
involvement.
? There is a lot more information available to all, whereas before it (was) all
kept under the table. None of it was top secret they just didn’t bother to pass it
on.
? Attitudes have changed, it was always ‘them and us’ and now it’s not like
that, there is communication between us - it seems to have been recognised that
we all work for one company.
? There is definitely more of a view that it is best to consult first and get the
employees' feelings on certain things to enable change, to get their buy-in.
Source: All respondents in the final interviews
Respondents were asked at the final evaluation whether participation in the IWP
had led to changes in how managers dealt with absenteeism, disciplinary issues,
interpersonal conflict in the workplace and formal disputes between management
and employees. The extent to which new or amended procedures had been
introduced in these areas and whether these had any beneficial outcomes are
explored elsewhere in the report. The consideration here is whether respondents
saw improvements in how managers approached these issues. Eleven
respondents said there had been improvements, which included all the
interviewees at The Health Store who identified positive changes in these areas.
Nine respondents said they were unable to point to any specific changes and
none of the respondents at the SHA felt able to answer the question.
In a number of the organisations issues outside the IWP project had a bearing on
responses. For example, at BEM it was identified that as a result of the IWP, prior
to the pay dispute, there had been an improvement in the number of grievances.
Changes in the interpretation of the long term sickness procedure led to reported
changes in how management dealt with such cases at Caterpillar, and at The
Health Store, as already identified, there was less need for managerial
involvement in conflict resolution and more issues being resolved at an informal
level. At the same organisation it was reported that, from an employee
perspective, return to work interviews by managers were helping absenteeism to
be better managed. There was a less consistent view at Northampton College
where respondents were divided about whether or not there had been
improvements among the majority of managers and there was no evidence
provided to illustrate any changes.
3.3 Benefits to Individuals in their Jobs
Asked whether there had been IWP outcomes which had helped respondents in
their own jobs, the responses varied considerably both across and within the
participating organisations. Where individuals had been gatekeepers and gained
42
new skills, they identified that these would be of benefit to them in their present
or future work roles. Where respondents held a central organisational role, for
example as an HR Manager, they were more likely to report clear benefits such
as greater ease in obtaining the employee perspective and their involvement in
new initiatives at The Health Store or the ability to delegate more to staff at
Liquid Control. Others expressed a more general view that the impact of
improved staff engagement would help them in their roles or to deal with future
changes observed this HR Director:
“I’m hoping that staff will come to us and work with us rather than against us
through the difficult times we have ahead …. In that way it will help my job.”
A small number of respondents found it difficult to identify how the outcomes of
the IWP had helped them in their immediate jobs although a number thought the
experience could be useful in the future or saw benefits in continuing networking
arrangements with other participating organisations. There was more likely to be
a shared view among interviewees in the smaller businesses, for example at
Liquid Control and The Health Store, that the IWP’s outcomes had benefited them
in their work roles. Here the outcomes could be more directly visible to the
workforce as illustrated by this elected employee representative’s comment:
“It’s a lot easier to talk and obviously I’m involved now with the meetings -
there’s no secrets anymore so there’s a lot more understanding which makes my
job a lot easier.”
3.4 Unanticipated Outcomes
Respondents were asked in the final interviews about any unanticipated positive
or negative organisational outcomes as a result of participation in the IWP. In
the telephone interviews with gatekeepers six months earlier, nine had reported
unanticipated positive outcomes compared to four who identified none. Six had
said there had been unanticipated negative outcomes, whereas seven reported
none. In the final evaluation it was possible to probe more deeply into this area
and to include respondents other than the gatekeepers. Again reported
unanticipated positive outcomes outweighed any negative outcomes.
Gatekeepers, as a result of their direct involvement throughout their projects
were more likely to identify where things had not gone to plan or there had been
a negative reaction. One such example was where team members had returned
from a workplace forum meeting with enthusiasm to apply agreed new ground
rules for the working environment but had received a demotivating, adverse
reception from other employees about what was being proposed. This particular
incident was reported by the gatekeepers as a critical learning point which had
illustrated the importance of channelling forum outcomes through supervisory
team leaders.
In three organisations, a positive unanticipated outcome had been the speed with
which employee engagement had improved once a forum had been established or
individual employees had begun to take ownership and propose changes in work
methods. Other outcomes were described as unanticipated in that they had not
formed part of the organisation’s action plan but had arisen as a result of an
improved, more open environment. For example, at Caterpillar it was reported
that it had been agreed that the members of one team should set their own
43
targets; an approach that had worked well so was being applied across other
teams. Because of the nature of their organisational role, HR Managers were
often best placed to see any positive and negative outcomes beyond the action
plan. At The Health Store this was reported as employees being more proactive
about health and safety matters. In contrast, at Mahle Powertrain a negative
outcome from the HR perspective was that the scope of the initial workplace
project had not included its manufacturing operations at another site.
There were instances where an outcome was reported as having both positive
and negative aspects. For example, at one organisation steps to address labour
utilisation had resulted in two redundancies; an outcome that was detrimental to
the individuals involved but in line with the organisation’s objective of achieving
better labour utilisation. A lack of direction about their role and what was
expected of the project at the outset gave rise to problems for the gatekeepers at
the SHA, but the final interview revealed that their difficulties led to a ‘lack of
communication’ being identified as a critical issue in terms of achieving the action
plan. It was then possible to begin to address this, with the assistance of the
Acas facilitator, through a series of focus groups.
3.5 Constraints on Progress
At the beginning of their projects, respondents were asked to identify any
particular obstacles or constraints to achieving their action plans. This question
was revisited in the final evaluation interviews. Not unexpectedly, there was a
varied range of responses. The recession had impacted in different ways on all
the private sector organisations but none were untouched (even if the effect was
in another part of the business); for example, there was short time working and
redundancies at two organisations and cash flow problems at another. Other
factors, such as a pay dispute at BEM and changes in management at a couple of
the organisations, had an impact on the progress of organisational projects. A
defining and innovative feature of the IWP was that it provided organisational
support over an extended period of time through the network events, action
learning and workplace Acas facilitation. But this also meant it was vulnerable to
changes of personnel within the organisation and there were instances of an
individual project faltering where a senior manager and/or a gatekeeper, who had
acted as a champion for the IWP, left or moved to a different role.
In contrast, the two public sector organisations appeared to experience the most
difficulty in clarifying the aims and scope of their action plans at the outset,
partly due to the presence of other related, and potentially overlapping,
organisational initiatives being underway at the same time as the IWP, for
example, a leadership development programme. Until issues of integration and
alignment with these other initiatives were addressed, there was some difficulty
in clarifying the focus of the IWP workplace project, the appropriate processes
and what the gatekeepers should be doing.
3.6 Levels of Support
The initial interviews identified that most gatekeepers’ had a concern about the
level of support they would receive from senior management and a couple were
44
sceptical about support from a work force who were viewed as apathetic and
disengaged due to low morale as a result of redundancies, changes of ownership
or restructuring. In the final interviews the level of support was revisited;
respondents were asked to consider the extent to which they agreed or disagreed
that there had been a lot of support from a) senior management, b) line
management, c) employees and to provide examples.
The levels of support reported by gatekeepers in the final interviews are
illustrated in Figure One:
Figure One: Gatekeepers’ reported levels of organisational support
(number of gatekeepers giving each answer)
0
2
4
6
8
10
Strongly
disagree
Tend to
disagree
Neither agree
nor disagree
Tend to agree Strongly agree
Support from senior management Support from line management
Support from employees
Source: All respondents in the final interviews
Figure One reveals that, in practice, the majority of all respondents reported in
the final evaluation that there had been a lot of support from senior managers
although four respondents felt that this had not been the case. Where support
was forthcoming at the outset, facilitators reported that there was more rapid
progress against the action plan as evidenced at BEM, until tensions arose due to
a pay dispute. Four gatekeepers reported that senior management support had
increased as their projects had developed and positive benefits became evident,
as one gatekeeper put it:
“Senior management did not show any particular support but to be fair nor did
they stand in the way of what we were trying to do …. once it became clear that
the pilot was working they have been right behind it.”
There were less initial concerns about the level of support from line managers
and responses at the final evaluation confirmed that this had been forthcoming
with the exception of Pendragon (see below) and one respondent at the SHA who
disagreed on the grounds that they would have been more supportive if the IWP’s
benefits had been better explained at the outset. At Pendragon, all three
respondents identified that the problems of support among line managers
stemmed from their lack of involvement, as one gatekeeper explained:
45
“We were probably a couple of months down the line when we started trying to
involve them, a lot of them were actually turning round and saying ‘I have no
idea what you’re talking about’. So I think they probably would have given their
support if they’d have had the opportunity to….but you know, that’s a learning
experience for us.”
Low morale and a lack of engagement led a number of respondents to initially
identify that the workforce would be apathetic and even cynical about the IWP’s
aims but the final interviews revealed that these fears were unrealised;
employees were reported as supportive and forthcoming in focus groups and at
other forums which provided opportunities for their ‘voice’ to be heard. Whilst the
gatekeeper at the SHA was of this view, there was a perception among the other
respondents that the employee support would, as with line management, have
been greater if they had understood the value of what was being proposed from
the beginning. The overall finding was that concerns about levels of support
diminished as individual projects progressed and the majority of gatekeepers
reported being satisfied with levels of internal support at the final evaluation. The
issue of ensuring that appropriate managerial support is in place at the outset is
returned to in the discussion of the programme design and delivery.
3.7 Learning Transfer
The extent to which learning transfer had taken place throughout the IWP was
reviewed in relation to:
a) the transfer of learning from the gatekeepers to others within their
organisation,
b) the transfer of learning between the gatekeepers from the participant
organisations.
The overall finding of the evaluation process was that learning transfer across the
participant organisations from gatekeepers sharing experiences at their network
meetings and the initial three and half day course was more frequently reported
than learning transfer from the gatekeepers to others within their individual
organisations. Questions about learning transfer within the organisation were
asked of all respondents whereas the question about learning transfer from the
other participating organisations was explored through the gatekeepers’ reported
experiences from attending the participant network events and the action
learning sets. Learning from individuals in other organisations was, however,
reported as one benefit of the Acas short courses which had been attended by
delegates from the organisations beyond just the gatekeepers.
Transfer of Learning within the Organisation
Asked whether or not skills and knowledge had been transferred from the
gatekeepers to others in the organisation, 65 per cent of respondents said this
had taken place. Not unsurprisingly, the most positive responses were from
gatekeepers with 83 per cent reporting that this had taken place.
Respondents other than the gatekeepers were divided in their perceptions of
learning transfer; 50 per cent reported that they perceived this had taken place,
29 per cent indicating that this had not happened and 21 per cent were unsure.
46
This led to an exploration of how gatekeepers’ learning from the IWP was
communicated to others in their organisations. The majority reported that this
had been communicated through the newly introduced information and
consultation mechanisms, for example:
“[The Gatekeeper] talks to me and has a meeting with the senior staff, then he’ll
have a meeting with his forum … and that’s how he’s passing the information
through.” Health Store
Others mentioned that this occurred informally through ‘word of mouth’ or by
putting information on the ‘project server’ but three respondents felt that the
outcomes were only really known to the direct participants on the programme as
one observed:
“The closer people were to the gatekeepers involved the more likely they were to
know about the learning from the project.” Caterpillar
One of the interventions which facilitated the sharing of learning was the various
short courses provided by Acas and attended by both the gatekeepers and
others.
At one organisation, where the gatekeeper was the Employee Relations Manager
for other Divisions as well as the participating Division, the skills and knowledge
she acquired from participating in the IWP were transferred more widely in the
company as illustrated by the following:
“… in a couple of businesses, we’ve definitely gone in and used some of the
things that we learnt from the forum to try and get other organisations or other
businesses to do the same.” Pendragon
Transfer of Learning Across Participant Organisations
The opportunity for gatekeepers to share and learn from each other’s experiences
was an integral part of the design of the initial three and half day course and a
central feature of the participant network meetings organised and facilitated by
UKWON, reflecting the action learning philosophy of the IWP. The contribution
made directly by the action learning sets as an element of the IWP programme is
explored later in the report. This issue is discussed further in 3.12, Evaluation of
the IWP Methodology).
Learning transfer and the sharing of knowledge between gatekeepers was
explored at the final interview stage. Respondents were asked to identify how
useful it had been to share knowledge and experience with participants from
other organisations on the programme on the following scale:
1) Not at all useful; 2) Of some use; 3) Useful; 4) Extremely useful.
11 of the 13 gatekeepers reported that sharing knowledge with others had been
either ‘useful’ or ‘extremely useful’ and their perception was that this had
occurred to a greater extent than learning transfer within their organisations
although the two aspects of learning transfer are clearly interrelated. They
illustrated their responses with the following examples of the benefits that had
resulted from sharing experiences with participants from other organisations:
47
“I was ... able to learn how different organisations approached setting up of focus
groups; we were a little behind in this area.” Liquid Control
“I gained a realisation that I tended to ramble and there are better ways to
communicate.” Thorpe Kilworth
“Sharing information helped us to benchmark; we saw that we were not really
that bad at all.” Pendragon
“Sharing helps to prioritise actions, so you don’t sink under the volume and to
realise that change is difficult.” Mahle
3.8 The Personal Development of the Gatekeepers
The degree of personal development emerging from any management
development intervention is notoriously difficult to establish as is the impact of
such development on outcomes (Mumford and Gold 2004:157). Nevertheless,
gatekeepers were asked in the telephone interviews in May 2010 what the
personal benefits had been for them at the point in time when, for most of them,
the different elements of the IWP programme had just been completed.
All of the gatekeepers identified some personal benefits from participation in the
IWP; examples included ‘ more confidence in speaking and chairing meetings’,
‘increased participation in group and team work’, ‘working more closely with
senior leaders’ and ‘the ability to utilise tools and techniques’. One gatekeeper
was so encouraged and motivated by his introduction ‘to the world of learning’ on
the IWP programme that he had begun a higher level ILM/NVQ Level 7
qualification in management and leadership, as he explained:
“If it had not been for this project and the insights I gained, I just would not
have pursued further development of myself as a manager and I would not be on
this ILM Level 7.” Health Store
The main personal benefits of the IWP identified by 7 of the 13 gatekeepers in
telephone interviews are those reported in Table Seven; namely the opportunity
to ‘network’, ‘share issues, problems and achievements’ with others participants
on the programme. Learning that other organisations of a different complexity
and size could face similar issues and challenges was described as reassuring but
also as developmental in terms of sharing how these were being approached.
Several gatekeepers felt this had “helped their self confidence” illustrated by the
participant who observed “learning what others were doing helped me to
challenge what we were doing”.
The facilitators also reported the IWP had improved the self confidence of the
gatekeepers they had worked with in their reporting of its outcomes.
In the final interviews gatekeepers were asked to identify the ‘three best
outcomes for them as participants on the programme’ and these are summarised
in Table Seven.
48
Table Seven: Summary of Gatekeepers’ reported three best outcomes
from IWP participation
Organisation 1 2 3
Brush
Electrical
Machines
(BEM)
Sharing challenges
and difficulties
Help and advice
from Acas on how to
deal with problems
Introducing the
news letter
Caterpillar
Getting facility
leaders to do more
employee
consultations with
the employees
rather than making
assumptions
Having a different
viewpoint and
mindset, being able
to think outside the
box
Implementation of
the employee
forum
Health Store
Engaging with
further education
Improved levels of
trust between
gatekeeper and
senior managers
Seeing
improvements in
morale
Liquid Control
Gatekeeper One
Developed new skills
about having
difficult
conversations
The practical
assignments done
with Acas
Learning about
communication
from visit to
Thorntons plc
Liquid Control
Gatekeeper Two
Networking with
others
Learning from the
deliverers
particularly from
case studies
Learning tools and
techniques
Mahle
Building
relationships with
others
Working with new
and different people
in my organisation
and building
organisational
knowledge
Seeing some
things actually
implemented
Northampton
College
Gatekeeper One
Getting the
organisational
project started
Appreciating how
difficult it is to get
people engaged in
improvement
activity
Recognised the
value of
networking
Northampton
College
Gatekeeper Two
Getting HR on board Focus groups
helping better
alignment so
helping to move us
forward
Seeing the senior
management
become aware of
the need to get
involved
Pendragon
Gatekeeper One
Learning from
mistakes
Positive outcomes
from the team
building events
Improved
relationships with
senior leaders
49
Pendragon
Gatekeeper Two
Getting teams to
work together more
effectively
Getting people
working together
and understanding
things
Introduction of a
customer
satisfaction survey
Strategic
Health
Authority
(NHS East
Midlands)
Networking with
other companies
Learning new skills Realising the
importance of
communication
Thorpe
Gatekeeper One
Networking with
people
The realisation that
we were a pretty
good company
Learning points
picked up
throughout the
project
Thorpe
Gatekeeper Two
Interacting with
other people, other
organisations
Learning new skills Seeing the
company starting
to be more
efficient and begin
to save money
Source: Final interviews with gatekeepers
3.9 Outcome Measures
The estimated economic impact of the IWP on the participating organisations is
summarised in Section Four of this report; the full analysis of the economic
impact is provided in Ecorys’ report in Appendix D. This analysis is based on data
drawn from the pre-project questionnaire and post-project questionnaire
completed by each of the participating organisations. These were sent to the
individual identified by each organisation as best placed to gather and provide
the information requested by the survey and to report on any changes that had
taken place in the 12 months between September 2009 and September 2010 as
a result of IWP participation.
In addition to the questionnaire, all respondents in the initial interviews were
asked how outcomes from the programme would be measured to see whether
participation in the programme had contributed to the performance of the
organisation. The question of measurable outcomes was revisited in the final
interviews where all the respondents were asked whether or not there were any
measurable outcomes that showed that participation in the programme had led to
improvements in organisational performance (in three instances this referred to
the workplace involved in the IWP). Where there were measurable outcomes,
respondents in each organisation were asked to provide examples. In practice the
responses to the post-project questionnaires and the final interviews revealed
that respondents had difficulty in disentangling the impact of the IWP from other
influences on organisational performance.
Seven organisations provided before and after data relating to Employment
Tribunal applications, six did so for employee grievances and five for disciplinary
sanctions in the post project questionnaires. There was a reported rise in
employee grievances and a reduction in disciplinary sanctions but no change in
Employment Tribunal claims. But organisational experiences were very different.
For example, one smaller business (with just over one hundred employees)
50
reported a reduction of twenty disciplinary sanctions over the 12 month period, a
fall they attributed directly to improved communication and employee
involvement from participation in the IWP. At another much larger organisation
there had been increased numbers of grievances and disciplinary sanctions
during the same period which may have been related to internal differences over
other workplace issues unrelated to the IWP. Whilst the data revealed no change
in the number of Employment Tribunal claims, given the potentially protracted
nature of such procedures, the origins of these where likely to have predated the
IWP and any conclusion on the programme impact of these would require a
longer time frame.
Of the five organisations who were able to respond to the questions on levels of
employee turnover and days lost through absence in the post project
questionnaire, two reported a decline in staff turnover and a further two reported
a reduction in days lost through absence since participating in the IWP. In terms
of the earlier reported changes to HR policies and procedures that had taken
place as a result of involvement in the IWP, there was no evidence provided to
show that these had impacted on organisational performance. But there could
have been an indirect contribution which was not easily identifiable. One such
example was the reduced absence levels reported at one workplace which was
possibly due to the adoption of a less rigid workplace interpretation of a company
absence procedure which had been an outcome of the employee forum
established as a result of the IWP.
Whilst the survey data presented a mixed picture, there was the perception
among respondents in managerial roles, mentioned previously, that problems
that had taken up their time prior to the IWP were being resolved at a lower level
and without their involvement saving managerial time.
The changes in numbers employed indicated a variation between public and
private sector organisations with increased voluntary departure at both the public
sector organisations although there was a large increase in employees leaving
due to redundancies at one whilst a slight decrease at the other. In contrast,
there was a decrease in all categories of departure in the private sector
organisations although, again, causality can not be disentangled from broader
economic influences.
As well as disentangling the impact of broader factors, such as the economic
downturn, analysis of this data is also impacted by the large variation in size of
participating organisation, the largest, having in excess of 1,000 employees
whilst the smallest, employed only twenty, and did not supply employment
change data.
When respondents were asked in the final interviews what they considered to be
the best measure of improvements resulting from their action plans, their
responses provided a range of both objective and subjective measures. For
example, a senior manager at BEM reported that the company was now doing
better financially although it was difficult to quantify how much this was due to
the IWP. At Thorpe Kilworth, one gatekeeper could point to employees’ proposals
resulting in a saving of between four and six thousand pounds. At Caterpillar
Logistics, the 10 per cent improvement in employee engagement specified in the
organisation’s action plan had been achieved in 2010 and further improvements
51
were anticipated in the next employee survey. Other measures were more
subjective as illustrated by this manager’s observation:
“I think the staff seem happier with their jobs they’re doing so there is a better
atmosphere. So I would say that is a sort of measure.”
In terms of adopting measures to demonstrate the impact of the IWP the
evaluation process offered a number of insights. Firstly, the initial interview
questions about final measurable outcomes revealed few of the respondents had
identified how they would measure the organisational benefits from participating
in the IWP. Three said this could be through an employee survey; two further
organisations had identified this as the measurement as part of their action plan
and for one there was a target figure for an improved score for employee
engagement which was achieved.
Secondly, for the majority, it was reported that success would be demonstrated
by achieving what they set out to do in the workplace action plan or evidenced by
improved communication, workforce morale, increased employee involvement
and the development of management and leadership skills. But it was not
identified how the outcomes in these difficult to measure areas would be
assessed. Notwithstanding, it was reported that improvements had been made to
a very large extent in all these areas by the end of the project. What was evident
is that the ‘yardstick’ adopted by respondents for measuring successful outcomes
from the IWP placed more emphasis on qualitative changes in the workplace than
the quantitative measures commonly used to gauge workplace improvements.
This may be partly due to diverse nature of the gatekeepers’ roles; if the
gatekeepers had all been senior managers the criteria for success might have
been very different. The final observation as previously reported is that
respondents in over half the participating organisations said that things would
have been worse without their IWP participation
3.10 Probability of Change without the IWP
Respondents were asked to identify the extent to which they thought the
organisational changes reported would have occurred without participation in the
IWP using a five point ranking scale of:
a) Completely (100%)
b) To a large extent (75%)
c) Partially (50%)
d) To some extent (25%)
e) Not at all (0%).
None of the respondents thought the changes would have been completely
achieved without the IWP although four respondents (all in managerial roles)
thought that these would have happened ‘to a large extent’. There was an even
split between the 20 respondents who thought the changes would have ‘partially
happened’ or ‘to some extent’, and only at The Health Store was it reported that
the changes would not have happened by all the interviewees. The responses
52
highlighted a difference of perception between managers, gatekeepers and
employee representatives; the latter two groups of respondents being more likely
to identify that the changes would have not occurred without the IWP. At Mahle
Powertrain who had left the project, it was reported that the IWP had been the
stimulus for changes which would otherwise not have occurred.
The availability of similar support to that provided by the IWP and the degree to
which the participating organisations would have taken this up was explored in
the final survey; the responses are analysed in the Economic Assessment of the
IWP in Appendix D. Despite respondents reporting that these changes would have
occurred to varying degrees, the survey data reveals it as unlikely that
participants would have taken up an alternative source of support. This raises the
question as to whether or not the organisations could have achieved the reported
outcomes without external support, particularly as the gatekeepers frequently
described the IWP as the ‘catalyst for change’ within their organisations - a view
that was shared by both delivery partners and the Acas facilitators in their
reflections on what the IWP had achieved.
3.11 Sustainability
As already identified, it was evident from the respondents in the final evaluation
interviews that they were increasingly viewing the workplace achievements
resulting from their involvement in the IWP as part of ‘on going’ programmes of
change. However, a third of the organisations were concerned that, without the
IWP, the momentum would not be sustained. This was particularly reported
where there had been a loss of individuals who had championed the project due
to staff turnover.
3.12 Evaluating the IWP Methodology
As part of the evaluation process, the gatekeepers were asked to evaluate the
contribution of each element of the programme provided by the IWP. As outlined
in Section One, these consisted of:
? an initial three and a half day course
? six participant network events
? eight action learning sets (six of which took place during the participant
network days)
? the Acas facilitation process
? additional inputs from Acas or UKWON (largely short Acas courses).
During the telephone interviews the gatekeepers were asked to rank the
usefulness of these different elements of the programme. A summary of their
responses at that point in time is provided in Table Eight below:
53
Table Eight: Gatekeepers’ perceptions of the usefulness of the different
elements of the programme
Questions relating to the Individual
elements of the programme
Extremely Useful
to a
large
extent
Partly
useful
Not
at all
useful
How useful was the initial delivered
element of the programme (the three
and half days leadership and
management course)?
2
6
4
1
How useful was the initial launch event?
4
8
0
0
How useful were the Participant
Network meetings?
5
4
4
0
How useful were the ‘Action Learning
Sets’?
3
4
4
1
How useful was the ACAS facilitation
element of the programme?
3
5
4
0
Source: Telephone Interviews with gatekeepers
In the final interviews there was a more ‘in depth’ exploration of the perceived
value of the different elements of the programme with the 13 gatekeepers who
were asked:
a) The contribution they felt each element had made to the final outcomes for
the organisation and for them personally;
b) To what extent they felt the key achievements of the IWP would have
occurred without this element of the programme;
c) What changes, if any, they would propose for any element of the programme.
The responses of the gatekeepers on the extent to which they felt the key
achievements of their workplace projects would have occurred without this
element of the programme are summarised in Table Nine.
54
Table Nine: The extent to which the key achievements would have
occurred without each element of the programme
Element of Programme
Not at all To some
Extent
To a
large
extent
Completely
Initial three and a half day
leadership and management
course
4
2
4
3
Participant network
meetings
3
4
5
1
The action learning sets
4
2
4
3
Acas facilitation
6
6
-
1
Source: Final interviews with gatekeepers
It was interesting to identify the extent of any changes in the gatekeepers’
perceptions of the value of the different elements six months after the telephone
interviews when the outcomes of their individual project had become clearer.
Their responses regarding each element in both the telephone and the final
evaluation interviews are considered below.
The I nitial Short Development Course
The initial three and half day course ‘Managing for the Workplace of Tomorrow’
was delivered by New College Nottingham from June to September 2009. It
offered an ILM qualification for those who wished to pursue this. 95 per cent of
participants were satisfied with the general content and delivery of the
programme and were particularly positive where ‘tools and techniques’ were
provided that could be easily transferred back to the business.
By the time of the telephone interviews 61 per cent indicated that they felt this
element of the programme had been ‘extremely useful’ or ‘useful to a large
extent’ with a further 31 per cent indicating that it had been ‘partly useful’. Only
one respondent expressed a view that it had ‘not been useful at all’. Positive
views related to the ‘bite size’ nature of different subjects which one gatekeeper
indicated ‘helped to get things started’. The content also seemed to provide a
‘focus’ for some gatekeepers ‘steering them to the right thinking’ sentiments
summed up by the following comment:
“… you need the focus and need the way forward. You need a bit of help
sometimes in achieving your goals, and in some form or another, I wouldn’t say
all of it but some parts of it we have actually used.” Thorpe Kilworth
More negative responses related to the content’s ‘level’ and its ‘relevance’,
illustrated by the following feedback:
55
“Nothing new in terms of content. I do it as part of the day job.” Pendragon
“I found some of it quite frustrating really because it was below the level I’d
already achieved…I think there was no attempt made to try and work out the
levels that different people were at.” Northampton College
However, some of the respondents for whom the content was revisiting existing
knowledge could see benefits, for example;
“We weren’t taught anything that we didn’t already know but it did highlight
those things and bring them to the forefront to make us think about it…reignited
the memory.” Caterpillar
As Table Nine reveals in the final interviews with the gatekeepers, 4 reported that
the achievements of their organisational projects would not have occurred at all
without the initial short programme (all from smaller businesses), 2 that these
would have to some extent, 4 to a large extent, 3 that they would have been
achieved without this element and 1 felt unable to answer the question. In
personal development terms the participants variously identified the ‘shared
experiences’, ‘being able to put things into context’, ‘helped to focus energies’,
whilst others were of the view that the programme’s workshops ‘would have been
of more help to others with less experience’. Responses from the two public
sector organisations were consistently the least positive about the initial
programme, possibly due to the complexity and size of their organisations or a
greater degree of developmental opportunities traditionally being made available
to employees in this sector.
The Proj ect Launch Event
One element of the programme singled out in the telephone interviews which was
not revisited in the final interviews was the IWP project’s launch event. Whilst not
a specific element of the project design, Table Eight illustrates that this was the
most highly rated aspect of the programme, rated as either ‘extremely useful’ or
‘useful to a large extent’ by all those gatekeepers who attended (one did not
attend). It is worth examining why respondents rated it so highly. It took place
on 23 September 2009 and the detail of its content is described in Section One.
An invitation to attend this launch event was extended to participant
gatekeepers, their senior managers and employee representatives from
participant organisations. The event was identified by the gatekeepers as
providing the whole project with ‘credibility’ particularly in the eyes of key
individuals in their organisations whose support was needed. These perceptions
are illustrated by the following quotes:
“Excellent, the people, the real case studies it was so informative…it was
inspiring.” Thorpe Kilworth
“It was particularly helpful that we were able to bring a wider audience from our
company and we managed to bring someone from HR and a senior union rep. So
it helped to explain the project …the speakers were brilliant.” BEM
“I think it was a really good opportunity to gain insights from other businesses
and companies.” Caterpillar
56
“We were able to take people from the business which gave credence to the
project and we were able to hear from other companies, disappointed in the
limited places.” SHA
The Participant Network Meetings
Six whole day network meetings facilitated by UKWON were held from October
2009 to March 2010. The design for these days, provided by UKWON, was that
the morning session of each day was devoted to a thematic discussion of
common problems and the afternoon was spent with the participants in action
learning sets in which each organisation could present progress, achievements,
obstacles and dilemmas for peer review and discussion. These were followed by
two half day meetings of the action learning sets, again facilitated by UKWON, in
May and September.
The location of these meetings varied in that different participant workplaces
were chosen to host these network events. The schedule was as outlined below:
October 2009 Overview of employee engagement and workplace
innovation
November 2009 Managing successful change (Donington Park)
December 2009 Case studies of workplace innovation
January 2010 Employee engagement event with Nita Clarke (BEM)
February 2010 Progress review on action plans, building trust and
breaking down silos.’
March 2010 Sustainability of change; reviewing the role of Gatekeepers
- (SHA)
Source: Programme of Network Meetings
In the telephone interviews, 9 respondents indicated that they felt the meetings
had been ‘extremely useful’ or ‘useful to a large extent’. The remaining 4
respondents rated the meetings as ‘partly useful’ (Table 8).
The most beneficial aspect articulated by respondents during both the telephone
and final interviews (confirmed by the supplementary information on the IWP
provided by some of the gatekeepers to UKWON), was once again, the
opportunity to share issues, problems and experiences with other gatekeepers.
This aspect was mentioned by 8 of the respondents
Table Nine reveals that by the final interviews, 3 gatekeepers reported that the
achievements of their organisational projects would not have occurred ‘at all’
without the network events, 4 that these would have to ‘some extent’, 5 ‘to a
large extent’, and 1 that they would have been achieved ‘without’ the network
events. In response to the question about the contribution of the Participant
Network meetings to the final outcomes either to the organisation or to the
participants personal ‘sharing of experiences’ was the single most reported
benefit.
Whilst the respondents predominantly highlighted the sharing opportunities
presented by the network meetings, learning from the thematic content of the
network meetings was evident when a formal review took place at a half day
57
learning set meeting held on 20
th
May 2010 (observed by members of the
evaluation team). Participants during this session articulated the learning they
had acquired which had relevance for them as individuals and for their
organisations.
Action Learning Sets
The responses in the telephone interviews set out in Table Eight reveal that the
majority of gatekeepers viewed the action learning sets as either ‘extremely
useful’ or ‘useful to a large extent’.
By the final interviews of the 13 gatekeepers, 4 reported that the achievements
of their organisational projects would not have occurred ‘at all’ without the action
learning events, 2 that these would have to ‘some extent’, 4 to a ‘large extent’
and 3 that they would have been achieved ‘without’ this element. Responses that
mirrored those for the initial action learning short programme which suggests
that this is a type of learning works well for some but not for others.
The responses from gatekeepers were generally positive in their view of the
action leaning sets as a useful mechanism for sharing information with one
another. However, it was apparent in the telephone and the final interviews that
a number of respondents were not altogether clear about what constituted the
action learning set element of the programme. To specifically find out about this
as a core element of the programme design, there were questions relating to the
action learning sets in the evaluation interviews but the gatekeepers’ difficulties
in singling these out from other events at the participant network meetings are
illustrated by the following observations:
“Same as the morning sessions giving and receiving advise.” BEM
“Sometimes good sometimes not so good, been a bit of a mixture.” Liquid
Control
Varying levels of attendance may have led to logistical difficulties in having two
learning sets on each occasion. This may have led to difficulties for certain
respondents in distinguishing between the morning event and the afternoon
action learning. The preference of some gatekeepers was for smaller action
learning sets as illustrated by the following comment:
“smaller groups certainly benefit the likes of me. I don’t like speaking up in larger
groups. Small groups let us get to know one another better and we bounced
ideas off one another…” Liquid Control
Other gatekeepers felt the action learning sets were useful but would have been
of even more value if there had been organisations in the same industry/sector
as themselves, for example:
“they were very useful for me. I have picked up a hell of a lot from people who
talked, but would have liked other similar organisations to have been involved.”
SHA
58
Acas Facilitation
At the time of the telephone interviews, when the facilitation had ended with the
exception of Northampton College, 62 per cent of respondents perceived the Acas
facilitation to be either ‘extremely useful’ or ‘useful to a large extent’. 3 indicated
the facilitation had been ‘extremely useful’, 5 that it had been ‘useful to a large
extent’ and 4 that it had been ‘partly useful’. Acas facilitation was reported as
taking a variety of forms which included assisting with the introduction of joint
forums, the facilitation of focus groups, producing reports, meetings with
members of the organisation and gatekeepers, holding joint problem solving
groups, a skills audit, organising bespoke training events, as well as email and
telephone contact.
A number of gatekeepers reported that they had not used their facilitator early
enough or made as much use of this element of the programme as they
subsequently realised they should have done, for example:
“It was our fault that we possibly didn’t use the facilitator as much as we could
have done. April (the end of the programme) arrived too quickly. We needed to
involve the facilitator earlier in the process.” Liquid Control
“We didn’t have long enough. Support for a full year would have been
welcomed.” Pendragon
Part of the explanation for such comments might lay in the timing of the initial
involvement of the facilitators with their respective participant organisations, all
the facilitators reported in their evaluation of the IWP that their earlier
involvement with the IWP would have been helpful to both parties, particularly in
establishing terms of reference for the organisational projects and ensuring that
there was senior management support. (Facilitator interviews)
A couple of respondents also identified that ‘personal chemistry’ between
facilitator and recipient gatekeeper did not always result in an immediate
productive relationship but could take time to develop as evidenced by the
following observations:
“It was partly useful. It took a long time to get going. They probably found us
frustrating to work with. I didn’t feel they were particularly pro-active.”
Northampton College
The issue of relationships taking time to develop is particularly illustrated by the
evaluation provided on this element at Northampton College where the
facilitation continued for much longer than elsewhere due to a slow start. The
gatekeepers’ perception of the facilitation process changed radically as the
process progressed as illustrated by the following feedback at the final
evaluation:
“It has made the biggest contribution for us. It was useful for the college to have
independent people who were experienced facilitators, with a proper reputation
and the kudos of Acas.” Northampton College
Six months following the telephone interviews, the final evaluation revealed that
respondents had become increasingly positive about the Acas facilitation with 12
59
(92 per cent) identifying that they would have not achieved their project’s
outcomes ‘at all’ or only ‘to some extent’ without the facilitation input with the
exception of one gatekeeper whose reported experience differed from the other
gatekeeper in the same organisation. The following observations typify the
positive feedback on this element of the programme:
“It was good. It couldn’t have happened without the Acas facilitation.”
Pendragon
“I would say our adviser had a very important role. Although they were back
stage they were driving us how to approach it in a very subtle way. Basically they
equipped us.” SHA.
“…feel overall that that was one of the key strengths out of all the different
elements…” Thorpe Kilworth
One explanation for the increasingly positive reporting of the impact of the Acas
facilitation was that its contribution not only took time to become apparent but
also that certain facilitation activity took time to generate results. Another factor
may be that open Acas courses had been attended which could have had a
positive influence on gatekeepers’ perceptions; these are now considered.
Acas Training
Whilst not an element of the initial design for the IWP project, Acas provided a
number of training courses for participants in areas of identified need as follows:
Acas ‘Open’ training events for the IWP
Date Title of event No of delegates
9
th
March 2010
Meetings, consultation and
Forums – some useful basics
6
10
th
March 2010
Having difficult conversations
6
12
th
March 2010
Having difficult conversations
6
13
th
July 2010
Facilitation Skills Training
11
30
th
September,
1
st
, 7
th
October 2010
Developing Mediation Skills
7
According to the attendance records a total of 23 delegates, including 10
gatekeepers, from across the participating organisations took advantage of
attending one or more of these events; one organisation had employees
attending all four courses.
60
At the telephone interviews Table Eight reveals that out of 8 gatekeepers
responses 5 respondents rated these courses as ‘extremely useful and 3 rated
them as ‘useful to a large extent’.
During the final interviews gatekeepers were asked, ‘did you or others in your
organisation receive inputs in addition to those already mentioned e.g. any
additional training organised through Acas or UKWON?’
Three respondents indicated that their organisations had receive some bespoke
training interventions from Acas: The Health Store and Pendragon had training
for their employee representatives; Caterpillar for the members of its new
employee forum; and Pendragon had training for its team leaders. A further two
organisations had received bespoke development interventions from UKWON. At
Thorpe Kilworth input on team working was provided whilst at Northampton
College UKWON provided assistance to help clarify the way forward with the
project. UKWON also arranged a visit to the chocolate manufacturer, Thortons
plc, as a developmental opportunity for the gatekeepers. Those who attended
this visit reported very positively on the event.
3.13 Messages for the Future
The messages or ‘lessons learnt’ from the evaluation of this pilot initiative for
similar projects in the future are drawn from the following sources - the
gatekeepers’ suggestions; the reflections of the delivery partners and the
reflections of the Acas facilitators.
Gatekeeper Suggestions for Future Programmes
During the final interviews the gatekeepers were asked what changes, if any,
they would propose to the different elements for any future programme. This
question did not include any additional inputs that had been provided as the IWP
project progressed.
Whilst some respondents were satisfied with the programme content, others
made proposals for change. As might be expected their responses reflected their
own particular learning style, level of knowledge, size of their organisation and
role. Gatekeepers’ comments received are reflected in Table Ten.
61
Table Ten: Gatekeepers’ proposals for future programmes
? Delivery interspersed with action including pre-work
? Smaller groups
? Better understanding of the organisational projects before the action learning
delivery input so enabling the input to better support the projects so adding
maximum value
? Greater tailoring to particular organisational contexts
? Short course input throughout the duration of the project instead of all at the
beginning
? Clearer structure to the network days and action learning sets
? Longer and earlier support from facilitators
? Participant organisations from the same sector
? One venue for the network days and action learning sets
? More specialist inputs at events e.g. speakers on relevant topics
Reflections of the Delivery Partners
The Acas Project Manager and representatives from UKWON (‘the delivery
partners’) were interviewed face to face at the beginning of the IWP about its
aims and objectives. They were interviewed again, post project, to gain their
reflections on its outcomes and to identify any changes or areas to be addressed
for similar programmes in the future. In terms of the IWP’s overall impact, both
delivery partners felt that a major achievement of the project had been that it
had raised awareness about the value of employee engagement and moved the
participant organisations towards greater employee involvement; a view that was
shared by the Acas facilitators. Their perception, supported by the evaluation
evidence, was that as participants gained in their understanding of the
importance of employee engagement for the business, the more they recognised
that this took time and effort to achieve and sustain. As already reported,
participants increasingly viewed the outcomes from their individual action plans
as part of an ongoing programme of improvements.
Asked what they would wish to review or change in future programmes, in no
order of priority, the key points identified by the delivery partners were:
? Allow more time to recruit; this had been constrained in the IWP due to
budgetary considerations. In addition, UKWON suggested that a self
assessment questionnaire could be used in the recruitment process. It was
felt that this would help applicants to clarify their objectives and provide
the opportunity for self reflection in terms of what they were seeking from
participation.
? Provide more detailed structured information about the programme once
the final gatekeepers had been selected. There was a lack of knowledge
about the IWP and what it involved among some of the finally appointed
gatekeepers due to last minute organisational changes and insufficient
internal briefing from those who had taken part in the selection process.
? Establish the level of commitment from senior managers at the outset. It
was reported that, whilst this might have been articulated at the selection
62
stage, there were instances where it was not evident when the workplace
project got underway - a situation that was exacerbated when there were
changes in senior management during the project. Senior management
support was also identified as a critical success factor by the facilitators
and is discussed further below.
? Consider introducing mechanisms to maintain contact and discuss
progress with senior management during the project as well as afterwards
to sustain momentum, for example periodic meetings.
? Ensure that there was sufficient Acas facilitation expertise available to
support participant organisations and consider ways in which this could be
developed more widely without impacting on the quality of provision.
Reflections of the Acas’ Facilitators
Telephone interviews were held with the Senior Acas Advisers involved in the IWP
in order to capture their reflections on the initiative and their own experiences as
facilitators. These took place when the facilitation process had ended with the
exception of Northampton College where it was still ongoing. The information
gained from these interviews was supplemented by end of project reports
completed by the individual facilitators which were made available to the
evaluators. In addition, throughout the IWP, member(s) of the evaluation team
attended the facilitator network meetings which took place from September 2009
to September 2010. These different sources generated a considerable amount of
data about the facilitation element of the programme. They revealed a very high
level of consistency among the facilitation team in terms of what they felt needed
to be reviewed for similar projects in the future. All the facilitators identified that
the key achievement of IWP programme had been that it had acted as a catalyst
for more attention to be paid to issues of employee involvement. In addition half
of them reported that the organisational role of the gatekeeper had a bearing on
their ability to progress workplace action plans
Undertaking the facilitator’s role was described both as ‘personally
developmental’ and ‘very worthwhile’ by facilitators. They reported that they had
welcomed the opportunity to work collaboratively, and in depth, with
organisations to improve workplace relationships in order to enhance
organisational performance. Having a protracted period of time to support
workplace projects was seen as a real opportunity to make a difference, and the
approach was seen as a chance to fully support the principles embedded in the
Acas Model Workplace (Acas, 2005). It was reported that it would have assisted
the facilitators in their roles if the different elements of the IWP had been more
clearly integrated and if they had been more aware of what was involved in other
areas of the programme. A key learning outcome reported by nearly all the
facilitators was the importance of ‘getting to grips’ with the culture of the
organisations and the pace at which things could be progressed. As one very
experienced facilitator observed:
“ I learnt that I had to go backwards to go forwards which paid dividends in
terms of gaining trust in the longer term … but it all took time.”
In terms of their experiences of the IWP, the following issues were identified as
areas for attention in any future initiative. (A number of these issues were also
63
touched upon by the delivery partners and, most particularly, by the Acas Project
Manager).
? Put in place agreed ‘Terms of Reference’ signed off with senior
management (with the involvement of the allocated facilitator) for each
workplace project before it began. The rationale for this proposed action
being that it would reduce the reported difficulties facilitators experienced
in clarifying and understanding the focus of the organisational project.
This action would address the issue of executive level support and also
their common experience that, where there had been a lack of clarity
about the aims of a project, it had seriously impacted on progress. This
view is supported by the evaluation evidence which identified that many
projects made slow progress in the initial months and that ‘getting things
started’ was a key role for the facilitators at the beginning of the
programme.
? Involve the facilitators as early as possible in any future project so that
they could develop their relationship with the organisations they were to
work with and their understanding of its issues and culture.
? Consider how facilitation experience and skills could best be developed,
particularly in terms of being able to be flexible, innovative and resilient
when things did not go to plan or organisations are less responsive than
anticipated. It was acknowledged that the level of experience of
undertaking the role varied across the team and sharing learning and
specific experiences were considered a vital part of developing facilitation
skills.
? Provide inputs from another experienced facilitator, including their
presence at meetings in the workplace, where there were particularly
difficulties or a lack of progress. In practice, as the IWP progressed a
couple of organisations had two facilitators working with them which
overcame some of the issues which could face a facilitator when operating
alone.
3.14 Paying for Future Programmes
The IWP project was free to participants, whilst the mainstream in-company
advisory support provided by Acas is charged. In order to gain some indication of
organisations’ willingness to pay for such initiatives in the future, and to
contribute to future planning, the survey explored what the participating
companies would be willing to pay for a similar project to the IWP and those
factors which would prevent them paying for such a project in the future. Asked
what would be considered a low price, but a level that would not give rise to
concerns about quality, the lowest price was £5,000 but the survey respondents
in the three smaller companies were unable to answer the survey question. When
asked what price would be considered too expensive to consider participation, the
responses varied from £10,000 to a maximum of £20,000 but, again, the smaller
businesses did not answer the question. Only four of the larger organisations
identified factors that would prevent them paying for a similar project in the
future; these were reported as the economic climate, their ability to pay and that
participation would depend on evidence that the programme would increase
64
turnover. The smaller companies, who consistently reported the most positive
outcomes from participation, had identified that they would not have been able to
pay for the support provided by the IWP in the final evaluation interviews; a
position that was reinforced by their responses in the final survey.
65
SECTION FOUR – ECONOMIC IMPACT
4.1 Summary of the Economic Impact of the IWP
This section summarises an analysis of the economic impacts of the emda
funded, Acas led 'Innovative Workplaces' project, developed in line with the
principles set out in emda's evaluation toolkit. The analysis is primarily based on
a baseline and follow-up questionnaire undertaken with participants of the
projects. Full details including all calculations which feed into the estimates in
this section are in Appendix D.
As outlined earlier, eleven organisations participated in the Innovative
Workplaces project, although three withdrew over the course of delivery,
resulting in a total of eight completing the project. The analysis is restricted to
eight completers, though it is acknowledged that further impacts were achieved
amongst the three who did not complete the project.
4.2 Availability and Take-up of Alternative Sources of Support
A key consideration in assessing the impact of the IWP is the extent to which
participants would have taken up a similar alternative source of support in the
absence of the project. If participants would have obtained similar support
elsewhere, then it is likely that any such outcomes would have been achieved
anyway. Overall, the evidence suggested participants would not have accessed
similar services elsewhere, with an estimated probability that organisations would
not have taken up alternative support of 84 percent.
4.3 Improvements made by Participating Organisations
Respondents were asked to report whether they implemented new or improved
human resource policies or procedures (or planned to make such improvements
in the future) during or since participating in the Innovative Workplaces project.
Seven of the eight organisations surveyed had implemented at least one new or
improved process and all had at least plans to implement new or improved
processes. Four reported that they had plans to implement improvements in the
future.
To identify how far these improvements were made as a result of the Innovative
Workplaces projects, respondents to the postal questionnaires were asked to
report the extent to which they would have made these improvements without
the support they received from Acas. On the basis of survey responses, it was
estimated that there was a 32 percent probability that participating organisations
would not have implemented improved HR procedures without Acas support.
4.4 Gross Additional Impacts of Improvements
Respondents were asked to report how far the improvements made had resulted
in improvements in productivity, or helped them create or safeguard jobs. They
were also asked to report if those improvements would have such effects in the
66
future. Estimates of the gross additional economic impacts of the Innovative
Workplaces project are set out in the table below (i.e. those changes in
productivity, employment and Gross Value Added (GVA)
1
that would not have
occurred without the project).
Table Eleven: Gross additional economic impacts
Impact a)
Additionality
of the
project
support
b)
Additionality
of actions
c) Economic
impacts
improvements
Gross
additional
economic
impacts
Employment impacts
Jobs safeguarded 0.84 0.32 2.0 0.5
Potential jobs created 0.84 0.32 13.5 3.7
GVA impacts
GVA safeguarded (£
per annum) 0.84 0.32 72,000 19,592
GVA created through
productivity gains (£
per annum) 0.84 0.32 566,665 154,195
Total GVA created to
date - - 638,665 173,787
Total potential GVA
created
(£ per annum) 0.84 0.32 486,000 132,245
Source: Ecorys analysis (gross additional economic impacts are estimated as the
economic impacts attributable to improvements x additionality of support x
additionality of actions: a*b*c).
The above table summarises data from three aspects of improved organisational
performance resulting from the project:
1. Employment growth. Organisations were asked to report whether the changes
they had made had resulted in them recruiting any additional workers (if
employment had increased) or protected any jobs (if employment had remained
the same or decreased). As noted in previous sections, the majority of
participating organisations had seen a contraction in their workforce, with one
organisation reporting that the changes they had made had helped safeguard two
jobs as a result of the changes made. Applying average GVA per worker in the
region (£36,000), this equates to GVA safeguarded of £72,000.
1
GVA is measure of the total output of the organisation, the value added by the
organisation to the goods and services it absorbs in the production process. In practical
terms, this is measured by subtracting the value an organisation's purchases of goods and
services (ranging from raw materials through to expenditures on property services) from
the value of what an organisation produces (which can be approximated by the value of
sales).
67
2. Future impacts on employment. The follow-up postal questionnaire was
undertaken six months following organisation's initial participation in the IWP.
Many of the process improvements delivered by organisations will take time to
implement (and as highlighted, some participants were still at the planning stage
with respect to some areas of improvement). Additionally, the effects of
improvements may take time to arise, so it is likely that a focus on the economic
impacts of the IWP to date understate the total effects of the project.
In order to capture potential future employment impacts of the Innovative
Workplaces project, participants were asked to report whether they would recruit
any additional workers over the next twelve months as a result of the
improvements they made to their business. Two of the eight organisations
reported that they planned to recruit a total of 13.5 workers in the next 12
months as a result of implementing process improvements, equating to a
potential per annum GVA impact of £486,000 (again applying GVA per worker in
the East Midlands of £36,000).
3. Increased productivity. Productivity is a measure of the output (GVA) an
organisation can produce for a given level of inputs, and is typically measured by
GVA per worker. GVA per worker might rise (among other reasons) if workers
become more efficient or skilled, if organisations replace workers with capital
equipment or machinery, or if organisations adopt more efficient production
processes. Two organisations reported a (measured) growth in turnover per
worker of £9,417 and £30,500, and applying the ratio of GVA to turnover in the
East Midlands (34 percent) this equates to productivity growth of £3,201 and
£10,374 per worker respectively. Aggregating this across each organisations
number of workers gives an overall estimate of gross additional GVA through
productivity gains of £567,000 (having applied the estimated average probability
that productivity gains would not have been achieved without the changes made,
but before applying the estimated project and support additionally probability).
4.5 Leakage, Displacement, Substitution and Multiplier Effects
In order to move from gross additional to net additional economic impacts
requires consideration of leakage, displacement, substitution effects and
multiplier effects:
? Leakage: this looks at how far the intervention has resulted in impacts
leaking outside of the target area. All organisations participating in the
project were based in the East Midlands, so there is no leakage of GVA
impacts. 85 percent of participants' employees lived in the East Midlands,
implying a leakage of employment impacts of 15 percent.
? Displacement: this is where improvements in the performance of assisted
organisations come at the expense of those not receiving support, and for
this project displacement was assumed to be in the region of 20 percent,
in line with wider evaluation evidence of enterprise support initiatives in
the East Midlands.
? Multiplier effects: where organisations are able to improve their
performance through greater sales or productivity, they will consume
more goods and services provided by other organisations based in the
68
East Midlands, generating wider economic impacts (supply chain multiplier
effects). Equally, where individuals fill any jobs created as a result of the
project or are able to increase their earnings as a result of becoming more
productive, there will be further multiplier effects as they spend their
additional income in regional businesses (induced multiplier effects).
Multiplier effects were estimated at 1.39 on the basis of the industrial profile of
organisations receiving support from the Innovative Workplaces project.
4.6 Present Value of GVA Impacts
Table Twelve: Net additional economic impacts per annum
Impact Gross
additional
economic
impacts
Leakage Displace
ment
Multiplier
effects
Net
additional
economic
impacts
Employment impacts
Jobs safeguarded 0.5 0.15 0.20 1.39 0.5
Potential jobs created 3.7 0.15 0.20 1.39 3.5
GVA impacts
GVA safeguarded (£ per
annum) 19,592 0.00 0.20 1.39 21,786
GVA created through
productivity gains (£ per
annum)
154,195 0.00 0.20 1.39 171,465
Total GVA created to
date
173,787 - - - 193,251
Total potential GVA
created (£ per annum)
132,245 0.00 0.20 1.39 147,057
Source: Ecorys analysis (Net additional economic impacts at the regional level are estimated
as: Gross impact x Additionality x (1 – Leakage) x (1 – Displacement) x Multiplier Effects).
Estimates of the net additional economic impacts (per annum) of the IWP are set
out in Table Twelve above. Total net additional GVA created or safeguarded (per
annum) is estimated at £193,000, with a further potential £147,000 per annum
created by November 2011.
The GVA impacts of enterprise support initiatives will grow and endure for a
period of time, though in general, such effects are expected to decay at a certain
point. The effects of enterprise support should be assumed to endure for three
years, in line with guidance issued by BIS. The table below shows the projected
time profile of GVA impacts (together with the present value of those impacts,
applying the 3.5 percent value for social time preference recommended by the
HM Treasury Green Book)
1
.
1
The value of GVA is discounted to reflect a social preference for income today over an
equivalent income in the future. The 3.5 percent discount rate suggests that society as a
69
Table Thirteen: Present value of GVA benefits
Year Present value of GVA
impacts created to
date (£)
Present value of
potential GVA impacts
(£)
Present value of
total GVA impacts
(£)
2009/10 193,251 - 193,251
2010/11 186,716 142,084 328,799
2011/12 180,402 137,279 317,681
2012/13 - 132,637 132,637
Total 560,368 411,999 972,368
Source: Ecorys analysis
4.7 Return on Investment
Acas figures indicate that a total of £244,035 was spent delivering the IWP, and
the project was estimated to have created economic impacts with a present value
of £972,400. This gives an overall return on investment to public sector (in
regional economic impacts) of £4 for every £1 of public sector expenditure. emda
contributed £227,437 of total project costs or 93 percent of total public sector
expenditure. In line with OffPAT guidance, it is assumed that 93 percent of the
overall impact of the project (£906,232) is attributable to emda's expenditure.
A total of £108,152 of costs represented project management and evaluation
costs, which due to the pilot nature of the project are likely to reduce if replicated
in the future. The table below provides ROI figures under scenarios in which
these costs fall by 25 percent and 50 percent.
Table Fourteen: Return on Investment
Costs & ROI Cost (£) Economic
Impacts
(£s)
Return on
Investment
Emda 227,437 906,232 4.0
Total public sector 244,035 972,368 4.0
Potential ROI for future replication of the project, if evaluation & project management
staff costs reduced by:
25% 216,997 972,368 4.5
50% 189,959 972,368 5.1
whole is indifferent between £100 of income (or costs) today and £103.50 of income/costs
a year later.
70
Costs and benefits that have not been quantified
While this analysis suggests that the IWP has generated a positive return on
investment over the short period over which impacts have been allowed to
accrue, it takes a narrow economic view of the benefits of the project. There are
a range of wider effects that would be considered in a full cost-benefit analysis of
the intervention.
? Economic impacts: There are a number of economic impacts of the project
that have not been quantified in this analysis, including productivity
safeguarded and any future employee disputes and Employment Tribunal
claims avoided in the future.
? Social impacts: There may also be a range of social benefits of
improvements in HR processes over and above those described in the
economic impact assessment.
? Project costs for the participating organisations: Estimates of return on
investment outlined above do not factor in the costs participants incur in
implementing improvements, such as management time.
? Disbenefits: Where the Innovative Workplaces has facilitated growth or
raised productivity, there will be a range of social costs that are not
captured in the analysis such as wider externalities associated with
economic growth including the environmental impacts associated with
greater utilisation of resources or congestion costs caused by more
intensive use of transport infrastructure.
With regard to economic impacts not quantified, it should be noted that the
present value of GVA impacts and return on investment figures that have been
calculated only include economic impacts that could be specifically identified and
measured in the course of the project evaluation using this framework. Further,
it was only possible to attribute economic impacts to four of the eight
organisations which took part in the final evaluation of the project (by using
questionnaire answers given by the organisations). It is assumed that all other
organisations had no economic impacts as a result of participation, when the true
economic impact of these organisations is unknown. An economic impact of zero
is, of course, not realistic – however, the report can only quote figures which
have been reported and a cautious approach is taken and so estimated GVA
impacts can therefore be considered a minimum return from this project.
71
SECTION FIVE – CONCLUSION
5.1 Why the Messages from the IWP Evaluation are of Value
The funding available for the IWP meant it could only be offered to a small
number of organisations in the East Midlands. However, the project offers
valuable insights and messages for similar future initiatives. A number of
dimensions make the IWP especially interesting for wider application.
a) The project adopted an innovative approach by offering different forms of
workplace support and development, including customised workplace
facilitation, to the participating organisations.
b) Although the IWP programme lasted for a relatively short time, it was longer
than many developmental or business interventions and was focussed at the
workplace level.
c) The evaluation methodology was designed to ensure that a record was made
of all changes that took place within the participating organisations over the
life time of the project and six months after its core elements had ceased;
and that lessons learned were captured from the perspective of multiple
stakeholders.
5.2 The Project’s Impact on the Participant Organisations
The evaluation identified that the IWP’s major impact on the participating
organisations had been improvements to workplace communication and
employee engagement; identified as the key ingredient for riding ‘the economic
storm’ in the results of the Sunday Times 100 best companies in 2010 (Sunday
Times, 2010). There are recognised difficulties in defining what is meant by
employee engagement; indeed MacLeod and Clarke in their 2009 review
identified a plethora of different definitions which included a practitioner definition
of engagement as ‘when the business values the employee and the employee
values the business.’ (2009:7). This interpretation of employee engagement is
probably the closest to the IWP participants’ reported understanding of
engagement during the evaluation. The key components of this definition,
evidenced by the workplace changes during the IWP, were regarded as
communication with and the involvement of the workforce, providing mechanisms
for employee voice and adopting a joint problem solving approach.
These changes led to business benefits. The economic impact assessment of the
IWP reported an overall minimum return on investment of £4 for every £1 of
public sector expenditure and a measurable positive economic impact of
approximately £906,000 attributable to the IWP’s expenditure.
It was estimated that 50 percent of the productivity gains reported in the final
survey data by the participating organisations would not have occurred without
the improvements resulting from the IWP. The perception of respondents was
that the project had improved productivity even though other factors, not least
the severe recession for much of the project, meant that the reality for a number
of the participating companies was a drop in turnover. Notwithstanding, the
72
message from the participant organisations was that the situation would have
been considerably worse without their participation in the project; a perception
that reinforced their commitment to continue to participate in the IWP if at all
possible. The overall evidence from the IWP’s outcomes is that it improved
employee engagement can enhance performance which supports the conclusion
of MacLeod and Clarke’s (2009) review of employee engagement.
Multiple benefits were found to be associated with enhanced employee
involvement. For example, managers reported that they found they could
delegate more, and a frequently identified benefit was a reduction in the time
they spent handling individual disputes. The economic impact estimated that the
participating firms saw a reduction in the time spent on grievances, disciplinary
sanctions and Employment Tribunal claims, issues which, as Gibbons (2007)
observes, have significant costs for GB businesses. The introduction of improved
or new mechanisms for employee involvement and consultation outlined in
Section Three and the case studies stemmed from the Acas facilitation process.
The evaluation revealed that, whilst these provided forums for improved formal
communication with employees, they also led to improved informal
communications. This was particularly identified in the smaller businesses but it
was also reported where ‘siloed’ working or divisions between groups of staff had
been the focus of an organisation’s action plan. The evaluation findings do,
however, lend support to the findings of Hall et al. (2009) that there can be
particular business benefits of informality in approaches to sharing information
and employee consultation in medium sized firms.
Tangible outputs from the project were evidenced by seven of the eight
organisations reporting the implementation of at least one new or improved
human resource procedure or process improvement. These were most frequently
in processes for informing and consulting with employees and absence
management and all had plans for future improvements following participation in
the IWP. Other positive outcomes associated with improved employee relations
climate (reported by the majority of organisations) included improvements in the
management approach and increase in workplace morale and trust between
employees and management since the IWP began. In two organisations which
had undertaken recent employee surveys there were increased scores for
employee engagement compared to surveys prior to the project. Respondents
found it difficult to single out the extent to which reported improvements in
workplace morale and levels of trust were due to the IWP although it was widely
described by participants as having been the catalyst for the reported
improvements. Where such improvements were not identified it was explained
that there were other events negatively impacting on morale, for example, as
already mentioned workplace redundancies or uncertainty about the
organisation’s future.
Forums for employee involvement had encouraged employees to contribute ideas
and their views on a range of issues. These mechanisms emerged as a key factor
in stimulating increased engagement in the IWP, supporting the findings of Truss
et al. (2006: 39) that allowing people to feed their views and opinions upwards is
the single most important driver of engagement. This had led to greater
proactivity among some of the organisations’ employees in terms of raising
issues and contributing ideas even in areas that had not necessarily been
anticipated in their initial action plans.
73
Although the evaluation revealed an increased commitment to an ‘on going’
programme of improvements as the project developed, there were concerns
about sustaining the momentum without the project and fears that there could be
a return to the situation prior the project. This was especially the case where
individuals who had acted as champions for the internal projects had moved on
or left the organisation.
The most positive outcomes from the IWP were consistently reported by the
SMEs with less than 200 employees and there was also more uniformity of view
across the respondents from these organisations of what had been achieved and
the value of the IWP programme. This may be due to the greater visibility of
improvements in smaller businesses and the ease with which these can be
communicated. Another explanation may lie in less previous take up of other
sources of external support because of the costs involved.
5.3 Developing Management and Leadership Skills
One of the keys aims of the project was to improve business performance
through the development of essential management and leadership skills primarily
through ‘cascading’ the gatekeepers’ learning to others in their organisations. The
final evaluation revealed that the majority of respondents felt this had happened
partly or to a large extent both through the gatekeepers. The additional input of
Acas short courses provided to others in the organisation also contributed to this.
The workplace benefits resulting from the development of these skills were again
reported most positively by the smaller businesses. This could be partly due to
the roles of the gatekeepers being more central within these organisations so
they were better placed to both share and apply their learning or because of less
previous development of these skills. It was the case that some of the larger
organisations felt that the programme did not offer them much that was new in
this area but this is not to suggest that they were of the view that these skills did
not require further development.
Whilst part of the criteria suggested to the participant organisations for selecting
their gatekeepers was that they should be proactive individuals who ‘would get
things done’, the evaluation’s findings revealed that in order to drive workplace
changes forward, the level of responsibility and job status of the gatekeeper were
important factors; this is an issue for consideration in any similar future projects.
For all the gatekeepers the project provided an opportunity for considerable
personal development as reported in Section Three. In some instances individuals
had benefited significantly but it did depend on the background, prior knowledge
and (as mentioned) the job status of the gatekeepers. Whilst some felt the
learning they had gained from participating could not be immediately applied in
their current roles, all anticipated that this would be of benefit in the future.
There were also reported wider benefits for some of the gatekeepers such as
improved self confidence, enhanced organisational knowledge, working
collaboratively outside their own area and the networking relationships they had
developed with participants from other organisations.
74
5.4 The Value of the Different Elements of the IWP
Perceptions of the value of each element of the programme varied for each
organisation and the gatekeepers’ perceptions depended on what they were
seeking from the project. The need to accommodate this diversity was recognised
by UKWON and had informed the design of the elements they provided. Their
stated initial aim was to ‘ground pre-existing knowledge in the task at hand’ for
gatekeepers with prior knowledge and experience and ‘to provide sufficient
actionable knowledge on employee involvement and participation’ for those with
less knowledge and experience. The evaluation evidence suggests that this was
largely achieved but it was essentially the combination of the different elements
of the overall programme that had led to the final outcomes from participation in
the IWP. In practice, each organisation took what it needed at different times
from the project.
Although the impact of the project stemmed from the ‘sum of its parts’, by the
time of the final evaluation it emerged that the customised Acas facilitation
process was regarded as particularly useful in terms of its contribution to the final
outcomes. Perceptions of the value of the facilitation element of the programme
did increase with time, possibly when it became more apparent what it had
provided. This may explain why some participants wished they had made more
use of their facilitator before the facilitation process ended and they would have
done so if they had realised earlier what it could offer. The majority of
gatekeepers reported that they would have liked continued support from an Acas
facilitator but only those from larger organisation felt that there would be a
willingness to pay for this if there was evidence that it clearly improved
performance.
There were areas of this provision where the Acas facilitators themselves
identified the need for changes in any future programmes. These can be summed
up as: ensuring their earliest possible involvement; having initial terms of
reference agreed with senior management in place before workplace projects
began to provide the focus and clarity essential to progress; facilitators having
access to additional support when difficulties arose; and strategies to deepen and
widen facilitation capability among Acas advisers. The evaluation findings support
the view that the facilitators need to be involved at the earliest stage, and that
senior management support in workplace projects is critical. The need for clarity
about the aims of workplace projects also emerged as important as did the
significance of the facilitation element being well integrated with the other
elements of the programme. The evaluation also identified that the IWP’s
facilitation element provided a unique opportunity to support the embedding of
the Acas Model Workplace’s principles into an organisation and provides further
evidence that Acas activities can bring longer-term improvements to the
employment relations climate (Meadows, 2007:3).
The participant network events were highly valued for the network opportunities
they provided to the gatekeepers and the action learning element of these days
provided the means of sharing progress and problems, testing out thinking with
others and learning about the approaches of other organisations. These were
identified as key benefits of the IWP by a number of gatekeepers. Whilst the
IWP’s aim was to have as diverse a range of organisations as possible, this did
lead to some gatekeepers reporting that it was difficult to transfer their learning
75
about the practices of other participants on the IWP because their organisations
were so dissimilar in size, their product or service.
Whilst gatekeepers reported that there was always some value for them in
attending network events, a number wanted more structure and clarity about the
action learning sets. It was also suggested that some input from organisations in
the same sector or industry would have been a benefit, for example, as a mentor
or coach to participants from organisations in the same industry. It is also worth
noting that the delivered inputs from leading speakers and practitioners were
particularly highly rated by participants as illustrated by the feedback on the
launch event; more such inputs could be considered for future programmes.
One question explored by the evaluation was whether the same support as
provided by the IWP could have been obtained from another source. It was the
respondents in managerial roles who reported that this might be the case but this
was not the opinion of other respondents who shared the view that this would not
have happened and that the changes would not have occurred without the
project. The evaluation findings offers little in the way of insights into what fees
could be charged for future similar projects other than a clear finding that only
the larger organisations felt this would be a possibility but that this would depend
on the economic climate, their ability to pay and clear evidence that such a
project would increase turnover. Despite consistently identifying the most
benefits from their participation in the IWP, the smaller organisations reported
that it was very unlikely that they would be able to pay for such a programme.
Notwithstanding, all the participants identified that if they were to consider such
a project in the future, it would be important to them that it was provided by
Acas on the grounds of its neutrality, reputation and that it offered a ‘well
respected quality Kite mark’.
The overall conclusion that can be drawn from the evaluation evidence is that the
IWP led to improvements in all the participating organisations particularly in
communication and employee engagement which in turn, had positive
performance outcomes. To varying degrees, the IWP met its objective of
developing leadership and management skills among its participants and it
proved to be a vehicle for the identification of skills gaps in a number of
individual organisations. It was successful in introducing improved arrangements
for employee involvement and employee voice resulting in many of the benefits
associated with good employee engagement. Whilst the project was small in
nature involving just ten organisations, it resulted in a positive return on
investment; for every £1 spent on the project IPW generated £4 of direct benefit
to the regional economy. But the key message that can be drawn from the IWP’s
evaluation is that similar future projects, customised to meet the needs of
individual organisations, would offer valuable support to those organisations
where employers are seeking to engage their work forces and work
collaboratively with them to improve efficiency and changes to their working
practices.
76
References
The Acas Model Workplace (2005). London: Acas
Anderson, V. (2007) The value of learning: A new model of value and evaluation,
CIPD Change Agenda series. London: Chartered Institute of Personnel and
Development
Ashton, D. and Sung, J. (2002) Workplace Learning for High Performance
Working. Geneva: ILO
Boxall., P. and Purcell, J. (2008) Strategy and Human Resource Management.
2nd edition, Palgrave Macmillan
Gibbons, M. (2007) Better Dispute Resolution: A review of employment dispute
resolution in Great Britain. Department of Trade and Industry Report, London:
DTI
Guba, G. and Lincoln, Y.S. (1999) Fourth Generation Evaluation. Edition 11, Sage
Publications
Hall, M., Hutchinson, S., Purcell, J., Terrry, M. and Parker, J. (2009)
Implementing information and consultation: developments in medium-sized
organisations. Employment Relations Research Series No.106, London: BIS
MacLeod, D. and Clarke, N. (2009) ‘Enhancing performance through employee
engagement’. A report to Government, London: BIS
Meadows, P. (2007) A Review of the Economic Impact of Employment Relations
Services delivered by ACAS. London: National Institute of Economic and Social
Research
Mumford, A. and Gold, J. (2004) Management Development: Strategies for
Action. 4
th
ed, London: CIPD
Purcell, J., Kinnie, N., Hutchinson, S., Rayton., B and Swart, J., (2003)
Understanding the people and performance Link: Unlocking the black box.
London: CIPD
Sisson, K. (2005) Improving work organisation - the case for a regional action
programme. Industrial Relations Research Unit, Warwick Business School.
www.emda.org.uk/research
Sunday Times 100 Best Companies to work for 2010. Sunday Times:
http://business.timesonline.co.uk
Truss, K., Soane, E., Edwards, C., Wisdom, K., Croll, A. and Burnette, J. (2006)
Working Life: Employee Attitudes and Engagement 2006. Research Report,
London: CIPD.
77
Appendix A: Case Studies of the IWP Participating Organisations
1. Brush Electrical Machines (BEM) Ltd
The Organisation
Brush Electrical Machines Ltd (BEM) is a manufacturer of generators for steam
and gas turbines based in Loughborough, Leicestershire with 800 employees at
the start of the project. Other separate Brush plants also occupy the site. Along
with its current sister company in the Czech Republic, BEM is the world's largest
independent manufacturer of turbine generators. Prior to the project, a takeover
had led to a new management which had instituted 'lean production' system
resulting in a series of redundancies. There were further redundancies during the
project due to BEM’s merger with a Brush Transformers plant at the site which
had previously been under separate ownership. Rationalisation occurred across
the two companies resulting in a new round of redundancies and, by the end of
the project, BEM had a workforce of 700.
The company’s nominated gatekeepers were a Business Analyst and a Senior
Unite Trade Union representative. Participation in the program was first
suggested by a full time Unite Official and was the only case where a gatekeeper
was also a workplace union representative.
The I ssues
Following the change of ownership, the company’s aim in participating in the
project was to improve morale, trust and employee engagement. It was hoped
that its participation would assist BEM to become a more innovative work
organisation and reduce resistance to change. In particular, the takeover, change
in management and other changes had resulted in low employee morale which
was reflected in a disappointing response rate in an employee survey in 2009.
The union felt that poor communication, in addition to the issues of change and
ownership, had contributed to the lack of trust and poor morale. At the time of
joining the project, the company’s particular concern was that, whilst the
workforce was highly skilled, there was a need to develop a less functional, more
process driven perspective among the workforce to take the organisation
forward.
Action Plan
The initial action plan aimed to improve two way communication across the site
through establishing focus groups to allow the workforce to "feel valued, heard
and motivated." The involvement in the project was initiated by employees at
BEM feeling that they, together with middle managers, were not being listened to
following changes in senior management arising from a series of takeovers.
The Outcomes
Initially the program was very successful with the establishment of a steering
committee and eight focus groups, each of ten employees, to improve lateral
communication across the site. Whilst employees reported feeling undervalued, a
lack of communication and poor handling of redundancies, there was also
continuing pride in the Company’s history and its products. The final evaluation
revealed that the production of a newsletter had improved communication across
the plant. The employer further identified that the company’s increase in
78
productivity and its ability to ride out the recent economic downturn could, in
part, be attributed to its involvement in the project. But the final evaluation also
revealed resurgence of division between management and employee
representatives and the absence of any gatekeepers to take the project forward.
One of the gatekeepers, the senior shop steward, withdrew from the programme
following the breakdown of pay negotiations and concerns about his role as a
gatekeeper whilst the other, a business analyst, left the organisation for
employment elsewhere towards the end of the project. Notwithstanding, both
parties viewed the project itself very positively and, in particular, the contribution
made by Acas. A recent round of redundancies has been an impediment to
improving levels of workplace trust but senior management’s view is that the
situation would have been far worse without involvement in the project.
79
2. Caterpillar Logistics
The Organisation
Caterpillar Logistics is a wholly owned subsidiary of Caterpillar Inc. which
provides third party warehousing and distribution, and other services. It has a
workforce of 600 employees in the East Midlands. In late 2007 Caterpillar, the
USA based company, took over the Hinckley site from Quinton Hazell (QH), a
company distributing spare parts to the motor industry which remains the
Hinckley sites main client. Around 130 QH employees were the subject of a TUPE
transfer to Caterpillar Logistics although 90 employees remained in the
employment of the client organisation on the Hinckley site. A further 20
Caterpillar managers and other salaried staff moved to the Hinckley site as a
result of the takeover. Caterpillar has three other sites in the region although
these are predominantly concerned with manufacture rather than distribution. As
a result of these changes in ownership in 2007, the site employs a mixture of
Caterpillar management, including an HR manager who provides services to three
other local Caterpillar sites, transferred QH staff and a significant number of
external staff. The two nominated gatekeepers were the Logistics Centre Manager
and an HR Manager who had responsibilities across the three other Caterpillar
sites in the region. The Logistics Centre manager left the site towards the end of
the project to work in South Africa.
The I ssues
The site was a mixture of rather different business and working cultures. At the
time of applying to take part in the project, issues of communication at the
Hinckley site had been identified as being aggravated by the presence of different
cultures resulting from the takeover. Previous US ownership had led to
employees losing participation in a share scheme and pension entitlements. This
was seen as contributing to workforce scepticism about organisational change,
new ideas and initiatives. Furthermore it was felt that a lack of trust in Caterpillar
existed among employees who had been involved in the TUPE transfer. The new
management spent a year introducing Caterpillar systems, including 'continuous
improvement' under the Caterpillar Production System, but found that whilst the
metrics for the site were comparable with other company sites, it scored
significantly lower on measures of employee engagement. For example, in
response to a question asking if they would recommend someone to come and
work at the organisation, a large percentage of employees had answered that
they would not.
The Action Plan
The action plan proposed introducing measures to increase the employee
engagement score in the company employee survey by 10 percentage points,
from the original 62% to 72%. However it was recognised by management that
the poor score for engagement in the employee survey reflected deeper issues
which reflected the divided culture at the Hinckley site.
The Outcomes
An initial series of employee focus groups organised by the Acas facilitator was
reported as cathartic in raising employee concerns across both day and night
shifts. The Acas facilitator used the outcomes of the focus groups to produce a
report for management and employees. A more flexible approach was quickly
80
adopted by managers in their application of the sickness absence policy and the
absence policy was amended accordingly. An employee forum was established
and, other than a short lapse resulting from changes in management, this has
met monthly. An employee survey was carried out in January 2010 which
indicated a 10 percentage point improvement in the engagement score, the
immediate target of the action plan. Further improvement is anticipated in the
annual company survey to be carried out in early 2011.
81
3. The Health Store
The Organisation
The Health Store is a co-operative wholesale and distribution centre for health
foods with a production unit producing its own label dried goods. It has been
trading for over 77 years and is one of the leading health food wholesalers in the
UK supplying stores throughout the UK, Ireland and many parts of Europe. It has
a workforce of 102 employees. The numbers employed did not change during the
project. The Operations Director and the Warehouse Manager were the
Company’s selected gatekeepers; although the Warehouse Manager left the
programme at its outset, he remained highly involved and committed to the
project as it developed.
The I ssues
The market pressures on the company are considerable. 12 competitor
wholesalers have ceased trading in recent times and, at the time of applying to
join the programme, the Company had identified the need to find operational
efficiencies and to increase sales to sustain the business. It had identified that to
maintain and grow the business required team work and, in particular, employee
engagement, across the workforce, with the direction and priorities of the
business. It also recognised the value of trained managers who were effective
communicators and understood employees’ concerns. The programme was seen
as a means of developing a ‘forward thinking workforce able to contribute to the
future of the business’. Due to the growth of the business, The Health Store had
moved to purpose-built, larger premises in 2007. This move brought with it
significant change not least to the close working arrangements that had been a
feature of working practices and employee relations at the previous site. When
the programme began, workforce morale was identified as generally poor
compared to what it had been when the company had been a smaller operation
at its previous premises.
The Action Plan
The main objective of the company’s action plan was to increase employee
engagement through better two way communication. The view was that more
effective and open dialogue between management and staff would mean that
staff would be better informed, able to share their concerns with managers and
more likely to put forward any suggestions they had for improvements to working
practices. It was intended to achieve this by establishing consultative processes
within the organisation and by appointing elected staff representatives. The scope
of the initial action plan was adjusted and the project was implemented as a pilot
in The Health Store’s operations department which is its largest department and
includes both production and the warehouse.
The Outcomes
By the time the formal programme ended in April 2010, two employee
representatives had been elected by the workforce and trained by Acas; joint
management and staff forum meetings had been established in the production
and warehouse areas and employee representatives were attending monthly
management team meetings. The momentum continued and six months after the
project had ended, the final evaluation revealed improved morale and
relationships between management and operations staff and, over a 12 month
82
period, reduced absenteeism. There had been a very significant fall (77
percentage points) in instances of formal disciplinary action and numerous
examples of improvements to working practices as a result of suggestions made
by the workforce. Possibly reflecting the size of the company, these changes
were identified as being largely due to better informal communication and a
greater openness between staff and management but the formal consultative
processes that had been put in place were being used where this was felt to be
appropriate. As the company is about to embark on a major change programme
in its operational system, it feels it is better equipped to face the challenges that
this will present as a result of its participation of the project but views it as a
process of continuous improvement.
83
4. Liquid Control
The organisation
Liquid Control Ltd (LCL) is a private company (whose owner lives in Dubai) and is
one of a group of companies known as the KK Group. It designs, builds and
supplies standard and custom built machines for processing (metering, mixing
and dispensing) single and multi-component liquids/pastes which are used in a
wide range of manufacturing processes. The company is also a distributor of a
number of products which can be used in the systems it provides or as ‘stand
alone’ dispensing products. The main applications covered are: - Structural
Bonding, Sealing, Vacuum Encapsulating, Potting, Moulding, Casting, Resin
Infusion, Resin Injection, Laminating, Filling and Packaging. At the beginning and
the end of the project, Liquid Control Ltd employed 20 people in the UK; all based
at its operations in Wellingborough with the exception one service engineer
working from home in Scotland. Two people were made redundant during the
period and they have taken on a new apprentice and a new Laboratory
Supervisor. The nominated gatekeepers were the Service Manager and the
Technical Manager.
The I ssues
At the time of applying to join the project, the company was seeking ISO9001
Certification and looking for ways to improve the way the business was managed.
After years of stagnation in its management and working methods, its
management had identified that to achieve the productivity and efficiency gains
needed to sustain the business, it was vital to ensure that that workforce was
supportive and engaged with changes in working practices and played an active
role in developing the business for the future. The company has an ageing
workforce, many of whom have long service and there is a perceived resistance
to change which has been exacerbated by the company being sold three times in
five years in recent times.
Historically, decisions had been made by just a few individuals in the business
without the involvement of the workforce. It was recognised that to grow the
business, the workforce needed to be involved in decisions about ‘what, when
and how’ things were done, and it was known from experience that change is not
always readily accepted, even if it is recognised as being for the benefit of all
concerned.
The Action Plan
The initial Action Plan was specifically to:
? To provide training where necessary so that employees can multi-task
? To employ apprentices to train up and fill the gap left by those employees
who will shortly be retiring
? To obtain ISO 9001 by the end of 2010
? To undertake a Stress Questionnaire amongst employees.
84
The Outcomes
There have been a number of outcomes relating directly to the initial Action Plan
including the conduct of a Skills Analysis of employees to identify training needs.
As a results of the Skills Analysis, the company made two people redundant but
also employed a new apprentice and a new Laboratory Supervisor. As a result of
the training needs analysis, approximately 50% of the staff have now completed
NVQ level 3 courses, ranging from computer/electronics skills, to customer
service and management skills. In terms of employee
communications/engagement, the Acas facilitator was instrumental in the
implementation of an Employee Engagement Survey, the results of which were
mostly positive. Focus groups were held to discuss findings and there were some
issues raised around communications and training and development, which are
being addressed. For example, they have introduced development appraisals for
everyone in the workforce and quarterly company meetings to keep everyone
informed. The process of achieving ISO 9001 status is on-going.
As well as the above, the company specifically identified that the following
outcomes have been undertaken as a result of the project:
? Various areas have been identified, such as Departmental Structure,
Product and Facilities, for working parties of employees to meet and produce
recommendations for improvement within these fields;
? One employee has received IOSH training to become a competent H&S
Officer;
? A member of staff from each department has become a Fire
Warden/Officer;
? Organised quarterly presentations by external personnel are taking place
across the company to improve health awareness;
? Weekly departmental meetings have been introduced to de-brief on the
previous week and discuss work-loads for the forthcoming week;
? An additional apprentice is to be recruited.
85
5. NHS East Midlands Strategic Health Authority (SHA)
The Organisation
NHS East Midlands is the Strategic Health Authority (SHA) providing leadership of
the NHS provision for the region’s population of 4.3 million spread across the
counties of Derbyshire, Leicestershire and Rutland, Lincolnshire,
Northamptonshire and Nottinghamshire. Overseeing a budget of £6 billion, the
SHA helps to ensure that local health systems operate effectively and efficiently
for the population. Established in July 2006 the SHAs have specific
responsibilities for relaying and explaining national policy, setting direction and
supporting and developing the region’s NHS Trust bodies. At the commencement
of the initiative, the East Midlands SHA had a workforce of 350 employees spread
across three sites including the Headquarters located in Sandiacre,
Nottinghamshire. There were originally two gatekeepers the Planning & Projects
Workstream Lead, IM&T and the Business Manager, Public Health. This latter
gatekeeper went on maternity leave during the course of the project.
The I ssues
The organisation sought to improve workforce flexibility and to take greater
account of employees' views, including any issues or concerns so promoting a
greater sense of fairness and consistency. It was believed that this could be
achieved by maximising the benefits of the Electronic Staff Record (ESR) system,
in particular by transferring the information ownership to the individual so
providing higher quality information. This was to be achieved through the
implementation of an innovative Self Service functionality adhering to
governance standards in relation to information access and data protection. The
Oracle Learning Management System (OLM) would be available through both
Management and Employee Self Service. Consequently, there was a need to train
all staff to be able to access the corporate Knowledge, Skills and Competencies
database (OLM), so enabling all employees to access their learning and
development details, and book themselves on internal courses with the
authorisation of their line managers. Such a change to organisational practice, it
was perceived, would increase feelings of value amongst staff as they become
increasingly responsible for their training and development. This in turn would
assist the organisation move towards a culture where everybody would be
encouraged to learn new skills which would help identify goals which might
contribute to the organisation's vision.
The Action Plan
The action plan envisaged maximising the ESR by transferring personal
information data ownership to the individual so enhancing the provision of high
quality information. It was believed that this change in working practice would
boost employee engagement through better communication in turn improving
workforce flexibility. Part of the project’s objectives were to enable managers to
maintain relevant personal information regarding their employees in such areas
as absence, terminations and personal details so reducing levels of information
handling. The action plan envisaged that increased employee utilisation of the
new system and its enhanced reporting facilities would enable more timely
recording of absence which in turn would enable managers to more effectively
manage both levels of absence and its cost. Maximising the utility of the ESR
86
would also devolve access, provide more rapid authorisation of information and
reduce paper based transactions in turn resulting in a reduction of labour
intensive administrative duties in both Payroll and HR. The action plan highlighted
the need to integrate the Acas IWP with the wider SHA ESR project. It was
anticipated that the Acas project would facilitate the Gatekeepers (within this
wider project) to help engage staff and to secure buy-in throughout the
organisation and support the effective and efficient utilisation of the ESR system
being introduced. The gatekeepers were operating in two pilot areas which would
increasingly be looking to enhance engagement of staff and line mangers with
the new ESR system. Consequently, it was identified at an early stage in the
project plan that communication was a key issue and this became the critical
focus of the internal project and facilitation process.
The Outcomes
By April 2010 significant progress had been made. Following some early
communication difficulties encountered as a result of the involvement of so many
different stakeholders, the ESR system became operational. Acas’ involvement
led to the facilitation of a number of focus groups of end-users which enabled all
‘voices’ to be heard and this employee participation reshaped training and
development initiatives which assisted the implementation of the ESR System.
Six months following the end of the project an increasing number of employees
were beginning to engage with the ESR up-dating their own personal data. This
has had three major effects: staff took greater ownership of their data and their
personal development, HR information is more accurate, and data handling has
become more efficient. This has led to improved reliability, productivity and
efficiency in the handling of personal data.
87
6. Northampton College
Northampton College is a provider of Further and Higher Education to some
12,000 learners. 4,500 of these are full-time students predominantly 16-19 years
old. The remainder are predominantly adult learners taking vocational and
professional qualifications. A small area of the college is concerned with the
higher education providing qualifications in Leadership & Management to middle
and senior managers at levels 4 to 8 (HND to PhD). This unit also works with
employers through organisational-development programmes and provides
consultancy and leadership coaching. At the commencement of the project, the
College employed 1003 staff (404 FT and 599 PT) At the conclusion of the project
it had a workforce of 1039 staff (374 FT and 665 PT). The college operates across
four sites, Booth Lane (Main Site), Lower Mounts (HE campus), Daventry campus
and Westbury Court (the Business Centre) all in Northamptonshire. The Manager
for Professional Development (Leadership & Management) and a Management
Lecturer and Part-Time Co-ordinator were chosen as the selected Gatekeepers for
the project.
The I ssues
As a result of an Ofsted inspection in April 2009, the College was graded at an
overall level 3 (Satisfactory). The College has a strategic vision to become an
Outstanding Grade 1 college by 2013. In order to achieve this its senior
management had identified that there was a need to achieve a culture change
from ‘satisfactory is good enough’ to one which promoted excellence across the
range of its activities. There were three major strands of the college’s strategy to
become Grade 1 which were relevant to the IWP. These were: to raise teaching
standards, develop leadership and management capability; to provide clear
direction and better mentoring, coaching and support for staff; and to radically
improve any areas of ‘failing provision’ as measured against the benchmarks of
the national FE ‘Success Rates’. An additional challenge for the college was that,
at the time of joining the project, it was in the early stages of demolishing and
rebuilding its main campus without temporarily relocating its operations
The Action Plan
The Action Plan at the outset of the project centred around a Leadership and
Management Development Programme, which had begun in October 2009. This
had been designed to develop capability in leadership and management. A
further Action Plan (see below) relating to the issue of employee engagement
emerged as a result of participation in the project and this led to the Acas
facilitation element of the programme continuing for longer than at other
participating organisations.
The Outcomes
The Acas facilitators held a number of diagnostic workshops involving functional
groups which included representatives from the trade unions. These workshops
were based around the results of a Staff Engagement Survey in 2009 which
identified that employee engagement was fundamental to the college achieving
the culture change it was seeking. A pivotal point for the project was when the
connection was made between the Leadership Programme and the survey’s
employee feedback on issues of leadership and management. Considerable
progress has been reported in the development of leadership skills, and their
application to achieve performance improvements, among college managers at all
88
levels. In terms of employee engagement there have been a number of outcomes
which are on-going. A further Action Plan has been developed between senior
managers and representatives from the two main trade unions following a joint
meeting facilitated by Acas. This addresses issues of employee consultation,
communications, student behaviour and teaching/learning observation. The
action plan agenda is being addressed by different task groups. Two joint
problem solving training sessions are to take place for the managers and staff
who are taking part in Task Groups.
89
7. Pendragon Contracts and National Fleet Solutions (Derby)
The Organisation
Pendragon PLC is the UK's Leading Automotive Retail Network. The automotive
retail outlets trade as Stratstone, Evans Halshaw and Chatfield’s, offering a large
selection of new and used vehicles. These brands represent over 20 franchises for
passenger vehicles, motorcycles, commercials and trucks operating from over
300 retail sites. Pendragon Corporate Division embraces Pendragon’s Contract
Hire and Pendragon Fleet Leasing business in the UK. The Division is located in
Derby. At the commencement of the project Pendragon Corporate Division
employed 98 team members (38 males and 60 females). At the conclusion of the
project team member numbers had reduced by 7 to 91 in total. Pendragon’s
Employee Relations Leader and Customer Services Director were the company’s
selected gatekeepers.
The I ssues
The primary business of Pendragon Corporate Division is client account
management and building relationships with external customers. The
increasingly adverse economic climate had severely impacted the industry, and
Pendragon Corporate Division had been subjected to a series of restructures and
redundancies. Consequently, team members had experienced a number of
changes, both structurally and to their own roles and responsibilities. This had
left a lot of team members with feelings of unrest and uncertainty regarding their
future job security (sometimes referred to by employees in the company as
‘survivor syndrome’). It was believed that this uncertainty had led to some team
members becoming disengaged, which in turn was having an impact on their
personal performance. Given the importance of client relationships, management
believed that team members who were disengaged would adversely impact the
external perception of the Division and ultimately adversely impact the overall
financial state of the organisation. The opportunity to participate in the
programme was seen as a possible catalyst to addressing the engagement issue;
would be an opportunity to set the foundations of better employee relations for
the Contracts Division going forward; and provide a chance to develop
approaches to engagement which could be utilised across Pendragon PLC.
The Action Plan
Consequently, the focus of the Corporate Division’s action plan was ‘to increase
business performance through improved team member engagement’. It was
believed that the project provided an opportunity to improve team member
engagement, encourage better team participation and departmental interaction
so increasing customer service.
The Outcomes
By the end of the project in April 2010 Pendragon Contracts had established a
team member forum. Initially this forum was facilitated by Acas but later became
self-facilitating. This forum agreed and instituted beneficial changes to a number
of what the gatekeepers called ‘house keeping policies’ relating to such issues as
an equitable car parking protocol, and the standardisation of the no-smoking
policy. The successful implementation of these changes paved the way to
enhanced more challenging issues such as improved team member
communication throughout the Division. A number of team building events were
90
held to meet this objective and improved team member’s engagement. Team
leaders had begun to meet on a monthly basis and a more recent employee
engagement survey displayed improved results. This enhanced communication
and engagement provided the confidence for the Division to resume their
customer service survey and gain commitment from other divisions to begin a
similar project internally. Six months following the end of the project increased
employee communications and engagement was helping the Division face the
severe economic climate. With imposed structural changes and changes to
personnel, the perceived challenge going forward appeared to be sustaining the
achievements made so far.
91
8. Thorpe Kilworth
The Organisation
Thorpe Kilworth was a designer, manufacturer and installer of high quality
furniture for education, laboratory and health care establishments with 116
employees at its site in Corby, Northamptonshire. The company’s nominated
gatekeepers were its Organisational Development Manager and Production
Manager.
Throughout the duration of the project, the company experienced difficulties
which resulted in short time working and redundancies. This was largely due to
the seasonal nature of its work and the unexpected loss of work from major
clients despite existing agreements. As a result, the headcount reduced to 80
employees during the project. A combination of cash flow problems and delays in
clients confirming contracts culminated in the company going into administration
in June 2010. As it had completed the IWP, it continued to be included in the
evaluation process. Since then, 27 former employees have formed a co-operative
company known as Thorpe Learning.
The I ssues
The company had identified that it needed to be more responsive to changes in
the market place and to the needs of its clients. It was seeking to increase
workforce versatility and flexibility so that individuals could be moved from one
process to another in order to meet fluctuations in demand. This required
employees to develop their skills and to be more adaptable in client facing
situations, as well as in project management, design, manufacture and
installation. In addition, the company was aware that they needed to improve its
competitiveness by maintaining quality and reducing waste and inefficiencies
The Action Plan
The aim was to improve the company’s competitive edge by maintaining quality
but improving manufacturing efficiency with the key objective of having all
materials and consumables ready and clearly identified for manufacture. This
particularly involved:
? Obtaining relevant information for production in a timely fashion
? Reducing double handling and other efficiency losses
? More consistency in design for manufacture
? Ensuring the delivery of all relevant materials to meet production
schedules on as near to a ‘just in time’ basis as possible
? Implementing relevant revised procedures with relevant training
? Measuring before & after for cost-benefit analysis.
A key element of the action plan was to improve employee engagement in a
workforce viewed as loyal and responsive but with concerns about changing long
established ways of working. Improving two way communications was seen as
essential so that the workforce understood and was committed to the changes in
the business.
92
The Outcomes
By May 2010 before the company went into administration, a significant amount
of the action plan had been achieved. Elements of lean manufacturing had been
introduced and the Stores area had been re-organised resulting in improved
security, improved facilities and better stock control. Throughout this re-
organisation there had been employee participation. A cross-functional working
party had been established to identify some of the weaknesses of the current
stock system. Acas facilitators had run a workshop for the company on joint
working/problem solving, and an employee engagement survey had been
developed. Six team leaders from staff in manufacturing and the office had been
appointed with a supporting training programme and the existing six employee
representatives had been trained. The reconstituted Staff Consultation Forum
highlighted numerous problems and inefficiencies such as the lack of standard
procedures and poor team-working. These issues were in the process of being
addressed prior to the closure of the business.
93
Appendix B: Acas short courses provided to participants
Innovative workplaces – emda project
Training events
Meetings, Consultation & Forums – some useful basics
Tuesday 9
th
March 2010
Represented
Organisation Number of attendees Gatekeepers
SHA NHS East Midlands 2 2
The Health Store 1 1
Thorpe Kilworth 2 2
Mahle 1 1
Having Difficult Conversations
Wednesday 10
th
March 2010
Represented
Organisation Number of attendees Gatekeepers
SHA NHS East Midlands 1 1
Thorpe Kilworth 1 0
Mahle 2 1
Pendragon 2 0
Caterpillar 2 1
Having Difficult Conversations
Friday 12
th
March 2010
Represented
Organisation Number of attendees Gatekeepers
SHA NHS East Midlands 1 1
Thorpe Kilworth 3 1
Pendragon 3 0
Liquid Control 2 2
The Health Store 1 1
Facilitation Skills Training
Tuesday 13
th
July
Represented
Organisation Number of attendees Gatekeepers
SHA NHS East Midlands 1 1
Thorpe Kilworth 3 1
Pendragon 2 0
Liquid Control 2 2
The Health Store 1 1
Caterpillar 2 0
94
Developing Mediation Skills
30
th
September, 1
st
October and 7
th
October 2010
Represented
Organisation Number of attendees Gatekeepers
SHA NHS East Midlands 1 1
Pendragon 2 1
Liquid Control 2 2
Caterpillar 2 1
In addition some facilitators delivered in-company training as follows:
? Pendragon - Essential Skills for Supervisors; Training for Workplace reps
? Caterpillar - Training for Workplace reps;
? The Health Store - Training for Workplace reps
95
Appendix C: Anderson’s Evaluation Model
Model of Value Contribution Evaluation – Anderson (2007)
Learning Function Efficiency Measures
Focus on assessing efficiency and
effectiveness of the learning function.
Although the term ‘learning function’ is
often taken to mean a specialist
department, the term can equally well
be used in a wider sense to refer to ‘the
body of Learning and Development
activity that has to be provided for an
organisation and the people most
directly responsible for that provision’
(Harrison, 2005:206).
Return on Expectation Measures
Focus on assessing the extent to which
the anticipated benefits of the learning
investment have been realised. Key
questions underpinning a return on
expectation approach are:
What were the original expectations of
organisational stakeholders for the
learning or training? Have those
expectations since changed?
What changes have occurred as a
result of the learning process?
To what extent have stakeholder
expectations been met?
Return on investment measures
Focus on assessment of the benefits of
specific learning and training
interventions compared with the costs
incurred. They may also involve an
assessment of the pay back period for
specific learning or training
investments. Key issues here are the
extent to which learning is directly
contributing to the achievement of
defined performance targets.
Key performance indicators and
benchmark measures
Focus on the more general evaluation
of HR processes and performance
through a comparison with key
performance indicators or external
standards of ‘good practice’ or
‘excellence’. These approaches may be
undertaken as a one-off’ but are more
useful when treated as a continuous
process in which the organisation
continually seeks to challenge and
improve its processes.
Source: Anderson, V. (2007) The value of learning: A new model of value and
evaluation, CIPD Change Agenda series, London: Chartered Institute of Personnel
and Development.
96
Innovative Workplaces – Economic Impact
Assessment
December 2010
97
Contents
1.0 Introduction............................................................................................. 1
1.1 Innovative Workplaces ..............................................................................................1
1.2 Analytical Framework ................................................................................................1
1.3 Assessing additionality .............................................................................................5
2.0 Economic Impact Assessment .............................................................. 6
2.1 Organisations participating in the Innovative Workplaces project ........................6
2.2 Availability and take-up of alternative sources of support .....................................6
2.3 Improvements made by participating organisations ..............................................7
2.4 Changes in organisational performance resulting from improvements................8
2.4.1 Staff retention and absence .........................................................................................9
2.4.2 Changes in grievances, disciplinary sanctions, and ET claims...................................10
2.4.3 Productivity.................................................................................................................11
2.4.4 Employment growth....................................................................................................13
2.5 Future impacts on employment ..............................................................................13
2.6 Gross additional economic impacts .......................................................................14
2.7 Leakage, displacement, substitution and multiplier effects.................................14
2.7.1 Leakage .....................................................................................................................14
2.7.2 Displacement..............................................................................................................15
2.7.3 Substitution effects.....................................................................................................15
2.7.4 Multiplier effects .........................................................................................................15
2.8 Net additional economic impacts per annum........................................................16
2.9 Present value of GVA impacts ................................................................................16
2.10 Return on investment ..............................................................................................17
2.11 Costs and benefits that have not been quantified.................................................18
List of figures
Figure 1.1 Analytical Framework.................................................................................4
Figure 2.1 Improvements made to Human Resource Policies by Participating
Organisations ...............................................................................................................7
Figure 2.2 Improvements in organisational performance resulting from process
improvements...............................................................................................................9
98
List of tables
Table 2.1 Estimated probability organisations would not have taken up an
alternative source of similar support in the absence of the Innovative Workplaces
project ..........................................................................................................................6
Table 2.2 Estimated probability organisations would not have implemented
improved HR procedures without the support they received (additionality of
actions).........................................................................................................................8
Table 2.3 Changes in grievances, disciplinary sanctions, and ET claims, and
associated management savings ...............................................................................10
Table 2.4 Estimated probability organisations would not have seen productivity
gains if they had not implemented process improvements to their business..............11
Table 2.5 Gross additional productivity gains............................................................12
Table 2.6 Gross additional economic impacts...........................................................14
Table 2.7 Multiplier Effects ........................................................................................16
Table 2.8 Net additional economic impacts...............................................................16
Table 2.9 Present value of GVA benefits ..................................................................17
Table 2.10 Return on Investment ..............................................................................18
99
1
1.0 Introduction
This report sets out the economic impacts of the emda funded Innovative Workplaces project that was
delivered by the Advisory, Conciliation, and Arbitration Service (Acas) between 2009 and 2010. The
analysis is designed to provide an estimate of the impacts of the project on participating organisations in
terms of employment and GVA that is aligned with the principles set out in emda's evaluation toolkit. The
analysis draws on a survey of participating organisations undertaken by Nottingham Trent University (who
are undertaking a full evaluation of the project for Acas).
1.1 Innovative Workplaces
The 'Innovative Workplaces – Developing Organisations' is a project aiming to improve the performance
of workplaces through greater employee involvement and engagement, and improved workplace
practices and procedures. Ten organisations from the public and private sectors in the East Midlands
initially participated in the project. Each received a bespoke package of support from senior Acas
advisors and the UK Work Organisation Network (UKWON). The project has involved addressing a range
of organisational issues, including:
? boosting employee engagement;
? improving communications processes;
? improving morale and motivation;
? managing change and restructuring;
? workforce flexibility;
? performance management and measurement; and,
? innovation and adaptation.
As well as guidance and support from Acas, the ten participating organisations also benefited from an
accredited management training programme, and peer support from fellow participants. The combination
of networking support, facilitated by UKWON, coupled with on-site practical help from Acas, was
designed to help participant companies to address some of the difficulties that have held back
performance. Eight of the participating organisations ultimately completed the programme.
1.2 Analytical Framework
This section sets out our analytical framework for assessing the economic impacts of the Innovative
Workplaces project in terms of net employment and GVA created and safeguarded. The Innovative
Workplaces is expected to generate economic impacts through the following mechanisms:
? Implementation of improvements to workplace processes: The programme of support provided to
participating organisations was designed to encourage them to implement and deliver improvements
to their HR and other related workplace practices.
? Gross changes in performance: Over the period following the implementation of improvements, the
organisation may see the performance of their business change. This could encompass either growth
100
2
or decline in their sales or turnover, or changes in the productivity of their workforce. These changes
will also be reflected in terms of:
? Changes in GVA: GVA is measure of the total output of the organisation, the value added by
the organisation to the goods and services it absorbs in the production process. In practical
terms, this is measured by subtracting the value an organisation's purchases of goods and
services (ranging from raw materials through to expenditures on property services
1
) from the
value of what an organisation produces (which can be approximated by the value of sales)
2
.
Changes in GVA can be driven by the following:
? Changes in GVA driven by sales growth: If demand for an organisation's products or
services increases, then the organisation will need to produce more of those products or
services to satisfy that demand. Other things being equal, the organisation will need to
employ further workers in order to deliver greater levels of output, resulting in impacts on
employment, with the converse applying if demand for the organisation's output falls.
? Changes in GVA driven by productivity growth: Overall GVA may also increase or
decrease without changes in employment levels through changes in productivity, i.e. the
levels of output that each worker in an organisation is able to produce over a given period of
time. This could occur through workers themselves increasing their skills or being organised
more efficiently, but also if the organisation is able to produce similar levels of output while
using less goods and services produced by other organisations (i.e. reducing their cost
base).
? Additionality: The objective of the economic impact assessment is to estimate the share of gross
changes in performance outlined above that can be attributed to the support provided through the
Innovative Workplaces project
3
. There are several dimensions to consider:
? How far participants would have taken up a similar alternative source of support: In the
absence of Innovative Workplaces, participants may have taken up similar support if it was
available from another source, provided by the public or the private sector. If this is the case,
then it is likely that the organisations would have implemented similar improvements to their
processes (providing the quality of the advice was comparable), resulting in similar outcomes.
? How far organisations would have implemented process improvement without support:
There is also the possibility that the support provided encouraged organisations to implement
improvements that they would have undertaken anyway.
1
Utilisation of capital items, such as computers and machinery for example, is accounted for under the alternative
measure Net Value Added, which also adjusts for the depreciation of those items over their normal working lives.
2
Valuing the output (GVA) of public sector organisations is typically more challenging as such organisations
generally do not provide products or services that are traded in the market, creating difficulties in estimating the
value of those products and services.
3
Deadweight can be seen as the reverse of this concept, referring to the gross changes in performance that
would have occurred in the absence of the project.
101
? How far changes in business performance can be attributed to the changes made: Finally,
there is a question as to how far changes in business performance can be attributed to the
process improvements made. This could occur in a range of ways for the Innovative
Workplaces project, with the most immediate effects likely to be felt in terms of productivity. If
process improvements help to improve workplace motivation, morale or employee engagement,
this might be felt in terms of improved staff retention or reduced numbers of staff days lost to
absence. Productivity will rise through a lower shares of revenue being allocated to recruitment
spending, together with a reduction in lost GVA due to either positions remaining vacant or staff
being absent from work. These effects (along with other factors such as improved staff
leadership skills or creativity) may also lead to sales growth (or help protect existing sales). For
example, if a organisation is able to retain experienced sales staff with established relationships
with customers, then sales growth may be enhanced over the longer term.
The Innovative Workplaces was a regionally targeted intervention, and there are a range of wider factors
that need to be considered in establishing estimates of economic impacts:
? Leakage: If the economic impacts of the intervention accrue to residents of other regions, then these
are considered to have 'leaked' outside the East Midlands. This occurs if businesses based outside
the East Midlands benefit from the support provided, or if those filling any vacancies created are not
residents of the region.
? Displacement: Participating organisations may also get a competitive advantage over other
organisations based in the East Midlands as a result of the support provided. This may result in a loss
of market share for these other organisations. These negative consequences (displacement) should
also be taken into account.
? Multiplier effects: Multiplier effects occur through two mechanisms. Firstly, to increase GVA,
organisations will need to procure goods and services from other organisations. To the extent they
purchase from other organisations in the East Midlands, these positive effects will help enhance
turnover and employment throughout the supply chain (indirect or supply chain multiplier effects).
Additionally, where wages are paid to any additional employees, their spending of this income in
regionally based organisations will have a similar effect.
? Net additional impacts: Net additional economic impacts at the regional level are estimated as:
Gross impact x Additionality x (1 – Leakage) x (1 – Displacement) x Multiplier Effects.
A diagram setting out our analytical framework is below:
3
102
4
E
C
O
T
E
C
Innovative
Workplaces
Peer support
Addi tionality (1):
How far would
participants have
found similar
support elsewhere?
Additionality (2):
How far would these
improvements have
been made without
Acas support?
Improved
employee
engagement
mechanisms
Improved HR
policies and
processes
Additionality (3):
How far would these
outcomes have been
achieved without the
improvements?
Improved staff
morale or
motivation
Reduced staff
absenteeism
Improved staff
creativity
Increased sales
Reduced costs
(legal, recruitment,
management time,
training)
Lost output (GVA)
recovered
Leakage,
displacement, and
multiplier effects
Net economic
effects on GVA
Enhanced
productivity (GVA
per worker) and
profitability
Acas activities Outputs Outcomes Impacts
Improved staff
retention
Practical on-site
advice on
process
improvements
Improved
communication
Improved staff
leadership skills
Figure 1.1 Analytical Framework
103
5
1.3 Assessing additionality
In general, the most robust approach to assessing the impact of public sector interventions is to adopt a
quasi-experimental approach in which comparisons are made between those receiving support are
compared to a control group to identify how far improvements can be attributed to participation in support.
This approach could not be applied to the Innovative Workplaces project: with eight organisations
completing the project, quasi-experimental approaches would not be sufficiently robust to generate any
statistically significant findings. Additionally, quasi-experimental evaluation studies are highly resource
intensive, and given the scale of the intervention, such costs would be difficult to justify.
Estimates of additionality are instead based on assigning assumptions of the extent to which impacts can
be attributed to interventions (in this case the innovative workplace project) based on the reported
perceptions of the participating organisations. The overall approach and assumptions utilised are
consistent with emda's Evaluation Toolkit, and reflect a commonly utilised approach to evaluation in such
contexts. However, the approach is not without caveats. Firstly, estimates are based on self-reporting,
which may not be reliable since respondents may have an incentive to over- or understate the impact of
the support they received, or in some cases, may not be in a position to assess the impact of support.
Additionally, the approach is based on ascribing a quantitative measure of probability to responses given
on a qualitative scale of impact (e.g. 'Very likely' – 'Very Unlikely'). The reliability of this approach
depends on how far the assumptions utilised are aligned with respondents perceptions of the impact
associated with their responses on average.
104
2.0 Economic Impact Assessment
This section sets out an analysis of the economic impacts of the emda funded, Acas led Innovative
Workplaces project. The economic impact assessment has been developed in alignment with the
methodological principles set out in emda's evaluation toolkit.
2.1 Organisations participating in the Innovative Workplaces project
Eleven organisations participated in the Innovative Workplaces project, although three withdrew over the
course of delivery, resulting in a total of eight completing the project. This analysis is restricted to the
eight completers, although, as other chapters in this report discuss, further impacts were achieved
amongst the three dropping out.
2.2 Availability and take-up of alternative sources of support
A key consideration in assessing the impact of the project is the extent to which the outcomes achieved
by participating organisations is the extent to which they would have taken up a similar alternative source
of support in the absence of the Innovative Workplaces project. If participants would have obtained similar
support elsewhere, then it is likely that any such outcomes would have been achieved anyway.
Respondents to the survey were asked to report whether they felt they would have been able to obtain
similar support from an alternative source, where they would have been able to obtain that support, and
how likely they would have been able to take up that support if the support provided through Innovative
Workplaces was unavailable. Of the eight respondents to the survey, three reported that they would have
been able to find a similar level of support elsewhere, for which there would have been fees. However,
two of the three respondents reporting they would be 'unlikely' to take up this alternative support, and
respondents were less clear on where they would have been able to obtain this support, citing
unspecified consultants and the Chamber of Commerce. There were also doubts about whether all
organisations would have the ability to pay for such a service.
Overall, the evidence suggests in the main, participants would not have accessed similar services
elsewhere, with an overall estimated probability that organisations would not have taken up alternative
support of 84 percent. Details of this estimate and the assumptions made are set out in the table below.
Table 2.1 Estimated probability organisations would not have taken up an alternative source of
similar support in the absence of the Innovative Workplaces project
Response to 'Would you have been able to find a
similar level of support elsewhere?'
a) Percentage of
respondents
b) Additionality
assumption/
weighting
c) Probability
that
organisations
would not have
found/used
equivalent
alternati ve
support
No 62.5 1.00 0.625
Yes 37.5 - -
6
105
Response to 'How likely is that you would have taken up this alternative support?'
Definitely 0 0.00 -
Likely 12.5 0.25 0.031
Neither likely nor unlikely 0 0.50 -
Unlikely 25 0.75 0.188
Definitely not 0 1.00 -
Estimated probability that organisations would not have taken up alternati ve support: 0.84
Source: Participant Survey, Nottingham Trent University (The probability organisations would not have found
alternative support is based on the proportion of respondents giving a particular response weighted by the
additionality assumption – i.e. column a x column b)
2.3 Improvements made by participating organisations
Respondents were asked to report whether they implemented new or improved human resource policies
or procedures (or planned to make such improvements in the future) during or since participating in the
Innovative Workplaces project. Seven of the eight organisations surveyed had implemented at least one
new or improved process and all had at least plans to implement new or improved processes.
Improvements in procedures for informing and consulting with employees were most commonly reported
by organisations, followed by 'other' processes, which might include equality or absence management
policies.
Four of the eight reported they had plans to implement improvements in the future, with improvements in
discipline and grievance policies most commonly reported, suggesting that some impacts of the project
might be felt at a later date.
Figure 2.1 Improvements made to Human Resource Policies by Participating Organisations
0
1
2
3
4
5
Discipline Grievances Redundancy Informing and
Consulting Employees
Other (such as
Equality Policy and
Absence
Management)
Type of procedure of policy improved
N
u
m
b
e
r
o
f
f
i
r
m
s
New
Improved
Planned
Source: Participant Survey, Nottingham Trent University
7
106
In order to identify how far these improvements were made as a result of the Innovative Workplaces
projects, respondents to the postal survey were asked to report the extent to which they would have
made these improvements without the support they received from Acas. None of the participating
organisations felt they definitely would not have implemented the main changes without the support they
received, although a quarter reported they probably would not have done so. Using the additionality
assumptions set out in Table 2.2 below, it is estimated that there was a 32 percent probability overall that
participating organisations would not have implemented improved HR procedures without Acas support.
Table 2.2 Estimated probability organisations would not have implemented improved HR
procedures without the support they received (additionality of actions)
Response to 'How likely is that you would
have introduced the main changes without
the support you recei ved from Acas?
a) Percentage of
respondents
b) Additionality
assumption/
weighting
c) Probability that
organisations
would not have
introduced the
main changes
without Acas
support?
Definitely 25 0.00 -
Probably 37.5 0.25 0.09
Probably not 25 0.75 0.19
Definitely not 0 1.00 -
Other (*) 12.5 0.33 0.04
Estimated probability organisations would not have made improvements without the
support they recei ved
0.32
Source: Participant Survey, Nottingham Trent University (*) one respondent reported that of three procedures
implemented, they would have definitely have implemented two without Acas support, and definitely not have
implemented the third, so an additionality rate of one in three (33 percent) is assumed in this case. The probability
that respondents would not have implemented improvements without the support they received is estimated by
percentage of respondents giving particular responses, weighted by the additionality assumption (i.e. column a x
column b).
2.4 Changes in organisational performance resulting from improvements
Participating organisations were also asked to report how the improvements they made resulted in
changes to the performance of their organisation or business. Most frequently reported were
improvements in communication within their organisations, increased staff engagement, and improved
leadership skills, with secondary effects on staff creativity and turnover.
8
107
Figure 2.2 Improvements in organisational performance resulting from process improvements
0
1
2
3
4
5
6
7
Improved
communication
within the
organisation
Increased staff
engagement
Improved
leadership skills
for some
employees
Improved staff
morale/motivation
Increased staff
creativity
Greater
sales/turnover
Type of improvement
N
u
m
b
e
r
o
f
f
i
r
m
s
Source: Participant Survey, Nottingham Trent University
The impacts outlined above generally relate to the intermediate outcomes of the improvements made
(with the exception of impacts on sales or turnover). Where these short term effects have led on to an
economic impact on the organisations concerned, these will be ultimately be observed through
improvements in productivity and sales growth. Three approaches to valuing economic impacts have
been considered:
? Impacts on productivity driven by reductions in staff retention and absence: Improvements to
HR processes may have had an immediate effect on rates of staff retention and absence via improved
staff morale and motivation. Such improvements would have an impact on GVA through increasing
productivity (through avoiding the output lost through staff absence or replacing workers that have left
the organisation).
? General productivity growth: The improvements in staff engagement and communication may have
also helped to boost productivity more generally through enabling staff to complete tasks more
efficiently or more effectively, resulting in further GVA impacts.
? Turnover and employment growth: Finally, the improvements made by organisations may have
ultimately resulted in greater sales and the recruitment of additional staff in order to meet demand.
These impacts are explored in turn in the following sections.
2.4.1 Staff retention and absence
Two organisations (from the five able to respond) reported they had seen a decline in the number of staff
leaving voluntarily, while a further two (again, from five able to respond) reported they had seen a fall in
the proportion of working days lost due to absence since participating in the project. However, no
respondents reported that the process improvements they had made as a result of Acas support had any
9
108
10
influence over these aspects of business performance. Therefore no economic impacts are estimated to
arise through reduced staff retention and absence.
2.4.2 Changes in grievances, disciplinary sanctions, and ET claims
In addition to benefits resulting from changes in staff absence and retention, if the Innovative Workplaces
project resulted in a decrease in the number of grievances, disciplinary sanctions, or Employment
Tribunal claims then organisations will avoid a range of costs (ranging from management costs potentially
through to legal costs).
The survey of firms indicated that organisations saw an increase in the number of employees utilising
grievance procedures, the number of cases where disciplinary sanctions were applied, and virtually no
change in the number of ET claims, as set out in the table below. In order to estimate the level of
management time associated with these, the number of estimated days of management time associated
with each type of case (as estimated from the 2007 CIPD Managing Conflict at Work survey and the 2008
Survey of Employment Tribunal Applications) was applied to the net change in the number of cases
observed across the sample. Overall, it was estimated that participating firms saw a increase in the level
of management time associated with grievances, disciplinary sanctions and ET claims of 158 days.
However around 80 percent of these days can be attributed to a single organisation. In all other
organisation, the volumes of disputes as measured through grievances, disciplinary sanctions and ET
claims had either remained stable or decreased (in some instances notably so).
Management time was value in terms of managers gross hourly pay (£18.52) and average hours worked
per week (37.5), giving an estimate of £138.90 per day
1
, and applied to the estimated net change in
management time associated with grievances, disciplinary sanctions and ET claims. Overall additional
costs were estimated at £21,946.
Table 2.3 Changes in grievances, disciplinary sanctions, and ET claims, and associated
management savings
Number in
6 months
before
project
Number in
6 months
after
project
Net
change
Manageme
nt Costs
(Days)
Overall
change in
days spent
Manageme
nt wage
costs per
day (£)
Total
Saving (£s)
Grievance
procedure
8 23 15 9 135 138.9 18,752
Disciplinary
Sanctions
43 44 1 13 13 138.9 1,806
ET claims 4 6 2 5 10 138.9 1,389
Overall
change
- - - -
158 21,946
Source: Participant survey, SETA 2008, CIPD Managing Conflict At Work 2007 and ASHE 2010
Notwithstanding the fact that the rise in incidence of disputes was entirely accounted for in one
organisation, seeing a rise in reported disputes may well be one of the apparently paradoxical outcomes
of improved procedures. It may be that a more transparent and accessible procedures create an
1
This assumes that the value of a managers work is equal to the value of their hourly earnings, but ignores
additional taxes (such as Employer's National Insurance contribution) and costs (such as training).
109
11
environment in which individuals are more aware, or feel more able to raise problems at work. This
outcome (though not apparently applicable in the case of the Innovative Workplaces Initiative) may result
in an increase of observable disputes. In the absence of these opportunities and mechanisms, conflict is
arguably more likely to manifest itself in much higher levels of employee turnover.
While these effectively offset the productivity gains estimated above, it is not possible to include these
savings in the overall economic impact assessment. We have no evidence that these additional costs are
necessarily attributable to the Innovative Workplaces project since, as owing to constraints on the length
of the questionnaire, respondents were not asked to attribute changes to the support they received.
Secondly, the additional costs are also negligible in the context of the estimates of GVA impacts through
wider productivity gains as set out below.
2.4.3 Productivity
Productivity is a measure of the output (GVA) an organisation can produce for a given level of inputs, and
is typically measured by GVA per worker. GVA per worker might rise (among other reasons) if workers
become more efficient or skilled, if organisations replace workers with capital equipment or machinery, or
if organisations adopt more efficient production processes
1
. Though reductions in staff absences or
grievances, or increases in staff retention, will increase productivity through improving the efficiency of the
organisation, the Innovative Workplaces project may have resulted in wider productivity benefits, for
example if the project resulted in more creative or motivated workers.
All participants felt that the productivity of their staff had risen over the past twelve months. The range of
responses given when asked to report how far productivity gains were due to the improvements they
made as a result of Acas funded support are set out in the table below. Based on the assumptions
outlined in the table, it is estimated that 50 percent of productivity gains seen by participants would not
have occurred without the improvements they made.
Table 2.4 Estimated probability organisations would not have seen productivity gains if they had
not implemented process improvements to their business
Response to ‘Has your productivity improved as a
result of the changes you made to your business?’
a) Percentage
of respondents
b) Additionality
assumption/
weighting
c) Probability
productivity
would have
improved
without
improvements
No 0 0.00 0
Yes 100 - -
Response to 'How likely is it that you would have seen these productivity gains if you had not implemented these
1
The overall change in GVA within an organisation can be measured by: ?Y = (Y/N) ?N + N x ?(Y/N), where Y is
output (GVA) and N is employment. GVA (Y) can be measured by an organisations turnover less expenditure on
intermediate goods and services. Owing to constraints on the length of the questionnaire, expenditure on
intermediate goods and services is assumed to be a constant proportion of turnover, implying a change in
turnover per worker is reflected in an equivalent proportional change in productivity. However, this definition has
obvious difficulties in application to the public sector – as highlighted in Section 1 – as the turnover of public
sector organisations is not generally driven by sales but by public sector budgeting. Though this general model is
applied to all participants in the Innovative Workplaces projects, these difficulties may result in some mis-
measurement of productivity effects with respect to public sector organisations.
110
changes?’
Definitely 0 0.00 0
Likely 37.5 0.25 0.094
Neither likely nor unlikely
12.5
0.50 0.063
Unlikely
37.5
0.75 0.281
Definitely not
0
1.00
No response
12.5
0.50 0.063
Estimated probability productivity gains would not have been achieved without
improvements made (row total)
0.50
Source: Participant Survey, Nottingham Trent University, the probability productivity gains would not have been
achieved without the improvements made is estimated on by the proportion of respondents giving each response,
weighted by the relevant additionality assumption (i.e. column a x column b)
While this suggests a strong impact, comparisons between the results of the baseline questionnaire
administered six months prior to the delivery of the Innovative Workplaces project and the follow-up
questionnaire delivered six months post intervention suggest that organisations have in reality faced
difficulties over the intervention period.
Three of the five organisations able to provide pre and post projects values for turnover reported that
productivity (as measured by turnover per worker) actually fell over the intervening period, despite the
above participant perceptions to the contrary. Organisations had not responded by reducing employment
to the same extent as turnover, culminating in productivity losses.
However, two organisations reported a (measured) growth in turnover per worker of £9,417 and £30,500,
and applying the ratio of GVA to turnover in the East Midlands (34 percent) this equates to productivity
growth of £3,201 and £10,374 per worker respectively. Aggregating this across each organisations
number of workers and applying the estimated average probability that productivity gains would not have
been achieved without the changes made, gives an overall estimate of gross additional GVA through
productivity gains of £567,000. These estimates are provided in detail in the table below.
Table 2.5 Gross additional productivity gains
Organisation a) Growth in
turnover per
worker (£)
b) Growth in
GVA per
worker (£)
c) Number of
workers
d) Growth in
GVA due to
producti vity
gains (£)
e) Additionality
of producti vity
gains
f) Gross
additional
GVA due to
producti vity
gains (£)
Org 1 9,417 3,202 98 313,766 0.5 156,883
Org 2 30,512 10,374 79 819,563 0.5 409,782
Total - - 177 1,133,330 - 566,665
Source: Ecorys analysis, column f = column a x 0.34 x column c x column e.
Given respondents perceptions of improved productivity (in spite of the reality that turnover per worker
had decreased in three of the five organisations), one of the actual effects of the project may have been
to slow down decline in productivity. Additionally, turnover has been used as proxy measurement for the
value of the goods or services produced by the participating organisations, and some participants may
have produced goods that have been left unsold. As such, it is worthwhile considering possible effects in
terms of productivity safeguarded by the project under the scenario that all participants saw similar
relative productivity gains.
12
111
13
On average, the relevant organisations saw a decline in turnover per worker of £82,900 per worker or a
decline in GVA per worker of £28,100
1
. The two participants whose productivity was observed to grow
saw an average increase in GVA per worker of 19.6 percent
2
, of which 9.8 percent was attributable to the
improvements made (i.e. 19.6 x 0.50). If it is assumed that the project had similar effects amongst those
participants seeing productivity decline, then GVA per worker may have been 9.8 percent lower in the
absence of the improvements made, leading to productivity safeguarded of £2,751 per worker).
Aggregating these effects across the 1,343 workers employed by the relevant workplaces making
improvements following support suggests that productivity effects could rise as high as £4.3m (i.e. £2,751
x 1,343 (productivity safeguarded) + £567,000 (productivity increased)).
However, these findings of this alternative scenario are not carried through to the final economic impact
assessment as they are based on the assumption that productivity effects are uniform across participants.
In order to develop a more reliable measure of productivity safeguarded, it would be necessary to
establish how much lower employment and turnover would have been in the absence of the project.
Owing to the constraints set by the format of the questionnaire, a decision was made not to request
participants to report the impacts of support on turnover.
2.4.4 Employment growth
Organisations were finally asked to report whether the changes they had made had resulted in them
recruiting any additional workers (if employment had increased) or protected any jobs (if employment had
remained the same or decreased). As noted above, the majority of participating organisations had seen a
contraction in their workforce, with one organisation reporting that the changes they had made had
helped safeguard two jobs as a result of the changes made. Applying average GVA per worker in the
region (£36,000), this equates to GVA safeguarded of £72,000.
2.5 Future impacts on employment
The follow-up postal survey was undertaken six months following organisation's initial participation in the
Innovative Workplaces project. Many of the process improvements delivered by organisations will take
time to implement (and as highlighted, some participants were still at the planning stage with respect to
some areas of improvement). Additionally, the effects of improvements may take time to arise, so it is
likely that a focus on the economic impacts of the Innovative Workplaces project to date understate the
total effects of the project.
In order to capture the potential future impacts of the Innovative Workplaces project, participants were
asked to report whether they would recruit any additional workers over the next twelve months as a result
of the improvements they made to their business. Two of the eight organisations reported that they
planned to recruit a total of 13.5 workers in the next 12 months as result of implementing process
improvements, equating to a potential per annum GVA impact of £486,000 (again applying GVA per
worker in the East Midlands of £36,000).
1
Again, using ratio of GVA to turnover derived from the 2008 Annual Business Inquiry of 34 percent in the East
Midlands.
2
The combined turnover of the two participants fell from £35.2m to £35.0m, while numbers of workers fell from 213 to
177, implying turnover per worker rose by 19.6 percent from £165,000 to £198,000.
112
14
2.6 Gross additional economic impacts
Estimates of the gross additional economic impact of the Innovative Workplaces project are set out in the
table below, combining estimates of the probability that organisations would have taken up alternative
sources of support (additionality of support), the probability that organisations would have made process
improvements without Acas support (additionality of actions), and the economic impacts on GVA resulting
from those process improvements. The gross additional GVA per impact of the Innovative Workplaces
project to date is estimated at £174,000 per annum, resulting primarily from productivity growth. A further
£132,000 per annum in GVA per annum is expected as a result of creating a potential gross additional 4.2
jobs per annum over the 12 months following the postal survey (i.e. by November 2011).
To summarise, in a purely economic view, and not withstanding impacts that have not been measured
(such as productivity safeguarded) or gains from improvements that organisations may not yet have
made, the project is estimated to have generated a minimum gross additional GVA impact of £173,700
per annum, and over £300,000 including future impacts. It must also be noted that a wide-array of
benefits from the project were cited but cannot be translated into economic impacts, but are set out in
other chapters of this report.
Table 2.6 Gross additional economic impacts
Impact a) Additionality
of the project
support
b) Additionality
of actions
c) Economic
impacts
improvements
1
Gross
additional
economic
impacts
Employment impacts
Jobs safeguarded 0.84 0.32 2.0 0.5
Potential jobs created 0.84 0.32 13.5 3.7
GVA impacts
GVA safeguarded (£ per annum) 0.84 0.32 72,000 19,592
GVA created through productivity
gains (£ per annum) 0.84 0.32 566,665 154,195
Total GVA created to date - - 638,665 173,787
Total potential GVA created (£ per
annum) 0.84 0.32 486,000 132,245
Source: Ecorys analysis (gross additional economic impacts are estimated as the economic impacts attributable to
improvements x additionality of support x additionality of actions: a*b*c).
2.7 Leakage, displacement, substitution and multiplier effects
In order to move from gross additional to net additional economic impacts requires consideration of
leakage, displacement, substitution effects and multiplier effects. This section deals with each of these in
turn.
2.7.1 Leakage
In evaluating the impact of area-based initiatives (the Innovative Workplaces project was targeted at the
East Midlands), it is important to consider how far the intervention has resulted in impacts leaking outside
1
The figures in this column relate to the estimates of economic impacts provided in sections 2.42, 2.43, and 2.5.
113
15
of the target area. This typically refers to any jobs created (or safeguarded) being taken up by residents
outside the East Midlands, or any GVA created being produced by organisations that are located outside
the region (as GVA is generally measured on a workplace basis).
All organisations participating in the project were based in the East Midlands, so there is no leakage of
GVA impacts. To assess leakage of employment impacts, participants were asked to report the
proportion of their employees that live in the East Midlands. On average (weighted by employment),
participating organisations reported that 85 percent of their employees live in the East Midlands, implying
a value for leakage of 15 percent. This is higher than might be typically expected for organisations based
in the region (an evaluation all emda funded activity between 1999/00 and 2006/07 undertaken by
ECOTEC suggested around 5 percent of organisations employees live outside the region), though this
figure was somewhat skewed by a large electrical manufacturer reporting that 20 percent of their
employees lived outside the region.
2.7.2 Displacement
Displacement occurs where improvements in the performance of publicly assisted businesses comes at
the expense of non supported organisations, for example if an organisation is able to increase their
market share, taking away sales from other companies in the East Midlands. Though the bulk of
quantifiable economic impacts were due to productivity rather than turnover growth, this nevertheless
implies that organisations are able to produce more output for a given size of their workforce, potentially
taking sales away from other organisations.
A postal questionnaire approach was adopted in the evaluation, and a decision was made in
questionnaire design to exclude questions that would enable a direct estimate of displacement, to avoid
making the questionnaire too lengthy with potential negative consequences for response rates. However,
ECOTEC's
1
evaluation of emda suggested that enterprise support initiatives typically result in
displacement rates in the region of 20 percent, and it has been assumed that this applies in the case of
the Innovative Workplaces project except in relation to the two public sector participants.
2.7.3 Substitution effects
Substitution effects relate to organisations substituting one activity for another in order to take advantage
of public sector subsidies and support. This is most relevant in consideration of employment support and
job brokerage initiatives, where organisations may recruit employment programme participants at the
expense of others in the labour market as a result of publicly funded intervention. However, this may also
occur in enterprise support interventions if one production process is substituted for another to take
advantage of public sector support (an example might be energy organisations switching to renewables
as a result of tax breaks or subsidies). Substitution effects are not deemed to be relevant in relation to the
Innovative Workplaces Initiative.
2.7.4 Multiplier effects
Where organisations are able to improve their performance through greater sales or productivity, they will
consume more goods and services provided by other organisations based in the East Midlands,
generating wider economic impacts (supply chain multiplier effects). Equally, where individuals fill any
1
ECOTEC has since been renamed Ecorys.
114
jobs created or are able to increase their earnings as a result of becoming more productive, there will be
further multiplier effects as they spend their additional income in regional businesses (induced multiplier
effects).
In line with the approach set out in the emda evaluation toolkit, multiplier effects have been estimated in
line with the regional multipliers utilised in the Experian regional economic model of the East Midlands.
Average multiplier effects (weighted by industry sector) are estimated at 1.39 as set out in the table
below.
Table 2.7 Multiplier Effects
Sector a) Percentage of
respondents
b) Composite
multiplier effects
Multiplier
calculation (a*b)
Manufacturing 25 1.35 0.34
Private services 50 1.40 0.7
Public services 25 1.42 0.36
Estimated project multiplier effects - 1.39 1.39
Source: Ecorys analysis, Regional Economic Model of the East Midlands, Experian
2.8 Net additional economic impacts per annum
Estimates of the net additional economic impacts (per annum) of the Innovative Workplaces project are
set out in the table below. Total net additional GVA created or safeguarded (per annum) is estimated at
£193,000, with a further potential £147,000 per annum created by November 2011.
Table 2.8 Net additional economic impacts
Impact Gross
additional
economic
impacts
Leakage Displacement Multiplier
effects
Net
additional
economic
impacts
Employment impacts
Jobs safeguarded 0.5 0.15 0.20 1.39 0.5
Potential jobs created 3.7 0.15 0.20 1.39 3.5
GVA impacts
GVA safeguarded (£ per annum) 19,592 0.00 0.20 1.39 21,786
GVA created through productivity gains
(£ per annum)
154,195 0.00 0.20 1.39 171,465
Total GVA created to date 173,787 - - - 193,251
Total potential GVA created (£ per
annum)
132,245 0.00 0.20 1.39 147,057
Source: Ecorys analysis (Net impact = Gross additional impact x (1 – Leakage) x (1 – Displacement) x Multiplier
effects)
2.9 Present value of GVA impacts
The GVA impacts of enterprise support initiatives will grow and endure for a period of time, though in
general, such effects are expected to decay at a certain point. At the time of the survey, six months had
elapsed since organisations had participated in the initiative, so it is not possible at this stage to
determine how long impacts might endure. In such circumstances, guidance issued by BIS on
16
115
17
implementing the evaluation of RDA funded initiatives, suggests that the effects of enterprise support
should be assumed to endure for three years.
The table below shows the projected time profile of GVA impacts (together with the present value of those
impacts, applying the 3.5 percent value for social time preference recommended by the HM Treasury
Green Book)
1
. Overall it is estimated that the Innovative Workplace will have a total GVA impact of
£1,020,900 with a present value (with a baseline of 2009/10) of £972,400.
Table 2.9 Present value of GVA benefits
Year 1a) Value
of GVA
impacts
created to
date (£)
1b) Value
of
potential
GVA
impacts (£)
1c) Total
GVA
impacts (£)
Discount
factor
2a) Present
value of
GVA
impacts
created to
date (£)
2b) Present
value of
potential
GVA
impacts (£)
2c)
Present
value of
total GVA
impacts
(£)
2009/10 193,251 - 193,251 1.00 193,251 - 193,251
2010/11 193,251 147,057 340,307 0.97 186,716 142,084 328,799
2011/12 193,251 147,057 340,307 0.93 180,402 137,279 317,681
2012/13 - 147,057 147,057 0.90 - 132,637 132,637
Total
579,752 441,169 1,020,922 - 560,368 411,999 972,368
Source: Ecorys analysis (present value of benefits are measured by multiplying the value of present and future GVA
impacts by the discount factor, which in turn is calculated by (1 / ((1 + 0.035)^n) where n is the number of years from
the baseline year). Column 1c = 1a + 1b, Columns 2a, 2b and 2c = 1a, 1b, 1c x Discount Factor
2.10 Return on investment
Acas figures indicate that a total of £244,035 was spent delivering the Innovative Workplaces project, and
the project was estimated to have created economic impacts with a present value of £972,400. This gives
an overall return on investment to public sector (in regional economic impacts) of £4.0 for every £1 of
public sector expenditure.
emda contributed £227,437 of total project costs, or 93 percent of total public sector expenditure. In line
with OffPAT guidelines on attributing impacts between public sector funders, it is assumed that 93
percent of the overall impact of the project (£906,232) is attributable to emda's expenditure. The
remainder of expenditure were in-kind contributions made by Acas.
A total of £60,000 of costs represented project management, and £48,152 on evaluation costs. Innovative
Workplaces was a pilot project, and such costs may be reduced if the project was replicated in the future.
The table below also provides ROI figures under scenarios in which evaluation and project management
costs are reduced by 25 percent and 50 percent, which results in an increase in ROI to £4.5 and £5.1 per
£1 of public sector expenditure respectively, on the basis that these savings have no impact on the
overall effectiveness of the project.
1
The value of GVA is discounted to reflect a social preference for income today over an equivalent income in the
future. The 3.5 percent discount rate suggests that society as a whole is indifferent between £100 of income (or
costs) today and £103.50 of income/costs a year later.
116
Table 2.10 Return on Investment
Costs & ROI Cost (£) Economic
Impacts (£s)
Return on
Investment
emda 227,437 906,232 4.0
Total public sector 244,035 972,368 4.0
Potential ROI for future replication of the project, if evaluation & project management staff costs reduced by:
25% 216,997 972,368 4.5
50% 189,959 972,368 5.1
2.11 Costs and benefits that have not been quantified
While this analysis suggests that the Innovative Workplaces project has generated a positive return on
investment over the short period over which impacts have been allowed to accrue, it takes a narrow
economic view of the benefits of the Innovative Workplaces project. There are a range of wider effects
that would be considered in a full cost-benefit analysis of the intervention:
? Economic impacts: There are a number of economic impacts of the project that have not been
quantified in this analysis. As already discussed, any productivity safeguarded by the intervention has
been excluded from the final return on investment figures. In addition, in the future, the project may
also help organisations avoid costly employee disputes and employment tribunal claims, generating
further productivity benefits.
? Social impacts: There may also be a range of social benefits of improvements in HR processes over
and above those described in the economic impact assessment, but are expanded on elsewhere in
the report. These are most likely to incorporate any improvements in employee welfare that are not felt
directly through increases in wages following productivity gains.
? Project costs for the participating organisations: At the same time, the estimates of return on
investment outlined above do not factor in the full costs of implementing the Innovative Workplaces
project. In particular, there participants will incur costs in implementing improvements that have been
not been captured. These are most likely to take the form of opportunity costs, with managers and
other staff taking time to develop new processes and improvements, which could otherwise have been
diverted to other productive activities.
? Disbenefits: Where the Innovative Workplaces has facilitated growth or raised productivity, there will
be a range of social costs that are not captured in the analysis. These costs might be incurred by
employees (for example, transport and childcare costs associated with those filling vacancies – though
the economic impacts of such expenditures are captured through the application of multiplier effects),
or could relate to wider externalities associated with economic growth such as the environmental
impacts associated with greater utilisation of resources or congestion costs caused by more intensive
use of transport infrastructure.
With regard to economic impacts not quantified, it should be noted that the present value of GVA impacts
and return on investment figures that have been calculated only include economic impacts that could be
specifically identified and measured in the course of the project evaluation using this framework. It was
18
117
19
only possible to attribute economic impacts to four of the eight organisations which took part in the final
evaluation of the project (by using questionnaire answers given by the organisations), so the GVA impact
of £1,020,900 (with a present value at 2009/10 of £972,400) is based on the benefits attributed to the
project from The Health Store, Liquid Control, Caterpillar and Thorpe Kilworth. It is assumed that all other
organisations had no economic impacts as a result of participation, when the true economic impact of
these organisations is unknown. An economic impact of zero is, of course, not realistic – however, the
report can only quote figures which have been reported and a cautious approach (cautious in that the
economic impact was positive for four organisations and negative for none) is taken that assumes no
financial impacts were achieved where organisations have been able to fully report impacts. Estimates of
GVA impacts (with a present value of £972,400) can therefore be seen as an estimate of the minimum
return from this project.
118
Published by Acas
Copyright ©Acas
doc_101191709.pdf
The Innovative Workplaces Project fits within such a ‘business support’ model in that it offered a range of in-depth support to ten diverse organisations in the East Midlands.
Research Summary
Unlocking Engagement:
A Review of the ‘Innovative Workplaces’ Initiative nitiative nitiative
Report prepared by
Lynette Harris, Alan Tuckman, Derek Watling and Bernadette Downes
Nottingham Business School
Nottingham Trent University
March 2011
For any further information on this study, or other aspects of the Acas
Research and Evaluation programme, please telephone 020 7210 3673
or email [email protected]
Acas research publications can be found at
www.acas.org.uk/researchpapers
Unlocking Engagement:
A Review of the ‘Innovative Workplaces’ Initiative
Report prepared by
Lynette Harris, Alan Tuckman, Derek Watling and Bernadette
Downes
Nottingham Business School
Nottingham Trent University
March 2011
Acknowledgements
Particular thanks go to Gill Dix and Jonathan Cooper of the Acas National
Research and Evaluation Section for their inputs and positive support throughout
the evaluation process. We also wish to thank, Noel Lambert and Kate Nowicki of
the Acas East Midlands office, Peter Totterdill and Rosemary Exton of UKWON,
Vanessa Fraser-Davis of New College, Nottingham and the Acas facilitators for
their many contributions to the evaluation process. Thank you also to Christopher
Hale from Ecorys for his work on the economic impact assessment of the project.
However, our special thanks must go to the Gatekeepers and their colleagues in
the participating organisations for their invaluable contribution to the project’s
evaluation and to the East Midlands Development Agency for providing the
funding which made the project possible in the first place.
2
Table of Contents
Acknowledgements...............................................................................2
Foreword from Ed Sweeney, Acas Chair................................................6
SUMMARY.................................................................................................7
SECTION ONE – INTRODUCTION............................................................14
1.1 The Project..................................................................................... 14
1.2 Project Aims and Objectives.............................................................. 15
1.3 The Context.................................................................................... 16
1.4 The Recruitment Process .................................................................. 16
1.5 The Participating Organisations ......................................................... 17
1.6 Programme Design and Delivery........................................................ 20
SECTION TWO – THE EVALUATION.........................................................24
2.1 The Evaluation Process..................................................................... 24
2.2 Outline Methodology and Data Sources............................................... 26
2.3 Reflections on the Evaluation Process ................................................. 27
SECTION THREE – EVALUATION FINDINGS............................................28
3.1 Changes and Benefits ...................................................................... 28
3.2 Approach to Management ................................................................. 41
3.3 Benefits to Individuals in their Jobs.................................................... 42
3.4 Unanticipated Outcomes................................................................... 43
3.5 Constraints on Progress.................................................................... 44
3.6 Levels of Support ............................................................................ 44
3.7 Learning Transfer ............................................................................ 46
3.8 The Personal Development of the Gatekeepers .................................... 48
3.9 Outcome Measures .......................................................................... 50
3.10 Probability of Change without the IWP .............................................. 52
3.11 Sustainability ................................................................................ 53
3.12 Evaluating the IWP Methodology...................................................... 53
3.13 Messages for the Future ................................................................. 61
3.14 Paying for Future Programmes......................................................... 64
SECTION FOUR – ECONOMIC IMPACT.....................................................66
4.1 Summary of the Economic Impact of the IWP...................................... 66
4.2 Availability and Take-up of Alternative Sources of Support .................... 66
4.3 Improvements made by Participating Organisations ............................. 66
4.4 Gross Additional Impacts of Improvements ......................................... 66
3
4.5 Leakage, Displacement, Substitution and Multiplier Effects .................... 68
4.6 Present Value of GVA Impacts........................................................... 69
4.7 Return on Investment ...................................................................... 70
SECTION FIVE – CONCLUSION ...............................................................72
5.1 Why the Messages from the IWP Evaluation are of Value....................... 72
5.2 The Project’s Impact on the Participant Organisations........................... 72
5.3 Developing Management and Leadership Skills .................................... 74
5.4 The Value of the Different Elements of the IWP.................................... 75
References .............................................................................................77
Appendix A: Case Studies of the IWP Participating Organisations..........78
1. Brush Electrical Machines (BEM) Ltd ............................................ 78
2. Caterpillar Logistics................................................................... 80
3. The Health Store ...................................................................... 82
4. Liquid Control .......................................................................... 84
5. NHS East Midlands Strategic Health Authority (SHA)...................... 86
6. Northampton College ................................................................ 88
7. Pendragon Contracts and National Fleet Solutions (Derby).............. 90
8. Thorpe Kilworth........................................................................ 92
Appendix B: Acas short courses provided to participants.......................94
Appendix C: Anderson’s Evaluation Model ..............................................96
Appendix D: Ecorys Economic Assessment Report .................................97
4
List of Tables
Table One: Evaluation Sources and Schedule 26
Table Two: Reported key organisational achievements as a result of IWP
participation
28
Table Three: The most important reported organisational change as a result
of the IWP
32
Table Four: Changes or planned changes to HR Policies and Procedures 36
Table Five: IWP’s contribution to the development of Management and
Leadership Skills
40
Table Six: Six examples of changes in the approach to management 42
Table Seven: Summary of Gatekeepers’ reported three best outcomes from
IWP participation
49
Table Eight: Gatekeepers’ perceptions of the usefulness of the different
elements of the programme
54
Table Nine: The extent to which the key achievements would have occurred
without each element of the programme
55
Table Ten: Gatekeepers’ proposals for future programmes 62
Table Eleven: Gross additional economic impacts 67
Table Twelve: Net additional economic impacts per annum 69
Table Thirteen: Present value of GVA benefits 70
Table Fourteen: Return on Investment 70
List of Figures
Figure One: Gatekeepers’ reported levels of organisational support 45
5
Foreword from Ed Sweeney, Acas Chair
Throughout the recent recession Acas has operated more than ever as an enabler
to economic growth. Part of this role has been to provide a variety of tailored
support for organisations; to help them introduce new forms of work organisation
and improve employment relations – and in so doing to bring about increased
competitiveness and productivity.
The Innovative Workplaces Project fits within such a ‘business support’ model in
that it offered a range of in-depth support to ten diverse organisations in the East
Midlands. Funded by the East Midland Development Agency (emda) and run in
partnership with the UK Work Organisation Network (UKWON), Acas has
successfully delivered an innovative workplace support project which, as this
report sets out, has delivered many benefits. At an overall level, there was a
positive ratio of benefits to costs for the economy. At a micro level, the benefits
for participating workplaces included improved communication, innovation in
work organisation, better management and leadership and increased staff
engagement.
The findings of the project support the view of David MacLeod and Nita Clarke in
their review of employee engagement
1
- that organisations which fully engage
with their employees benefit greatly through better innovation, productivity and
performance. However employee engagement was only part of the Innovative
Workplaces story. A key strength of this project has been its practical and
tailored approach to helping organisations tackle challenging workplace problems.
The Innovative Workplaces Project was conceived in a positive economic climate
but by the start of the initiative, the economy had entered a recession.
Undertaking such an innovative project during a financial crisis might not on the
face of it seem well-timed – but these are in fact the very times when
organisations can benefit most from such business support projects. The IWP
project design – with its combination of leadership and management training, in-
company support and networking across organisations – was a pilot. Its results
tell us that the more organisations which Acas can help through similar projects
in the future, the more robust those organisations will be to survive future
economic turbulence.
This report provides a valuable resource for future reference. It maps both the
structure and outcomes of the Innovative Workplaces Project; and provides a
unique insight into the factors contributing to workplace change from the
perspective of multiple stakeholders. On behalf of Acas I wish to thank colleagues
from the Nottingham Trent University for their work in evaluating the project.
1
Macleod, D. and Clarke, N. (2009) ‘Enhancing performance through employee
engagement’
6
SUMMARY
A review of the Acas led business development initiative,
‘Innovative workplaces - Developing organisations for the future’
The overall purpose of the Innovative Workplaces project (IWP), the subject of
this evaluation report, was to offer ‘in depth’ support to some ten diverse
organisations in the East Midlands region who were seeking to change workplace
practices and increase employee engagement as a means of improving
organisational performance. This project, funded by the East Midlands
Development Agency (emda) in 2009/10, was managed and delivered by Acas in
East Midlands. The United Kingdom Work Organisation Network (UKWON) was
the delivery partner on the project and the Nottingham Trent University Business
School (NTU) was responsible for its evaluation.
The IWP approach
The IWP was innovative in a number of respects. It set out to:
? stimulate organisational change,
? be workplace focused,
? provide customised organisational support,
? develop management and leadership skills through a practical, rather than
a theoretical approach,
? provide an integrated evaluation of the impact of this pilot initiative.
The IWP offered direct support to two nominated representatives (referred to as
gatekeepers) from each of the ten participating organisations through a
combination of different organisational support interventions provided from June
2009 to May 2010, although some of these were extended to September 2010.
The gatekeepers were the focal point of the project, not least because
management and leadership was a priority for emda. The underpinning rationale
was that a handful of key individuals could successfully be the catalyst for
sustainable organisational change. The gatekeepers were therefore nominated
and selected for their ability to get things done rather than simply their formal
job role.
The IWP began with the gatekeepers undertaking a three and half day course,
spread over three months, to develop their management and leadership skills,
devise their action plans and begin to establish the interactions necessary for
successful action learning.
An Acas facilitator (and a back up facilitator) was linked to each organisation and
they helped the gatekeepers draw up their action plans. They also provided
customised practical support for bringing these action plans to life back in the
participants’ workplaces. Additional support was provided where there was an
identified need, for example through Acas training.
7
Full day monthly participant network meetings facilitated by UKWON incorporated
action learning sets in which the participants were invited to both support and
challenge one another when devising and implementing their action plans.
The action plans were pivotal as they informed the projects within each
organisation and provided the basis for evaluating the IWP’s impact. The
gatekeepers presented their plans to their fellow gatekeepers in action learning
sets on the morning of 23
rd
September 2009. That afternoon the project was
formally launched at an event featuring national and local business figures able to
pass on the benefit of their experiences of change through engagement.
The organisational benefits subsequently achieved proved to be considerably
wider than the scope of the initial action plans for many of the participating
organisations.
Changes and Benefits
All the participating organisations reported that the IWP had led, not only to the
achievement of some of the workplace changes sought in their initial plans, but
also to improvements in the wider employee relations climate. For the majority
what they had sought from participating in the IWP was achieved to a large
extent and a range of different, but frequently related, organisational issues had
been addressed; these included improving levels of employee engagement,
morale, communications between management and employees or staff in
different functional areas, workforce flexibility and the implementation of change.
Respondents from the smaller organisations were more positive and more likely
to have a shared view within the organisation about the outcomes of the IWP and
its business benefits. In the SME business context the impact of what had been
achieved was, arguably, easier to identify and more visible to the workforce. In
contrast, the two public sector organisations appeared to experience the most
difficulty in clarifying the aims and scope of their action plans at the outset,
partly due to the presence of other related, and potentially overlapping,
organisational initiatives being underway at the same time as the IWP (for
example, a leadership development programme).
The following summary identifies the reported key outcomes of the project for
the eight organisations that completed the IWP and the participant’s perceptions
of its contribution in terms of achieving the changes they were seeking. The UK
was in recession for almost all of the IWP; an economic context that had a
bearing both on continuing participation and the progress of individual
organisational projects.
Communication and Engagement
Improved communication was identified by respondents from all the participating
organisations as the ‘single most important change’ resulting from the IWP by
December 2010. This was the view of the managers, employee representatives
and the gatekeepers who took part in the project’s final evaluation. In six of the
organisations improved communication was identified as leading directly to
increased levels of employee engagement. In each organisation, improvements in
communication and employee engagement stemmed from the adoption of
8
mechanisms for capturing direct inputs from the workforce and listening to
employees’ views. Mechanisms for improving employee voice ranged from the
establishment of a workplace employee forum, to a steering group with employee
representatives with reporting task groups and/or employee focus groups
facilitated by Acas.
The IWP had resulted in seven out of eight organisations putting in place
mechanisms to promote and capture new ideas. The smaller businesses found it
rather easier to provide instances where ideas from the workforce had been
proposed, explored and implemented. At five organisations, where Acas
facilitated focus groups had taken place, respondents reported that these had
resulted in increased levels of employee engagement and a greater willingness to
contribute ideas. There was a more mixed message from some other
respondents who identified that, whilst the IWP had encouraged the sharing of
ideas, they had concerns about longer term levels of organisational interest and
engagement with the ideas put forward by employees.
The organisational benefits associated with improved communication varied
depending on the issues facing each organisation. For example, participation in
the IWP had enabled one organisation to return to levels of productive, informal
communication that had characterised the business prior to its expansion and
move to a larger premises. At another participation had led to the achievement of
one of its main aims in joining the project; a 10 percentage point improvement in
the employee engagement score in its annual company employee survey.
Managers in half of the participating organisations reported that one outcome of
their involvement in the IWP was that issues formerly referred directly to them
were now being resolved at a lower level in the management chain or by
employees themselves. This was identified by respondents as saving
management time with consequent improvements in efficiency and productivity.
These benefits were particularly identified by participants in the smaller
businesses and were seen to be the result of increased employee involvement.
For example, one SME manager, who was a gatekeeper on the programme,
reported a 75 per cent reduction in the time he personally spent addressing
workplace disciplinary and grievance issues.
HR policies and procedures
Seven of the eight organisations reported the implementation of at least one new
or improved human resource policy or procedure and all had plans for future
improvements following their participation in the IWP. The most widely reported
were improvements to processes for informing and consulting with employees
and absence management.
Workplace climate
Identifying those factors which contribute to improved morale is complex.
Notwithstanding, the majority of respondents identified that workplace morale
had improved following participation in the IWP but it was not always possible to
identify whether or not this improvement could be attributed directly to the
impact of the project. There were also a number of organisations where events
had or were taking place, frequently due to the economic climate, which were felt
9
to be impacting negatively on workforce morale; these included a pay freeze and
redundancies.
Management and Leadership Skills
In terms of improving management and leadership skills, the final evaluation
revealed that the majority of respondents felt this had happened either partly or
to a large extent as a result of their organisation’s participation. The benefits
associated with improvements in management included improved trust levels
between employees and management. This was reported by the majority of
respondents across all the organisations evaluated although this was not
necessarily a shared view across all the respondents from the same organisation.
The reasons for this varied; for example, at one organisation the level of
management and leadership skills had not been identified as an issue to be
addressed and, at another, a dispute over pay had led to internal differences
between management and employees.
Asked whether they could have obtained the support provided by the IWP from
another source, respondents in managerial roles felt this might have been the
case. However, the survey data revealed that it was highly unlikely that such
support would have been taken up with the participants in the smaller businesses
most consistently reporting that this would not have happened and that the
reported changes would not have occurred without the IWP. Most significantly,
the IWP was held to have acted as the catalyst for organisational change by the
vast majority of respondents; a view shared by both the delivery partners and
the Acas facilitators.
Economic Impact
The economic impact assessment of the IWP reported an overall minimum return
on investment of £4 for every £1 of public sector expenditure and a measurable
positive economic impact of approximately £906,000 attributable to the IWP’s
expenditure. Participating organisations attributed proportions of improvements
in staff productivity and present and future employment growth directly to the
IWP. The economic impacts quoted can be thought of as cautious estimates, due
to the fact that it was not possible to fully measure all benefits – such as
participating organisations reporting that their difficulties (often related to the
recession) would have been considerably greater without the organisation’s
involvement in the IWP, but being unable to quantify such impacts.
Evaluating the IWP Methodology
As described, there were a number of components to the IWP approach. Whilst
the impact of the IWP overall stemmed from the ‘sum of its parts’, the
customised facilitation process was regarded as particularly useful in terms of its
contribution to the final outcomes; indeed the majority reported that they would
have liked continued support from an Acas facilitator. The evaluation revealed
that the facilitation process was increasingly valued as time progressed when it
became more apparent what it had actually contributed. This may explain why a
number of participants said that they wished they had made more use of their
facilitator before the facilitation process ended and that they would have done so
10
if they had realised earlier what this could offer. From the perspective of the
facilitators themselves, it was unanimously identified that having terms of
reference agreed with senior management in place at the outset would have
enabled more rapid progress against the action plans and reduced the time they
personally spent in developing an understanding of what some of the workplace
projects were aiming to achieve.
A feature of the IWP was that it had intentionally included a diverse range of
organisations. From the perspective of the participants this diversity had the
advantage of offering new perspectives. The network events and action learning
sets facilitated by UKWON were particularly valued as an opportunity to share
experiences and their individual project’s progress but gatekeepers also reported
that learning about the practices adopted by other participating organisations
tended to have limited relevance to their own situations because of the differing
nature of their businesses or sectors. One suggestion from gatekeepers at a
couple of the participating organisations was that having access to a mentor from
a similar industry or sector would be a valuable addition to the interventions
offered by the project.
The initiative achieved changes in a relatively short period of time, particularly in
terms of improving employee engagement through enhanced communication in
many of the participating organisations. The evaluation findings suggest that this
is most likely due to the mix of interventions that were offered which could be
adapted to suit the needs of individual organisations and their representatives.
This approach worked particularly well where there was a gatekeeper
representing the organisation who had the ability and sufficient influence in their
workplace to progress their project, particularly when there were obstacles to be
dealt with.
The programme was designed to be sufficiently flexible to accommodate the
needs of the different organisations, their emergent action plans and the
development needs of their gatekeepers. This meant that its interventions were
loosely structured but some gatekeepers reported that they would have
welcomed more delivered inputs which provided specific information that they
could take back to their organisation, for example, the presentations provided by
the speakers at the highly successful IWP launch event on employee
engagement.
Perceptions of the value of each element of the programme varied for each
organisation and the individual gatekeepers depending on what they were
seeking. The need to accommodate this diversity was recognised by UKWON and
had informed the design of the elements they provided. Their aim was to ‘ground
pre-existing knowledge in the task at hand’ for gatekeepers with prior knowledge
and experience and ‘to provide sufficient actionable knowledge on employee
involvement and participation’ for those with less knowledge and experience. The
evidence from the evaluation is that this was largely achieved. But the overall
message from the IWP is that it was essentially the combination of the different
elements that led to the final outcomes for both the participating organisations
and the gatekeepers. In practice, the participants took what they needed at
different times from the IWP.
11
Levels of support for the IWP
Senior management of the organisations signed the application forms for IWP to
register their support for the project but it was always intended that the
organisation gatekeepers and Acas facilitators would have to work together to
build support for the project within the organisations, including from senior
management. Levels of support for the IWP from senior management, and from
some employees where there was identified low morale, emerged as a particular
concern for over half the nominated gatekeepers in the initial interviews. But, the
final evaluation revealed a positive message about the actual levels of support for
the project from senior managers, middle managers and employees. In four
organisations it was reported that management support had grown as a result of
the inputs from the Acas facilitation process and as the benefits stemming from
the IWP became apparent. The degree of senior management support, and
ensuring their early engagement, were identified as critical factors in progressing
organisational projects and in achieving their aims, not only by the gatekeepers,
but also by the delivery partners and the Acas facilitators in their final reflections
on the IWP.
Sustainability
There was clear evidence in the final evaluation that respondents increasingly
viewed the workplace achievements resulting from their involvement in the IWP
as part of ‘on going’ programmes of change. However, a third of the
organisations were concerned that, without the IWP, the momentum would not
be sustained. This was particularly reported where there had been a loss of
individuals who had championed the project due to staff turnover.
Looking to the Future
Asked about paying for a similar programme to the IWP in the future, only the
larger organisations felt this might be a possibility but that this would depend on
factors such as the economic climate, their ability to pay and (for the private
sector companies) evidence that such an intervention would increase turnover.
Despite identifying the most positive outcomes from the IWP, the smaller
businesses reported that it was very unlikely that they would be able to pay for
such a programme. Notwithstanding, all the participants identified that if they
were to consider such a project in the future, it would be important to them that
it was provided by Acas on the grounds of its reputation and impartiality.
Overall, the evidence from the evaluation reveals that the Innovative Workplace
Project led to improvements in the participating organisations, predominantly in
the areas of workplace communications, employee involvement and engagement
as well as other dimensions of employee relations which impacted positively on
their organisation. The outcomes of the IWP lend support to the key conclusion
of MacLeod and Clarke’s 2009 review of employee engagement for the
Department of Business; that improving employee engagement can improve
business performance. To varying degrees, the IWP also met its objective of
developing leadership and management skills and was a vehicle for the
identification of skills gaps in several organisations. Furthermore, the economic
impact assessment analysis revealed that these changes, when aggregated,
12
generated a positive return for the organisations as a whole in difficult economic
times.
A key message from the evaluation of the IWP is that similar future initiatives,
customised to meet the needs of individual organisations, could offer valuable
support to employers who are seeking to engage their work forces and work
collaboratively in initiatives to improve efficiency and changes to working
practices.
13
SECTION ONE – INTRODUCTION
1.1 The Project
The purpose of this report is to provide an independent evaluation of an Acas
initiative ‘Innovative Workplaces - Developing Organisations for the Future’
(referred to hereafter as ‘IWP’) which took place during 2009 and 2010. The
overall aim of the initiative was to offer in-depth support to a number of
organisations in the East Midlands region who were seeking to change workplace
practices and increase employee engagement as a means of improving
organisational performance. The project was funded by the East Midlands
Development Agency (emda), and managed by Acas in the East Midlands region.
Acas also delivered the project programme in collaboration with the United
Kingdom Work Organisation Network (UKWON), its appointed delivery partner.
Additional strategic input was provided by the Acas National Research and
Evaluation Section (RES) and Acas’ Academic Adviser Professor John Purcell. The
East Midlands Employment Relations Forum (ERF) steering group acted in an
advisory capacity throughout the project.
Over a period of twelve months, the Innovative Workplace Project (IWP) provided
a number of different interventions to ten participating organisations. The IWP
began in April 2009 with the recruitment and selection of organisations seeking
to participate. The core elements of the IWP were: an initial three and half day
short course designed both to develop leadership and management skills and to
build trust and dialogue between participants; monthly meetings of the
participants; and facilitation by senior Acas advisers to provide customised
workplace support. A key requirement of the initiative was that there should be
an independent evaluation of its impact in terms of the experiences and
outcomes for the participating organisations and an identification of lessons
learnt for the purposes of wider dissemination. In this sense, the IWP was
regarded as a pilot. Nottingham Trent University was commissioned to lead the
evaluation. The independent research agency, Ecorys, was appointed towards the
end of the IWP to undertake an analysis of the economic impact of the project
which used, and complemented, the data collated by the Nottingham Trent
evaluation
1
.
The report sets out the key findings from the evaluation which culminated in final
review interviews focusing on the outcomes for the participating organisations
and individuals (designated as the ‘gatekeepers’). These final interviews took
place during November and December 2010, six months after the conclusion of
the main elements of the IWP.
The report is divided into five sections. Section One outlines the IWP’s aims and
objectives, the design and rationale for the different elements of the delivered
programme, the participating organisations and what they were hoping to gain
from the IWP. Section Two details the evaluation process. Section Three reports
on the outcomes for the participating organisations, the individual gatekeepers
and their evaluation of the IWP programme. Section Four provides a summary of
the economic impact of the IWP and Section Five identifies the main conclusions
1
A summary of the economic impact analysis is in section four, and the full Ecorys report
is in Appendix D.
14
that can be drawn from the project in terms of its impact and any wider
messages that can be drawn from its outcomes which could inform similar
interventions in the future.
To avoid confusion in the report’s terminology, the overall initiative is referred to
as the project or the IWP whereas the delivered elements provided by the project
are referred to as ‘the programme’; ‘individual projects’ refers to those initiatives
that took place within the participating organisations and their nominated
representatives on the programme are described as the ‘gatekeepers’. As this
was the descriptor used throughout the life of project, it has the merit of being
familiar to all those involved when it came to evaluation. Where ‘respondents’ are
referred to in the report this includes all the individuals, including the
gatekeepers, who were interviewed as part of the evaluation process in the
participating organisations.
1.2 Project Aims and Objectives
The IWP’s overall purpose was to enable a diverse ‘pilot’ group of organisations
from across the East Midlands to increase profitability and competitiveness, as
well as improving working lives, through the development of enhanced leadership
competencies, new forms of work organisation and increased employee
involvement and engagement.
The IWP builds on research evidence that employee involvement and
participative forms of work organisation contribute to organisational performance
and enhanced competitiveness (Purcell et al., 2003; Boxall and Purcell, 2007;
Ashton and Sung, 2002). Existing research, reviewed by Sisson (2005) in a
report for emda, indicates that the return on investment in skills development is
not fully realised unless firms develop working practices based on job enrichment
and employee initiative. Sisson argues that the term “workplace innovation”, part
of the project’s rationale, is commonly used to describe the introduction of
workplace practices that are new to the company itself and which have to be
reinvented within each organisational context. It is this definition of workplace
innovation that is reflected in the project’s title.
The initial aims and objectives developed by Acas and emda for the IWP were to:
? Facilitate sustainable organisational change and improve business
performance by focusing on the development of essential management
and leadership skills, better work organisation and employee involvement
and engagement;
? Capture a body of evidence by gathering the stories from each of the
intended ten participating organisations, to provide a guide to effective
organisational change drawn from practice;
? Share the lessons learned and the outcomes achieved in order to
encourage other organisations both in the East Midlands and nationally to
adopt similar interventions;
? Influence policy makers and stakeholders as to the importance of
management and leadership skills development, work organisation and
employee engagement for business productivity;
? Provide an example of how Acas, working in partnership, can improve
business productivity and working lives in a sub-national economy.
15
1.3 The Context
The initiative was conceived of prior to the published findings of the MacLeod
Review on Employee Engagement (MacLeod and Clarke, 2009). But MacLeod’s
findings together with the principles of the Acas Model Workplace (2005) and
UKWON’s international involvement in workplace innovation initiatives provided a
backdrop for the IWP’s design and the delivery of its different elements. By the
time recruitment for the IWP began in 2009, the UK economy was in a recession
which deepened during the life of the initiative; a factor which impacted to
varying degrees on the participating organisations and the individuals selected as
the gatekeepers on the IWP. For example, one company had to withdraw when it
went into receivership; another went into receivership with a subsequent
management buy out; at another, a gatekeeper was made redundant. This
uncertain and challenging environment needs to be taken into account in any
consideration of the initiative’s final outcomes. It was encouraging that, despite
the various pressures experienced by the participating organisations, there was a
continuing commitment to their action plans and to their involvement in the
project.
1.4 The Recruitment Process
In Spring 2009, the opportunity to participate in the IWP was widely advertised
through emda, Acas and UKWON with a series of open access familiarisation
sessions for organisations interested in learning more about the initiative.
Following a process of application, a number of organisations from across the
East Midlands region were invited by Acas to face-to-face discussions during May
and June 2009. The Project Manager was variously joined by Acas and UKWON
colleagues for the discussions which took place with a mix of senior managers,
potential gatekeepers and HR professionals from organisations interested in
participating. Organisations were asked in the discussions to set out their overall
objective in seeking to join the IWP; why they felt it would benefit them; and to
demonstrate their commitment to engaging and staying with the IWP from start
to finish. The latter was an especially important criteria in determining which
organisations would be invited to become involved. The outcome of these
discussions was that eleven organisations were recruited to participate in the
IWP. These represented considerable diversity in terms of their size, sector and
geographical location across the region.
Two employees were nominated as gatekeepers by each organisation to attend
the programme and act as the catalyst in taking their plans for action forward
with the support provided by Acas and UKWON as the delivery partners. The
suggested criteria for selecting the gatekeepers were that they should be
proactive individuals who ‘would get things done’. One company decided to
withdraw from the programme at the beginning of the initial short management
and leadership course. The ten remaining organisations, the size of the workforce
at the workplace involved in the IWP and what they were seeking from their
participation are described briefly below. Further details of those organisations
that completed the IWP can be found in the short case studies in Appendix A.
16
1.5 The Participating Organisations
Brush Electrical Machines Ltd (BEM), a manufacturer of generators for steam
and gas turbines based in Loughborough, Leicestershire with 800 employees at
the start of the project. Following a recent change of ownership, the company’s
aim in participating in the IWP was to improve morale, trust and employee
engagement. It was hoped that its participation would assist BEM to become a
more innovative work organisation and reduce resistance to change. At the time
of joining the IWP, the company’s particular concerns were that, whilst the
workforce was highly skilled, there was a need to develop a less functional, more
process driven perspective among the workforce to take the organisation
forward. The nominated gatekeepers were a Business Analyst and a Senior Unite
Trade Union representative. Participation in the IWP was first suggested by a full
time Unite Official and was the only instance where a gatekeeper was also a
workplace trade union representative.
Caterpillar Logistics, a wholly owned subsidiary of Caterpillar Inc. providing
third party warehousing and distribution and other services. At the start of the
project it had 87 employees at its Hinckley site where the project took place. The
Hinckley operation resulted from a takeover of a Caterpillar customer. The
Company’s key aim for the project was to specifically improve upon the level of
employee engagement, reported as 62% in its 2008 employee survey, to a target
of 72% or more through improvements to employee participation and
cooperation at all levels. At the time of applying for the IWP, issues of
communications at the site had been identified as being aggravated by the
presence of different cultures at the Hinckley site resulting from the takeover.
Several changes in employer were identified as contributing to a high degree of
workforce scepticism with regard to change, new ideas and initiatives. The
nominated gatekeepers were the Logistics Centre Manager and an HR Manager
who also had responsibilities across three other Caterpillar sites.
Donington Park Leisure (DPL), an events management company known for its
race circuit events for motor sports and ‘would be home’ of the British Formula
One Grand Prix with 33 employees based in Castle Donington, Derbyshire. Its
rationale for participating in the IWP was to support its aim of being at the
forefront of customer service through an investment in staff and ultimately to be
‘Investors in People’ accredited. The company was particularly seeking to
enhance customer service through improvements to employee engagement and
internal communications. Its nominated gatekeepers were the Sales Manager and
the HR Manager. Unfortunately DPL had to leave the project as it went into
administration in November 2009 following its unsuccessful bid to host the 2010
British Formula One Grand Prix.
Liquid Control, a builder and supplier of standard and custom built machines for
processing (metering, mixing and dispensing) single and multi-component
liquids/pastes based in Northamptonshire with a workforce of 20 employees. At
the time of applying to join the IWP, the company was looking at ways to
improve the way the business was managed. It had identified that productivity
and efficiency gains were needed to sustain the business and the IWP was seen
as a means of increasing workforce productivity and efficiency through greater
employee engagement. The nominated gatekeepers were the Engineering
Manager and the Service Manager.
17
MAHLE Powertrain, an engineering and consultancy firm developing
technologies and products for reducing fuel consumption which also offers
traditional engineering services for the automotive and engine industry. With 300
employees at its operations in Northampton where their project took place,
Mahle’s aim in participating in the IWP was to retain skilled staff in difficult
economic times through increased empowerment and employee voice and to
improve morale, trust and a sense of recognition among the workforce. Mahle
intended to establish a cross functional employee forum to improve two way
communications. Progress with their project was delayed due to short time
working and redundancies at the site as a result of the impact of the recession on
the business. The nominated gatekeeper was a Senior Principal Analysis
Engineer; there were initially two gatekeepers but one of the gatekeepers left the
IWP not long after it began due to redundancy. Concerns about commercial and
organisational sensitivities resulted in an executive decision by the company in
April 2010 that they would not continue to participate in the IWP.
Notwithstanding, their project continued internally and the organisation made a
contribution to the final evaluation of the IWP.
NHS East Midlands, the strategic health authority (SHA) for the region
providing leadership of the NHS to ensure that health systems operate effectively
and efficiently for a population of 4.5 million spread across Derbyshire,
Leicestershire, Lincolnshire, Northamptonshire, Nottinghamshire and Rutland.
Established in July 2006 with a role to relay and explain national policy, set
direction and support and develop all NHS Trust bodies, its workforce of 350
employees are mainly based at the SHA’s Headquarters in Sandiacre,
Nottinghamshire where the project took place. The SHA’s initial aim in
participating in the IWP was to maximise the benefits of electronic staff records
(ESR) by transferring ownership to individuals and providing high quality
information for employees and line managers. The intention was that, if this led
to a greater staff utilisation of the SHA’s Oracle Learning Manager (OLM), it would
support the implementation of the organisation’s Talent Management Framework
and NHS East Midlands’ commitment to “grow its own” leaders and managers for
the future. It was identified at an early stage in the project that communication
was a key issue and this became the critical focus. The nominated gatekeepers
for the IWP were a Business Manager in Public Health (who left during the IWP
for a period of maternity leave) and the Workstream Leader (Planning and
Projects) in the Information Management and Technology Department.
Northampton College, a provider of Further and Higher Education to around
12,000 learners split over four sites in Northampton which employs some 1,000
employees. The project took place across all four sites. The College’s specific aim
in participating in the IWP was to attain an improved Ofsted grade by 2013. They
saw the IWP as providing an opportunity to achieve the internal culture change
identified as necessary to improve the 2009 ranking of Satisfactory. There were
challenges facing the College in terms of its student success rates, developing
management and leadership capability with little history of centralised strategy
and corporate coherence leading to long established local practices. In addition,
the college had a number of recognised trade unions resulting in a range of
consultative processes. An additional challenge for the college, at the time of
joining the IWP, was that it was in the early stages of demolishing and rebuilding
its main campus without relocating its operations. The nominated gatekeepers
were the Manager for Professional Development and a Management
Lecturer/Team Co-ordinator.
18
Pendragon Contract Hire and Fleet Solutions is the commercial and
contracts leasing division of an automotive retail network for new and used
vehicles also trading as Stratstone, Evans Halshaw and Chatfields. This part of
the business, located in Derby, employed 98 staff at the start of the IWP. Due to
the economic climate and its impact on the industry, there had been a series of
restructures and redundancies within the company which had led to feelings of
uncertainty in terms of job security and a resultant loss of employee engagement
which impacted on overall business outcomes. The company’s aim in
participating in the IWP was to improve employee engagement primarily through
the establishment of a consultative forum to gain the views of the workforce.
Having completed its first employee survey shortly before applying to join the
IWP, it was seeking to increase the 50% response rate and the level of positive
responses. The IWP was seen as a means of improving employee relations and
moving the business forward by developing better people management. The two
gatekeepers nominated for the IWP were the Customer Services Director and the
HR Manager.
The Health Store, a customer owned cooperative health food wholesaler and
distribution centre for its members, with a production unit producing own label
dried goods based in Nottinghamshire which employs 102 employees. The
company’s aim in participating in the IWP was to gain a forward thinking and
trained workforce able to contribute to the future of the business through
improved employee engagement and improved communications between
management and employees. At the time the company joined the IWP it had
identified that morale had dropped since moving, in 2007, to its new, purpose-
built premises, and was seeking to improve this as a result of improved
communications between management and employees. The nominated
gatekeepers were the Operations Director and the Warehouse Manager. One of
the gatekeepers ceased his formal participation in the programme at the time of
the initial short course but remained very involved in the internal project within
the workplace.
Thorpe Kilworth, a designer, manufacturer and installer of furniture for
education, laboratory and health care establishments with 99 employees based in
Corby, Northamptonshire. At the time of joining the IWP, the company had
identified the need to be more responsive to changes in the market and the
needs of its clients. Its aim was to use their project as the means of addressing
these issues through increased workforce versatility and flexibility enabling them
to move individuals from one process to another in order to meet fluctuations in
demand. The aim was to improve the company’s competitive edge by maintaining
quality but reducing waste and inefficiencies. One particular focus was to improve
employee engagement in a workforce viewed as loyal and responsive but with
concerns about changing long established ways of working. Improving two way
communications so that the workforce understood and were committed to
changes in the business was seen as a key factor in increasing employee
engagement. The nominated gatekeepers for the IWP were the Organisational
Development Manager and the Production Manager. Thorpe Kilworth went into
administration in June 2010 but had completed the IWP and continued to be
included in the evaluation process.
19
1.6 Programme Design and Delivery
The Programme Design
Acas and UKWON’s stated principles for the design of the Innovative Workplaces
project were as follows:
a) The effectiveness of support for companies individually (through facilitation or
consultancy) is considerably enhanced by group-based learning and
knowledge exchange combined with peer-review of change proposals and
implementation processes.
b) Participative work organisation and employee involvement can lead to
convergence between improved performance and improved quality of working
life.
c) Change Entrepreneurs able to instigate dialogue, mobilise diverse actors and
work between formal organisational structures can play a key role in securing
effective and sustainable change. But to be effective in this role, individuals
need to be able to see themselves as entrepreneurial and to receive high-
level support even when they challenge established practice.
These design principles led to the ten participating organisations receiving three
strands of support:
1. An initial short course of three and half days delivered over three months
which was designed to enable participants to learn about good practice,
develop their leadership skills, evaluate their own organisations with
reference to the Acas Model Workplace and to formulate an action plan for
change.
2. Six full day participant network meetings consisting of half-day thematic
inputs and discussion on shared issues and problems and a half-day of action
learning sets in which each organisation presented its progress,
achievements, obstacles and dilemmas for peer review and discussion. There
were two additional half day meetings of the action learning sets.
3. Acas facilitation for a period of up to eleven days where an Acas Senior
Adviser gave practical in-company support and Acas advice and guidance.
This was supplemented by Acas short courses and further inputs from UKWON
where a particular need was identified.
The Acas Project Manager was responsible for the coordination of the different
elements and monitoring progress in the participating organisations. The latter
was undertaken through the Acas facilitators and UKWON, who between them
were in regular contact with all the participants. Taking the above core
objectives, and the design principles together, the different elements of the IWP
were each designed to achieve a range of objectives, as set out below:
Build Commitment
Building and maintaining the commitment of the organisations’ nominated
participants was an important strand. At the outset the two gatekeepers from
each organisation took part in the short, action-learning based programme. The
course, designed and extensively piloted in a previous project by UKWON, was
delivered over a period of weeks by New College, Nottingham and provided
optional Institute of Leadership and Management accreditation for the
20
participants. Each organisation devised and refined its action plan during this
stage of the programme. Plans were then presented by each gatekeeper and
subjected to scrutiny by their peers during the final half day of the course on the
morning of 23
rd
September 2009.
During the afternoon of 23
rd
September, the IWP was publically launched with an
inception event. A range of guests from the participant organisations, including
some senior managers and employee representatives as well as the gatekeepers,
the project managers and delivery partners and the Acas facilitators attended the
event. The key speakers for the event included David MacLeod, co-author of the
MacLeod Review on Employee Engagement (MacLeod and Clarke, 2009), Warren
Glover of Lindum Construction (an East Midlands employer from the Times Top
100) and John Purcell (Strategic Adviser to Acas with a specialism in the link
between employee involvement and organisational performance). The event, as
well as providing an opportunity to publicise the initiative, was also designed to
provide further stimulus to the project participants in preparation for their return
to their workplaces to implement their action plans. An end of project
dissemination event took place on 20 October 2010; this was an interactive
symposium on employee engagement with guest speakers and inputs from the
gatekeepers about the IWP’s impact on their organisations and their own
development.
Facilitate Change
Identifying and facilitating change, as fitted the individual organisations’
priorities, were core objectives. In their role as facilitators the Acas Senior
Advisers provided practical in-company advice and guidance in accordance with
the Facilitator’s Brief. In all cases this began with initial meetings to explore and
discuss the action plans. The Advisers used their skills and experience to
facilitate the process of turning ideas and aspirations into practical strategies.
This facilitation role was core to the relationship between the Senior Advisers and
the organisations. In many cases the initial advice was followed by diagnostic
workshops and focus groups, facilitated by the Senior Adviser. These provided
powerful information and gave a sound basis for the direction which the
individual projects then took. As well as initial advice and workshops, Acas also
gave advice on setting up consultative forums, on staff surveys, and on wider
policy development. The contact with the Acas Adviser provided a central core of
support which was supplemented by additional Acas training (see Appendix B).
As supplements to the core programme, UKWON provided additional guidance to
Northampton College, at a point where greater clarity was needed, and on work
organisation to Thorpe Kilworth. Acas made its open access training courses
available to all the organisations. They were developed and provided in
consultation with the participant organisations. Details of this provision and
attendance are provided in Appendix B.
In some cases the Acas Senior Adviser provided specific in-company bespoke
training to support individual projects. Pendragon received training on ‘Essential
Skills for Supervisors’ and ‘Training for Workplace Representatives’ and
Caterpillar and The Health Store each received bespoke training for their new
employee representatives.
21
Provide Peer Learning and Review
Gatekeepers took part in monthly network meetings designed to provide a
greater depth of understanding in relation to specific aspects of work organisation
and employee involvement. The network meetings aimed to facilitate the
exchange of knowledge and experience between participants. The content of
these meetings was, as far as possible, responsive to needs expressed in the
action learning sets and issues raised by the Senior Acas Advisers. UKWON
organised and facilitated the network meetings which were also attended by the
Acas Project Manager.
In the afternoons, following the network meetings, action learning sets facilitated
by UKWON enabled participants to reflect on progress and refine their action
plans based on the exchange of ideas between gatekeepers and peer review.
The programme structure provided a framework within which the gatekeepers
could reflect on the strengths and weaknesses of existing practices in their
organisations, learn from and crucially challenge each other, test ideas and
proposals in a safe and supportive environment, and share both problems and
achievements as their work progressed.
Evaluate the Project
Evaluation was a key element of the IWP and it was a requirement of the emda
funding that there was an evaluation of the IWP’s outcomes. In practice,
evaluation processes were woven into all aspects of the delivery, diagnostic and
learning processes with a view to identifying:
a) the impact of the IWP (including its economic impact) from multiple
perspectives within the organisation, to include specific benefits to
participants and their organisations and unforeseen outcomes;
b) the effectiveness of the development interventions; for example, the
workplace facilitation and the network forums from the perspective of the
participants;
c) information for dissemination about an innovative project based on
partnership working, particularly about any ‘lessons learnt’ for future
interventions, for organisations, delivery partners and other regions
considering similar projects.
Disseminate the Lessons Learnt
There was a focus throughout the IWP to capture both its outcomes and
experiences of the participants to:
? enable other organisations in the East Midlands and nationally to follow
suit;
? publicise the approach and achievements of the individual organisational
projects to policy makers, strategists and stakeholders regionally and
nationally;
? provide opportunity for the region, the organisations, participants and
those involved in the delivery to share their experiences of an innovative
project which champions a partnership approach.
22
? aside from a final written report, the main intended vehicles for the
dissemination process were:
o regionally the Acas Employment Relations Forums in the East
Midlands, a themed local event;
o internal Acas meetings, such as senior management team
gatherings;
o ad hoc meetings with policy makers, strategists and stakeholders;
o short filmed summaries including a project overview from the
managers and delivery partners, and short (3-4 minutes) filmed
accounts of the work and impact of the IWP in some, if not all, of
the participant organisations;
o short case studies of the participating organisations;
o the Acas website, particularly the East Midlands pages.
23
SECTION TWO – THE EVALUATION
2.1 The Evaluation Process
The evaluation element of the IWP was designed to reflect the different
approaches to assessing learning value contribution based on Anderson’s model
(2007) of value and evaluation set out in Appendix C. There was a particular
focus on evaluating
? the return on expectations in terms of the extent to which intended
organisational outcomes were realised
? the economic impact/return on investment through a range of
performance indicators
? the extent to which the wider aims of the intervention had been achieved,
? the efficiency and effectiveness of the learning and development process
and activities.
A ‘cornerstone’ of the monitoring/evaluation approach was obtaining a ‘pluralistic
perspective’ of each organisation’s individual project and its outcomes. As a
result interviews took place with a range of stakeholders at each participant
organisation in addition to the nominated gatekeepers at the beginning of the
project and six months after the programme had finished. These usually included
a senior manager and/or line manager, an HR manager and an employee
representative.
A multi-method research design was adopted to generate both qualitative and
quantitative data in order to evaluate the IWP’s overall impact against its four
overall aims. This required an evaluation of a number of specific outcomes from
different stakeholder perspectives with a particular focus on:
? the organisational changes from participating in the IWP, including any
unforeseen outcomes;
? the development of the individual gatekeepers;
? the extent to which skills and knowledge had been transferred from the
gatekeepers to others within the organisation;
? the extent of sharing learning and knowledge between the gatekeepers on
the programme;
? the effectiveness of the different development interventions provided by
the programme from the perspective of the participating gatekeepers;
? the lessons learnt from the IWP in terms of what worked well and less well
to help for the purposes of wider dissemination;
? the cost/benefits to the participating organisations and a set of questions
designed specifically to calculate the economic impact of the IWP.
The Evaluation Stages
The evaluation consisted of the following four stages.
Stage One
At the beginning of the programme, short postal questionnaires were completed
by each participating organisation (in most instances by an HR Manager) to gain
initial baseline data. Once the participating organisations had drawn up their
24
action plans, face-to-face interviews were conducted in October 2009 to identify
different organisational stakeholders’ expectations of the organisational benefits
to be gained from involvement in the IWP, and their perspectives on the
proposed changes in their organisation’s action plan. This data formed the basis
for the final evaluation of the IWP’s overall impact. In addition face-to-face or
telephone interviews were conducted with all the delivery partners to explore
their expectations for and perspectives on the IWP and its objectives.
Stage Two
The analysis throughout the IWP of any feedback data on the delivered elements
of the programme which included feedback forms from the initial short course,
participant network meetings (which included the half day action learning sets)
and any supplementary Acas short course attendance. It had been intended to
also gain information about the learning of participants and how this was being
shared and applied in the workplaces through the maintenance of learning logs
while they were undertaking the IWP. In practice, these learning logs were not
completed although a couple of gatekeepers had maintained a log of their
personal learning.
Stage Three
Telephone interviews at the end of the facilitation process (in all but one
organisation) with all the gatekeepers to obtain an evaluation of progress against
their action plans and their experiences of the different elements of the IWP.
These interviews took place in May 2010 and referred back to the participants’
initial plans and expectations. In addition, telephone interviews were conducted
with the Acas facilitator for each participant organisation. These interviews were
supplemented by reflective summary reports provided by each member of the
Acas facilitation team.
Stage Four
The final stage of the evaluation took place during the last three months of 2010,
approximately six months after the programme delivery had finished.
Questionnaires, completed by the participant organisations, revisited the areas
addressed in the base line survey undertaken at Stage One and included a
number of questions designed specifically to evaluate the economic impact of the
organisations participating in the IWP at an organisation and economy-wide level.
Final interviews were conducted in November/December 2010 with, wherever
possible, the same individuals at each organisation interviewed at Stage One.
End of project interviews were also conducted with the UKWON and Acas delivery
partners which included the Acas Project Manager. The key purpose of the final
evaluation stage was to identify the overall impact of the IWP for the participant
organisations and the individual gatekeepers. The data collated was compared to
the data provided at Stage One. The aim was to explore the extent of changes,
achieved or ongoing, as a result of the IWP.
A key challenge for the evaluation process was that there had been other
changes at a number of the participant organisations, including staffing, whilst
undertaking the IWP. Acknowledging that many outcomes/impacts would be
ongoing and realised in the future, the optimal time chosen to balance the
reporting deadline, the availability of data and allowing sufficient time for change
to take place was six months after the delivered elements of IWP had largely
finished.
25
2.2 Outline Methodology and Data Sources
Various methods were adopted to measure the outcomes of the IWP. Using the
range of instruments summarised in Table One, and supplementary sources of
data generated by the IWP (for example the Acas Facilitators’ reports) helped to
secure more robust evidence for the analysis of organisational impact as well as
provide opportunities for the triangulation of certain results. Qualitative and
quantitative, baseline and post-project data drawn from questionnaires were
inputted into a simple database for analysis. Interview data was transcribed and
coded for scrutiny using Nvivo software for qualitative analysis.
Table One: Evaluation Sources and Schedule
Evaluation Research
Instrument
Source N Date
Baseline Pre-project
Questionnaire
One competed by each
participating organisation
10 September 2009
Initial Interviews in
Participant Organisations
Gatekeepers, Manager, HR
Manager, Employee
Representative at each
participating organisation.
38 October 2009
Initial Interviews with
Project Manager,
Delivery Partners
Acas, UKWON, Course
Tutor (New College) and
emda
6 November 2009
Telephone Interviews
with Gatekeepers at the
end of project delivery.
Gatekeepers at each
participating organisation
13 April/May 2010
Facilitator Telephone
Interviews
Acas Senior Advisers acting
as facilitators
5 June 2010
Final interviews with
Project Manager and
Delivery Partners
Acas and UKWON 4 September/
October 2010
Final Post-project
Questionnaire
One completed by each
participating organisation
8 November 2010
Final Interview Schedule
Gatekeepers, Manager, HR
Manager, Employee
Representative at each
participating organisation
where possible
29 October/
November 2010
Supplementary Data Sources:
? Gatekeeper Feedback Questionnaires on ILM course
? Gatekeeper Feedback Questionnaires on Participant Network and Action
Learning Meetings
? Evaluator Observation at Acas Facilitators’ Meetings
? Facilitators’ End of Project Reports
? Organisational Action Plans
26
Table One also provides a summary of the sources of data that were used and
when these were obtained. One major challenge for the evaluators was that
during the period of the project there were organisational changes and events
(for example redundancies, maternity leave or staff leaving) which impacted on
those involved in the project in each organisation. The differences in the number
of respondents in the initial interviews and those who took part in the final
interviews is partly due to the reduction from ten to eight organisations but is
also due to these changes in personnel. For example the 21 gatekeepers
interviewed at Stage One of the evaluation had reduced to 13 by the final
evaluation interviews. The research instruments can be requested by interested
parties from Jonathan Cooper or Gill Dix in the Acas Research and Evaluation
team.
2.3 Reflections on the Evaluation Process
In terms of reflections on the evaluation process, there were a number of
learning points for future evaluations of similar projects.
? Earlier knowledge of the participating organisations and their individual
action plans to develop the evaluation team’s understanding of each
organisation’s context and issues. This would have been assisted by the
evaluation team being present when the organisations presented their
individual actions plans at the launch event and would have provided an
early opportunity to gain ‘the buy in’ of the participating organisations and
individuals into the evaluation process (Guba and Lincoln 1989).
? A comprehensive briefing on the rationale for and the nature of the
evaluation strategy for all the nominated gatekeepers at the outset of the
IWP would have assisted the evaluation process and helped to address
any concerns of individual gatekeepers about its purpose.
? The research design was based on evaluating participants’ learning logs
and observing the action learning elements of the programme. This was
not possible with the result that it was more difficult to report on the
gatekeeper learning that took place and how this was applied during their
participation in the IWP.
27
SECTION THREE – EVALUATION FINDINGS
3.1 Changes and Benefits
At the end of their initial short course, the participating companies presented
their action plans at the IWP launch event in September 2009. Whilst these plans
informed the projects within each organisation and provided the basis for
evaluating the IWP’s impact, the reality was that the organisational changes
reported as a result of participation in the IWP were wider than those sought in
the initial action plans. These changes are examined throughout this section and
the key achievements reported by the respondents at each organisation in the
final interviews and the post-project questionnaires, are summarised in Table
Two below.
Table Two: Reported key organisational achievements as a result of IWP
participation
Organisation Initial Action Plan Reported Achievements
Brush Electrical
Machines Ltd
(BEM)
? Improve two way
communication
? Enhance managerial
awareness of the employee
perspective
? Improve employee
awareness of management’s
perspective
? Establishment of a steering
committee and focus groups
? The introduction of a company
newsletter to assist
communications
? Better equipped to meet the
challenges of an increasingly
difficult economic climate
Caterpillar
Logistics
? Introduce measures to
enhance employee
engagement
? Increase the employee
engagement score in the
company employee survey
by 10 percentage points
? Improve communication
between different groups of
staff
? Establishment of an employee
forum
? Improved communication
between staff groups
? Changes to the application of
the absence policy
? Employee engagement score
improved by 10 percentage
points
Liquid Control ? Develop workforce flexibility
? Identify skills gaps and
employ apprentices to fill the
gaps left by employees due
to retire
? Obtain ISO 9001 by the end
of 2010
? Undertake a Stress
Questionnaire amongst
employees
? Workforce skills analysis
? Introduction of developmental
appraisals for all employees
? Workforce training which has
increased flexibility
? Recruitment of apprentice(s)
? Implementation of an
employee engagement survey
? The introduction of quarterly
company meetings
? The introduction of weekly
departmental meetings
28
Northampton
College
? Initial action plan - to
enhance Leadership and
Management capability
? Later action plan – to
address issues of employee
consultation, communication
and involvement
Outcomes from the IWP are still
evolving. It is reported that they
are likely to be:
? Enhanced employee
involvement
? Development of leadership
skills among managers at all
levels
? The introduction of joint
problem solving task groups
Pendragon
? Improve team member
engagement
? Encourage better team
participation and
departmental interaction
? Improve customer Service
? The establishment of a
employee forum
? Introduction of team building
events
? Improved employee
engagement
? Improved team member
(employee) communication
throughout the division
? Review and revision of
‘housekeeping’ policies and
practices
? Introduction of monthly team
leader meetings
? Re-introduction of a customer
service survey
Strategic Health
Authority
(NHS East
Midlands)
? Engage staff to maximise the
use of the Electronic Staff
Records System (ESR)
? Transfer ownership of
personal data to individuals
? Enable managers to better
maintain employee data
? Reduce levels of data
handling to enhance
administrative efficiency
? Improved facility for ‘employee
voice’
? Increased staff usage of the
ESR
? Increasing staff ownership of
personal development
? More accurate HR information
? Improved reliability,
productivity and efficiency in
the handling of personal data
The Health Store ? Increase employee
engagement
? Improve two way
communication
? Establish an employee forum
? Elect employee
representatives
? Encourage employee
suggestions for innovation
? Elected and trained employee
representatives
? Establishment of a joint
management and employee
forum (production and
warehouse areas)
? Employee representatives
attendance at monthly
management meetings
? Improved workplace
communication and morale
and employee engagement
? Significant decrease in the
number of disciplinary cases
? Improved working practices as a
result of employee suggestions
29
Thorpe Kilworth
? Improve the company’s
competitive edge
? Improve manufacturing
efficiency
? Enhance employee
engagement and
communication
? Facilitate challenge to long
held beliefs and working
practices
? The establishment of a cross-
functional working party
? The establishment of a staff
consultative forum
? The introduction of employee
representative training
? Enhanced problem solving
capability
? Re-organisation of the stores
department
? Introduction of elements of
‘lean manufacturing’
? Development of an employee
engagement survey
Sources: Organisational Action Plans, initial interviews, final interviews and post-
project questionnaires
Achievements of Action Plans
Identifying what the organisations were seeking to achieve from their IWP
workplace projects was a focus of the initial interviews; the extent of progress
against their action plans was explored in the end of programme telephone
interviews with the gatekeepers and revisited in the final interviews. As a result,
the extent to which the workplace action plans were achieved is taken as the
starting point for identifying organisational changes and their benefits as a result
of organisational participation in the IWP. In addition to the summary of key
achievements provided in Table Two, further details of the issues, action plans
and outcomes are provided in the short case studies of the participating
organisation in Appendix A.
In practice, the initial action plans underwent subsequent development and
modification with input from the allocated Acas facilitators. The reasons for these
changes varied; in several instances the facilitation process revealed a need for
greater clarity about what the organisation was seeking to achieve or it became
evident that the scope of a project needed to be adjusted. At the final evaluation,
gatekeepers and other respondents from the participating companies were asked
to identify to what extent their organisations had achieved their action plans
using the following ranking scale: a) Completely (100%), b) To a large extent
(75%), c) Partially (50%) and d) To some extent (25%).
Across the organisations the majority of respondents (69 per cent) reported that
their action plans had been achieved ‘to a large extent’. Other than one
gatekeeper at Thorpe Kilworth, no one identified that their action plans had been
‘completely achieved.’ One explanation for this is that the respondents regarded
the interventions in their action plans as elements of a programme of continuous
improvement. For example, whilst identifying very positive outcomes, Pendragon,
The Health Store and Liquid Control all reported that they were seeking further
improvements than originally envisaged in their action plans. The perception that
the workplace projects were ongoing organisational initiatives to strengthen the
business was more in evidence at the final evaluation stage than in the telephone
interviews conducted with gatekeepers six months earlier. Using the same
ranking scale, three gatekeepers had then reported their action plans as
‘completely achieved’, four ‘to a large extent’ and six as ‘partially achieved’. The
30
shift towards a longer term view of the project as a programme of continuing
improvement had taken place across all the organisations by December 2010.
The final evaluation revealed differences between the responses of the
gatekeepers and other respondents who were more likely to report that an action
plan’s aims had been achieved ‘partially’ or ‘to some extent’. Responses at The
Health Store, Liquid Control and the SHA were both the most positive and
consistent across interviewees from the same organisation. One likely
explanation for the variation in responses between gatekeepers and other
respondents is the gatekeepers’ greater knowledge of the action plans and the
final outcomes; another is that their generally more positive view of the final
outcomes was shaped by their direct involvement in and engagement with the
project.
The Most I mportant Organisational Change
Respondents in each organisation were asked to single out what they saw as the
most important organisational change(s) in the final interviews. Table Three
provides a summary of their responses.
31
Table Three: The most important reported organisational change as a
result of the IWP
? BEM - improved communication as a result of the reintroduction of a
newsletter with increased managerial awareness of the employee perspective as
well as more employee awareness of business priorities (a view of improved
communication not shared by the gatekeeper who had left the programme).
? Caterpillar - following the establishment of an employee forum, improved
communications and closer working between the two distinct staff groups which
had resulted from a merger in 2007, evidenced by improved employee
engagement scores in the 2010 company employee survey.
? Liquid Control – greater workforce flexibility and utilisation of staff
resources as a result of improved communication and a skills analysis undertaken
by Acas which had led to better knowledge of individual job roles, closer working
and targeted training for skills development.
? Northampton College - due to delays in identifying the key issues and
facilitation continuing up to the final evaluation, outcomes arising from the IWP
are still evolving. It is envisaged that the most important changes to be reported
are likely to be enhanced employee involvement through improved
communications assisted by the College’s programme for developing leadership
skills among college managers at all levels.
? Pendragon - following the establishment of an employee forum,
improvements in communications at divisional level which are supported by
improved results in the most recent employee engagement survey.
? NHS East Midlands Strategic Health Authority - improved
communication arising from Acas facilitated focus groups which enabled the
implementation and integration of an Electronic Staff Records (ESR) System; this
had resulted in individuals beginning to take ownership of their own personal
data which in turn has begun to facilitate a more efficient handling of individual
data within directorates.
? The Health Store – the introduction of a joint management and staff
forum and elected employee representatives was the catalyst for improved
communications, levels of trust and employee engagement across the Production
and Warehouse operations, evidenced by growing examples of employee
proposals for improving working methods to increase productivity.
? Thorpe Kilworth - improved workplace communication due to
establishing a cross functional working party and a staff consultative forum with
employee representatives; one outcome of these improvements being a re-
organisation of the stores which had been of benefit across the business.
Source: Final interviews with all respondents
32
At Mahle Powertrain the most important change reported as a result of their
involvement was a recognition that to retain and develop the creativity and
innovation of its workforce, communication needed to more effective as a means
of improving employee voice, morale and recognition. This had resulted in the
establishment of a steering group with task groups to address particular issues.
Their project was ongoing internally but again was viewed as a continuous
improvement process.
Communications and Engagement
All eight of the participating organisations reported improved communications
arising from their participation in the IWP with six organisations identifying this
as leading to improved levels of employee engagement. Improved workplace
communications was the one change identified by all the participants in the final
survey data and in the final interviews. As Table Three illustrates it was also
reported as the most important organisational changes that had taken place as a
consequence of the IWP. But the nature of the reported achievements in this area
varied considerably across the participating organisations as a result of the
different issues to be addressed, their size, sector and individual histories. A
further influencing factor was the degree of business turbulence that was
experienced during the life time of the project. For example, the anticipated
improvements in employee engagement were adversely impacted upon in two
organisations as a result of workplace redundancies and the participation of
another was suspended for three months due to the impact of the recession on
its business.
In several instances, a particular change in the organisation prior to the IWP had
created problems in communication. For example, at Caterpillar a change in
ownership in 2007 and a TUPE transfer of staff had led to different working
cultures among its staff group; at The Health Store an expansion and relocation
to a larger purpose built premises in 2007 had led to the loss of previous close
working relationships, a situation described in the following terms by the
gatekeeper:
“We followed all the management practices that they tell you to follow and we
got nothing except a big shed with a lot of miserable people in it who were doing
just enough.”
One benefit of improved levels of communication and employee engagement
identified by a number of respondents in managerial roles was a reduction in the
time they spent on matters that could be resolved without their direct
involvement; these were now being addressed at a lower level as illustrated by
the following responses:
“There is definitely better communication in the business now. I just don’t get all
the tittle tattle and everything in my office any more. Things are being sorted
outside of management really. Production is a really good example. They put
plans together, they propose the plans and then actions are taken. That area has
come on significantly…” Senior Manager - BEM
“Since the project got into its stride there have been six developments that have
been gained from the employee engagement process which could well have been
driven by a manager, supervisor or through Health and Safety. But they certainly
33
wouldn’t have occurred at the pace they have without the employees latching on
to it and driving it themselves.” Operations Director - Health Store
For others, improved communication was seen as reducing divisions and
improving collaborative working between different parts of the organisation
although such improvements were widely acknowledged to be work in progress,
for example:
“I think the way we operate and communicate has probably got better. We work
closer together rather than being segmented. … I don’t think communication is
great but we do work better across departments now.” Manager - Liquid Control
“The key achievement would definitely be that we’ve managed to get some cross
functional or cross department dialogue going on which was a massive issue for
us in the first instance. We always struggle to get the likes of Finance and
Customer Service to actually talk to one another. We are not 100% there,
absolutely not, but I think they work slightly more closely together so that’s been
a massive achievement.” Gatekeeper - Pendragon
Capturing New I deas
Respondents were asked whether new mechanisms had been put in place to
encourage, exchange and capture new ideas and new ways of doings things as a
result of participating in the IWP and to identify what these were. Seven of the
eight organisations identified that they had introduced some form of employee
involvement arrangement, providing examples of where this had occurred as a
result of the focus groups facilitated by Acas, or through an employee forum or a
steering committee with task groups involving staff and management. In three
organisations it was pointed out that there were existing processes for capturing
employees’ ideas, for example a suggestion card scheme at Caterpillar, a
recognition scheme at Pendragon as well as a relaunched scheme called CANI
(Constant and Never-ending Improvement), and a number of processes for
improvement at the SHA which included a ‘Sounding Board’ and a forum called
‘Connect’ set up to capture any new ideas.
At The Health Store six examples were reported of instances where employees’
proposals for operational improvements had been considered and implemented.
Whilst some of these were broad in their scope, for example an employee’s
proposals was in progress for reorganising the warehouse, others could be
regarded as small improvements. These included the disposal of waste material
and revising purchase order documentation to make it more effective. However,
their cumulative impact was reported as making a real difference to productivity.
They also illustrated increasing levels of employee engagement with change
processes and the ownership of the operations they were involved in. One
employee representative elected as a result of the IWP explained how employees’
ideas were being progressed in the following terms:
“They come to me if there are any issues or there is anything they want to put
forward ... then we have a meeting so I can put their ideas forward and we will
sit and discuss which is the best route to go down.”
It was pointed out that new ideas were also increasingly being communicated
directly to managers as a result of the improved informal communications, a
34
dimension identified earlier and one which was seen as a key achievement of the
IWP.
At Liquid Control it was reported that appraisals had begun to take place again
(as a result of the IWP) which provided employees with the opportunity to put
forward ideas about their jobs and that input from the workforce was now pro-
actively sought in machine design and development, for example:
“We‘re encouraging the guys on the shop floor to come forward and have their
input into how machines are designed and built in the first place - they know how
things go together.”
There was less clarity about the adoption of new ideas at Northampton College.
Here respondents felt that it was too early to tell whether the consultative groups
or the open forums that were taking place on different themes (for example, the
new build) would lead to the adoption of ideas put forward by the workforce.
Where focus groups had taken place as a result of Acas facilitation, respondents
at five organisations reported very positively on their impact in terms of
encouraging individuals to put forward ideas and the extent of employee
contribution. There was a more mixed message from four respondents at three of
these organisations about maintaining longer term levels of interest and
engagement with the ideas put forward by employees. The concern was that,
whilst the IWP had created the ‘buzz’ reported by both gatekeepers and the Acas
facilitators, its impact would not be sustained and there would be a return to the
status quo which had prevailed prior to the project, described by one respondent
as “sticking an idea in a post box somewhere and nothing really ever happens.”
Communication of Outcomes
As part of the exploration of workplace communication in the evaluation, all
respondents were asked in the final interviews how the outcomes from their
organisational projects had been communicated to the workforce. Whilst most
organisations had indicated that they had introduced new or improved
mechanisms for employee communication, when it came to the communication of
the outcomes, the methods adopted prior to the IWP were generally utilised to
communicate the outcomes, for example notice boards, e-mail and organisational
intranets. The actual reported knowledge of project outcomes was patchy in
three of the organisations among respondents other than the gatekeepers.
HR Policies and Procedures
In the final survey respondents were asked to report changes that had taken
place or were planned, in HR policies or procedures as a result of participating in
the IWP. These are examined in more detail in the economic assessment impact
undertaken by Ecorys, and presented in Appendix D but the final questionnaire
results are provided in Table Four.
35
Table Four: Changes or planned changes to HR Policies and Procedures
HR Policies and
Procedures
Number of organisations reporting
Types of HR
policy/procedure
New Policy Improved
Policy
Planned
Policy
Review
Discipline 2 1 3
Grievance 1 1 3
Redundancy 0 1 1
Informing and Consulting
Employees
1 4 1
Other (e.g. equality and
absence management)
1 3 1
Source: Post - project questionnaires
Table Four reveals that at least one change in an HR policy or procedure was
reported as having taken place by respondents at seven of the eight participating
organisations.
The main reported changes were explored in more depth in the final evaluation
interviews. These confirmed that the changes involved mostly new or improved
processes for informing and consulting with employees or changes in approaches
to managing absence. Some of the participant organisations were part of multi-
site operations and were subject to policies and procedures determined at
corporate level which meant that they could not make changes independently at
site level, as this respondent from Pendragon pointed out:
“our company policies, they’re set from above. So none of those are going to
change because that would affect our handbook … and our handbooks are
standard.”
Issues stemming from standardised company policies and procedures were
fundamental to the problems which had led to Caterpillar Logistics applying to
participate in the IWP. As a result of its involvement in IWP, increased employee
involvement and communication had led to a reinterpretation of a previously
rigidly applied company sickness absence policy.
What was evident in the reported changes to HR processes was that where two
way communications had improved through a range of mechanisms, specific
problems and concerns could be identified and addressed as illustrated by the
observation of this gatekeeper at The Health Store:
“There were things that we’ve brought in that probably (we) hadn’t got round to
before because we didn’t know what was happening.”
Similarly better communication had led to beneficial changes to certain
procedures or practices at other organisations, for example, to the appraisal
systems at Northampton College, the SHA and Mahle Powertrain. Whilst the
interview data indicated that improvements in communication had allowed long
running, yet suppressed, problems to be aired and resolved this was not
36
manifest, in all cases by formal changes to policies and procedures. In some
instances it emerged in more informal ways.
Further evidence from the final evaluation interviews suggested that where a
climate of greater openness resulted from an individual project, it had
encouraged more organisational proactivity in addressing issues identified by
employees. It should be noted, however, that due to the challenging economic
climate and consequent wider changing organisational context, respondents
sometimes found difficulties in identifying where there were changes in HR
policies and procedures that could be directly attributable to the IWP as opposed
to this wider changing organisational context.
Workplace Climate
At the outset and at the end of the IWP respondents were asked to describe
workplace morale at their organisations on the following scale:
a) Very poor; b) Poor; c) Good d) Very good
By December 2010, six months after the project had finished, eighteen
respondents (69 per cent) reported that workplace morale was either ‘good’ or
‘very good’. Although the majority felt morale was good, only three respondents
saw it as ‘very good’. Perceptions of workplace morale had improved significantly
since the initial interviews in Autumn 2009 when only five respondents had
described workplace morale as ‘good’ and none described it as ‘very good’.
Respondents were then asked to identify whether or not workplace morale had
improved as a result of participation in the IWP. A number of respondents found
this a difficult question to answer either because there were wider external
factors impacting on the organisation (for example, uncertainty about the future
of strategic health authorities in the light of the Government’s review of NHS
structures) or the presence of internal issues (such as a pay freeze or recent
redundancies) which were felt to be having an adverse effect on morale.
Notwithstanding such factors, the majority of interviewees identified that the
improvement in workplace morale was as a result of participation in the IWP; 58
per cent reported it had led to an improvement and 19 per cent that it had not,
with the remaining 23 per cent feeling unable to answer the question due to
other issues impacting on morale.
The most consistently positive perceptions of the impact of the IWP on morale
came from the smaller businesses, reinforcing the earlier suggestion that the
impact of the IWP was more immediately evident in these organisations. With the
exception of a totally shared view among respondents at The Health Store and at
Thorpe Kilworth that there had been an improvement, gatekeepers were
generally the most positive in their view of the IWP’s impact on morale, possibly
again as a result of their closer knowledge of the outcomes and their personal
commitment to the IWP.
Reflecting upon the impact that participation in the IWP had on morale within
their workplaces, six respondents in managerial roles identified that morale would
have been considerably worse without the IWP, a view illustrated by these
managers’ observations:
37
“I think if we hadn’t been doing the talking, if we hadn’t been doing these things
then it would have been worse.” BEM
“Taking into account that we’re coming out of recession and the year preceding
that everyone was thinking they were going to be made redundant its hard to
quantify but it would all have been much more of a challenge without this
project.” Health Store
Asked to give examples to illustrate improvements in workplace morale,
respondents tended to describe a more positive employee relations climate rather
than providing specific instances to support their view. Gatekeepers found it
easier to point to an internal change or action as a result of the IWP which had
contributed to improved morale and all but three were able to do so, for
example:
“Probably the team events we have done, I think that boosts morale because
there is something at the end of it. We’ve done various out of work team member
events and social activities and done some internally. I think that has definitely
had a knock on effect.” Pendragon
“People are putting themselves out to sort of help the company you know and
make better work practices ... we get a lot of continuous improvements
suggestions as well, a lot of ideas. We’re meant to get three ideas per person
per year and we’re now getting about six or so.” Caterpillar
“The guys on the benches were working more as a team, helping one another
and taking the trouble to go to the stores once we had reorganised it as opposed
to ‘Oh I can’t be bothered to go over there because I’ll never find it anyway’ … it
was a benefit.” Thorpe Kilworth
Levels of Trust
At the final evaluation respondents were asked to describe the present level of
trust between managers and employees on the same scale used to describe
workplace morale:
a) Very poor; b) Poor; c) Good d) Very good
The majority of respondents (54 per cent) described it is as ‘good’ or ‘very good’.
Responses were the most consistent at Thorpe Kilworth, the SHA and The Health
Store where all interviewees reported it to be ‘good’ and at Liquid Control where
trust levels were perceived to be ‘very good’. At BEM it was described as poor
due the internal issues already referred to; at Pendragon, respondents all felt
that levels of trust varied from good to poor but were agreed that there had been
improvements in some areas.
Improved levels of trust between managers and employees were reported at all
eight organisations since the IWP began but this was not necessarily a view
shared by all respondents at the same organisation. The exception to this was in
the smaller businesses (Liquid Control, The Health Store and Thorpe Kilworth)
who all felt that the IWP had contributed to improved levels of trust. The pay
dispute at BEM had led to an observed divergence in respondents’ views and only
one respondent (who reported an improvement) at the SHA felt able to answer
this question. There were different views among respondents at Pendragon
38
depending on whether the question was concerned with trust between employees
and their direct management or interdepartmental trust; the perception was that
levels of trust were higher within than across departments although across the
workplace it had been described as ‘quite good’ when the IWP began.
At Caterpillar trust levels were described as ‘good and improved’ by three
respondents but a fourth respondent did not feel it had changed and was still
‘poor’. Respondents at Northampton College reported trust levels as ‘different in
different areas’ and that ‘it all depended who you talked to’ but nonetheless
described it as either ‘good’ or ‘quite good’ but, despite identifying that trust had
improved in the past year, did not feel it was possible to single out what might
account for this.
Management and Leadership Development
A core aim of the project was the development of management and leadership
skills. Respondents were asked whether this had been an outcome of the IWP in
the post-project questionnaire and in the final interviews where it was also
explored from the perspective of whether it was felt there had been changes in
approaches to management as a result of the project. Both dimensions are
examined below.
Developing Management and Leadership Skills
A key objective of the IWP was to facilitate long term organisational change by
focusing on the development of organisational management and leadership skills
through the gatekeepers in the first instance. The intention was that gatekeepers
would transfer their learning from the programme to stimulate a wider workplace
enhancement of these skills. Respondents were asked to what extent they felt
the IWP had met its objective of facilitating organisational change by enhancing
management and leadership skills (not just of those in managerial roles) and to
identify whether this had occurred to:
a) To a large extent; b) Partly; c) Not at all; d) Don’t know
At the final evaluation the majority of respondents (61 per cent) said this had
happened either ‘partly’ or to ‘a large extent’ with seven respondents reporting
that this had happened to ‘a large extent’, ten that it had partly occurred, five felt
that it had not happened, five did not know with one respondent unable to
answer. Table Five provides reported examples of how the project contributed to
the development of management and leadership skills with the participating
organisations.
39
Table Five: IWP’s contribution to the development of Management and
Leadership Skills
? It made the management team of the operations department at least look
at themselves and say well yeah, maybe I’m not as good as I think I am and
I’d lump myself in there as well.
? Because as gatekeepers we report to the operations manager, we’ve
shown him what we’ve learnt and it’s pinpointed that the same mistakes have
been made by management over again.
? People that are involved in it feel more empowered to do things off their
own back and to do things without me saying ‘can you do this because I want
XYZ.’ Now they come to me and say do you think? I say ‘go for it’.
? If nothing else the training courses that were offered have enhanced at
least some of the department leaders in dealing with difficult issues …from a
purely selfish point of view, dealing with some of the conflict that used to lead
to me getting involved, if nothing else that will have helped them.
Source: All respondents in the final interviews
Not unexpectedly the majority of the gatekeepers (78 per cent) reported that the
IWP had ‘to a large extent’ or ‘partly’ achieved its objective of developing
management and leadership skills, but it was encouraging that 43 percent of
other respondents reported that the IWP had ‘to a large extent’ or ‘partly’
achieved this aim. Only at The Health Store did all the respondents report that
this had been achieved to a ‘large extent’. This may have been more evident
because the gatekeeper was in a senior managerial role which, by definition,
required such skills. At Liquid Control the majority view was that this had been
achieved to a ‘large extent’ but here one respondent reported not knowing
whether or not these skills had been developed, in common with a respondent at
Caterpillar.
The same question had been explored with the gatekeepers during the telephone
interviews six months earlier; by the final evaluation the number of gatekeepers
who thought the IWP had met this objective ‘to a large extent’ had increased
from three to seven, an increase which may be due to further developmental
interventions taking place in the intervening period, predominantly through
attendance at Acas short courses by gatekeepers and other attendees from the
participating organisations.
The question proved to be particularly challenging for some gatekeepers who,
whilst reporting that they had personally developed as a result of the IWP, found
it more difficult to identify whether they had developed management and
leadership skills which had facilitated organisational change. The benefits they
reported from being participants on the IWP and the value they obtained from its
different elements are discussed later in the evaluation of the different elements
of the programme. All of them, however, had experienced the initial development
programme ‘Managing the Workplace of Tomorrow’ accredited by the Institute of
Leadership and Management (ILM) at NVQ level 3. As previously mentioned this
offered the option of completing the requirements for the qualification, 12 out of
the original 20 attendees chose to do so and were all successful.
40
3.2 Approach to Management
The final evaluation found that 16 of the 26 respondents in the participating
organisations thought the approach to management had changed as a result of
the IWP. Eight thought there had been no real changes and two felt unable to
answer. Compared to other areas of evaluation there was rather greater
consistency among respondents within the same organisation where it was
perceived that changes had taken place, for example at Liquid Control, The
Health Store and Northampton College. This was also the case where it was felt
changes had not occurred, for example at Pendragon where it was reported ‘that
nothing much had changed but there wasn’t really a problem to start with.’
The types of changes reported were dominated by better communication, more
delegation and a shift towards a more consultative management style. Whilst not
a complete list, Table Six provides the type of changes respondents commonly
identified had taken place in the approach to management as a result of IWP
participation.
41
Table Six: Six examples of changes in the approach to management
? The management style is less authoritarian with more explanation and
information about the reasons for asking for something to be done. There is less
of ‘Well I’m the manager, this (is) how we do things ... managers are more
approachable.
? There is greater delegation due to better knowledge of the skills set - the
work is better spread.
? Managers have seen the benefits of one to one dialogue and employee
involvement.
? There is a lot more information available to all, whereas before it (was) all
kept under the table. None of it was top secret they just didn’t bother to pass it
on.
? Attitudes have changed, it was always ‘them and us’ and now it’s not like
that, there is communication between us - it seems to have been recognised that
we all work for one company.
? There is definitely more of a view that it is best to consult first and get the
employees' feelings on certain things to enable change, to get their buy-in.
Source: All respondents in the final interviews
Respondents were asked at the final evaluation whether participation in the IWP
had led to changes in how managers dealt with absenteeism, disciplinary issues,
interpersonal conflict in the workplace and formal disputes between management
and employees. The extent to which new or amended procedures had been
introduced in these areas and whether these had any beneficial outcomes are
explored elsewhere in the report. The consideration here is whether respondents
saw improvements in how managers approached these issues. Eleven
respondents said there had been improvements, which included all the
interviewees at The Health Store who identified positive changes in these areas.
Nine respondents said they were unable to point to any specific changes and
none of the respondents at the SHA felt able to answer the question.
In a number of the organisations issues outside the IWP project had a bearing on
responses. For example, at BEM it was identified that as a result of the IWP, prior
to the pay dispute, there had been an improvement in the number of grievances.
Changes in the interpretation of the long term sickness procedure led to reported
changes in how management dealt with such cases at Caterpillar, and at The
Health Store, as already identified, there was less need for managerial
involvement in conflict resolution and more issues being resolved at an informal
level. At the same organisation it was reported that, from an employee
perspective, return to work interviews by managers were helping absenteeism to
be better managed. There was a less consistent view at Northampton College
where respondents were divided about whether or not there had been
improvements among the majority of managers and there was no evidence
provided to illustrate any changes.
3.3 Benefits to Individuals in their Jobs
Asked whether there had been IWP outcomes which had helped respondents in
their own jobs, the responses varied considerably both across and within the
participating organisations. Where individuals had been gatekeepers and gained
42
new skills, they identified that these would be of benefit to them in their present
or future work roles. Where respondents held a central organisational role, for
example as an HR Manager, they were more likely to report clear benefits such
as greater ease in obtaining the employee perspective and their involvement in
new initiatives at The Health Store or the ability to delegate more to staff at
Liquid Control. Others expressed a more general view that the impact of
improved staff engagement would help them in their roles or to deal with future
changes observed this HR Director:
“I’m hoping that staff will come to us and work with us rather than against us
through the difficult times we have ahead …. In that way it will help my job.”
A small number of respondents found it difficult to identify how the outcomes of
the IWP had helped them in their immediate jobs although a number thought the
experience could be useful in the future or saw benefits in continuing networking
arrangements with other participating organisations. There was more likely to be
a shared view among interviewees in the smaller businesses, for example at
Liquid Control and The Health Store, that the IWP’s outcomes had benefited them
in their work roles. Here the outcomes could be more directly visible to the
workforce as illustrated by this elected employee representative’s comment:
“It’s a lot easier to talk and obviously I’m involved now with the meetings -
there’s no secrets anymore so there’s a lot more understanding which makes my
job a lot easier.”
3.4 Unanticipated Outcomes
Respondents were asked in the final interviews about any unanticipated positive
or negative organisational outcomes as a result of participation in the IWP. In
the telephone interviews with gatekeepers six months earlier, nine had reported
unanticipated positive outcomes compared to four who identified none. Six had
said there had been unanticipated negative outcomes, whereas seven reported
none. In the final evaluation it was possible to probe more deeply into this area
and to include respondents other than the gatekeepers. Again reported
unanticipated positive outcomes outweighed any negative outcomes.
Gatekeepers, as a result of their direct involvement throughout their projects
were more likely to identify where things had not gone to plan or there had been
a negative reaction. One such example was where team members had returned
from a workplace forum meeting with enthusiasm to apply agreed new ground
rules for the working environment but had received a demotivating, adverse
reception from other employees about what was being proposed. This particular
incident was reported by the gatekeepers as a critical learning point which had
illustrated the importance of channelling forum outcomes through supervisory
team leaders.
In three organisations, a positive unanticipated outcome had been the speed with
which employee engagement had improved once a forum had been established or
individual employees had begun to take ownership and propose changes in work
methods. Other outcomes were described as unanticipated in that they had not
formed part of the organisation’s action plan but had arisen as a result of an
improved, more open environment. For example, at Caterpillar it was reported
that it had been agreed that the members of one team should set their own
43
targets; an approach that had worked well so was being applied across other
teams. Because of the nature of their organisational role, HR Managers were
often best placed to see any positive and negative outcomes beyond the action
plan. At The Health Store this was reported as employees being more proactive
about health and safety matters. In contrast, at Mahle Powertrain a negative
outcome from the HR perspective was that the scope of the initial workplace
project had not included its manufacturing operations at another site.
There were instances where an outcome was reported as having both positive
and negative aspects. For example, at one organisation steps to address labour
utilisation had resulted in two redundancies; an outcome that was detrimental to
the individuals involved but in line with the organisation’s objective of achieving
better labour utilisation. A lack of direction about their role and what was
expected of the project at the outset gave rise to problems for the gatekeepers at
the SHA, but the final interview revealed that their difficulties led to a ‘lack of
communication’ being identified as a critical issue in terms of achieving the action
plan. It was then possible to begin to address this, with the assistance of the
Acas facilitator, through a series of focus groups.
3.5 Constraints on Progress
At the beginning of their projects, respondents were asked to identify any
particular obstacles or constraints to achieving their action plans. This question
was revisited in the final evaluation interviews. Not unexpectedly, there was a
varied range of responses. The recession had impacted in different ways on all
the private sector organisations but none were untouched (even if the effect was
in another part of the business); for example, there was short time working and
redundancies at two organisations and cash flow problems at another. Other
factors, such as a pay dispute at BEM and changes in management at a couple of
the organisations, had an impact on the progress of organisational projects. A
defining and innovative feature of the IWP was that it provided organisational
support over an extended period of time through the network events, action
learning and workplace Acas facilitation. But this also meant it was vulnerable to
changes of personnel within the organisation and there were instances of an
individual project faltering where a senior manager and/or a gatekeeper, who had
acted as a champion for the IWP, left or moved to a different role.
In contrast, the two public sector organisations appeared to experience the most
difficulty in clarifying the aims and scope of their action plans at the outset,
partly due to the presence of other related, and potentially overlapping,
organisational initiatives being underway at the same time as the IWP, for
example, a leadership development programme. Until issues of integration and
alignment with these other initiatives were addressed, there was some difficulty
in clarifying the focus of the IWP workplace project, the appropriate processes
and what the gatekeepers should be doing.
3.6 Levels of Support
The initial interviews identified that most gatekeepers’ had a concern about the
level of support they would receive from senior management and a couple were
44
sceptical about support from a work force who were viewed as apathetic and
disengaged due to low morale as a result of redundancies, changes of ownership
or restructuring. In the final interviews the level of support was revisited;
respondents were asked to consider the extent to which they agreed or disagreed
that there had been a lot of support from a) senior management, b) line
management, c) employees and to provide examples.
The levels of support reported by gatekeepers in the final interviews are
illustrated in Figure One:
Figure One: Gatekeepers’ reported levels of organisational support
(number of gatekeepers giving each answer)
0
2
4
6
8
10
Strongly
disagree
Tend to
disagree
Neither agree
nor disagree
Tend to agree Strongly agree
Support from senior management Support from line management
Support from employees
Source: All respondents in the final interviews
Figure One reveals that, in practice, the majority of all respondents reported in
the final evaluation that there had been a lot of support from senior managers
although four respondents felt that this had not been the case. Where support
was forthcoming at the outset, facilitators reported that there was more rapid
progress against the action plan as evidenced at BEM, until tensions arose due to
a pay dispute. Four gatekeepers reported that senior management support had
increased as their projects had developed and positive benefits became evident,
as one gatekeeper put it:
“Senior management did not show any particular support but to be fair nor did
they stand in the way of what we were trying to do …. once it became clear that
the pilot was working they have been right behind it.”
There were less initial concerns about the level of support from line managers
and responses at the final evaluation confirmed that this had been forthcoming
with the exception of Pendragon (see below) and one respondent at the SHA who
disagreed on the grounds that they would have been more supportive if the IWP’s
benefits had been better explained at the outset. At Pendragon, all three
respondents identified that the problems of support among line managers
stemmed from their lack of involvement, as one gatekeeper explained:
45
“We were probably a couple of months down the line when we started trying to
involve them, a lot of them were actually turning round and saying ‘I have no
idea what you’re talking about’. So I think they probably would have given their
support if they’d have had the opportunity to….but you know, that’s a learning
experience for us.”
Low morale and a lack of engagement led a number of respondents to initially
identify that the workforce would be apathetic and even cynical about the IWP’s
aims but the final interviews revealed that these fears were unrealised;
employees were reported as supportive and forthcoming in focus groups and at
other forums which provided opportunities for their ‘voice’ to be heard. Whilst the
gatekeeper at the SHA was of this view, there was a perception among the other
respondents that the employee support would, as with line management, have
been greater if they had understood the value of what was being proposed from
the beginning. The overall finding was that concerns about levels of support
diminished as individual projects progressed and the majority of gatekeepers
reported being satisfied with levels of internal support at the final evaluation. The
issue of ensuring that appropriate managerial support is in place at the outset is
returned to in the discussion of the programme design and delivery.
3.7 Learning Transfer
The extent to which learning transfer had taken place throughout the IWP was
reviewed in relation to:
a) the transfer of learning from the gatekeepers to others within their
organisation,
b) the transfer of learning between the gatekeepers from the participant
organisations.
The overall finding of the evaluation process was that learning transfer across the
participant organisations from gatekeepers sharing experiences at their network
meetings and the initial three and half day course was more frequently reported
than learning transfer from the gatekeepers to others within their individual
organisations. Questions about learning transfer within the organisation were
asked of all respondents whereas the question about learning transfer from the
other participating organisations was explored through the gatekeepers’ reported
experiences from attending the participant network events and the action
learning sets. Learning from individuals in other organisations was, however,
reported as one benefit of the Acas short courses which had been attended by
delegates from the organisations beyond just the gatekeepers.
Transfer of Learning within the Organisation
Asked whether or not skills and knowledge had been transferred from the
gatekeepers to others in the organisation, 65 per cent of respondents said this
had taken place. Not unsurprisingly, the most positive responses were from
gatekeepers with 83 per cent reporting that this had taken place.
Respondents other than the gatekeepers were divided in their perceptions of
learning transfer; 50 per cent reported that they perceived this had taken place,
29 per cent indicating that this had not happened and 21 per cent were unsure.
46
This led to an exploration of how gatekeepers’ learning from the IWP was
communicated to others in their organisations. The majority reported that this
had been communicated through the newly introduced information and
consultation mechanisms, for example:
“[The Gatekeeper] talks to me and has a meeting with the senior staff, then he’ll
have a meeting with his forum … and that’s how he’s passing the information
through.” Health Store
Others mentioned that this occurred informally through ‘word of mouth’ or by
putting information on the ‘project server’ but three respondents felt that the
outcomes were only really known to the direct participants on the programme as
one observed:
“The closer people were to the gatekeepers involved the more likely they were to
know about the learning from the project.” Caterpillar
One of the interventions which facilitated the sharing of learning was the various
short courses provided by Acas and attended by both the gatekeepers and
others.
At one organisation, where the gatekeeper was the Employee Relations Manager
for other Divisions as well as the participating Division, the skills and knowledge
she acquired from participating in the IWP were transferred more widely in the
company as illustrated by the following:
“… in a couple of businesses, we’ve definitely gone in and used some of the
things that we learnt from the forum to try and get other organisations or other
businesses to do the same.” Pendragon
Transfer of Learning Across Participant Organisations
The opportunity for gatekeepers to share and learn from each other’s experiences
was an integral part of the design of the initial three and half day course and a
central feature of the participant network meetings organised and facilitated by
UKWON, reflecting the action learning philosophy of the IWP. The contribution
made directly by the action learning sets as an element of the IWP programme is
explored later in the report. This issue is discussed further in 3.12, Evaluation of
the IWP Methodology).
Learning transfer and the sharing of knowledge between gatekeepers was
explored at the final interview stage. Respondents were asked to identify how
useful it had been to share knowledge and experience with participants from
other organisations on the programme on the following scale:
1) Not at all useful; 2) Of some use; 3) Useful; 4) Extremely useful.
11 of the 13 gatekeepers reported that sharing knowledge with others had been
either ‘useful’ or ‘extremely useful’ and their perception was that this had
occurred to a greater extent than learning transfer within their organisations
although the two aspects of learning transfer are clearly interrelated. They
illustrated their responses with the following examples of the benefits that had
resulted from sharing experiences with participants from other organisations:
47
“I was ... able to learn how different organisations approached setting up of focus
groups; we were a little behind in this area.” Liquid Control
“I gained a realisation that I tended to ramble and there are better ways to
communicate.” Thorpe Kilworth
“Sharing information helped us to benchmark; we saw that we were not really
that bad at all.” Pendragon
“Sharing helps to prioritise actions, so you don’t sink under the volume and to
realise that change is difficult.” Mahle
3.8 The Personal Development of the Gatekeepers
The degree of personal development emerging from any management
development intervention is notoriously difficult to establish as is the impact of
such development on outcomes (Mumford and Gold 2004:157). Nevertheless,
gatekeepers were asked in the telephone interviews in May 2010 what the
personal benefits had been for them at the point in time when, for most of them,
the different elements of the IWP programme had just been completed.
All of the gatekeepers identified some personal benefits from participation in the
IWP; examples included ‘ more confidence in speaking and chairing meetings’,
‘increased participation in group and team work’, ‘working more closely with
senior leaders’ and ‘the ability to utilise tools and techniques’. One gatekeeper
was so encouraged and motivated by his introduction ‘to the world of learning’ on
the IWP programme that he had begun a higher level ILM/NVQ Level 7
qualification in management and leadership, as he explained:
“If it had not been for this project and the insights I gained, I just would not
have pursued further development of myself as a manager and I would not be on
this ILM Level 7.” Health Store
The main personal benefits of the IWP identified by 7 of the 13 gatekeepers in
telephone interviews are those reported in Table Seven; namely the opportunity
to ‘network’, ‘share issues, problems and achievements’ with others participants
on the programme. Learning that other organisations of a different complexity
and size could face similar issues and challenges was described as reassuring but
also as developmental in terms of sharing how these were being approached.
Several gatekeepers felt this had “helped their self confidence” illustrated by the
participant who observed “learning what others were doing helped me to
challenge what we were doing”.
The facilitators also reported the IWP had improved the self confidence of the
gatekeepers they had worked with in their reporting of its outcomes.
In the final interviews gatekeepers were asked to identify the ‘three best
outcomes for them as participants on the programme’ and these are summarised
in Table Seven.
48
Table Seven: Summary of Gatekeepers’ reported three best outcomes
from IWP participation
Organisation 1 2 3
Brush
Electrical
Machines
(BEM)
Sharing challenges
and difficulties
Help and advice
from Acas on how to
deal with problems
Introducing the
news letter
Caterpillar
Getting facility
leaders to do more
employee
consultations with
the employees
rather than making
assumptions
Having a different
viewpoint and
mindset, being able
to think outside the
box
Implementation of
the employee
forum
Health Store
Engaging with
further education
Improved levels of
trust between
gatekeeper and
senior managers
Seeing
improvements in
morale
Liquid Control
Gatekeeper One
Developed new skills
about having
difficult
conversations
The practical
assignments done
with Acas
Learning about
communication
from visit to
Thorntons plc
Liquid Control
Gatekeeper Two
Networking with
others
Learning from the
deliverers
particularly from
case studies
Learning tools and
techniques
Mahle
Building
relationships with
others
Working with new
and different people
in my organisation
and building
organisational
knowledge
Seeing some
things actually
implemented
Northampton
College
Gatekeeper One
Getting the
organisational
project started
Appreciating how
difficult it is to get
people engaged in
improvement
activity
Recognised the
value of
networking
Northampton
College
Gatekeeper Two
Getting HR on board Focus groups
helping better
alignment so
helping to move us
forward
Seeing the senior
management
become aware of
the need to get
involved
Pendragon
Gatekeeper One
Learning from
mistakes
Positive outcomes
from the team
building events
Improved
relationships with
senior leaders
49
Pendragon
Gatekeeper Two
Getting teams to
work together more
effectively
Getting people
working together
and understanding
things
Introduction of a
customer
satisfaction survey
Strategic
Health
Authority
(NHS East
Midlands)
Networking with
other companies
Learning new skills Realising the
importance of
communication
Thorpe
Gatekeeper One
Networking with
people
The realisation that
we were a pretty
good company
Learning points
picked up
throughout the
project
Thorpe
Gatekeeper Two
Interacting with
other people, other
organisations
Learning new skills Seeing the
company starting
to be more
efficient and begin
to save money
Source: Final interviews with gatekeepers
3.9 Outcome Measures
The estimated economic impact of the IWP on the participating organisations is
summarised in Section Four of this report; the full analysis of the economic
impact is provided in Ecorys’ report in Appendix D. This analysis is based on data
drawn from the pre-project questionnaire and post-project questionnaire
completed by each of the participating organisations. These were sent to the
individual identified by each organisation as best placed to gather and provide
the information requested by the survey and to report on any changes that had
taken place in the 12 months between September 2009 and September 2010 as
a result of IWP participation.
In addition to the questionnaire, all respondents in the initial interviews were
asked how outcomes from the programme would be measured to see whether
participation in the programme had contributed to the performance of the
organisation. The question of measurable outcomes was revisited in the final
interviews where all the respondents were asked whether or not there were any
measurable outcomes that showed that participation in the programme had led to
improvements in organisational performance (in three instances this referred to
the workplace involved in the IWP). Where there were measurable outcomes,
respondents in each organisation were asked to provide examples. In practice the
responses to the post-project questionnaires and the final interviews revealed
that respondents had difficulty in disentangling the impact of the IWP from other
influences on organisational performance.
Seven organisations provided before and after data relating to Employment
Tribunal applications, six did so for employee grievances and five for disciplinary
sanctions in the post project questionnaires. There was a reported rise in
employee grievances and a reduction in disciplinary sanctions but no change in
Employment Tribunal claims. But organisational experiences were very different.
For example, one smaller business (with just over one hundred employees)
50
reported a reduction of twenty disciplinary sanctions over the 12 month period, a
fall they attributed directly to improved communication and employee
involvement from participation in the IWP. At another much larger organisation
there had been increased numbers of grievances and disciplinary sanctions
during the same period which may have been related to internal differences over
other workplace issues unrelated to the IWP. Whilst the data revealed no change
in the number of Employment Tribunal claims, given the potentially protracted
nature of such procedures, the origins of these where likely to have predated the
IWP and any conclusion on the programme impact of these would require a
longer time frame.
Of the five organisations who were able to respond to the questions on levels of
employee turnover and days lost through absence in the post project
questionnaire, two reported a decline in staff turnover and a further two reported
a reduction in days lost through absence since participating in the IWP. In terms
of the earlier reported changes to HR policies and procedures that had taken
place as a result of involvement in the IWP, there was no evidence provided to
show that these had impacted on organisational performance. But there could
have been an indirect contribution which was not easily identifiable. One such
example was the reduced absence levels reported at one workplace which was
possibly due to the adoption of a less rigid workplace interpretation of a company
absence procedure which had been an outcome of the employee forum
established as a result of the IWP.
Whilst the survey data presented a mixed picture, there was the perception
among respondents in managerial roles, mentioned previously, that problems
that had taken up their time prior to the IWP were being resolved at a lower level
and without their involvement saving managerial time.
The changes in numbers employed indicated a variation between public and
private sector organisations with increased voluntary departure at both the public
sector organisations although there was a large increase in employees leaving
due to redundancies at one whilst a slight decrease at the other. In contrast,
there was a decrease in all categories of departure in the private sector
organisations although, again, causality can not be disentangled from broader
economic influences.
As well as disentangling the impact of broader factors, such as the economic
downturn, analysis of this data is also impacted by the large variation in size of
participating organisation, the largest, having in excess of 1,000 employees
whilst the smallest, employed only twenty, and did not supply employment
change data.
When respondents were asked in the final interviews what they considered to be
the best measure of improvements resulting from their action plans, their
responses provided a range of both objective and subjective measures. For
example, a senior manager at BEM reported that the company was now doing
better financially although it was difficult to quantify how much this was due to
the IWP. At Thorpe Kilworth, one gatekeeper could point to employees’ proposals
resulting in a saving of between four and six thousand pounds. At Caterpillar
Logistics, the 10 per cent improvement in employee engagement specified in the
organisation’s action plan had been achieved in 2010 and further improvements
51
were anticipated in the next employee survey. Other measures were more
subjective as illustrated by this manager’s observation:
“I think the staff seem happier with their jobs they’re doing so there is a better
atmosphere. So I would say that is a sort of measure.”
In terms of adopting measures to demonstrate the impact of the IWP the
evaluation process offered a number of insights. Firstly, the initial interview
questions about final measurable outcomes revealed few of the respondents had
identified how they would measure the organisational benefits from participating
in the IWP. Three said this could be through an employee survey; two further
organisations had identified this as the measurement as part of their action plan
and for one there was a target figure for an improved score for employee
engagement which was achieved.
Secondly, for the majority, it was reported that success would be demonstrated
by achieving what they set out to do in the workplace action plan or evidenced by
improved communication, workforce morale, increased employee involvement
and the development of management and leadership skills. But it was not
identified how the outcomes in these difficult to measure areas would be
assessed. Notwithstanding, it was reported that improvements had been made to
a very large extent in all these areas by the end of the project. What was evident
is that the ‘yardstick’ adopted by respondents for measuring successful outcomes
from the IWP placed more emphasis on qualitative changes in the workplace than
the quantitative measures commonly used to gauge workplace improvements.
This may be partly due to diverse nature of the gatekeepers’ roles; if the
gatekeepers had all been senior managers the criteria for success might have
been very different. The final observation as previously reported is that
respondents in over half the participating organisations said that things would
have been worse without their IWP participation
3.10 Probability of Change without the IWP
Respondents were asked to identify the extent to which they thought the
organisational changes reported would have occurred without participation in the
IWP using a five point ranking scale of:
a) Completely (100%)
b) To a large extent (75%)
c) Partially (50%)
d) To some extent (25%)
e) Not at all (0%).
None of the respondents thought the changes would have been completely
achieved without the IWP although four respondents (all in managerial roles)
thought that these would have happened ‘to a large extent’. There was an even
split between the 20 respondents who thought the changes would have ‘partially
happened’ or ‘to some extent’, and only at The Health Store was it reported that
the changes would not have happened by all the interviewees. The responses
52
highlighted a difference of perception between managers, gatekeepers and
employee representatives; the latter two groups of respondents being more likely
to identify that the changes would have not occurred without the IWP. At Mahle
Powertrain who had left the project, it was reported that the IWP had been the
stimulus for changes which would otherwise not have occurred.
The availability of similar support to that provided by the IWP and the degree to
which the participating organisations would have taken this up was explored in
the final survey; the responses are analysed in the Economic Assessment of the
IWP in Appendix D. Despite respondents reporting that these changes would have
occurred to varying degrees, the survey data reveals it as unlikely that
participants would have taken up an alternative source of support. This raises the
question as to whether or not the organisations could have achieved the reported
outcomes without external support, particularly as the gatekeepers frequently
described the IWP as the ‘catalyst for change’ within their organisations - a view
that was shared by both delivery partners and the Acas facilitators in their
reflections on what the IWP had achieved.
3.11 Sustainability
As already identified, it was evident from the respondents in the final evaluation
interviews that they were increasingly viewing the workplace achievements
resulting from their involvement in the IWP as part of ‘on going’ programmes of
change. However, a third of the organisations were concerned that, without the
IWP, the momentum would not be sustained. This was particularly reported
where there had been a loss of individuals who had championed the project due
to staff turnover.
3.12 Evaluating the IWP Methodology
As part of the evaluation process, the gatekeepers were asked to evaluate the
contribution of each element of the programme provided by the IWP. As outlined
in Section One, these consisted of:
? an initial three and a half day course
? six participant network events
? eight action learning sets (six of which took place during the participant
network days)
? the Acas facilitation process
? additional inputs from Acas or UKWON (largely short Acas courses).
During the telephone interviews the gatekeepers were asked to rank the
usefulness of these different elements of the programme. A summary of their
responses at that point in time is provided in Table Eight below:
53
Table Eight: Gatekeepers’ perceptions of the usefulness of the different
elements of the programme
Questions relating to the Individual
elements of the programme
Extremely Useful
to a
large
extent
Partly
useful
Not
at all
useful
How useful was the initial delivered
element of the programme (the three
and half days leadership and
management course)?
2
6
4
1
How useful was the initial launch event?
4
8
0
0
How useful were the Participant
Network meetings?
5
4
4
0
How useful were the ‘Action Learning
Sets’?
3
4
4
1
How useful was the ACAS facilitation
element of the programme?
3
5
4
0
Source: Telephone Interviews with gatekeepers
In the final interviews there was a more ‘in depth’ exploration of the perceived
value of the different elements of the programme with the 13 gatekeepers who
were asked:
a) The contribution they felt each element had made to the final outcomes for
the organisation and for them personally;
b) To what extent they felt the key achievements of the IWP would have
occurred without this element of the programme;
c) What changes, if any, they would propose for any element of the programme.
The responses of the gatekeepers on the extent to which they felt the key
achievements of their workplace projects would have occurred without this
element of the programme are summarised in Table Nine.
54
Table Nine: The extent to which the key achievements would have
occurred without each element of the programme
Element of Programme
Not at all To some
Extent
To a
large
extent
Completely
Initial three and a half day
leadership and management
course
4
2
4
3
Participant network
meetings
3
4
5
1
The action learning sets
4
2
4
3
Acas facilitation
6
6
-
1
Source: Final interviews with gatekeepers
It was interesting to identify the extent of any changes in the gatekeepers’
perceptions of the value of the different elements six months after the telephone
interviews when the outcomes of their individual project had become clearer.
Their responses regarding each element in both the telephone and the final
evaluation interviews are considered below.
The I nitial Short Development Course
The initial three and half day course ‘Managing for the Workplace of Tomorrow’
was delivered by New College Nottingham from June to September 2009. It
offered an ILM qualification for those who wished to pursue this. 95 per cent of
participants were satisfied with the general content and delivery of the
programme and were particularly positive where ‘tools and techniques’ were
provided that could be easily transferred back to the business.
By the time of the telephone interviews 61 per cent indicated that they felt this
element of the programme had been ‘extremely useful’ or ‘useful to a large
extent’ with a further 31 per cent indicating that it had been ‘partly useful’. Only
one respondent expressed a view that it had ‘not been useful at all’. Positive
views related to the ‘bite size’ nature of different subjects which one gatekeeper
indicated ‘helped to get things started’. The content also seemed to provide a
‘focus’ for some gatekeepers ‘steering them to the right thinking’ sentiments
summed up by the following comment:
“… you need the focus and need the way forward. You need a bit of help
sometimes in achieving your goals, and in some form or another, I wouldn’t say
all of it but some parts of it we have actually used.” Thorpe Kilworth
More negative responses related to the content’s ‘level’ and its ‘relevance’,
illustrated by the following feedback:
55
“Nothing new in terms of content. I do it as part of the day job.” Pendragon
“I found some of it quite frustrating really because it was below the level I’d
already achieved…I think there was no attempt made to try and work out the
levels that different people were at.” Northampton College
However, some of the respondents for whom the content was revisiting existing
knowledge could see benefits, for example;
“We weren’t taught anything that we didn’t already know but it did highlight
those things and bring them to the forefront to make us think about it…reignited
the memory.” Caterpillar
As Table Nine reveals in the final interviews with the gatekeepers, 4 reported that
the achievements of their organisational projects would not have occurred at all
without the initial short programme (all from smaller businesses), 2 that these
would have to some extent, 4 to a large extent, 3 that they would have been
achieved without this element and 1 felt unable to answer the question. In
personal development terms the participants variously identified the ‘shared
experiences’, ‘being able to put things into context’, ‘helped to focus energies’,
whilst others were of the view that the programme’s workshops ‘would have been
of more help to others with less experience’. Responses from the two public
sector organisations were consistently the least positive about the initial
programme, possibly due to the complexity and size of their organisations or a
greater degree of developmental opportunities traditionally being made available
to employees in this sector.
The Proj ect Launch Event
One element of the programme singled out in the telephone interviews which was
not revisited in the final interviews was the IWP project’s launch event. Whilst not
a specific element of the project design, Table Eight illustrates that this was the
most highly rated aspect of the programme, rated as either ‘extremely useful’ or
‘useful to a large extent’ by all those gatekeepers who attended (one did not
attend). It is worth examining why respondents rated it so highly. It took place
on 23 September 2009 and the detail of its content is described in Section One.
An invitation to attend this launch event was extended to participant
gatekeepers, their senior managers and employee representatives from
participant organisations. The event was identified by the gatekeepers as
providing the whole project with ‘credibility’ particularly in the eyes of key
individuals in their organisations whose support was needed. These perceptions
are illustrated by the following quotes:
“Excellent, the people, the real case studies it was so informative…it was
inspiring.” Thorpe Kilworth
“It was particularly helpful that we were able to bring a wider audience from our
company and we managed to bring someone from HR and a senior union rep. So
it helped to explain the project …the speakers were brilliant.” BEM
“I think it was a really good opportunity to gain insights from other businesses
and companies.” Caterpillar
56
“We were able to take people from the business which gave credence to the
project and we were able to hear from other companies, disappointed in the
limited places.” SHA
The Participant Network Meetings
Six whole day network meetings facilitated by UKWON were held from October
2009 to March 2010. The design for these days, provided by UKWON, was that
the morning session of each day was devoted to a thematic discussion of
common problems and the afternoon was spent with the participants in action
learning sets in which each organisation could present progress, achievements,
obstacles and dilemmas for peer review and discussion. These were followed by
two half day meetings of the action learning sets, again facilitated by UKWON, in
May and September.
The location of these meetings varied in that different participant workplaces
were chosen to host these network events. The schedule was as outlined below:
October 2009 Overview of employee engagement and workplace
innovation
November 2009 Managing successful change (Donington Park)
December 2009 Case studies of workplace innovation
January 2010 Employee engagement event with Nita Clarke (BEM)
February 2010 Progress review on action plans, building trust and
breaking down silos.’
March 2010 Sustainability of change; reviewing the role of Gatekeepers
- (SHA)
Source: Programme of Network Meetings
In the telephone interviews, 9 respondents indicated that they felt the meetings
had been ‘extremely useful’ or ‘useful to a large extent’. The remaining 4
respondents rated the meetings as ‘partly useful’ (Table 8).
The most beneficial aspect articulated by respondents during both the telephone
and final interviews (confirmed by the supplementary information on the IWP
provided by some of the gatekeepers to UKWON), was once again, the
opportunity to share issues, problems and experiences with other gatekeepers.
This aspect was mentioned by 8 of the respondents
Table Nine reveals that by the final interviews, 3 gatekeepers reported that the
achievements of their organisational projects would not have occurred ‘at all’
without the network events, 4 that these would have to ‘some extent’, 5 ‘to a
large extent’, and 1 that they would have been achieved ‘without’ the network
events. In response to the question about the contribution of the Participant
Network meetings to the final outcomes either to the organisation or to the
participants personal ‘sharing of experiences’ was the single most reported
benefit.
Whilst the respondents predominantly highlighted the sharing opportunities
presented by the network meetings, learning from the thematic content of the
network meetings was evident when a formal review took place at a half day
57
learning set meeting held on 20
th
May 2010 (observed by members of the
evaluation team). Participants during this session articulated the learning they
had acquired which had relevance for them as individuals and for their
organisations.
Action Learning Sets
The responses in the telephone interviews set out in Table Eight reveal that the
majority of gatekeepers viewed the action learning sets as either ‘extremely
useful’ or ‘useful to a large extent’.
By the final interviews of the 13 gatekeepers, 4 reported that the achievements
of their organisational projects would not have occurred ‘at all’ without the action
learning events, 2 that these would have to ‘some extent’, 4 to a ‘large extent’
and 3 that they would have been achieved ‘without’ this element. Responses that
mirrored those for the initial action learning short programme which suggests
that this is a type of learning works well for some but not for others.
The responses from gatekeepers were generally positive in their view of the
action leaning sets as a useful mechanism for sharing information with one
another. However, it was apparent in the telephone and the final interviews that
a number of respondents were not altogether clear about what constituted the
action learning set element of the programme. To specifically find out about this
as a core element of the programme design, there were questions relating to the
action learning sets in the evaluation interviews but the gatekeepers’ difficulties
in singling these out from other events at the participant network meetings are
illustrated by the following observations:
“Same as the morning sessions giving and receiving advise.” BEM
“Sometimes good sometimes not so good, been a bit of a mixture.” Liquid
Control
Varying levels of attendance may have led to logistical difficulties in having two
learning sets on each occasion. This may have led to difficulties for certain
respondents in distinguishing between the morning event and the afternoon
action learning. The preference of some gatekeepers was for smaller action
learning sets as illustrated by the following comment:
“smaller groups certainly benefit the likes of me. I don’t like speaking up in larger
groups. Small groups let us get to know one another better and we bounced
ideas off one another…” Liquid Control
Other gatekeepers felt the action learning sets were useful but would have been
of even more value if there had been organisations in the same industry/sector
as themselves, for example:
“they were very useful for me. I have picked up a hell of a lot from people who
talked, but would have liked other similar organisations to have been involved.”
SHA
58
Acas Facilitation
At the time of the telephone interviews, when the facilitation had ended with the
exception of Northampton College, 62 per cent of respondents perceived the Acas
facilitation to be either ‘extremely useful’ or ‘useful to a large extent’. 3 indicated
the facilitation had been ‘extremely useful’, 5 that it had been ‘useful to a large
extent’ and 4 that it had been ‘partly useful’. Acas facilitation was reported as
taking a variety of forms which included assisting with the introduction of joint
forums, the facilitation of focus groups, producing reports, meetings with
members of the organisation and gatekeepers, holding joint problem solving
groups, a skills audit, organising bespoke training events, as well as email and
telephone contact.
A number of gatekeepers reported that they had not used their facilitator early
enough or made as much use of this element of the programme as they
subsequently realised they should have done, for example:
“It was our fault that we possibly didn’t use the facilitator as much as we could
have done. April (the end of the programme) arrived too quickly. We needed to
involve the facilitator earlier in the process.” Liquid Control
“We didn’t have long enough. Support for a full year would have been
welcomed.” Pendragon
Part of the explanation for such comments might lay in the timing of the initial
involvement of the facilitators with their respective participant organisations, all
the facilitators reported in their evaluation of the IWP that their earlier
involvement with the IWP would have been helpful to both parties, particularly in
establishing terms of reference for the organisational projects and ensuring that
there was senior management support. (Facilitator interviews)
A couple of respondents also identified that ‘personal chemistry’ between
facilitator and recipient gatekeeper did not always result in an immediate
productive relationship but could take time to develop as evidenced by the
following observations:
“It was partly useful. It took a long time to get going. They probably found us
frustrating to work with. I didn’t feel they were particularly pro-active.”
Northampton College
The issue of relationships taking time to develop is particularly illustrated by the
evaluation provided on this element at Northampton College where the
facilitation continued for much longer than elsewhere due to a slow start. The
gatekeepers’ perception of the facilitation process changed radically as the
process progressed as illustrated by the following feedback at the final
evaluation:
“It has made the biggest contribution for us. It was useful for the college to have
independent people who were experienced facilitators, with a proper reputation
and the kudos of Acas.” Northampton College
Six months following the telephone interviews, the final evaluation revealed that
respondents had become increasingly positive about the Acas facilitation with 12
59
(92 per cent) identifying that they would have not achieved their project’s
outcomes ‘at all’ or only ‘to some extent’ without the facilitation input with the
exception of one gatekeeper whose reported experience differed from the other
gatekeeper in the same organisation. The following observations typify the
positive feedback on this element of the programme:
“It was good. It couldn’t have happened without the Acas facilitation.”
Pendragon
“I would say our adviser had a very important role. Although they were back
stage they were driving us how to approach it in a very subtle way. Basically they
equipped us.” SHA.
“…feel overall that that was one of the key strengths out of all the different
elements…” Thorpe Kilworth
One explanation for the increasingly positive reporting of the impact of the Acas
facilitation was that its contribution not only took time to become apparent but
also that certain facilitation activity took time to generate results. Another factor
may be that open Acas courses had been attended which could have had a
positive influence on gatekeepers’ perceptions; these are now considered.
Acas Training
Whilst not an element of the initial design for the IWP project, Acas provided a
number of training courses for participants in areas of identified need as follows:
Acas ‘Open’ training events for the IWP
Date Title of event No of delegates
9
th
March 2010
Meetings, consultation and
Forums – some useful basics
6
10
th
March 2010
Having difficult conversations
6
12
th
March 2010
Having difficult conversations
6
13
th
July 2010
Facilitation Skills Training
11
30
th
September,
1
st
, 7
th
October 2010
Developing Mediation Skills
7
According to the attendance records a total of 23 delegates, including 10
gatekeepers, from across the participating organisations took advantage of
attending one or more of these events; one organisation had employees
attending all four courses.
60
At the telephone interviews Table Eight reveals that out of 8 gatekeepers
responses 5 respondents rated these courses as ‘extremely useful and 3 rated
them as ‘useful to a large extent’.
During the final interviews gatekeepers were asked, ‘did you or others in your
organisation receive inputs in addition to those already mentioned e.g. any
additional training organised through Acas or UKWON?’
Three respondents indicated that their organisations had receive some bespoke
training interventions from Acas: The Health Store and Pendragon had training
for their employee representatives; Caterpillar for the members of its new
employee forum; and Pendragon had training for its team leaders. A further two
organisations had received bespoke development interventions from UKWON. At
Thorpe Kilworth input on team working was provided whilst at Northampton
College UKWON provided assistance to help clarify the way forward with the
project. UKWON also arranged a visit to the chocolate manufacturer, Thortons
plc, as a developmental opportunity for the gatekeepers. Those who attended
this visit reported very positively on the event.
3.13 Messages for the Future
The messages or ‘lessons learnt’ from the evaluation of this pilot initiative for
similar projects in the future are drawn from the following sources - the
gatekeepers’ suggestions; the reflections of the delivery partners and the
reflections of the Acas facilitators.
Gatekeeper Suggestions for Future Programmes
During the final interviews the gatekeepers were asked what changes, if any,
they would propose to the different elements for any future programme. This
question did not include any additional inputs that had been provided as the IWP
project progressed.
Whilst some respondents were satisfied with the programme content, others
made proposals for change. As might be expected their responses reflected their
own particular learning style, level of knowledge, size of their organisation and
role. Gatekeepers’ comments received are reflected in Table Ten.
61
Table Ten: Gatekeepers’ proposals for future programmes
? Delivery interspersed with action including pre-work
? Smaller groups
? Better understanding of the organisational projects before the action learning
delivery input so enabling the input to better support the projects so adding
maximum value
? Greater tailoring to particular organisational contexts
? Short course input throughout the duration of the project instead of all at the
beginning
? Clearer structure to the network days and action learning sets
? Longer and earlier support from facilitators
? Participant organisations from the same sector
? One venue for the network days and action learning sets
? More specialist inputs at events e.g. speakers on relevant topics
Reflections of the Delivery Partners
The Acas Project Manager and representatives from UKWON (‘the delivery
partners’) were interviewed face to face at the beginning of the IWP about its
aims and objectives. They were interviewed again, post project, to gain their
reflections on its outcomes and to identify any changes or areas to be addressed
for similar programmes in the future. In terms of the IWP’s overall impact, both
delivery partners felt that a major achievement of the project had been that it
had raised awareness about the value of employee engagement and moved the
participant organisations towards greater employee involvement; a view that was
shared by the Acas facilitators. Their perception, supported by the evaluation
evidence, was that as participants gained in their understanding of the
importance of employee engagement for the business, the more they recognised
that this took time and effort to achieve and sustain. As already reported,
participants increasingly viewed the outcomes from their individual action plans
as part of an ongoing programme of improvements.
Asked what they would wish to review or change in future programmes, in no
order of priority, the key points identified by the delivery partners were:
? Allow more time to recruit; this had been constrained in the IWP due to
budgetary considerations. In addition, UKWON suggested that a self
assessment questionnaire could be used in the recruitment process. It was
felt that this would help applicants to clarify their objectives and provide
the opportunity for self reflection in terms of what they were seeking from
participation.
? Provide more detailed structured information about the programme once
the final gatekeepers had been selected. There was a lack of knowledge
about the IWP and what it involved among some of the finally appointed
gatekeepers due to last minute organisational changes and insufficient
internal briefing from those who had taken part in the selection process.
? Establish the level of commitment from senior managers at the outset. It
was reported that, whilst this might have been articulated at the selection
62
stage, there were instances where it was not evident when the workplace
project got underway - a situation that was exacerbated when there were
changes in senior management during the project. Senior management
support was also identified as a critical success factor by the facilitators
and is discussed further below.
? Consider introducing mechanisms to maintain contact and discuss
progress with senior management during the project as well as afterwards
to sustain momentum, for example periodic meetings.
? Ensure that there was sufficient Acas facilitation expertise available to
support participant organisations and consider ways in which this could be
developed more widely without impacting on the quality of provision.
Reflections of the Acas’ Facilitators
Telephone interviews were held with the Senior Acas Advisers involved in the IWP
in order to capture their reflections on the initiative and their own experiences as
facilitators. These took place when the facilitation process had ended with the
exception of Northampton College where it was still ongoing. The information
gained from these interviews was supplemented by end of project reports
completed by the individual facilitators which were made available to the
evaluators. In addition, throughout the IWP, member(s) of the evaluation team
attended the facilitator network meetings which took place from September 2009
to September 2010. These different sources generated a considerable amount of
data about the facilitation element of the programme. They revealed a very high
level of consistency among the facilitation team in terms of what they felt needed
to be reviewed for similar projects in the future. All the facilitators identified that
the key achievement of IWP programme had been that it had acted as a catalyst
for more attention to be paid to issues of employee involvement. In addition half
of them reported that the organisational role of the gatekeeper had a bearing on
their ability to progress workplace action plans
Undertaking the facilitator’s role was described both as ‘personally
developmental’ and ‘very worthwhile’ by facilitators. They reported that they had
welcomed the opportunity to work collaboratively, and in depth, with
organisations to improve workplace relationships in order to enhance
organisational performance. Having a protracted period of time to support
workplace projects was seen as a real opportunity to make a difference, and the
approach was seen as a chance to fully support the principles embedded in the
Acas Model Workplace (Acas, 2005). It was reported that it would have assisted
the facilitators in their roles if the different elements of the IWP had been more
clearly integrated and if they had been more aware of what was involved in other
areas of the programme. A key learning outcome reported by nearly all the
facilitators was the importance of ‘getting to grips’ with the culture of the
organisations and the pace at which things could be progressed. As one very
experienced facilitator observed:
“ I learnt that I had to go backwards to go forwards which paid dividends in
terms of gaining trust in the longer term … but it all took time.”
In terms of their experiences of the IWP, the following issues were identified as
areas for attention in any future initiative. (A number of these issues were also
63
touched upon by the delivery partners and, most particularly, by the Acas Project
Manager).
? Put in place agreed ‘Terms of Reference’ signed off with senior
management (with the involvement of the allocated facilitator) for each
workplace project before it began. The rationale for this proposed action
being that it would reduce the reported difficulties facilitators experienced
in clarifying and understanding the focus of the organisational project.
This action would address the issue of executive level support and also
their common experience that, where there had been a lack of clarity
about the aims of a project, it had seriously impacted on progress. This
view is supported by the evaluation evidence which identified that many
projects made slow progress in the initial months and that ‘getting things
started’ was a key role for the facilitators at the beginning of the
programme.
? Involve the facilitators as early as possible in any future project so that
they could develop their relationship with the organisations they were to
work with and their understanding of its issues and culture.
? Consider how facilitation experience and skills could best be developed,
particularly in terms of being able to be flexible, innovative and resilient
when things did not go to plan or organisations are less responsive than
anticipated. It was acknowledged that the level of experience of
undertaking the role varied across the team and sharing learning and
specific experiences were considered a vital part of developing facilitation
skills.
? Provide inputs from another experienced facilitator, including their
presence at meetings in the workplace, where there were particularly
difficulties or a lack of progress. In practice, as the IWP progressed a
couple of organisations had two facilitators working with them which
overcame some of the issues which could face a facilitator when operating
alone.
3.14 Paying for Future Programmes
The IWP project was free to participants, whilst the mainstream in-company
advisory support provided by Acas is charged. In order to gain some indication of
organisations’ willingness to pay for such initiatives in the future, and to
contribute to future planning, the survey explored what the participating
companies would be willing to pay for a similar project to the IWP and those
factors which would prevent them paying for such a project in the future. Asked
what would be considered a low price, but a level that would not give rise to
concerns about quality, the lowest price was £5,000 but the survey respondents
in the three smaller companies were unable to answer the survey question. When
asked what price would be considered too expensive to consider participation, the
responses varied from £10,000 to a maximum of £20,000 but, again, the smaller
businesses did not answer the question. Only four of the larger organisations
identified factors that would prevent them paying for a similar project in the
future; these were reported as the economic climate, their ability to pay and that
participation would depend on evidence that the programme would increase
64
turnover. The smaller companies, who consistently reported the most positive
outcomes from participation, had identified that they would not have been able to
pay for the support provided by the IWP in the final evaluation interviews; a
position that was reinforced by their responses in the final survey.
65
SECTION FOUR – ECONOMIC IMPACT
4.1 Summary of the Economic Impact of the IWP
This section summarises an analysis of the economic impacts of the emda
funded, Acas led 'Innovative Workplaces' project, developed in line with the
principles set out in emda's evaluation toolkit. The analysis is primarily based on
a baseline and follow-up questionnaire undertaken with participants of the
projects. Full details including all calculations which feed into the estimates in
this section are in Appendix D.
As outlined earlier, eleven organisations participated in the Innovative
Workplaces project, although three withdrew over the course of delivery,
resulting in a total of eight completing the project. The analysis is restricted to
eight completers, though it is acknowledged that further impacts were achieved
amongst the three who did not complete the project.
4.2 Availability and Take-up of Alternative Sources of Support
A key consideration in assessing the impact of the IWP is the extent to which
participants would have taken up a similar alternative source of support in the
absence of the project. If participants would have obtained similar support
elsewhere, then it is likely that any such outcomes would have been achieved
anyway. Overall, the evidence suggested participants would not have accessed
similar services elsewhere, with an estimated probability that organisations would
not have taken up alternative support of 84 percent.
4.3 Improvements made by Participating Organisations
Respondents were asked to report whether they implemented new or improved
human resource policies or procedures (or planned to make such improvements
in the future) during or since participating in the Innovative Workplaces project.
Seven of the eight organisations surveyed had implemented at least one new or
improved process and all had at least plans to implement new or improved
processes. Four reported that they had plans to implement improvements in the
future.
To identify how far these improvements were made as a result of the Innovative
Workplaces projects, respondents to the postal questionnaires were asked to
report the extent to which they would have made these improvements without
the support they received from Acas. On the basis of survey responses, it was
estimated that there was a 32 percent probability that participating organisations
would not have implemented improved HR procedures without Acas support.
4.4 Gross Additional Impacts of Improvements
Respondents were asked to report how far the improvements made had resulted
in improvements in productivity, or helped them create or safeguard jobs. They
were also asked to report if those improvements would have such effects in the
66
future. Estimates of the gross additional economic impacts of the Innovative
Workplaces project are set out in the table below (i.e. those changes in
productivity, employment and Gross Value Added (GVA)
1
that would not have
occurred without the project).
Table Eleven: Gross additional economic impacts
Impact a)
Additionality
of the
project
support
b)
Additionality
of actions
c) Economic
impacts
improvements
Gross
additional
economic
impacts
Employment impacts
Jobs safeguarded 0.84 0.32 2.0 0.5
Potential jobs created 0.84 0.32 13.5 3.7
GVA impacts
GVA safeguarded (£
per annum) 0.84 0.32 72,000 19,592
GVA created through
productivity gains (£
per annum) 0.84 0.32 566,665 154,195
Total GVA created to
date - - 638,665 173,787
Total potential GVA
created
(£ per annum) 0.84 0.32 486,000 132,245
Source: Ecorys analysis (gross additional economic impacts are estimated as the
economic impacts attributable to improvements x additionality of support x
additionality of actions: a*b*c).
The above table summarises data from three aspects of improved organisational
performance resulting from the project:
1. Employment growth. Organisations were asked to report whether the changes
they had made had resulted in them recruiting any additional workers (if
employment had increased) or protected any jobs (if employment had remained
the same or decreased). As noted in previous sections, the majority of
participating organisations had seen a contraction in their workforce, with one
organisation reporting that the changes they had made had helped safeguard two
jobs as a result of the changes made. Applying average GVA per worker in the
region (£36,000), this equates to GVA safeguarded of £72,000.
1
GVA is measure of the total output of the organisation, the value added by the
organisation to the goods and services it absorbs in the production process. In practical
terms, this is measured by subtracting the value an organisation's purchases of goods and
services (ranging from raw materials through to expenditures on property services) from
the value of what an organisation produces (which can be approximated by the value of
sales).
67
2. Future impacts on employment. The follow-up postal questionnaire was
undertaken six months following organisation's initial participation in the IWP.
Many of the process improvements delivered by organisations will take time to
implement (and as highlighted, some participants were still at the planning stage
with respect to some areas of improvement). Additionally, the effects of
improvements may take time to arise, so it is likely that a focus on the economic
impacts of the IWP to date understate the total effects of the project.
In order to capture potential future employment impacts of the Innovative
Workplaces project, participants were asked to report whether they would recruit
any additional workers over the next twelve months as a result of the
improvements they made to their business. Two of the eight organisations
reported that they planned to recruit a total of 13.5 workers in the next 12
months as a result of implementing process improvements, equating to a
potential per annum GVA impact of £486,000 (again applying GVA per worker in
the East Midlands of £36,000).
3. Increased productivity. Productivity is a measure of the output (GVA) an
organisation can produce for a given level of inputs, and is typically measured by
GVA per worker. GVA per worker might rise (among other reasons) if workers
become more efficient or skilled, if organisations replace workers with capital
equipment or machinery, or if organisations adopt more efficient production
processes. Two organisations reported a (measured) growth in turnover per
worker of £9,417 and £30,500, and applying the ratio of GVA to turnover in the
East Midlands (34 percent) this equates to productivity growth of £3,201 and
£10,374 per worker respectively. Aggregating this across each organisations
number of workers gives an overall estimate of gross additional GVA through
productivity gains of £567,000 (having applied the estimated average probability
that productivity gains would not have been achieved without the changes made,
but before applying the estimated project and support additionally probability).
4.5 Leakage, Displacement, Substitution and Multiplier Effects
In order to move from gross additional to net additional economic impacts
requires consideration of leakage, displacement, substitution effects and
multiplier effects:
? Leakage: this looks at how far the intervention has resulted in impacts
leaking outside of the target area. All organisations participating in the
project were based in the East Midlands, so there is no leakage of GVA
impacts. 85 percent of participants' employees lived in the East Midlands,
implying a leakage of employment impacts of 15 percent.
? Displacement: this is where improvements in the performance of assisted
organisations come at the expense of those not receiving support, and for
this project displacement was assumed to be in the region of 20 percent,
in line with wider evaluation evidence of enterprise support initiatives in
the East Midlands.
? Multiplier effects: where organisations are able to improve their
performance through greater sales or productivity, they will consume
more goods and services provided by other organisations based in the
68
East Midlands, generating wider economic impacts (supply chain multiplier
effects). Equally, where individuals fill any jobs created as a result of the
project or are able to increase their earnings as a result of becoming more
productive, there will be further multiplier effects as they spend their
additional income in regional businesses (induced multiplier effects).
Multiplier effects were estimated at 1.39 on the basis of the industrial profile of
organisations receiving support from the Innovative Workplaces project.
4.6 Present Value of GVA Impacts
Table Twelve: Net additional economic impacts per annum
Impact Gross
additional
economic
impacts
Leakage Displace
ment
Multiplier
effects
Net
additional
economic
impacts
Employment impacts
Jobs safeguarded 0.5 0.15 0.20 1.39 0.5
Potential jobs created 3.7 0.15 0.20 1.39 3.5
GVA impacts
GVA safeguarded (£ per
annum) 19,592 0.00 0.20 1.39 21,786
GVA created through
productivity gains (£ per
annum)
154,195 0.00 0.20 1.39 171,465
Total GVA created to
date
173,787 - - - 193,251
Total potential GVA
created (£ per annum)
132,245 0.00 0.20 1.39 147,057
Source: Ecorys analysis (Net additional economic impacts at the regional level are estimated
as: Gross impact x Additionality x (1 – Leakage) x (1 – Displacement) x Multiplier Effects).
Estimates of the net additional economic impacts (per annum) of the IWP are set
out in Table Twelve above. Total net additional GVA created or safeguarded (per
annum) is estimated at £193,000, with a further potential £147,000 per annum
created by November 2011.
The GVA impacts of enterprise support initiatives will grow and endure for a
period of time, though in general, such effects are expected to decay at a certain
point. The effects of enterprise support should be assumed to endure for three
years, in line with guidance issued by BIS. The table below shows the projected
time profile of GVA impacts (together with the present value of those impacts,
applying the 3.5 percent value for social time preference recommended by the
HM Treasury Green Book)
1
.
1
The value of GVA is discounted to reflect a social preference for income today over an
equivalent income in the future. The 3.5 percent discount rate suggests that society as a
69
Table Thirteen: Present value of GVA benefits
Year Present value of GVA
impacts created to
date (£)
Present value of
potential GVA impacts
(£)
Present value of
total GVA impacts
(£)
2009/10 193,251 - 193,251
2010/11 186,716 142,084 328,799
2011/12 180,402 137,279 317,681
2012/13 - 132,637 132,637
Total 560,368 411,999 972,368
Source: Ecorys analysis
4.7 Return on Investment
Acas figures indicate that a total of £244,035 was spent delivering the IWP, and
the project was estimated to have created economic impacts with a present value
of £972,400. This gives an overall return on investment to public sector (in
regional economic impacts) of £4 for every £1 of public sector expenditure. emda
contributed £227,437 of total project costs or 93 percent of total public sector
expenditure. In line with OffPAT guidance, it is assumed that 93 percent of the
overall impact of the project (£906,232) is attributable to emda's expenditure.
A total of £108,152 of costs represented project management and evaluation
costs, which due to the pilot nature of the project are likely to reduce if replicated
in the future. The table below provides ROI figures under scenarios in which
these costs fall by 25 percent and 50 percent.
Table Fourteen: Return on Investment
Costs & ROI Cost (£) Economic
Impacts
(£s)
Return on
Investment
Emda 227,437 906,232 4.0
Total public sector 244,035 972,368 4.0
Potential ROI for future replication of the project, if evaluation & project management
staff costs reduced by:
25% 216,997 972,368 4.5
50% 189,959 972,368 5.1
whole is indifferent between £100 of income (or costs) today and £103.50 of income/costs
a year later.
70
Costs and benefits that have not been quantified
While this analysis suggests that the IWP has generated a positive return on
investment over the short period over which impacts have been allowed to
accrue, it takes a narrow economic view of the benefits of the project. There are
a range of wider effects that would be considered in a full cost-benefit analysis of
the intervention.
? Economic impacts: There are a number of economic impacts of the project
that have not been quantified in this analysis, including productivity
safeguarded and any future employee disputes and Employment Tribunal
claims avoided in the future.
? Social impacts: There may also be a range of social benefits of
improvements in HR processes over and above those described in the
economic impact assessment.
? Project costs for the participating organisations: Estimates of return on
investment outlined above do not factor in the costs participants incur in
implementing improvements, such as management time.
? Disbenefits: Where the Innovative Workplaces has facilitated growth or
raised productivity, there will be a range of social costs that are not
captured in the analysis such as wider externalities associated with
economic growth including the environmental impacts associated with
greater utilisation of resources or congestion costs caused by more
intensive use of transport infrastructure.
With regard to economic impacts not quantified, it should be noted that the
present value of GVA impacts and return on investment figures that have been
calculated only include economic impacts that could be specifically identified and
measured in the course of the project evaluation using this framework. Further,
it was only possible to attribute economic impacts to four of the eight
organisations which took part in the final evaluation of the project (by using
questionnaire answers given by the organisations). It is assumed that all other
organisations had no economic impacts as a result of participation, when the true
economic impact of these organisations is unknown. An economic impact of zero
is, of course, not realistic – however, the report can only quote figures which
have been reported and a cautious approach is taken and so estimated GVA
impacts can therefore be considered a minimum return from this project.
71
SECTION FIVE – CONCLUSION
5.1 Why the Messages from the IWP Evaluation are of Value
The funding available for the IWP meant it could only be offered to a small
number of organisations in the East Midlands. However, the project offers
valuable insights and messages for similar future initiatives. A number of
dimensions make the IWP especially interesting for wider application.
a) The project adopted an innovative approach by offering different forms of
workplace support and development, including customised workplace
facilitation, to the participating organisations.
b) Although the IWP programme lasted for a relatively short time, it was longer
than many developmental or business interventions and was focussed at the
workplace level.
c) The evaluation methodology was designed to ensure that a record was made
of all changes that took place within the participating organisations over the
life time of the project and six months after its core elements had ceased;
and that lessons learned were captured from the perspective of multiple
stakeholders.
5.2 The Project’s Impact on the Participant Organisations
The evaluation identified that the IWP’s major impact on the participating
organisations had been improvements to workplace communication and
employee engagement; identified as the key ingredient for riding ‘the economic
storm’ in the results of the Sunday Times 100 best companies in 2010 (Sunday
Times, 2010). There are recognised difficulties in defining what is meant by
employee engagement; indeed MacLeod and Clarke in their 2009 review
identified a plethora of different definitions which included a practitioner definition
of engagement as ‘when the business values the employee and the employee
values the business.’ (2009:7). This interpretation of employee engagement is
probably the closest to the IWP participants’ reported understanding of
engagement during the evaluation. The key components of this definition,
evidenced by the workplace changes during the IWP, were regarded as
communication with and the involvement of the workforce, providing mechanisms
for employee voice and adopting a joint problem solving approach.
These changes led to business benefits. The economic impact assessment of the
IWP reported an overall minimum return on investment of £4 for every £1 of
public sector expenditure and a measurable positive economic impact of
approximately £906,000 attributable to the IWP’s expenditure.
It was estimated that 50 percent of the productivity gains reported in the final
survey data by the participating organisations would not have occurred without
the improvements resulting from the IWP. The perception of respondents was
that the project had improved productivity even though other factors, not least
the severe recession for much of the project, meant that the reality for a number
of the participating companies was a drop in turnover. Notwithstanding, the
72
message from the participant organisations was that the situation would have
been considerably worse without their participation in the project; a perception
that reinforced their commitment to continue to participate in the IWP if at all
possible. The overall evidence from the IWP’s outcomes is that it improved
employee engagement can enhance performance which supports the conclusion
of MacLeod and Clarke’s (2009) review of employee engagement.
Multiple benefits were found to be associated with enhanced employee
involvement. For example, managers reported that they found they could
delegate more, and a frequently identified benefit was a reduction in the time
they spent handling individual disputes. The economic impact estimated that the
participating firms saw a reduction in the time spent on grievances, disciplinary
sanctions and Employment Tribunal claims, issues which, as Gibbons (2007)
observes, have significant costs for GB businesses. The introduction of improved
or new mechanisms for employee involvement and consultation outlined in
Section Three and the case studies stemmed from the Acas facilitation process.
The evaluation revealed that, whilst these provided forums for improved formal
communication with employees, they also led to improved informal
communications. This was particularly identified in the smaller businesses but it
was also reported where ‘siloed’ working or divisions between groups of staff had
been the focus of an organisation’s action plan. The evaluation findings do,
however, lend support to the findings of Hall et al. (2009) that there can be
particular business benefits of informality in approaches to sharing information
and employee consultation in medium sized firms.
Tangible outputs from the project were evidenced by seven of the eight
organisations reporting the implementation of at least one new or improved
human resource procedure or process improvement. These were most frequently
in processes for informing and consulting with employees and absence
management and all had plans for future improvements following participation in
the IWP. Other positive outcomes associated with improved employee relations
climate (reported by the majority of organisations) included improvements in the
management approach and increase in workplace morale and trust between
employees and management since the IWP began. In two organisations which
had undertaken recent employee surveys there were increased scores for
employee engagement compared to surveys prior to the project. Respondents
found it difficult to single out the extent to which reported improvements in
workplace morale and levels of trust were due to the IWP although it was widely
described by participants as having been the catalyst for the reported
improvements. Where such improvements were not identified it was explained
that there were other events negatively impacting on morale, for example, as
already mentioned workplace redundancies or uncertainty about the
organisation’s future.
Forums for employee involvement had encouraged employees to contribute ideas
and their views on a range of issues. These mechanisms emerged as a key factor
in stimulating increased engagement in the IWP, supporting the findings of Truss
et al. (2006: 39) that allowing people to feed their views and opinions upwards is
the single most important driver of engagement. This had led to greater
proactivity among some of the organisations’ employees in terms of raising
issues and contributing ideas even in areas that had not necessarily been
anticipated in their initial action plans.
73
Although the evaluation revealed an increased commitment to an ‘on going’
programme of improvements as the project developed, there were concerns
about sustaining the momentum without the project and fears that there could be
a return to the situation prior the project. This was especially the case where
individuals who had acted as champions for the internal projects had moved on
or left the organisation.
The most positive outcomes from the IWP were consistently reported by the
SMEs with less than 200 employees and there was also more uniformity of view
across the respondents from these organisations of what had been achieved and
the value of the IWP programme. This may be due to the greater visibility of
improvements in smaller businesses and the ease with which these can be
communicated. Another explanation may lie in less previous take up of other
sources of external support because of the costs involved.
5.3 Developing Management and Leadership Skills
One of the keys aims of the project was to improve business performance
through the development of essential management and leadership skills primarily
through ‘cascading’ the gatekeepers’ learning to others in their organisations. The
final evaluation revealed that the majority of respondents felt this had happened
partly or to a large extent both through the gatekeepers. The additional input of
Acas short courses provided to others in the organisation also contributed to this.
The workplace benefits resulting from the development of these skills were again
reported most positively by the smaller businesses. This could be partly due to
the roles of the gatekeepers being more central within these organisations so
they were better placed to both share and apply their learning or because of less
previous development of these skills. It was the case that some of the larger
organisations felt that the programme did not offer them much that was new in
this area but this is not to suggest that they were of the view that these skills did
not require further development.
Whilst part of the criteria suggested to the participant organisations for selecting
their gatekeepers was that they should be proactive individuals who ‘would get
things done’, the evaluation’s findings revealed that in order to drive workplace
changes forward, the level of responsibility and job status of the gatekeeper were
important factors; this is an issue for consideration in any similar future projects.
For all the gatekeepers the project provided an opportunity for considerable
personal development as reported in Section Three. In some instances individuals
had benefited significantly but it did depend on the background, prior knowledge
and (as mentioned) the job status of the gatekeepers. Whilst some felt the
learning they had gained from participating could not be immediately applied in
their current roles, all anticipated that this would be of benefit in the future.
There were also reported wider benefits for some of the gatekeepers such as
improved self confidence, enhanced organisational knowledge, working
collaboratively outside their own area and the networking relationships they had
developed with participants from other organisations.
74
5.4 The Value of the Different Elements of the IWP
Perceptions of the value of each element of the programme varied for each
organisation and the gatekeepers’ perceptions depended on what they were
seeking from the project. The need to accommodate this diversity was recognised
by UKWON and had informed the design of the elements they provided. Their
stated initial aim was to ‘ground pre-existing knowledge in the task at hand’ for
gatekeepers with prior knowledge and experience and ‘to provide sufficient
actionable knowledge on employee involvement and participation’ for those with
less knowledge and experience. The evaluation evidence suggests that this was
largely achieved but it was essentially the combination of the different elements
of the overall programme that had led to the final outcomes from participation in
the IWP. In practice, each organisation took what it needed at different times
from the project.
Although the impact of the project stemmed from the ‘sum of its parts’, by the
time of the final evaluation it emerged that the customised Acas facilitation
process was regarded as particularly useful in terms of its contribution to the final
outcomes. Perceptions of the value of the facilitation element of the programme
did increase with time, possibly when it became more apparent what it had
provided. This may explain why some participants wished they had made more
use of their facilitator before the facilitation process ended and they would have
done so if they had realised earlier what it could offer. The majority of
gatekeepers reported that they would have liked continued support from an Acas
facilitator but only those from larger organisation felt that there would be a
willingness to pay for this if there was evidence that it clearly improved
performance.
There were areas of this provision where the Acas facilitators themselves
identified the need for changes in any future programmes. These can be summed
up as: ensuring their earliest possible involvement; having initial terms of
reference agreed with senior management in place before workplace projects
began to provide the focus and clarity essential to progress; facilitators having
access to additional support when difficulties arose; and strategies to deepen and
widen facilitation capability among Acas advisers. The evaluation findings support
the view that the facilitators need to be involved at the earliest stage, and that
senior management support in workplace projects is critical. The need for clarity
about the aims of workplace projects also emerged as important as did the
significance of the facilitation element being well integrated with the other
elements of the programme. The evaluation also identified that the IWP’s
facilitation element provided a unique opportunity to support the embedding of
the Acas Model Workplace’s principles into an organisation and provides further
evidence that Acas activities can bring longer-term improvements to the
employment relations climate (Meadows, 2007:3).
The participant network events were highly valued for the network opportunities
they provided to the gatekeepers and the action learning element of these days
provided the means of sharing progress and problems, testing out thinking with
others and learning about the approaches of other organisations. These were
identified as key benefits of the IWP by a number of gatekeepers. Whilst the
IWP’s aim was to have as diverse a range of organisations as possible, this did
lead to some gatekeepers reporting that it was difficult to transfer their learning
75
about the practices of other participants on the IWP because their organisations
were so dissimilar in size, their product or service.
Whilst gatekeepers reported that there was always some value for them in
attending network events, a number wanted more structure and clarity about the
action learning sets. It was also suggested that some input from organisations in
the same sector or industry would have been a benefit, for example, as a mentor
or coach to participants from organisations in the same industry. It is also worth
noting that the delivered inputs from leading speakers and practitioners were
particularly highly rated by participants as illustrated by the feedback on the
launch event; more such inputs could be considered for future programmes.
One question explored by the evaluation was whether the same support as
provided by the IWP could have been obtained from another source. It was the
respondents in managerial roles who reported that this might be the case but this
was not the opinion of other respondents who shared the view that this would not
have happened and that the changes would not have occurred without the
project. The evaluation findings offers little in the way of insights into what fees
could be charged for future similar projects other than a clear finding that only
the larger organisations felt this would be a possibility but that this would depend
on the economic climate, their ability to pay and clear evidence that such a
project would increase turnover. Despite consistently identifying the most
benefits from their participation in the IWP, the smaller organisations reported
that it was very unlikely that they would be able to pay for such a programme.
Notwithstanding, all the participants identified that if they were to consider such
a project in the future, it would be important to them that it was provided by
Acas on the grounds of its neutrality, reputation and that it offered a ‘well
respected quality Kite mark’.
The overall conclusion that can be drawn from the evaluation evidence is that the
IWP led to improvements in all the participating organisations particularly in
communication and employee engagement which in turn, had positive
performance outcomes. To varying degrees, the IWP met its objective of
developing leadership and management skills among its participants and it
proved to be a vehicle for the identification of skills gaps in a number of
individual organisations. It was successful in introducing improved arrangements
for employee involvement and employee voice resulting in many of the benefits
associated with good employee engagement. Whilst the project was small in
nature involving just ten organisations, it resulted in a positive return on
investment; for every £1 spent on the project IPW generated £4 of direct benefit
to the regional economy. But the key message that can be drawn from the IWP’s
evaluation is that similar future projects, customised to meet the needs of
individual organisations, would offer valuable support to those organisations
where employers are seeking to engage their work forces and work
collaboratively with them to improve efficiency and changes to their working
practices.
76
References
The Acas Model Workplace (2005). London: Acas
Anderson, V. (2007) The value of learning: A new model of value and evaluation,
CIPD Change Agenda series. London: Chartered Institute of Personnel and
Development
Ashton, D. and Sung, J. (2002) Workplace Learning for High Performance
Working. Geneva: ILO
Boxall., P. and Purcell, J. (2008) Strategy and Human Resource Management.
2nd edition, Palgrave Macmillan
Gibbons, M. (2007) Better Dispute Resolution: A review of employment dispute
resolution in Great Britain. Department of Trade and Industry Report, London:
DTI
Guba, G. and Lincoln, Y.S. (1999) Fourth Generation Evaluation. Edition 11, Sage
Publications
Hall, M., Hutchinson, S., Purcell, J., Terrry, M. and Parker, J. (2009)
Implementing information and consultation: developments in medium-sized
organisations. Employment Relations Research Series No.106, London: BIS
MacLeod, D. and Clarke, N. (2009) ‘Enhancing performance through employee
engagement’. A report to Government, London: BIS
Meadows, P. (2007) A Review of the Economic Impact of Employment Relations
Services delivered by ACAS. London: National Institute of Economic and Social
Research
Mumford, A. and Gold, J. (2004) Management Development: Strategies for
Action. 4
th
ed, London: CIPD
Purcell, J., Kinnie, N., Hutchinson, S., Rayton., B and Swart, J., (2003)
Understanding the people and performance Link: Unlocking the black box.
London: CIPD
Sisson, K. (2005) Improving work organisation - the case for a regional action
programme. Industrial Relations Research Unit, Warwick Business School.
www.emda.org.uk/research
Sunday Times 100 Best Companies to work for 2010. Sunday Times:
http://business.timesonline.co.uk
Truss, K., Soane, E., Edwards, C., Wisdom, K., Croll, A. and Burnette, J. (2006)
Working Life: Employee Attitudes and Engagement 2006. Research Report,
London: CIPD.
77
Appendix A: Case Studies of the IWP Participating Organisations
1. Brush Electrical Machines (BEM) Ltd
The Organisation
Brush Electrical Machines Ltd (BEM) is a manufacturer of generators for steam
and gas turbines based in Loughborough, Leicestershire with 800 employees at
the start of the project. Other separate Brush plants also occupy the site. Along
with its current sister company in the Czech Republic, BEM is the world's largest
independent manufacturer of turbine generators. Prior to the project, a takeover
had led to a new management which had instituted 'lean production' system
resulting in a series of redundancies. There were further redundancies during the
project due to BEM’s merger with a Brush Transformers plant at the site which
had previously been under separate ownership. Rationalisation occurred across
the two companies resulting in a new round of redundancies and, by the end of
the project, BEM had a workforce of 700.
The company’s nominated gatekeepers were a Business Analyst and a Senior
Unite Trade Union representative. Participation in the program was first
suggested by a full time Unite Official and was the only case where a gatekeeper
was also a workplace union representative.
The I ssues
Following the change of ownership, the company’s aim in participating in the
project was to improve morale, trust and employee engagement. It was hoped
that its participation would assist BEM to become a more innovative work
organisation and reduce resistance to change. In particular, the takeover, change
in management and other changes had resulted in low employee morale which
was reflected in a disappointing response rate in an employee survey in 2009.
The union felt that poor communication, in addition to the issues of change and
ownership, had contributed to the lack of trust and poor morale. At the time of
joining the project, the company’s particular concern was that, whilst the
workforce was highly skilled, there was a need to develop a less functional, more
process driven perspective among the workforce to take the organisation
forward.
Action Plan
The initial action plan aimed to improve two way communication across the site
through establishing focus groups to allow the workforce to "feel valued, heard
and motivated." The involvement in the project was initiated by employees at
BEM feeling that they, together with middle managers, were not being listened to
following changes in senior management arising from a series of takeovers.
The Outcomes
Initially the program was very successful with the establishment of a steering
committee and eight focus groups, each of ten employees, to improve lateral
communication across the site. Whilst employees reported feeling undervalued, a
lack of communication and poor handling of redundancies, there was also
continuing pride in the Company’s history and its products. The final evaluation
revealed that the production of a newsletter had improved communication across
the plant. The employer further identified that the company’s increase in
78
productivity and its ability to ride out the recent economic downturn could, in
part, be attributed to its involvement in the project. But the final evaluation also
revealed resurgence of division between management and employee
representatives and the absence of any gatekeepers to take the project forward.
One of the gatekeepers, the senior shop steward, withdrew from the programme
following the breakdown of pay negotiations and concerns about his role as a
gatekeeper whilst the other, a business analyst, left the organisation for
employment elsewhere towards the end of the project. Notwithstanding, both
parties viewed the project itself very positively and, in particular, the contribution
made by Acas. A recent round of redundancies has been an impediment to
improving levels of workplace trust but senior management’s view is that the
situation would have been far worse without involvement in the project.
79
2. Caterpillar Logistics
The Organisation
Caterpillar Logistics is a wholly owned subsidiary of Caterpillar Inc. which
provides third party warehousing and distribution, and other services. It has a
workforce of 600 employees in the East Midlands. In late 2007 Caterpillar, the
USA based company, took over the Hinckley site from Quinton Hazell (QH), a
company distributing spare parts to the motor industry which remains the
Hinckley sites main client. Around 130 QH employees were the subject of a TUPE
transfer to Caterpillar Logistics although 90 employees remained in the
employment of the client organisation on the Hinckley site. A further 20
Caterpillar managers and other salaried staff moved to the Hinckley site as a
result of the takeover. Caterpillar has three other sites in the region although
these are predominantly concerned with manufacture rather than distribution. As
a result of these changes in ownership in 2007, the site employs a mixture of
Caterpillar management, including an HR manager who provides services to three
other local Caterpillar sites, transferred QH staff and a significant number of
external staff. The two nominated gatekeepers were the Logistics Centre Manager
and an HR Manager who had responsibilities across the three other Caterpillar
sites in the region. The Logistics Centre manager left the site towards the end of
the project to work in South Africa.
The I ssues
The site was a mixture of rather different business and working cultures. At the
time of applying to take part in the project, issues of communication at the
Hinckley site had been identified as being aggravated by the presence of different
cultures resulting from the takeover. Previous US ownership had led to
employees losing participation in a share scheme and pension entitlements. This
was seen as contributing to workforce scepticism about organisational change,
new ideas and initiatives. Furthermore it was felt that a lack of trust in Caterpillar
existed among employees who had been involved in the TUPE transfer. The new
management spent a year introducing Caterpillar systems, including 'continuous
improvement' under the Caterpillar Production System, but found that whilst the
metrics for the site were comparable with other company sites, it scored
significantly lower on measures of employee engagement. For example, in
response to a question asking if they would recommend someone to come and
work at the organisation, a large percentage of employees had answered that
they would not.
The Action Plan
The action plan proposed introducing measures to increase the employee
engagement score in the company employee survey by 10 percentage points,
from the original 62% to 72%. However it was recognised by management that
the poor score for engagement in the employee survey reflected deeper issues
which reflected the divided culture at the Hinckley site.
The Outcomes
An initial series of employee focus groups organised by the Acas facilitator was
reported as cathartic in raising employee concerns across both day and night
shifts. The Acas facilitator used the outcomes of the focus groups to produce a
report for management and employees. A more flexible approach was quickly
80
adopted by managers in their application of the sickness absence policy and the
absence policy was amended accordingly. An employee forum was established
and, other than a short lapse resulting from changes in management, this has
met monthly. An employee survey was carried out in January 2010 which
indicated a 10 percentage point improvement in the engagement score, the
immediate target of the action plan. Further improvement is anticipated in the
annual company survey to be carried out in early 2011.
81
3. The Health Store
The Organisation
The Health Store is a co-operative wholesale and distribution centre for health
foods with a production unit producing its own label dried goods. It has been
trading for over 77 years and is one of the leading health food wholesalers in the
UK supplying stores throughout the UK, Ireland and many parts of Europe. It has
a workforce of 102 employees. The numbers employed did not change during the
project. The Operations Director and the Warehouse Manager were the
Company’s selected gatekeepers; although the Warehouse Manager left the
programme at its outset, he remained highly involved and committed to the
project as it developed.
The I ssues
The market pressures on the company are considerable. 12 competitor
wholesalers have ceased trading in recent times and, at the time of applying to
join the programme, the Company had identified the need to find operational
efficiencies and to increase sales to sustain the business. It had identified that to
maintain and grow the business required team work and, in particular, employee
engagement, across the workforce, with the direction and priorities of the
business. It also recognised the value of trained managers who were effective
communicators and understood employees’ concerns. The programme was seen
as a means of developing a ‘forward thinking workforce able to contribute to the
future of the business’. Due to the growth of the business, The Health Store had
moved to purpose-built, larger premises in 2007. This move brought with it
significant change not least to the close working arrangements that had been a
feature of working practices and employee relations at the previous site. When
the programme began, workforce morale was identified as generally poor
compared to what it had been when the company had been a smaller operation
at its previous premises.
The Action Plan
The main objective of the company’s action plan was to increase employee
engagement through better two way communication. The view was that more
effective and open dialogue between management and staff would mean that
staff would be better informed, able to share their concerns with managers and
more likely to put forward any suggestions they had for improvements to working
practices. It was intended to achieve this by establishing consultative processes
within the organisation and by appointing elected staff representatives. The scope
of the initial action plan was adjusted and the project was implemented as a pilot
in The Health Store’s operations department which is its largest department and
includes both production and the warehouse.
The Outcomes
By the time the formal programme ended in April 2010, two employee
representatives had been elected by the workforce and trained by Acas; joint
management and staff forum meetings had been established in the production
and warehouse areas and employee representatives were attending monthly
management team meetings. The momentum continued and six months after the
project had ended, the final evaluation revealed improved morale and
relationships between management and operations staff and, over a 12 month
82
period, reduced absenteeism. There had been a very significant fall (77
percentage points) in instances of formal disciplinary action and numerous
examples of improvements to working practices as a result of suggestions made
by the workforce. Possibly reflecting the size of the company, these changes
were identified as being largely due to better informal communication and a
greater openness between staff and management but the formal consultative
processes that had been put in place were being used where this was felt to be
appropriate. As the company is about to embark on a major change programme
in its operational system, it feels it is better equipped to face the challenges that
this will present as a result of its participation of the project but views it as a
process of continuous improvement.
83
4. Liquid Control
The organisation
Liquid Control Ltd (LCL) is a private company (whose owner lives in Dubai) and is
one of a group of companies known as the KK Group. It designs, builds and
supplies standard and custom built machines for processing (metering, mixing
and dispensing) single and multi-component liquids/pastes which are used in a
wide range of manufacturing processes. The company is also a distributor of a
number of products which can be used in the systems it provides or as ‘stand
alone’ dispensing products. The main applications covered are: - Structural
Bonding, Sealing, Vacuum Encapsulating, Potting, Moulding, Casting, Resin
Infusion, Resin Injection, Laminating, Filling and Packaging. At the beginning and
the end of the project, Liquid Control Ltd employed 20 people in the UK; all based
at its operations in Wellingborough with the exception one service engineer
working from home in Scotland. Two people were made redundant during the
period and they have taken on a new apprentice and a new Laboratory
Supervisor. The nominated gatekeepers were the Service Manager and the
Technical Manager.
The I ssues
At the time of applying to join the project, the company was seeking ISO9001
Certification and looking for ways to improve the way the business was managed.
After years of stagnation in its management and working methods, its
management had identified that to achieve the productivity and efficiency gains
needed to sustain the business, it was vital to ensure that that workforce was
supportive and engaged with changes in working practices and played an active
role in developing the business for the future. The company has an ageing
workforce, many of whom have long service and there is a perceived resistance
to change which has been exacerbated by the company being sold three times in
five years in recent times.
Historically, decisions had been made by just a few individuals in the business
without the involvement of the workforce. It was recognised that to grow the
business, the workforce needed to be involved in decisions about ‘what, when
and how’ things were done, and it was known from experience that change is not
always readily accepted, even if it is recognised as being for the benefit of all
concerned.
The Action Plan
The initial Action Plan was specifically to:
? To provide training where necessary so that employees can multi-task
? To employ apprentices to train up and fill the gap left by those employees
who will shortly be retiring
? To obtain ISO 9001 by the end of 2010
? To undertake a Stress Questionnaire amongst employees.
84
The Outcomes
There have been a number of outcomes relating directly to the initial Action Plan
including the conduct of a Skills Analysis of employees to identify training needs.
As a results of the Skills Analysis, the company made two people redundant but
also employed a new apprentice and a new Laboratory Supervisor. As a result of
the training needs analysis, approximately 50% of the staff have now completed
NVQ level 3 courses, ranging from computer/electronics skills, to customer
service and management skills. In terms of employee
communications/engagement, the Acas facilitator was instrumental in the
implementation of an Employee Engagement Survey, the results of which were
mostly positive. Focus groups were held to discuss findings and there were some
issues raised around communications and training and development, which are
being addressed. For example, they have introduced development appraisals for
everyone in the workforce and quarterly company meetings to keep everyone
informed. The process of achieving ISO 9001 status is on-going.
As well as the above, the company specifically identified that the following
outcomes have been undertaken as a result of the project:
? Various areas have been identified, such as Departmental Structure,
Product and Facilities, for working parties of employees to meet and produce
recommendations for improvement within these fields;
? One employee has received IOSH training to become a competent H&S
Officer;
? A member of staff from each department has become a Fire
Warden/Officer;
? Organised quarterly presentations by external personnel are taking place
across the company to improve health awareness;
? Weekly departmental meetings have been introduced to de-brief on the
previous week and discuss work-loads for the forthcoming week;
? An additional apprentice is to be recruited.
85
5. NHS East Midlands Strategic Health Authority (SHA)
The Organisation
NHS East Midlands is the Strategic Health Authority (SHA) providing leadership of
the NHS provision for the region’s population of 4.3 million spread across the
counties of Derbyshire, Leicestershire and Rutland, Lincolnshire,
Northamptonshire and Nottinghamshire. Overseeing a budget of £6 billion, the
SHA helps to ensure that local health systems operate effectively and efficiently
for the population. Established in July 2006 the SHAs have specific
responsibilities for relaying and explaining national policy, setting direction and
supporting and developing the region’s NHS Trust bodies. At the commencement
of the initiative, the East Midlands SHA had a workforce of 350 employees spread
across three sites including the Headquarters located in Sandiacre,
Nottinghamshire. There were originally two gatekeepers the Planning & Projects
Workstream Lead, IM&T and the Business Manager, Public Health. This latter
gatekeeper went on maternity leave during the course of the project.
The I ssues
The organisation sought to improve workforce flexibility and to take greater
account of employees' views, including any issues or concerns so promoting a
greater sense of fairness and consistency. It was believed that this could be
achieved by maximising the benefits of the Electronic Staff Record (ESR) system,
in particular by transferring the information ownership to the individual so
providing higher quality information. This was to be achieved through the
implementation of an innovative Self Service functionality adhering to
governance standards in relation to information access and data protection. The
Oracle Learning Management System (OLM) would be available through both
Management and Employee Self Service. Consequently, there was a need to train
all staff to be able to access the corporate Knowledge, Skills and Competencies
database (OLM), so enabling all employees to access their learning and
development details, and book themselves on internal courses with the
authorisation of their line managers. Such a change to organisational practice, it
was perceived, would increase feelings of value amongst staff as they become
increasingly responsible for their training and development. This in turn would
assist the organisation move towards a culture where everybody would be
encouraged to learn new skills which would help identify goals which might
contribute to the organisation's vision.
The Action Plan
The action plan envisaged maximising the ESR by transferring personal
information data ownership to the individual so enhancing the provision of high
quality information. It was believed that this change in working practice would
boost employee engagement through better communication in turn improving
workforce flexibility. Part of the project’s objectives were to enable managers to
maintain relevant personal information regarding their employees in such areas
as absence, terminations and personal details so reducing levels of information
handling. The action plan envisaged that increased employee utilisation of the
new system and its enhanced reporting facilities would enable more timely
recording of absence which in turn would enable managers to more effectively
manage both levels of absence and its cost. Maximising the utility of the ESR
86
would also devolve access, provide more rapid authorisation of information and
reduce paper based transactions in turn resulting in a reduction of labour
intensive administrative duties in both Payroll and HR. The action plan highlighted
the need to integrate the Acas IWP with the wider SHA ESR project. It was
anticipated that the Acas project would facilitate the Gatekeepers (within this
wider project) to help engage staff and to secure buy-in throughout the
organisation and support the effective and efficient utilisation of the ESR system
being introduced. The gatekeepers were operating in two pilot areas which would
increasingly be looking to enhance engagement of staff and line mangers with
the new ESR system. Consequently, it was identified at an early stage in the
project plan that communication was a key issue and this became the critical
focus of the internal project and facilitation process.
The Outcomes
By April 2010 significant progress had been made. Following some early
communication difficulties encountered as a result of the involvement of so many
different stakeholders, the ESR system became operational. Acas’ involvement
led to the facilitation of a number of focus groups of end-users which enabled all
‘voices’ to be heard and this employee participation reshaped training and
development initiatives which assisted the implementation of the ESR System.
Six months following the end of the project an increasing number of employees
were beginning to engage with the ESR up-dating their own personal data. This
has had three major effects: staff took greater ownership of their data and their
personal development, HR information is more accurate, and data handling has
become more efficient. This has led to improved reliability, productivity and
efficiency in the handling of personal data.
87
6. Northampton College
Northampton College is a provider of Further and Higher Education to some
12,000 learners. 4,500 of these are full-time students predominantly 16-19 years
old. The remainder are predominantly adult learners taking vocational and
professional qualifications. A small area of the college is concerned with the
higher education providing qualifications in Leadership & Management to middle
and senior managers at levels 4 to 8 (HND to PhD). This unit also works with
employers through organisational-development programmes and provides
consultancy and leadership coaching. At the commencement of the project, the
College employed 1003 staff (404 FT and 599 PT) At the conclusion of the project
it had a workforce of 1039 staff (374 FT and 665 PT). The college operates across
four sites, Booth Lane (Main Site), Lower Mounts (HE campus), Daventry campus
and Westbury Court (the Business Centre) all in Northamptonshire. The Manager
for Professional Development (Leadership & Management) and a Management
Lecturer and Part-Time Co-ordinator were chosen as the selected Gatekeepers for
the project.
The I ssues
As a result of an Ofsted inspection in April 2009, the College was graded at an
overall level 3 (Satisfactory). The College has a strategic vision to become an
Outstanding Grade 1 college by 2013. In order to achieve this its senior
management had identified that there was a need to achieve a culture change
from ‘satisfactory is good enough’ to one which promoted excellence across the
range of its activities. There were three major strands of the college’s strategy to
become Grade 1 which were relevant to the IWP. These were: to raise teaching
standards, develop leadership and management capability; to provide clear
direction and better mentoring, coaching and support for staff; and to radically
improve any areas of ‘failing provision’ as measured against the benchmarks of
the national FE ‘Success Rates’. An additional challenge for the college was that,
at the time of joining the project, it was in the early stages of demolishing and
rebuilding its main campus without temporarily relocating its operations
The Action Plan
The Action Plan at the outset of the project centred around a Leadership and
Management Development Programme, which had begun in October 2009. This
had been designed to develop capability in leadership and management. A
further Action Plan (see below) relating to the issue of employee engagement
emerged as a result of participation in the project and this led to the Acas
facilitation element of the programme continuing for longer than at other
participating organisations.
The Outcomes
The Acas facilitators held a number of diagnostic workshops involving functional
groups which included representatives from the trade unions. These workshops
were based around the results of a Staff Engagement Survey in 2009 which
identified that employee engagement was fundamental to the college achieving
the culture change it was seeking. A pivotal point for the project was when the
connection was made between the Leadership Programme and the survey’s
employee feedback on issues of leadership and management. Considerable
progress has been reported in the development of leadership skills, and their
application to achieve performance improvements, among college managers at all
88
levels. In terms of employee engagement there have been a number of outcomes
which are on-going. A further Action Plan has been developed between senior
managers and representatives from the two main trade unions following a joint
meeting facilitated by Acas. This addresses issues of employee consultation,
communications, student behaviour and teaching/learning observation. The
action plan agenda is being addressed by different task groups. Two joint
problem solving training sessions are to take place for the managers and staff
who are taking part in Task Groups.
89
7. Pendragon Contracts and National Fleet Solutions (Derby)
The Organisation
Pendragon PLC is the UK's Leading Automotive Retail Network. The automotive
retail outlets trade as Stratstone, Evans Halshaw and Chatfield’s, offering a large
selection of new and used vehicles. These brands represent over 20 franchises for
passenger vehicles, motorcycles, commercials and trucks operating from over
300 retail sites. Pendragon Corporate Division embraces Pendragon’s Contract
Hire and Pendragon Fleet Leasing business in the UK. The Division is located in
Derby. At the commencement of the project Pendragon Corporate Division
employed 98 team members (38 males and 60 females). At the conclusion of the
project team member numbers had reduced by 7 to 91 in total. Pendragon’s
Employee Relations Leader and Customer Services Director were the company’s
selected gatekeepers.
The I ssues
The primary business of Pendragon Corporate Division is client account
management and building relationships with external customers. The
increasingly adverse economic climate had severely impacted the industry, and
Pendragon Corporate Division had been subjected to a series of restructures and
redundancies. Consequently, team members had experienced a number of
changes, both structurally and to their own roles and responsibilities. This had
left a lot of team members with feelings of unrest and uncertainty regarding their
future job security (sometimes referred to by employees in the company as
‘survivor syndrome’). It was believed that this uncertainty had led to some team
members becoming disengaged, which in turn was having an impact on their
personal performance. Given the importance of client relationships, management
believed that team members who were disengaged would adversely impact the
external perception of the Division and ultimately adversely impact the overall
financial state of the organisation. The opportunity to participate in the
programme was seen as a possible catalyst to addressing the engagement issue;
would be an opportunity to set the foundations of better employee relations for
the Contracts Division going forward; and provide a chance to develop
approaches to engagement which could be utilised across Pendragon PLC.
The Action Plan
Consequently, the focus of the Corporate Division’s action plan was ‘to increase
business performance through improved team member engagement’. It was
believed that the project provided an opportunity to improve team member
engagement, encourage better team participation and departmental interaction
so increasing customer service.
The Outcomes
By the end of the project in April 2010 Pendragon Contracts had established a
team member forum. Initially this forum was facilitated by Acas but later became
self-facilitating. This forum agreed and instituted beneficial changes to a number
of what the gatekeepers called ‘house keeping policies’ relating to such issues as
an equitable car parking protocol, and the standardisation of the no-smoking
policy. The successful implementation of these changes paved the way to
enhanced more challenging issues such as improved team member
communication throughout the Division. A number of team building events were
90
held to meet this objective and improved team member’s engagement. Team
leaders had begun to meet on a monthly basis and a more recent employee
engagement survey displayed improved results. This enhanced communication
and engagement provided the confidence for the Division to resume their
customer service survey and gain commitment from other divisions to begin a
similar project internally. Six months following the end of the project increased
employee communications and engagement was helping the Division face the
severe economic climate. With imposed structural changes and changes to
personnel, the perceived challenge going forward appeared to be sustaining the
achievements made so far.
91
8. Thorpe Kilworth
The Organisation
Thorpe Kilworth was a designer, manufacturer and installer of high quality
furniture for education, laboratory and health care establishments with 116
employees at its site in Corby, Northamptonshire. The company’s nominated
gatekeepers were its Organisational Development Manager and Production
Manager.
Throughout the duration of the project, the company experienced difficulties
which resulted in short time working and redundancies. This was largely due to
the seasonal nature of its work and the unexpected loss of work from major
clients despite existing agreements. As a result, the headcount reduced to 80
employees during the project. A combination of cash flow problems and delays in
clients confirming contracts culminated in the company going into administration
in June 2010. As it had completed the IWP, it continued to be included in the
evaluation process. Since then, 27 former employees have formed a co-operative
company known as Thorpe Learning.
The I ssues
The company had identified that it needed to be more responsive to changes in
the market place and to the needs of its clients. It was seeking to increase
workforce versatility and flexibility so that individuals could be moved from one
process to another in order to meet fluctuations in demand. This required
employees to develop their skills and to be more adaptable in client facing
situations, as well as in project management, design, manufacture and
installation. In addition, the company was aware that they needed to improve its
competitiveness by maintaining quality and reducing waste and inefficiencies
The Action Plan
The aim was to improve the company’s competitive edge by maintaining quality
but improving manufacturing efficiency with the key objective of having all
materials and consumables ready and clearly identified for manufacture. This
particularly involved:
? Obtaining relevant information for production in a timely fashion
? Reducing double handling and other efficiency losses
? More consistency in design for manufacture
? Ensuring the delivery of all relevant materials to meet production
schedules on as near to a ‘just in time’ basis as possible
? Implementing relevant revised procedures with relevant training
? Measuring before & after for cost-benefit analysis.
A key element of the action plan was to improve employee engagement in a
workforce viewed as loyal and responsive but with concerns about changing long
established ways of working. Improving two way communications was seen as
essential so that the workforce understood and was committed to the changes in
the business.
92
The Outcomes
By May 2010 before the company went into administration, a significant amount
of the action plan had been achieved. Elements of lean manufacturing had been
introduced and the Stores area had been re-organised resulting in improved
security, improved facilities and better stock control. Throughout this re-
organisation there had been employee participation. A cross-functional working
party had been established to identify some of the weaknesses of the current
stock system. Acas facilitators had run a workshop for the company on joint
working/problem solving, and an employee engagement survey had been
developed. Six team leaders from staff in manufacturing and the office had been
appointed with a supporting training programme and the existing six employee
representatives had been trained. The reconstituted Staff Consultation Forum
highlighted numerous problems and inefficiencies such as the lack of standard
procedures and poor team-working. These issues were in the process of being
addressed prior to the closure of the business.
93
Appendix B: Acas short courses provided to participants
Innovative workplaces – emda project
Training events
Meetings, Consultation & Forums – some useful basics
Tuesday 9
th
March 2010
Represented
Organisation Number of attendees Gatekeepers
SHA NHS East Midlands 2 2
The Health Store 1 1
Thorpe Kilworth 2 2
Mahle 1 1
Having Difficult Conversations
Wednesday 10
th
March 2010
Represented
Organisation Number of attendees Gatekeepers
SHA NHS East Midlands 1 1
Thorpe Kilworth 1 0
Mahle 2 1
Pendragon 2 0
Caterpillar 2 1
Having Difficult Conversations
Friday 12
th
March 2010
Represented
Organisation Number of attendees Gatekeepers
SHA NHS East Midlands 1 1
Thorpe Kilworth 3 1
Pendragon 3 0
Liquid Control 2 2
The Health Store 1 1
Facilitation Skills Training
Tuesday 13
th
July
Represented
Organisation Number of attendees Gatekeepers
SHA NHS East Midlands 1 1
Thorpe Kilworth 3 1
Pendragon 2 0
Liquid Control 2 2
The Health Store 1 1
Caterpillar 2 0
94
Developing Mediation Skills
30
th
September, 1
st
October and 7
th
October 2010
Represented
Organisation Number of attendees Gatekeepers
SHA NHS East Midlands 1 1
Pendragon 2 1
Liquid Control 2 2
Caterpillar 2 1
In addition some facilitators delivered in-company training as follows:
? Pendragon - Essential Skills for Supervisors; Training for Workplace reps
? Caterpillar - Training for Workplace reps;
? The Health Store - Training for Workplace reps
95
Appendix C: Anderson’s Evaluation Model
Model of Value Contribution Evaluation – Anderson (2007)
Learning Function Efficiency Measures
Focus on assessing efficiency and
effectiveness of the learning function.
Although the term ‘learning function’ is
often taken to mean a specialist
department, the term can equally well
be used in a wider sense to refer to ‘the
body of Learning and Development
activity that has to be provided for an
organisation and the people most
directly responsible for that provision’
(Harrison, 2005:206).
Return on Expectation Measures
Focus on assessing the extent to which
the anticipated benefits of the learning
investment have been realised. Key
questions underpinning a return on
expectation approach are:
What were the original expectations of
organisational stakeholders for the
learning or training? Have those
expectations since changed?
What changes have occurred as a
result of the learning process?
To what extent have stakeholder
expectations been met?
Return on investment measures
Focus on assessment of the benefits of
specific learning and training
interventions compared with the costs
incurred. They may also involve an
assessment of the pay back period for
specific learning or training
investments. Key issues here are the
extent to which learning is directly
contributing to the achievement of
defined performance targets.
Key performance indicators and
benchmark measures
Focus on the more general evaluation
of HR processes and performance
through a comparison with key
performance indicators or external
standards of ‘good practice’ or
‘excellence’. These approaches may be
undertaken as a one-off’ but are more
useful when treated as a continuous
process in which the organisation
continually seeks to challenge and
improve its processes.
Source: Anderson, V. (2007) The value of learning: A new model of value and
evaluation, CIPD Change Agenda series, London: Chartered Institute of Personnel
and Development.
96
Innovative Workplaces – Economic Impact
Assessment
December 2010
97
Contents
1.0 Introduction............................................................................................. 1
1.1 Innovative Workplaces ..............................................................................................1
1.2 Analytical Framework ................................................................................................1
1.3 Assessing additionality .............................................................................................5
2.0 Economic Impact Assessment .............................................................. 6
2.1 Organisations participating in the Innovative Workplaces project ........................6
2.2 Availability and take-up of alternative sources of support .....................................6
2.3 Improvements made by participating organisations ..............................................7
2.4 Changes in organisational performance resulting from improvements................8
2.4.1 Staff retention and absence .........................................................................................9
2.4.2 Changes in grievances, disciplinary sanctions, and ET claims...................................10
2.4.3 Productivity.................................................................................................................11
2.4.4 Employment growth....................................................................................................13
2.5 Future impacts on employment ..............................................................................13
2.6 Gross additional economic impacts .......................................................................14
2.7 Leakage, displacement, substitution and multiplier effects.................................14
2.7.1 Leakage .....................................................................................................................14
2.7.2 Displacement..............................................................................................................15
2.7.3 Substitution effects.....................................................................................................15
2.7.4 Multiplier effects .........................................................................................................15
2.8 Net additional economic impacts per annum........................................................16
2.9 Present value of GVA impacts ................................................................................16
2.10 Return on investment ..............................................................................................17
2.11 Costs and benefits that have not been quantified.................................................18
List of figures
Figure 1.1 Analytical Framework.................................................................................4
Figure 2.1 Improvements made to Human Resource Policies by Participating
Organisations ...............................................................................................................7
Figure 2.2 Improvements in organisational performance resulting from process
improvements...............................................................................................................9
98
List of tables
Table 2.1 Estimated probability organisations would not have taken up an
alternative source of similar support in the absence of the Innovative Workplaces
project ..........................................................................................................................6
Table 2.2 Estimated probability organisations would not have implemented
improved HR procedures without the support they received (additionality of
actions).........................................................................................................................8
Table 2.3 Changes in grievances, disciplinary sanctions, and ET claims, and
associated management savings ...............................................................................10
Table 2.4 Estimated probability organisations would not have seen productivity
gains if they had not implemented process improvements to their business..............11
Table 2.5 Gross additional productivity gains............................................................12
Table 2.6 Gross additional economic impacts...........................................................14
Table 2.7 Multiplier Effects ........................................................................................16
Table 2.8 Net additional economic impacts...............................................................16
Table 2.9 Present value of GVA benefits ..................................................................17
Table 2.10 Return on Investment ..............................................................................18
99
1
1.0 Introduction
This report sets out the economic impacts of the emda funded Innovative Workplaces project that was
delivered by the Advisory, Conciliation, and Arbitration Service (Acas) between 2009 and 2010. The
analysis is designed to provide an estimate of the impacts of the project on participating organisations in
terms of employment and GVA that is aligned with the principles set out in emda's evaluation toolkit. The
analysis draws on a survey of participating organisations undertaken by Nottingham Trent University (who
are undertaking a full evaluation of the project for Acas).
1.1 Innovative Workplaces
The 'Innovative Workplaces – Developing Organisations' is a project aiming to improve the performance
of workplaces through greater employee involvement and engagement, and improved workplace
practices and procedures. Ten organisations from the public and private sectors in the East Midlands
initially participated in the project. Each received a bespoke package of support from senior Acas
advisors and the UK Work Organisation Network (UKWON). The project has involved addressing a range
of organisational issues, including:
? boosting employee engagement;
? improving communications processes;
? improving morale and motivation;
? managing change and restructuring;
? workforce flexibility;
? performance management and measurement; and,
? innovation and adaptation.
As well as guidance and support from Acas, the ten participating organisations also benefited from an
accredited management training programme, and peer support from fellow participants. The combination
of networking support, facilitated by UKWON, coupled with on-site practical help from Acas, was
designed to help participant companies to address some of the difficulties that have held back
performance. Eight of the participating organisations ultimately completed the programme.
1.2 Analytical Framework
This section sets out our analytical framework for assessing the economic impacts of the Innovative
Workplaces project in terms of net employment and GVA created and safeguarded. The Innovative
Workplaces is expected to generate economic impacts through the following mechanisms:
? Implementation of improvements to workplace processes: The programme of support provided to
participating organisations was designed to encourage them to implement and deliver improvements
to their HR and other related workplace practices.
? Gross changes in performance: Over the period following the implementation of improvements, the
organisation may see the performance of their business change. This could encompass either growth
100
2
or decline in their sales or turnover, or changes in the productivity of their workforce. These changes
will also be reflected in terms of:
? Changes in GVA: GVA is measure of the total output of the organisation, the value added by
the organisation to the goods and services it absorbs in the production process. In practical
terms, this is measured by subtracting the value an organisation's purchases of goods and
services (ranging from raw materials through to expenditures on property services
1
) from the
value of what an organisation produces (which can be approximated by the value of sales)
2
.
Changes in GVA can be driven by the following:
? Changes in GVA driven by sales growth: If demand for an organisation's products or
services increases, then the organisation will need to produce more of those products or
services to satisfy that demand. Other things being equal, the organisation will need to
employ further workers in order to deliver greater levels of output, resulting in impacts on
employment, with the converse applying if demand for the organisation's output falls.
? Changes in GVA driven by productivity growth: Overall GVA may also increase or
decrease without changes in employment levels through changes in productivity, i.e. the
levels of output that each worker in an organisation is able to produce over a given period of
time. This could occur through workers themselves increasing their skills or being organised
more efficiently, but also if the organisation is able to produce similar levels of output while
using less goods and services produced by other organisations (i.e. reducing their cost
base).
? Additionality: The objective of the economic impact assessment is to estimate the share of gross
changes in performance outlined above that can be attributed to the support provided through the
Innovative Workplaces project
3
. There are several dimensions to consider:
? How far participants would have taken up a similar alternative source of support: In the
absence of Innovative Workplaces, participants may have taken up similar support if it was
available from another source, provided by the public or the private sector. If this is the case,
then it is likely that the organisations would have implemented similar improvements to their
processes (providing the quality of the advice was comparable), resulting in similar outcomes.
? How far organisations would have implemented process improvement without support:
There is also the possibility that the support provided encouraged organisations to implement
improvements that they would have undertaken anyway.
1
Utilisation of capital items, such as computers and machinery for example, is accounted for under the alternative
measure Net Value Added, which also adjusts for the depreciation of those items over their normal working lives.
2
Valuing the output (GVA) of public sector organisations is typically more challenging as such organisations
generally do not provide products or services that are traded in the market, creating difficulties in estimating the
value of those products and services.
3
Deadweight can be seen as the reverse of this concept, referring to the gross changes in performance that
would have occurred in the absence of the project.
101
? How far changes in business performance can be attributed to the changes made: Finally,
there is a question as to how far changes in business performance can be attributed to the
process improvements made. This could occur in a range of ways for the Innovative
Workplaces project, with the most immediate effects likely to be felt in terms of productivity. If
process improvements help to improve workplace motivation, morale or employee engagement,
this might be felt in terms of improved staff retention or reduced numbers of staff days lost to
absence. Productivity will rise through a lower shares of revenue being allocated to recruitment
spending, together with a reduction in lost GVA due to either positions remaining vacant or staff
being absent from work. These effects (along with other factors such as improved staff
leadership skills or creativity) may also lead to sales growth (or help protect existing sales). For
example, if a organisation is able to retain experienced sales staff with established relationships
with customers, then sales growth may be enhanced over the longer term.
The Innovative Workplaces was a regionally targeted intervention, and there are a range of wider factors
that need to be considered in establishing estimates of economic impacts:
? Leakage: If the economic impacts of the intervention accrue to residents of other regions, then these
are considered to have 'leaked' outside the East Midlands. This occurs if businesses based outside
the East Midlands benefit from the support provided, or if those filling any vacancies created are not
residents of the region.
? Displacement: Participating organisations may also get a competitive advantage over other
organisations based in the East Midlands as a result of the support provided. This may result in a loss
of market share for these other organisations. These negative consequences (displacement) should
also be taken into account.
? Multiplier effects: Multiplier effects occur through two mechanisms. Firstly, to increase GVA,
organisations will need to procure goods and services from other organisations. To the extent they
purchase from other organisations in the East Midlands, these positive effects will help enhance
turnover and employment throughout the supply chain (indirect or supply chain multiplier effects).
Additionally, where wages are paid to any additional employees, their spending of this income in
regionally based organisations will have a similar effect.
? Net additional impacts: Net additional economic impacts at the regional level are estimated as:
Gross impact x Additionality x (1 – Leakage) x (1 – Displacement) x Multiplier Effects.
A diagram setting out our analytical framework is below:
3
102
4
E
C
O
T
E
C
Innovative
Workplaces
Peer support
Addi tionality (1):
How far would
participants have
found similar
support elsewhere?
Additionality (2):
How far would these
improvements have
been made without
Acas support?
Improved
employee
engagement
mechanisms
Improved HR
policies and
processes
Additionality (3):
How far would these
outcomes have been
achieved without the
improvements?
Improved staff
morale or
motivation
Reduced staff
absenteeism
Improved staff
creativity
Increased sales
Reduced costs
(legal, recruitment,
management time,
training)
Lost output (GVA)
recovered
Leakage,
displacement, and
multiplier effects
Net economic
effects on GVA
Enhanced
productivity (GVA
per worker) and
profitability
Acas activities Outputs Outcomes Impacts
Improved staff
retention
Practical on-site
advice on
process
improvements
Improved
communication
Improved staff
leadership skills
Figure 1.1 Analytical Framework
103
5
1.3 Assessing additionality
In general, the most robust approach to assessing the impact of public sector interventions is to adopt a
quasi-experimental approach in which comparisons are made between those receiving support are
compared to a control group to identify how far improvements can be attributed to participation in support.
This approach could not be applied to the Innovative Workplaces project: with eight organisations
completing the project, quasi-experimental approaches would not be sufficiently robust to generate any
statistically significant findings. Additionally, quasi-experimental evaluation studies are highly resource
intensive, and given the scale of the intervention, such costs would be difficult to justify.
Estimates of additionality are instead based on assigning assumptions of the extent to which impacts can
be attributed to interventions (in this case the innovative workplace project) based on the reported
perceptions of the participating organisations. The overall approach and assumptions utilised are
consistent with emda's Evaluation Toolkit, and reflect a commonly utilised approach to evaluation in such
contexts. However, the approach is not without caveats. Firstly, estimates are based on self-reporting,
which may not be reliable since respondents may have an incentive to over- or understate the impact of
the support they received, or in some cases, may not be in a position to assess the impact of support.
Additionally, the approach is based on ascribing a quantitative measure of probability to responses given
on a qualitative scale of impact (e.g. 'Very likely' – 'Very Unlikely'). The reliability of this approach
depends on how far the assumptions utilised are aligned with respondents perceptions of the impact
associated with their responses on average.
104
2.0 Economic Impact Assessment
This section sets out an analysis of the economic impacts of the emda funded, Acas led Innovative
Workplaces project. The economic impact assessment has been developed in alignment with the
methodological principles set out in emda's evaluation toolkit.
2.1 Organisations participating in the Innovative Workplaces project
Eleven organisations participated in the Innovative Workplaces project, although three withdrew over the
course of delivery, resulting in a total of eight completing the project. This analysis is restricted to the
eight completers, although, as other chapters in this report discuss, further impacts were achieved
amongst the three dropping out.
2.2 Availability and take-up of alternative sources of support
A key consideration in assessing the impact of the project is the extent to which the outcomes achieved
by participating organisations is the extent to which they would have taken up a similar alternative source
of support in the absence of the Innovative Workplaces project. If participants would have obtained similar
support elsewhere, then it is likely that any such outcomes would have been achieved anyway.
Respondents to the survey were asked to report whether they felt they would have been able to obtain
similar support from an alternative source, where they would have been able to obtain that support, and
how likely they would have been able to take up that support if the support provided through Innovative
Workplaces was unavailable. Of the eight respondents to the survey, three reported that they would have
been able to find a similar level of support elsewhere, for which there would have been fees. However,
two of the three respondents reporting they would be 'unlikely' to take up this alternative support, and
respondents were less clear on where they would have been able to obtain this support, citing
unspecified consultants and the Chamber of Commerce. There were also doubts about whether all
organisations would have the ability to pay for such a service.
Overall, the evidence suggests in the main, participants would not have accessed similar services
elsewhere, with an overall estimated probability that organisations would not have taken up alternative
support of 84 percent. Details of this estimate and the assumptions made are set out in the table below.
Table 2.1 Estimated probability organisations would not have taken up an alternative source of
similar support in the absence of the Innovative Workplaces project
Response to 'Would you have been able to find a
similar level of support elsewhere?'
a) Percentage of
respondents
b) Additionality
assumption/
weighting
c) Probability
that
organisations
would not have
found/used
equivalent
alternati ve
support
No 62.5 1.00 0.625
Yes 37.5 - -
6
105
Response to 'How likely is that you would have taken up this alternative support?'
Definitely 0 0.00 -
Likely 12.5 0.25 0.031
Neither likely nor unlikely 0 0.50 -
Unlikely 25 0.75 0.188
Definitely not 0 1.00 -
Estimated probability that organisations would not have taken up alternati ve support: 0.84
Source: Participant Survey, Nottingham Trent University (The probability organisations would not have found
alternative support is based on the proportion of respondents giving a particular response weighted by the
additionality assumption – i.e. column a x column b)
2.3 Improvements made by participating organisations
Respondents were asked to report whether they implemented new or improved human resource policies
or procedures (or planned to make such improvements in the future) during or since participating in the
Innovative Workplaces project. Seven of the eight organisations surveyed had implemented at least one
new or improved process and all had at least plans to implement new or improved processes.
Improvements in procedures for informing and consulting with employees were most commonly reported
by organisations, followed by 'other' processes, which might include equality or absence management
policies.
Four of the eight reported they had plans to implement improvements in the future, with improvements in
discipline and grievance policies most commonly reported, suggesting that some impacts of the project
might be felt at a later date.
Figure 2.1 Improvements made to Human Resource Policies by Participating Organisations
0
1
2
3
4
5
Discipline Grievances Redundancy Informing and
Consulting Employees
Other (such as
Equality Policy and
Absence
Management)
Type of procedure of policy improved
N
u
m
b
e
r
o
f
f
i
r
m
s
New
Improved
Planned
Source: Participant Survey, Nottingham Trent University
7
106
In order to identify how far these improvements were made as a result of the Innovative Workplaces
projects, respondents to the postal survey were asked to report the extent to which they would have
made these improvements without the support they received from Acas. None of the participating
organisations felt they definitely would not have implemented the main changes without the support they
received, although a quarter reported they probably would not have done so. Using the additionality
assumptions set out in Table 2.2 below, it is estimated that there was a 32 percent probability overall that
participating organisations would not have implemented improved HR procedures without Acas support.
Table 2.2 Estimated probability organisations would not have implemented improved HR
procedures without the support they received (additionality of actions)
Response to 'How likely is that you would
have introduced the main changes without
the support you recei ved from Acas?
a) Percentage of
respondents
b) Additionality
assumption/
weighting
c) Probability that
organisations
would not have
introduced the
main changes
without Acas
support?
Definitely 25 0.00 -
Probably 37.5 0.25 0.09
Probably not 25 0.75 0.19
Definitely not 0 1.00 -
Other (*) 12.5 0.33 0.04
Estimated probability organisations would not have made improvements without the
support they recei ved
0.32
Source: Participant Survey, Nottingham Trent University (*) one respondent reported that of three procedures
implemented, they would have definitely have implemented two without Acas support, and definitely not have
implemented the third, so an additionality rate of one in three (33 percent) is assumed in this case. The probability
that respondents would not have implemented improvements without the support they received is estimated by
percentage of respondents giving particular responses, weighted by the additionality assumption (i.e. column a x
column b).
2.4 Changes in organisational performance resulting from improvements
Participating organisations were also asked to report how the improvements they made resulted in
changes to the performance of their organisation or business. Most frequently reported were
improvements in communication within their organisations, increased staff engagement, and improved
leadership skills, with secondary effects on staff creativity and turnover.
8
107
Figure 2.2 Improvements in organisational performance resulting from process improvements
0
1
2
3
4
5
6
7
Improved
communication
within the
organisation
Increased staff
engagement
Improved
leadership skills
for some
employees
Improved staff
morale/motivation
Increased staff
creativity
Greater
sales/turnover
Type of improvement
N
u
m
b
e
r
o
f
f
i
r
m
s
Source: Participant Survey, Nottingham Trent University
The impacts outlined above generally relate to the intermediate outcomes of the improvements made
(with the exception of impacts on sales or turnover). Where these short term effects have led on to an
economic impact on the organisations concerned, these will be ultimately be observed through
improvements in productivity and sales growth. Three approaches to valuing economic impacts have
been considered:
? Impacts on productivity driven by reductions in staff retention and absence: Improvements to
HR processes may have had an immediate effect on rates of staff retention and absence via improved
staff morale and motivation. Such improvements would have an impact on GVA through increasing
productivity (through avoiding the output lost through staff absence or replacing workers that have left
the organisation).
? General productivity growth: The improvements in staff engagement and communication may have
also helped to boost productivity more generally through enabling staff to complete tasks more
efficiently or more effectively, resulting in further GVA impacts.
? Turnover and employment growth: Finally, the improvements made by organisations may have
ultimately resulted in greater sales and the recruitment of additional staff in order to meet demand.
These impacts are explored in turn in the following sections.
2.4.1 Staff retention and absence
Two organisations (from the five able to respond) reported they had seen a decline in the number of staff
leaving voluntarily, while a further two (again, from five able to respond) reported they had seen a fall in
the proportion of working days lost due to absence since participating in the project. However, no
respondents reported that the process improvements they had made as a result of Acas support had any
9
108
10
influence over these aspects of business performance. Therefore no economic impacts are estimated to
arise through reduced staff retention and absence.
2.4.2 Changes in grievances, disciplinary sanctions, and ET claims
In addition to benefits resulting from changes in staff absence and retention, if the Innovative Workplaces
project resulted in a decrease in the number of grievances, disciplinary sanctions, or Employment
Tribunal claims then organisations will avoid a range of costs (ranging from management costs potentially
through to legal costs).
The survey of firms indicated that organisations saw an increase in the number of employees utilising
grievance procedures, the number of cases where disciplinary sanctions were applied, and virtually no
change in the number of ET claims, as set out in the table below. In order to estimate the level of
management time associated with these, the number of estimated days of management time associated
with each type of case (as estimated from the 2007 CIPD Managing Conflict at Work survey and the 2008
Survey of Employment Tribunal Applications) was applied to the net change in the number of cases
observed across the sample. Overall, it was estimated that participating firms saw a increase in the level
of management time associated with grievances, disciplinary sanctions and ET claims of 158 days.
However around 80 percent of these days can be attributed to a single organisation. In all other
organisation, the volumes of disputes as measured through grievances, disciplinary sanctions and ET
claims had either remained stable or decreased (in some instances notably so).
Management time was value in terms of managers gross hourly pay (£18.52) and average hours worked
per week (37.5), giving an estimate of £138.90 per day
1
, and applied to the estimated net change in
management time associated with grievances, disciplinary sanctions and ET claims. Overall additional
costs were estimated at £21,946.
Table 2.3 Changes in grievances, disciplinary sanctions, and ET claims, and associated
management savings
Number in
6 months
before
project
Number in
6 months
after
project
Net
change
Manageme
nt Costs
(Days)
Overall
change in
days spent
Manageme
nt wage
costs per
day (£)
Total
Saving (£s)
Grievance
procedure
8 23 15 9 135 138.9 18,752
Disciplinary
Sanctions
43 44 1 13 13 138.9 1,806
ET claims 4 6 2 5 10 138.9 1,389
Overall
change
- - - -
158 21,946
Source: Participant survey, SETA 2008, CIPD Managing Conflict At Work 2007 and ASHE 2010
Notwithstanding the fact that the rise in incidence of disputes was entirely accounted for in one
organisation, seeing a rise in reported disputes may well be one of the apparently paradoxical outcomes
of improved procedures. It may be that a more transparent and accessible procedures create an
1
This assumes that the value of a managers work is equal to the value of their hourly earnings, but ignores
additional taxes (such as Employer's National Insurance contribution) and costs (such as training).
109
11
environment in which individuals are more aware, or feel more able to raise problems at work. This
outcome (though not apparently applicable in the case of the Innovative Workplaces Initiative) may result
in an increase of observable disputes. In the absence of these opportunities and mechanisms, conflict is
arguably more likely to manifest itself in much higher levels of employee turnover.
While these effectively offset the productivity gains estimated above, it is not possible to include these
savings in the overall economic impact assessment. We have no evidence that these additional costs are
necessarily attributable to the Innovative Workplaces project since, as owing to constraints on the length
of the questionnaire, respondents were not asked to attribute changes to the support they received.
Secondly, the additional costs are also negligible in the context of the estimates of GVA impacts through
wider productivity gains as set out below.
2.4.3 Productivity
Productivity is a measure of the output (GVA) an organisation can produce for a given level of inputs, and
is typically measured by GVA per worker. GVA per worker might rise (among other reasons) if workers
become more efficient or skilled, if organisations replace workers with capital equipment or machinery, or
if organisations adopt more efficient production processes
1
. Though reductions in staff absences or
grievances, or increases in staff retention, will increase productivity through improving the efficiency of the
organisation, the Innovative Workplaces project may have resulted in wider productivity benefits, for
example if the project resulted in more creative or motivated workers.
All participants felt that the productivity of their staff had risen over the past twelve months. The range of
responses given when asked to report how far productivity gains were due to the improvements they
made as a result of Acas funded support are set out in the table below. Based on the assumptions
outlined in the table, it is estimated that 50 percent of productivity gains seen by participants would not
have occurred without the improvements they made.
Table 2.4 Estimated probability organisations would not have seen productivity gains if they had
not implemented process improvements to their business
Response to ‘Has your productivity improved as a
result of the changes you made to your business?’
a) Percentage
of respondents
b) Additionality
assumption/
weighting
c) Probability
productivity
would have
improved
without
improvements
No 0 0.00 0
Yes 100 - -
Response to 'How likely is it that you would have seen these productivity gains if you had not implemented these
1
The overall change in GVA within an organisation can be measured by: ?Y = (Y/N) ?N + N x ?(Y/N), where Y is
output (GVA) and N is employment. GVA (Y) can be measured by an organisations turnover less expenditure on
intermediate goods and services. Owing to constraints on the length of the questionnaire, expenditure on
intermediate goods and services is assumed to be a constant proportion of turnover, implying a change in
turnover per worker is reflected in an equivalent proportional change in productivity. However, this definition has
obvious difficulties in application to the public sector – as highlighted in Section 1 – as the turnover of public
sector organisations is not generally driven by sales but by public sector budgeting. Though this general model is
applied to all participants in the Innovative Workplaces projects, these difficulties may result in some mis-
measurement of productivity effects with respect to public sector organisations.
110
changes?’
Definitely 0 0.00 0
Likely 37.5 0.25 0.094
Neither likely nor unlikely
12.5
0.50 0.063
Unlikely
37.5
0.75 0.281
Definitely not
0
1.00
No response
12.5
0.50 0.063
Estimated probability productivity gains would not have been achieved without
improvements made (row total)
0.50
Source: Participant Survey, Nottingham Trent University, the probability productivity gains would not have been
achieved without the improvements made is estimated on by the proportion of respondents giving each response,
weighted by the relevant additionality assumption (i.e. column a x column b)
While this suggests a strong impact, comparisons between the results of the baseline questionnaire
administered six months prior to the delivery of the Innovative Workplaces project and the follow-up
questionnaire delivered six months post intervention suggest that organisations have in reality faced
difficulties over the intervention period.
Three of the five organisations able to provide pre and post projects values for turnover reported that
productivity (as measured by turnover per worker) actually fell over the intervening period, despite the
above participant perceptions to the contrary. Organisations had not responded by reducing employment
to the same extent as turnover, culminating in productivity losses.
However, two organisations reported a (measured) growth in turnover per worker of £9,417 and £30,500,
and applying the ratio of GVA to turnover in the East Midlands (34 percent) this equates to productivity
growth of £3,201 and £10,374 per worker respectively. Aggregating this across each organisations
number of workers and applying the estimated average probability that productivity gains would not have
been achieved without the changes made, gives an overall estimate of gross additional GVA through
productivity gains of £567,000. These estimates are provided in detail in the table below.
Table 2.5 Gross additional productivity gains
Organisation a) Growth in
turnover per
worker (£)
b) Growth in
GVA per
worker (£)
c) Number of
workers
d) Growth in
GVA due to
producti vity
gains (£)
e) Additionality
of producti vity
gains
f) Gross
additional
GVA due to
producti vity
gains (£)
Org 1 9,417 3,202 98 313,766 0.5 156,883
Org 2 30,512 10,374 79 819,563 0.5 409,782
Total - - 177 1,133,330 - 566,665
Source: Ecorys analysis, column f = column a x 0.34 x column c x column e.
Given respondents perceptions of improved productivity (in spite of the reality that turnover per worker
had decreased in three of the five organisations), one of the actual effects of the project may have been
to slow down decline in productivity. Additionally, turnover has been used as proxy measurement for the
value of the goods or services produced by the participating organisations, and some participants may
have produced goods that have been left unsold. As such, it is worthwhile considering possible effects in
terms of productivity safeguarded by the project under the scenario that all participants saw similar
relative productivity gains.
12
111
13
On average, the relevant organisations saw a decline in turnover per worker of £82,900 per worker or a
decline in GVA per worker of £28,100
1
. The two participants whose productivity was observed to grow
saw an average increase in GVA per worker of 19.6 percent
2
, of which 9.8 percent was attributable to the
improvements made (i.e. 19.6 x 0.50). If it is assumed that the project had similar effects amongst those
participants seeing productivity decline, then GVA per worker may have been 9.8 percent lower in the
absence of the improvements made, leading to productivity safeguarded of £2,751 per worker).
Aggregating these effects across the 1,343 workers employed by the relevant workplaces making
improvements following support suggests that productivity effects could rise as high as £4.3m (i.e. £2,751
x 1,343 (productivity safeguarded) + £567,000 (productivity increased)).
However, these findings of this alternative scenario are not carried through to the final economic impact
assessment as they are based on the assumption that productivity effects are uniform across participants.
In order to develop a more reliable measure of productivity safeguarded, it would be necessary to
establish how much lower employment and turnover would have been in the absence of the project.
Owing to the constraints set by the format of the questionnaire, a decision was made not to request
participants to report the impacts of support on turnover.
2.4.4 Employment growth
Organisations were finally asked to report whether the changes they had made had resulted in them
recruiting any additional workers (if employment had increased) or protected any jobs (if employment had
remained the same or decreased). As noted above, the majority of participating organisations had seen a
contraction in their workforce, with one organisation reporting that the changes they had made had
helped safeguard two jobs as a result of the changes made. Applying average GVA per worker in the
region (£36,000), this equates to GVA safeguarded of £72,000.
2.5 Future impacts on employment
The follow-up postal survey was undertaken six months following organisation's initial participation in the
Innovative Workplaces project. Many of the process improvements delivered by organisations will take
time to implement (and as highlighted, some participants were still at the planning stage with respect to
some areas of improvement). Additionally, the effects of improvements may take time to arise, so it is
likely that a focus on the economic impacts of the Innovative Workplaces project to date understate the
total effects of the project.
In order to capture the potential future impacts of the Innovative Workplaces project, participants were
asked to report whether they would recruit any additional workers over the next twelve months as a result
of the improvements they made to their business. Two of the eight organisations reported that they
planned to recruit a total of 13.5 workers in the next 12 months as result of implementing process
improvements, equating to a potential per annum GVA impact of £486,000 (again applying GVA per
worker in the East Midlands of £36,000).
1
Again, using ratio of GVA to turnover derived from the 2008 Annual Business Inquiry of 34 percent in the East
Midlands.
2
The combined turnover of the two participants fell from £35.2m to £35.0m, while numbers of workers fell from 213 to
177, implying turnover per worker rose by 19.6 percent from £165,000 to £198,000.
112
14
2.6 Gross additional economic impacts
Estimates of the gross additional economic impact of the Innovative Workplaces project are set out in the
table below, combining estimates of the probability that organisations would have taken up alternative
sources of support (additionality of support), the probability that organisations would have made process
improvements without Acas support (additionality of actions), and the economic impacts on GVA resulting
from those process improvements. The gross additional GVA per impact of the Innovative Workplaces
project to date is estimated at £174,000 per annum, resulting primarily from productivity growth. A further
£132,000 per annum in GVA per annum is expected as a result of creating a potential gross additional 4.2
jobs per annum over the 12 months following the postal survey (i.e. by November 2011).
To summarise, in a purely economic view, and not withstanding impacts that have not been measured
(such as productivity safeguarded) or gains from improvements that organisations may not yet have
made, the project is estimated to have generated a minimum gross additional GVA impact of £173,700
per annum, and over £300,000 including future impacts. It must also be noted that a wide-array of
benefits from the project were cited but cannot be translated into economic impacts, but are set out in
other chapters of this report.
Table 2.6 Gross additional economic impacts
Impact a) Additionality
of the project
support
b) Additionality
of actions
c) Economic
impacts
improvements
1
Gross
additional
economic
impacts
Employment impacts
Jobs safeguarded 0.84 0.32 2.0 0.5
Potential jobs created 0.84 0.32 13.5 3.7
GVA impacts
GVA safeguarded (£ per annum) 0.84 0.32 72,000 19,592
GVA created through productivity
gains (£ per annum) 0.84 0.32 566,665 154,195
Total GVA created to date - - 638,665 173,787
Total potential GVA created (£ per
annum) 0.84 0.32 486,000 132,245
Source: Ecorys analysis (gross additional economic impacts are estimated as the economic impacts attributable to
improvements x additionality of support x additionality of actions: a*b*c).
2.7 Leakage, displacement, substitution and multiplier effects
In order to move from gross additional to net additional economic impacts requires consideration of
leakage, displacement, substitution effects and multiplier effects. This section deals with each of these in
turn.
2.7.1 Leakage
In evaluating the impact of area-based initiatives (the Innovative Workplaces project was targeted at the
East Midlands), it is important to consider how far the intervention has resulted in impacts leaking outside
1
The figures in this column relate to the estimates of economic impacts provided in sections 2.42, 2.43, and 2.5.
113
15
of the target area. This typically refers to any jobs created (or safeguarded) being taken up by residents
outside the East Midlands, or any GVA created being produced by organisations that are located outside
the region (as GVA is generally measured on a workplace basis).
All organisations participating in the project were based in the East Midlands, so there is no leakage of
GVA impacts. To assess leakage of employment impacts, participants were asked to report the
proportion of their employees that live in the East Midlands. On average (weighted by employment),
participating organisations reported that 85 percent of their employees live in the East Midlands, implying
a value for leakage of 15 percent. This is higher than might be typically expected for organisations based
in the region (an evaluation all emda funded activity between 1999/00 and 2006/07 undertaken by
ECOTEC suggested around 5 percent of organisations employees live outside the region), though this
figure was somewhat skewed by a large electrical manufacturer reporting that 20 percent of their
employees lived outside the region.
2.7.2 Displacement
Displacement occurs where improvements in the performance of publicly assisted businesses comes at
the expense of non supported organisations, for example if an organisation is able to increase their
market share, taking away sales from other companies in the East Midlands. Though the bulk of
quantifiable economic impacts were due to productivity rather than turnover growth, this nevertheless
implies that organisations are able to produce more output for a given size of their workforce, potentially
taking sales away from other organisations.
A postal questionnaire approach was adopted in the evaluation, and a decision was made in
questionnaire design to exclude questions that would enable a direct estimate of displacement, to avoid
making the questionnaire too lengthy with potential negative consequences for response rates. However,
ECOTEC's
1
evaluation of emda suggested that enterprise support initiatives typically result in
displacement rates in the region of 20 percent, and it has been assumed that this applies in the case of
the Innovative Workplaces project except in relation to the two public sector participants.
2.7.3 Substitution effects
Substitution effects relate to organisations substituting one activity for another in order to take advantage
of public sector subsidies and support. This is most relevant in consideration of employment support and
job brokerage initiatives, where organisations may recruit employment programme participants at the
expense of others in the labour market as a result of publicly funded intervention. However, this may also
occur in enterprise support interventions if one production process is substituted for another to take
advantage of public sector support (an example might be energy organisations switching to renewables
as a result of tax breaks or subsidies). Substitution effects are not deemed to be relevant in relation to the
Innovative Workplaces Initiative.
2.7.4 Multiplier effects
Where organisations are able to improve their performance through greater sales or productivity, they will
consume more goods and services provided by other organisations based in the East Midlands,
generating wider economic impacts (supply chain multiplier effects). Equally, where individuals fill any
1
ECOTEC has since been renamed Ecorys.
114
jobs created or are able to increase their earnings as a result of becoming more productive, there will be
further multiplier effects as they spend their additional income in regional businesses (induced multiplier
effects).
In line with the approach set out in the emda evaluation toolkit, multiplier effects have been estimated in
line with the regional multipliers utilised in the Experian regional economic model of the East Midlands.
Average multiplier effects (weighted by industry sector) are estimated at 1.39 as set out in the table
below.
Table 2.7 Multiplier Effects
Sector a) Percentage of
respondents
b) Composite
multiplier effects
Multiplier
calculation (a*b)
Manufacturing 25 1.35 0.34
Private services 50 1.40 0.7
Public services 25 1.42 0.36
Estimated project multiplier effects - 1.39 1.39
Source: Ecorys analysis, Regional Economic Model of the East Midlands, Experian
2.8 Net additional economic impacts per annum
Estimates of the net additional economic impacts (per annum) of the Innovative Workplaces project are
set out in the table below. Total net additional GVA created or safeguarded (per annum) is estimated at
£193,000, with a further potential £147,000 per annum created by November 2011.
Table 2.8 Net additional economic impacts
Impact Gross
additional
economic
impacts
Leakage Displacement Multiplier
effects
Net
additional
economic
impacts
Employment impacts
Jobs safeguarded 0.5 0.15 0.20 1.39 0.5
Potential jobs created 3.7 0.15 0.20 1.39 3.5
GVA impacts
GVA safeguarded (£ per annum) 19,592 0.00 0.20 1.39 21,786
GVA created through productivity gains
(£ per annum)
154,195 0.00 0.20 1.39 171,465
Total GVA created to date 173,787 - - - 193,251
Total potential GVA created (£ per
annum)
132,245 0.00 0.20 1.39 147,057
Source: Ecorys analysis (Net impact = Gross additional impact x (1 – Leakage) x (1 – Displacement) x Multiplier
effects)
2.9 Present value of GVA impacts
The GVA impacts of enterprise support initiatives will grow and endure for a period of time, though in
general, such effects are expected to decay at a certain point. At the time of the survey, six months had
elapsed since organisations had participated in the initiative, so it is not possible at this stage to
determine how long impacts might endure. In such circumstances, guidance issued by BIS on
16
115
17
implementing the evaluation of RDA funded initiatives, suggests that the effects of enterprise support
should be assumed to endure for three years.
The table below shows the projected time profile of GVA impacts (together with the present value of those
impacts, applying the 3.5 percent value for social time preference recommended by the HM Treasury
Green Book)
1
. Overall it is estimated that the Innovative Workplace will have a total GVA impact of
£1,020,900 with a present value (with a baseline of 2009/10) of £972,400.
Table 2.9 Present value of GVA benefits
Year 1a) Value
of GVA
impacts
created to
date (£)
1b) Value
of
potential
GVA
impacts (£)
1c) Total
GVA
impacts (£)
Discount
factor
2a) Present
value of
GVA
impacts
created to
date (£)
2b) Present
value of
potential
GVA
impacts (£)
2c)
Present
value of
total GVA
impacts
(£)
2009/10 193,251 - 193,251 1.00 193,251 - 193,251
2010/11 193,251 147,057 340,307 0.97 186,716 142,084 328,799
2011/12 193,251 147,057 340,307 0.93 180,402 137,279 317,681
2012/13 - 147,057 147,057 0.90 - 132,637 132,637
Total
579,752 441,169 1,020,922 - 560,368 411,999 972,368
Source: Ecorys analysis (present value of benefits are measured by multiplying the value of present and future GVA
impacts by the discount factor, which in turn is calculated by (1 / ((1 + 0.035)^n) where n is the number of years from
the baseline year). Column 1c = 1a + 1b, Columns 2a, 2b and 2c = 1a, 1b, 1c x Discount Factor
2.10 Return on investment
Acas figures indicate that a total of £244,035 was spent delivering the Innovative Workplaces project, and
the project was estimated to have created economic impacts with a present value of £972,400. This gives
an overall return on investment to public sector (in regional economic impacts) of £4.0 for every £1 of
public sector expenditure.
emda contributed £227,437 of total project costs, or 93 percent of total public sector expenditure. In line
with OffPAT guidelines on attributing impacts between public sector funders, it is assumed that 93
percent of the overall impact of the project (£906,232) is attributable to emda's expenditure. The
remainder of expenditure were in-kind contributions made by Acas.
A total of £60,000 of costs represented project management, and £48,152 on evaluation costs. Innovative
Workplaces was a pilot project, and such costs may be reduced if the project was replicated in the future.
The table below also provides ROI figures under scenarios in which evaluation and project management
costs are reduced by 25 percent and 50 percent, which results in an increase in ROI to £4.5 and £5.1 per
£1 of public sector expenditure respectively, on the basis that these savings have no impact on the
overall effectiveness of the project.
1
The value of GVA is discounted to reflect a social preference for income today over an equivalent income in the
future. The 3.5 percent discount rate suggests that society as a whole is indifferent between £100 of income (or
costs) today and £103.50 of income/costs a year later.
116
Table 2.10 Return on Investment
Costs & ROI Cost (£) Economic
Impacts (£s)
Return on
Investment
emda 227,437 906,232 4.0
Total public sector 244,035 972,368 4.0
Potential ROI for future replication of the project, if evaluation & project management staff costs reduced by:
25% 216,997 972,368 4.5
50% 189,959 972,368 5.1
2.11 Costs and benefits that have not been quantified
While this analysis suggests that the Innovative Workplaces project has generated a positive return on
investment over the short period over which impacts have been allowed to accrue, it takes a narrow
economic view of the benefits of the Innovative Workplaces project. There are a range of wider effects
that would be considered in a full cost-benefit analysis of the intervention:
? Economic impacts: There are a number of economic impacts of the project that have not been
quantified in this analysis. As already discussed, any productivity safeguarded by the intervention has
been excluded from the final return on investment figures. In addition, in the future, the project may
also help organisations avoid costly employee disputes and employment tribunal claims, generating
further productivity benefits.
? Social impacts: There may also be a range of social benefits of improvements in HR processes over
and above those described in the economic impact assessment, but are expanded on elsewhere in
the report. These are most likely to incorporate any improvements in employee welfare that are not felt
directly through increases in wages following productivity gains.
? Project costs for the participating organisations: At the same time, the estimates of return on
investment outlined above do not factor in the full costs of implementing the Innovative Workplaces
project. In particular, there participants will incur costs in implementing improvements that have been
not been captured. These are most likely to take the form of opportunity costs, with managers and
other staff taking time to develop new processes and improvements, which could otherwise have been
diverted to other productive activities.
? Disbenefits: Where the Innovative Workplaces has facilitated growth or raised productivity, there will
be a range of social costs that are not captured in the analysis. These costs might be incurred by
employees (for example, transport and childcare costs associated with those filling vacancies – though
the economic impacts of such expenditures are captured through the application of multiplier effects),
or could relate to wider externalities associated with economic growth such as the environmental
impacts associated with greater utilisation of resources or congestion costs caused by more intensive
use of transport infrastructure.
With regard to economic impacts not quantified, it should be noted that the present value of GVA impacts
and return on investment figures that have been calculated only include economic impacts that could be
specifically identified and measured in the course of the project evaluation using this framework. It was
18
117
19
only possible to attribute economic impacts to four of the eight organisations which took part in the final
evaluation of the project (by using questionnaire answers given by the organisations), so the GVA impact
of £1,020,900 (with a present value at 2009/10 of £972,400) is based on the benefits attributed to the
project from The Health Store, Liquid Control, Caterpillar and Thorpe Kilworth. It is assumed that all other
organisations had no economic impacts as a result of participation, when the true economic impact of
these organisations is unknown. An economic impact of zero is, of course, not realistic – however, the
report can only quote figures which have been reported and a cautious approach (cautious in that the
economic impact was positive for four organisations and negative for none) is taken that assumes no
financial impacts were achieved where organisations have been able to fully report impacts. Estimates of
GVA impacts (with a present value of £972,400) can therefore be seen as an estimate of the minimum
return from this project.
118
Published by Acas
Copyright ©Acas
doc_101191709.pdf