Managing To Improve Public Services

Description
Managing to Improve Public Services A report by the Advanced Institute of Management Research Public Service Fellows.

Managing to Improve
Public Services
A report by the Advanced Institute of Management Research
Public Service Fellows
Edited by Jean Hartley and Alan Pike
Written by:
Professor G Boyne, Cardiff Business School
Dr. P Collier, Aston Business School
Professor C Donaldson, University of Newcastle upon Tyne
Professor J Hartley, Warwick Business School
Professor P Longley, University College London
Professor M O’Mahony, Birmingham Business School
Professor M Pidd, Lancaster University Management School
Professor C Skelcher, INLOGOV – Birmingham University
Professor B Townley, University of Edinburgh Management School
Dr. R Walker, Cardiff Business School
Professor M Wallace, University of Bath
June 2005
Contents
Introduction 1
Innovation in governance and public services 3
Professor Jean Hartley
Organisational turnaround and public service excellence 6
Professor George Boyne
Excellence, performance and innovation 8
Dr. Richard M Walker
Modelling and systems aspects 10
of performance measurement
Professor Michael Pidd
The management of metrics: 13
performance measures in the public sector
Professor Barbara Townley
Metrics for service delivery 16
Professor Mary O’Mahony
Managing police performance 18
Dr. Paul M Collier
Health care priority setting 20
Professor Cam Donaldson
Governance and performance 22
Professor Chris Skelcher
Geodemographics and local service delivery 24
Professor Paul Longley
Managing complex and programmatic change 26
Professor Mike Wallace
AIM Public Service Fellows – contacts 30
About AIM 31
1
Introduction
Public service reform remains a central priority for the newly re-elected UK
Government. During the Government’s third term there is the prospect of
continuing changes in services ranging from schools, hospitals and the police
to local government’s relationships with both Westminster, Whitehall and
local communities.
The Advanced Institute of Management Research (AIM) Public Service Fellows seek to contribute
evidence and ideas to this process of change, and to the specialist and public debates accompanying
it, by advancing the research and theory-building that is needed to inform efforts to improve
public services.
This document offers an introduction to work that the AIM Fellows have recently undertaken on management topics related
to one or more of five priority issues. The priority issues, selected for their potential to help public sector policy-makers and
practitioners in their efforts to achieve service improvements, are:
Innovation and new ways of working, including the role of leadership in stimulating and supporting new practices; creating and
utilising metrics to measure performance for improved service delivery; identifying the incentives that matter for people working
in public services; managing the potentially conflicting demands for accountability, equity, responsiveness to client demand and
value for money; building capacity to manage change, especially for implementing innovations to improve the public services.
The AIM Fellows’ work is timely not only because the public sector is going through a period of exceptional change. As some
of the following contributions indicate, there is a shortage of good quality, research-based evidence on a number of the issues
covered by the five themes. This applies to both specific topics, for example the current trends towards new governance
structures in local organisations, and to broader ones such as innovation and improvement. The work of the Fellows has
identified significant gaps in available research and projects are being initiated to address some of them.
A shortage of information about the public sector in the academic literature has led to over-reliance on private sector models
when examining important aspects of public sector innovation and management. Research outlined in this document shows that,
while private sector strategies can sometimes work in the public services, the differences between the two sectors must always
be kept in mind. Among the most obvious differences are the role of commercial competition as a driver of change in the private
sector, and the function of market prices as a measure of performance. It is clear, however, that public expectations, as well as
political decisions, help create conditions that promote change in the public sector.
Today’s public sector managers are familiar with target setting, performance indicators and measurement. The research outlined
on these pages demonstrates the problems that can arise from confused goals in the design and implementation of performance
indicators; the risks of perverse outcomes; the case for less frequent changes in priorities, and the need for stronger links between
the quest for performance improvement and the allocation of resources. Publication of performance data is shown to have more
influence on the behaviour of managers in public sector organisations than on members of the public themselves, with performance
improvements not leading automatically to greater public trust.
Similarly, innovation does not necessarily lead to improvement, although even unsuccessful attempts at innovation may be a valuable
learning experience in public sector organisations. Change generates ambiguity and uncertainty, creating conditions where efforts to
bring it about may lead to unintended consequences. External factors, as much as managerial choices, can influence a failing public
sector organisation’s chances of improving. A number of the articles also raise, in different forms, the need to link national and local
priorities, recognise the importance of variable local circumstances, and improve learning between central and local organisations.
These are among the themes raised in the following contributions from the AIM Public Service Fellows.
2
About the AIM Public Service Fellows
? 11 leading academics in the field of public service management and organisation…
? Undertaking demonstration projects, case studies, systematic reviews of evidence and the development of new techniques…
? In co-operation with leading international specialists and UK policy-makers and public managers…
? Disseminating ideas, methods and evidence through workshops, seminars, reports, journal articles and the AIM website…
? To enhance the productivity and performance of UK governance and public services.
AIM has an innovative approach to creating conditions for synergy amongst management researchers through their collaborative
efforts to inform research investment in high priority areas of national policy. The rationale for initial Economic and Social Research
Council (ESRC) and Engineering and Physical Sciences Research Council (EPSRC) investment in the AIM Public Service
Developmental Fellowships is to harness experienced management researchers’ expertise in developing thematic research and
progressing a future research agenda for public services. Delivering better public services is an enduring priority focus of domestic
policy debate. UK research on management can and should make an important contribution to policy and practice through the
identification and exploitation of opportunities for increased productivity and enhanced performance – while at the same time
identifying issues that must be addressed in future research.
Public Service Fellowships are designed to be developmental. In contributing individual expertise to the collective enterprise,
Fellows mutually support each other in exploring new ideas and their practical relevance through active engagement with
policy-makers, practitioners and other researchers from the UK and overseas.
The Public Service Fellows were each appointed for between 12 and 18 months, starting between September 2003 and
January 2004. The Fellows have worked either full or part-time on research and development activities that draw on their individual
expertise and have produced a range of outputs. The Fellows are using a variety of approaches, including demonstration projects,
case studies, quantitative analysis of large data-sets, co-production of knowledge with public service managers, and systematic
literature reviews.
They are also collaborating in a range of seminars, workshops and publications. In addition to their individual contributions (details
obtainable from the Fellows through email or University website – see contacts, p30), they are engaged in two collaborative outputs:
? A report to the ESRC entitled Themes and Issues for Consideration for the ESRC Research Programme
on Excellence in Public Services (May 2004, see www.aimresearch.org). This is a discussion document
designed to inform the longer-term ESRC research agenda on public services;
? In addition, they have a contract to produce an edited book – Improving Public Services – which draws
on the Public Service Fellows’ work and examines key questions about productivity and performance,
innovation and improvement, and managing and measuring organisational and institutional change.
The following pages illustrate the key themes, issues and questions arising from the Fellows’ work,
to inform policy and practice.
The work of
the Fellows
has identified
significant gaps
in available
research and
projects are
being initiated
to address some
of them.
3
“ Governments sometimes treat innovation and improvement as
synonymous, as though innovations inevitably lead to improved
services and governance; the reality is that these are separate
concepts and questions ”
Professor Jean Hartley,
Warwick Business School
Innovation in governance and public services
The aims of this Fellowship were to:
? Review the literature about innovation, from both the public and private sectors, to identify key themes to contribute
to improving governance and public services;
? Contribute to the longer-term research agenda on the topic of innovation and improvement in public services;
? Identify and build international links with researchers interested in innovation and improvement;
? Develop the agenda for the research and disseminate findings through discussions, in seminars and workshops,
and with policy-makers and practitioners.
Policy context
Innovation has become ‘policy chic’ in that much discourse about public services reform and improvement appears to hinge on
the need for public service organisations to become more innovative. There has been an explosion of innovation language in policy
and planning documents, and the establishment of new units and organisations dedicated to improving the quality and quantity of
innovation in public service organisations. Yet innovation in the public service sector is poorly understood; there is an over-reliance
on models of innovation from the private sector, where the drivers, processes and outcomes of innovation may often be different.
There is an urgent need therefore to look beyond the rhetoric of innovation and to develop theories and models which are relevant
to the complex and often contested circumstances of public services, drawing on the best ideas from the private sector but also
clarifying where the public service context may suggest different processes and outcomes.
Activities carried out
I undertook a detailed, systematic review of the social science literature on innovation, and identified a number of themes
and questions:
1 What are the similarities and differences in innovation in the public and private sectors, and how might these condition
our understanding of innovation in the public sector?
2 What is innovation in governance and public services and what are its catalysts?
3 What are the varied relationships that can exist between innovation and improvement, and how can innovation be evaluated
in terms of its contribution to public value?
4 What are the processes of innovation development in public services, and what are its barriers and facilitators?
5 With the diffusion of innovation being a key element of public service innovation, what are the processes of knowledge
creation and transfer which underlie the sharing of good/promising practice?
These issues are explored further below.
I also worked closely with two AIM International Visiting Fellows on aspects of innovation and improvement:
? Professor Mark Moore from the Kennedy School of Government at Harvard University, who gave a number of AIM/IGPM
seminars and workshops with academics and practitioners at Warwick, in London and in Wales, and who produced an article
in a special issue of a journal on innovation from a USA perspective; I worked with Professor Moore on a longer-term research
agenda, especially in relation to the outcomes of innovation in terms of public value, and also designed with him a one-week
programme on Innovations in Governance in December 2004, which we ran for a group of public managers from the USA,
UK and developing countries;
4
? Dr. Anne McLennan from the School of Public and Development Management at the University of the Witwatersrand,
who produced a working paper on innovation and improvement from a South African perspective, and who gave a seminar
to academics and practitioners on innovation in educational policy and school improvement in South Africa.
I also spent time working at Harvard University with colleagues at the Ash Institute of Democratic Governance and Innovation,
and at the Hauser Center for Non-profit Organizations at the Kennedy School of Government, and also visited potential
collaborators and engaged in discussions at a number of Australian universities (Australian and New Zealand School of
Government, University of Melbourne, Australian National University, and Australian Graduate School of Management, Sydney.)
I tested out some of the literature in relation to the diffusion of innovation through analysing empirical data and writing papers
which examined some of the inter-organisational processes underlying the sharing of good practice, as a key element of the
diffusion of innovation.
I presented the emerging ideas and tested these with a range of practitioners in public services in England, Wales, Scotland
and Northern Ireland.
I developed research proposals for further research into innovation and improvement in public services
I advised the Minister for Local and Regional Government on further strengthening the Beacon Scheme, based on research
into the diffusion of innovation and the sharing of good practice between organisations.
Key themes about innovation
Public and private sector comparisons of innovation
There is still relatively little literature about public sector innovation, and hence an over-reliance on the private sector as the source
of theoretical and practical understanding. There are some similarities in innovation processes and outcomes (from which it is
important to learn), but also distinctive differences between innovation in private firms and in public organisations. Economic
competition is not the primary driving force for the public sector, though the pressures to innovate also derive from a dynamic
environment of changing public need and expectations. The unit of analysis for the costs and benefits of innovation has been,
until very recently, the individual firm while that for public service organisations is generally the institutional field (e.g. hospitals,
police services) or even the public service as a whole. So, inter-organisational networks of innovation are important in the public
sector, including the sharing of good/promising practice, within a framework of open collaboration. This task is more complex
than in the private sector, where innovations are predominantly within a single firm or within a closed network of strategic
partners, within a framework of competition.
Service innovations (in contrast to product innovations) require greater tacit knowledge; have less well defined system borders; are
less tractable to cost-benefit analysis; rarely have a dedicated development unit; are more difficult to trial; concern behaviours and
work relations, and are more subjectively assessed. Service innovations and improvements typically have high levels of ambiguity
and uncertainty since they are affected by the variability of the human elements of both service givers and receivers (the latter
are sometimes co-producers).
These features suggest that the transfer of theory and empirical findings from private firms to public services is far from
straightforward. Accordingly, there is a need for robust theory and evidence derived directly from public service organisations.
The nature and origins of innovation in the public service sector
Effective innovation is not only about having a new idea but about implementing it in the organisation. It is sometimes assumed
that public service organisations are poor at innovating because the driver of competition is largely absent. However, a dynamic
society can create many catalysts for innovation; analysis shows that there have been many major innovations in the public sector
over the last 50 years, in both governance and public services. The origins of innovation in the public sector are multi-level.
Innovations may be ‘top-down’ (through policy); ‘bottom-up’ (within the organisation), or ‘lateral’ (diffusion/sharing of good practice
across organisations). Each implies a different configuration of actors and institutions, with different roles for policy-makers,
managers and citizens, and different issues for addressing the political, compared with the managerial, pressures for change.
Innovation, improvement and public value
Policy language sometimes treats innovation and improvement as synonymous, as though innovations inevitably lead to improved
services and governance; the reality, however, is that these are separate concepts. It is possible to have innovation without
improvement, improvement without innovation, or to have neither or both. Furthermore, innovation in public services needs
to be valued to the extent that it leads to improvements, not it its own right. Research from the private sector suggests that
about a third of innovations fail to be implemented successfully and it might be more realistic for governments and public services
to recognise and learn from failure (or partial failure), rather than assume identity of innovation and improvement.
Of particular interest to me in this Fellowship was how to evaluate innovation in terms of improved public value. This requires
an analysis not only of improved outputs but of improved outcomes, whether in the social, economic or political spheres.
Mark Moore’s public value chain is one approach aimed at trying to understand the wider benefits of innovations for society
and not just for the individual organisation.
5
Innovation processes, barriers and facilitators
Research in the private sector shows that innovation can be supported through stages; clear decisions are made in some firms
about whether to take the innovation to the next stage (e.g. prototype to scaling up) and exit points. However, other writers point
to the limitations of a linear approach to innovation, emphasising instead the winding paths of serendipity, power and resource
allocation on the ‘innovation journey’. Research in public services also points to the limitations of the rational planning approach,
with added complexities deriving from the barriers and facilitators of political as well as managerial leadership.
Innovation is associated with particular organisational characteristics and processes, though some are better at supporting the
initial development of innovation while others are better at supporting the exploitation of innovation. How to achieve both
continuous and radical innovation remains a key issue for organisations.
The diffusion of innovation: sharing good practice
Sharing good practice, sometimes called the diffusion of innovation, is central to public sector innovation because the outcomes
sought relate to public value, which means that good ideas in one organisation or service area can be transferred across both
geographical space and services.
Until recently, the concept behind diffusion was based on simple models of benchmarking and sharing good practice.
This emphasised a ‘drag and drop’ approach to new practices and focused especially on explicit knowledge transfer.
However, my literature review and empirical research shows that adaption rather than adoption is central to the sharing
of good practices. In addition, exposure to sharing good practice may help accelerate existing plans for change. Innovation
continues to occur in the new location, even where the primary idea is a diffusion of an existing innovation. This suggests
that knowledge is both created and transferred in diffusion, and my AIM research developed further theory to help explain
why, where and how such knowledge transfer can be successful.
Implications for policy and practice
My systematic review of the literature is punctuated with sections on the implications for policy and practice. Some highlights are:
? Recognition of the potential of the public service sector to innovate, but arising from different drivers from the private sector;
? The need to separate innovation from improvement conceptually recognising that, while some innovations may fail, learning
may be an important outcome of such experiences which helps build future innovative capacity;
? Public sector innovation needs to be judged against different criteria of success to the private firm. While the measure of
success for the latter is organisational advantage, the measure for public services is public value which concentrates on
outcomes as well as outputs, and benefits for wider society rather than just the individual organisation;
? Diffusion of innovation is central for public services. However, the knowledge creation and transfer underpinning such spreading
of good practice needs to recognise the need for adaption not adoption as a key mechanism. This involves learning how to
make best use of innovation and tailoring it to local circumstances;
? The current emphasis by government on the spreading of good practice has focused mainly more on ‘lateral’ learning (between
professions and service organisations) but there is scope also for ‘vertical learning’ (between central government and local
public service organisations).
Main outputs of the Fellowship
I have written a number of papers on innovation and the diffusion of innovation in major academic journals, including British
Journal of Management, Public Management Review and Local Government Studies. Of particular interest is a special issue
of Public Money and Management which is based on an ESRC AIM workshop and includes contributions on innovation from
the UK and USA, from academics, policy-makers and managers, and from research into both the public and private sectors.
Other papers are in submission.
The AIM International Visiting Fellows have each contributed a working paper, available from Warwick Business School,
and one further paper has already been published.
I have disseminated findings and contributed to workshops for the Scottish Leadership Foundation, the National Assembly
for Wales, for Chartered Institute of Public Finance and Accountancy, Office of the Deputy Prime Minister, Office of
Government Commerce, and the Treasury.
I have written papers for the Review of Public Administration in Northern Ireland, and for the Minister for Local
and Regional Government.
I have undertaken seminars with a range of public managers, who have come from all levels of government and most service areas.
6
“ The initial results suggest that inspectors are more likely to judge
organisations as having recovered from failure if they ‘modernise’,
regardless of improvements in their service performance ”
Professor George Boyne,
Cardiff Business School
Organisational turnaround and public service excellence
Outline of AIM Fellowship
The main aims of this Fellowship were to:
? Evaluate the relevance of private sector turnaround strategies to public organisations;
? Develop models of turnaround in the public sector;
? Undertake empirical analyses of failing public organisations and their turnaround strategies.
Policy context
Public service turnaround is an issue of pressing practical as well as theoretical importance. An important ‘scene setting’ element
of this research was to identify all of the public organisations that have been officially labelled as ‘failures’ in recent years. More
than 1,000 organisations in England were categorised as failing between 2001-2004 including schools, hospitals, health care trusts,
prisons, fire brigades, social services departments and entire local authorities. The extent of failure varies markedly across local
areas, some of which have no failing organisations while others have more than 20. It is therefore important to understand why
public organisations fail and why some are more successful than others at achieving a turnaround in their performance.
Private sector strategies and public service turnaround
Very little theoretical or empirical work on public service turnaround has previously been undertaken. By contrast, a substantial
academic literature exists on recovery from failure in the private sector. This work is based largely on strategic management
theory. Three turnaround strategies appear to be effective in the private sector.
? Retrenchment – cutting costs and raising efficiency by quitting unprofitable markets, deleting weak products and selling assets;
? Repositioning – moving into new markets and products and redefining the mission and objectives of a company;
? Reorganisation – altering the internal characteristics of a failing firm, most often by changing the chief executive or the entire
senior management team.
Theoretical reasoning and case study evidence suggest that the scope for retrenchment and repositioning is more limited in the
public sector, largely because of legal and political constraints. Thus the default turnaround strategy for failing public services is
often reorganisation, accompanied by minor elements of retrenchment and repositioning.
New models of public service turnaround
The models of turnaround derived from the literature on private sector strategies rest on two assumptions: first, that the
achievement of turnaround is subject to managerial action and control; secondly, that judgements of organisational performance
are entirely ‘substantive’ and based on the results that are achieved (e.g. efficiency, effectiveness). These assumptions, which
may not be entirely valid even for private organisations, seem inappropriate for an analysis of public service turnaround. I have,
therefore, developed two new models:
7
Luck
This model relaxes the assumption that failing organisations are masters of their own destiny. An alternative view is that
performance is partly attributable to ‘luck’, which means changes in external circumstances that are beyond the control of an
organisation. This perspective on turnaround draws upon the population ecology view of organisations. A public organisation
may fail because the level of need for its services becomes overwhelming (e.g. as a result of a local economic crisis).
Similarly, a failing organisation may improve because service needs decline or inter-governmental revenues rise.
Legitimacy
This model relaxes the assumption that failing organisations have poor substantive performance and instead posits that they
lack formal legitimacy. This perspective draws on institutional theory which suggests that success or failure is strongly linked
to whether organisations adopt the ‘right’ managerial structures and processes, as judged by powerful external stakeholders.
A focus on legitimacy is especially relevant to the public sector in the UK, because most failing organisations have had this
label attached to them by government inspectors.
New evidence on public service failure and turnaround
Two existing data sets were used to pursue the empirical aims of the Fellowship.
First, organisational failure in English local government was analysed. This combined data on the performance of 120 local
authorities with socio-economic data and information from a survey of their management characteristics. The statistical
results show that failure is attributable to both ‘misfortune’ (especially economic deprivation and diverse service needs)
and ‘mismanagement’ (weak leadership and poor performance management).
Second, organisational turnaround in 140 Texas School Districts was evaluated by combining data on exam results with
information from two surveys of school superintendents. The statistical results show that luck is a significant influence on
recovery from failure (school districts were more likely to improve if the socio-economic characteristics of local pupils and inter-
governmental revenues changed in their favour). School districts that pursued repositioning were slightly more likely to recover,
but retrenchment was associated with continued failure. The most effective turnaround strategy was reorganisation, especially
the appointment of a new chief executive from inside the organisation and the recruitment of better front-line staff.
Finally, the role of legitimacy in turnaround was investigated through qualitative studies of failing housing and social services
departments in England. For each service, four failing organisations were selected, with two that have achieved turnaround being
compared with two that are still failing. The initial results suggest that inspectors are more likely to judge organisations as having
recovered from failure if they ‘modernise’, regardless of improvements in their service performance. This evidence confirms the
important role of legitimacy in public service turnaround.
Major implications for policy and practice
This research shows that:
? Failure and turnaround in the substantive performance of services is influenced at least as much
by external constraints as organisational choices;
? Private sector strategies of repositioning and reorganisation are feasible and potentially effective
in public organisations;
? Judgements of failure and turnaround in the UK public sector are influenced too strongly by the management arrangements
in organisations that provide services. More weight needs to be placed on their substantive achievements rather than the
extent to which their management structures and processes fit the expectations of central government and its agents.
Main outcomes of the research
Two articles have been published, four more have been accepted for publication in leading journals such as Administration and
Society, Journal of Public Administration Research and Theory, Public Administration and Public Administration Review, and others
are currently under review. A co-edited book entitled Public Service Performance will be published by Cambridge University Press
in 2006. In addition to my own work, four other staff at Cardiff University are now researching public service turnaround.
The Fellowship has also led to new research collaborations with leading public management scholars such as Professor Ken Meier
(Texas A&M University) and Professor Laurence O’Toole (University of Georgia). Dissemination to policy and practice includes
a report co-authored with Stuart Reid at the Audit Commission and published by the Office of the Deputy Prime Minister.
This report provides guidance on the turnaround strategies that may be effective in public organisations.
More than 1,000
organisations in
England were
categorised as
failing between
2001-2004…
8
“ Policy-makers, when planning new management reforms, need to think
more systematically about the likely chances of successful implementation
in different types of organisations in different places ”
Dr. Richard M Walker,
Cardiff Business School and Associate Professor, University of Hong Kong
Excellence, performance and innovation
My Fellowship focused on excellence, innovation and performance. After completing ten months of the Fellowship at Cardiff
University I took leave of absence and undertook the remaining eight months at the University of Hong Kong.
A major review of innovation and organisational performance was undertaken together with a number of empirical papers.
The empirical papers explored the relationship between management and organisational performance including strategy content,
representative bureaucracy and management reform. Other studies explored workforce diversity and environmental management
in English local government and the networking behaviour of managers in the UK and US, together with research on management
reforms and professionals. Methodological papers examined the use of multiple informants in survey research in public management
and methods in innovation-performance research; a research agenda for public management scholars has been published.
Implications for policy-makers
Governments in the UK and around the globe regularly promote new ways of doing things. While many of the strategies promoted
by governments involve the implementation of single metrics, an increasing proportion focus on a number of simultaneous
changes. All these changes are introduced with the aim of improving organisational performance. However, evidence on the
likely consequences of the strategies implemented by governments is as best incomplete and at worst absent.
With these concerns in mind, a major element of the Fellowship was to review the published empirical evidence on
innovation-performance relationships. While innovation is one the most widely researched phenomena in the social sciences
only 30 statistical refereed academic papers explore this area, of which a mere two focus on public organisations.
Though the evidence may be limited, a clear conclusion of these studies is that innovation is a route to better organisational
performance. However, the limited nature of the evidence means it is not possible to be precise about when, where and how
a strategy of innovation will pay off. A number of useful issues did emerge, nonetheless:
? The impact of innovation on organisational performance is typically mediated by other management practices;
? The benefits that accrue from being a first mover are often available to others, though the size of the impact may be slightly
more moderate;
? Public organisations need to implement both product and process innovations simultaneously: to implement just one type
of innovation has a much lower, and perhaps negative, impact on performance.
A number of these concerns informed the empirical research agenda pursued during the Fellowship. Empirical evidence has
reinforced the importance of innovation type. Prior public sector research on the diffusion of innovation and organisational
innovation has often been blind to innovation type, a significant oversight because these relationships are highly complex.
When governments promote innovation they need to pay attention to external environmental, organisational and diffusion
determinants because they have different effects on the likelihood of innovation being adopted.
For example, the adoption of new product innovations is more likely to be influenced by diffusion drivers (public pressures and
competition from other public providers) than organisational or environmental drivers. Partnership innovations, on the other hand,
are driven by environmental factors (the diversity of service need and the political disposition of a local authority area) and diffusion
drivers (public pressures and competition from other public providers). The relationship between organisational characteristics
and innovation type is equally complex. Preliminary analysis shows a positive relationship between new product innovation and
organisational performance, whereas for organisational innovations the direction is negative.
9
Empirical work has also been undertaken on the consequences of strategy predicated on innovation. The strategy content
framework for public organisations developed by George Boyne and myself argues that organisations are prospectors, defenders
or reactors. Prospectors are innovative, outward looking organisations. Defenders seek to maintain their existing strategies and
structures. Reactors do not possess a strategy, but rather await instructions on behaviour from external actors.
Empirical analysis of English local government demonstrates that prospectors are more likely to achieve higher levels of
organisational performance, while reactors produce lower levels. These relationships are maintained in studies exploring the
relationship between representative bureaucracy, strategy and performance. The evidence in English local government does not
support the argument that representative bureaucracies perform better if their workforces reflect the characteristics of constituent
populations. However, where organisations pursue a prospector strategy they are able to mitigate this negative relationship.
Given the innovation focus of prospector organisations, the Fellowship has begun to provide additional supportive evidence
on the innovation-performance relationship.
My co-authored research on management reform and the professionals alludes to these complex relationships. This work
concludes that doctors have been best able to resist managerial reform, with local authority social workers offering medium, and
housing managers minimal, levels of resistance. These conclusions, indicating that universal metrics for reform are inappropriate
across and within public organisations, are confirmed in other empirical findings. Evidence has been presented to indicate that
management reforms are driven by different factors at different levels within an organisation’s hierarchy. Different forms of
management reform are very differently influenced, while the performance impacts of programmes of management reform
are likely to vary substantially.
The UK Government’s public service Improvement strategy consists of four facets: minimum standards, devolution and delegation,
flexibility and incentives and enhanced choice for the consumer. Empirical analysis of this programme of management reform
indicates that only some aspects of this agenda directly affect a number of different types of organisational performance.
Common elements that matter for performance are the process of planning for minimum standards, organisational flexibility
and user choice. Policy-makers, when planning new management reforms, need to think more systematically about the likely
chances of successful implementation in different types of organisations in different places, and the relationship between
management reforms and their likely impact on organisational performance.
My final comments are reserved for the process of research in public management. It is clear from the literature review
undertaken during this Fellowship, supported by the views of my AIM International Visiting Fellows and the empirical analysis,
that quantitative skills still need to be enhanced in the academic community. This in return requires the availability of more
comprehensive datasets on the management and performance of public agencies. With this basic infrastructure in place,
it would be possible to undertake multiple informant (as against the elite surveys often used by government) and longitudinal
analysis that controls for internal and external factors, and explores moderate relationships necessary to clearly understand
the determinants of performance in public organisations. This would enable more comprehensive advice to be provided to
the world of policy and practice.
Two important capacity building events took place during the Fellowship. The first was titled ‘Building Capacity in Quantitative
Methods in Public Management’. This workshop was lead by Professor Kenneth J Meier, of Texas A&M University, and examined
issues including the role data archives, time series data, and optimisation techniques. The second was an international seminar
held at Cardiff University in May 2004, titled ‘The Determinants of Performance in Public Organisations’. The seminar attracted
more than 30 international academics who delivered papers exploring the assertion that management matters for the performance
of public organisations. Papers from this international seminar will be published in a special edition of Journal of Public
Administration Research and Theory and an edited Cambridge University Press book.
In addition to the important focus on tangible outputs, progression to knowledge development in the field of public management
was to be advanced through collaboration with international peers. To this end active research collaboration has been initiated
during the Fellowship with Professor Gene A. Brewer of the University of Georgia at Athens (on red tape); Professor Fariborz
Damanpour of Rutgers University (on innovation and performance); Professor Jae M Moon of Korea University (on common
source bias in studies of performance in public organisations and management and performance) and collaborate research has
developed apace with Professor Kenneth J. Meier and Professor Laurence O’Toole
1
. This empirical agenda is exploring the
contributions that strategy and networking make to organisational performance in the UK and the USA. Papers on aspects of the
research undertaken have been presented at University of Georgia at Athens, Texas A&M University, Georgia State University,
Korea University, Seoul National University, Huazhong University of Science and Technology, Public Management Research
Association conference, Greening of Industry Network conference, Xi’an Jiaotong University, Asia Pacific Network on Housing
Research, City University of Hong Kong, Swinburne University, International Symposium of Public Management and the
American Academy of Management Conference. Discussions have been held with local government practitioners and
policy-makers in London, Cardiff, Hong Kong and Australia.
1
All bar Professor Damanpour were AIM International Visiting Fellows
10
“ It seems that the publication of performance data has much more
effect on managers and providers of public services than on the public.
Performance indicators are a component in a control system”
Professor Michael Pidd,
Lancaster University Management School
Modelling and systems aspects of performance measurement
I used my Fellowship to examine ways in which performance measurement is used in the provision of public services
concentrating, as the period unfolded, on a small number of issues.
Perversity and dysfunctionality
For about 50 years people have written of the possible dysfunctional effects of performance measurement in organisations,
whether public or private. Nevertheless, the same mistakes are made time and time again. Examination of the literature and
discussions with practitioners and researchers reveal that the main reason for this relates to confused goals in the design and
implementation of performance measurement in the public services. Given the many different assumptions about how public
services should be managed this confusion is not surprising, but it should not be glossed over.
The Royal Statistical Society has suggested three reasons for measuring the performance of public services:
1 To see what works: the use of quantitative indicators as a foundation for evidence-based policy by measuring and comparing
the performance of different delivery and policy options;
2 To identify competences: the use of quantitative indicators to identify good performers (and, by implication, poor performers).
Often the resulting performance data are published in, for example, school league tables, star ratings of NHS hospital trusts
and the performance of local authority social services departments;
3 To support public accountability: this rests mainly on the publication of performance data to allow members of the public
to see whether services are being delivered properly and offer value for money. Since current fashion separates policy from
operations the accountability loop must be closed in some way, otherwise there will be no link between policy and action.
Properly done, all three reasons seem sensible, though there are many hidden reefs on which the good ship performance
measurement may founder. In reality, however, there are at least two other reasons for performance measurement’s current
popularity:
4 To enable central control: there is a current fashion for setting targets which agencies must meet, and this cannot be
accomplished without externally imposed performance measurement. This can, however, lead to a climate in which innovation
is stifled, caution prevails, and people focus only on what is being measured.
5 To show that something is being done: there are often symbolic reasons for being seen to measure the performance
of public services, and this is especially important for politicians.
One of the great dangers of any performance measurement is that it can lead to ‘performativity’, which has two aspects.
The first is that apparently improved performance can be virtual rather than real because people learn to play the game.
The second is that the measurement starts to redefine the task and can construct concepts of performance in its own image.
A performance measurement system can probably be designed to meet all or most of the above objectives, but only if it is
openly and deliberately done. There is no evidence to suggest that this is the case. Although tools exist to help the design
and implementation, notably Soft Systems Methodology, these do not seem to be in use. Hence it should be no surprise
that dysfunctionality and perverse outcomes occur.
11
Modelling approaches to improve performance measurement
Performance measurement usually involves the reduction of performance to numerical performance indicators (PIs). PIs are hence
models of the performance of an organisation. Any model is a simplification and this is not necessarily a bad thing, since it allows
people to focus on what is important. However, it is essential, when building a model, to be sure what approximations are being
made and what aspects are included or omitted.
Modelling offers ways of assessing the effects of introducing performance measurement systems. Though the use of
performance assessment frameworks is almost universal in the UK public sector, consideration is rarely given to the interactions
between performance indicators and their targets. For example, the NHS operates a performance assessment framework that,
in England, includes annual publication of star ratings for NHS Trusts. It seems that this regime has been effective in reducing
waiting times for hospital treatment. However, no one really knows how much of this is due to gaming on the part of managers,
or how feasible it would be to make the targets even more severe. For example, two of the targets have tightened as follows:
Given changing demographics, changes in GP contracts and the availability of the NHS Direct service, no one knows whether the
latest targets are actually feasible. No one knows whether making them even tighter in future might be possible. Queuing theory
suggests, however, there is a serious risk that squeezing health provision at one point may lead to worse problems in places
where performance is not part of the targets.
Modelling approaches can be a great help in understanding these effects of congestion and in developing approaches leading to
increases in overall effectiveness. One outcome from my research Fellowship is a research grant from EPSRC to investigate such
effects through computer simulation. This project, Modelling for Performance Measurement and Improvement to Meet Stochastic
Demand for Public Services: A Study of Acute Hospitals, started in May. I am collaborating with Professor Peter Smith (Health
Economics, University of York) and Professor Gwyn Bevan (Operational Research, LSE). We will develop generic dynamic simulation
models of patient flows in acute hospitals to understand the effects of the performance regime. While the focus will be on acute
hospitals there should be some generic lessons for other public services, even though the dynamic models will not be transferable.
The effect of publishing performance data
Public accountability is a commonly cited justification for the publication of performance indicators in the public sector. While it is
hard to argue that this is a bad thing, evidence of the effects of publication is mixed.
Consider, for example, the publication of PIs relating to the performance of health care systems. Do the public actually use this
data and, if so, what is its effect? There has been very little research into this but, according to the limited evidence, the public
is not much interested in the data. Once a brief flurry of interest at the time of publication passes, recall is
very limited and does not seem to affect peoples’ choices. It appears that, when assessing the apparent
quality of health care, people are likely to rely on their own experience, that of relatives and others, and
advice from GPs rather than the published data.
A related issue is public trust, which is an important concern in contemporary society. Anthony Giddens,
the former Director of the LSE, has argued that in modern societies peoples’ trust has shifted significantly.
Modern societies are no longer formed around stable norms and relationships, leading to a change from
trust based on personal relationships to trust based on abstract systems devised by experts. In addition,
suggests Giddens, the unique characteristic of knowledge in the modern era is that it is temporary,
being continually questioned in the light of new knowledge. Abstract systems are intended to fill this gap.
PIs form just such an abstract system, devised by experts and intended to demonstrate the performance
of public services.
Target 2001/02 March 2003 March 2004 December 2005
Patients should wait less than the 15 months 12 months 9 months 6 months
standard for an inpatient admission
Patients should wait less than the 26 weeks 21 weeks 17 weeks 3 months
standard for an outpatient appointment
Modern
societies are no
longer formed
around stable
norms and
relationships,
leading to a
change from
trust based
on personal
relationships…
12
Do they lead to increased public trust? The indications are not good. J H Hibbard and others report a study (Health Services
Research 37, 2002) of properly designed scorecards relating to clinical outcomes in US hospitals. Not only does it show that public
recall is low, there is also a hint that people tend to recall negative rather than positive data. This may be due to the psychological
power of negative framing (a 10% risk of complications) versus positive framing (a 90% chance of success). The study suggests
that, perversely, publication of PIs may lead to a perception that things are getting worse rather than better. This raises concerns
for democratic accountability and could be exploited by the unscrupulous.
It seems, in fact, that the publication of performance data has much more effect on managers and providers of public services
than on the public. PIs are a component in a control system, as two examples illustrate. First, there has been a natural, unplanned
experiment in the NHS. The same data on performance are collected in England, Northern Ireland, Scotland and Wales. This
information leads to publication of star ratings for hospitals in England, but not in Scotland and Wales. Performance of hospitals
in England, as measured in the factors contributing to star ratings, has improved considerably over the period of publication.
That is not true of the other parts of the UK.
A second example, also reported by Hibbard et al (Health Affairs, March/April, 2003), is based on work in Wisconsin where clinical
outcome data were collected for a number of common conditions. For some hospitals, this data was published on a scorecard that
clearly identified high and low performers. For the others, the data was collected but not published. Though there is no evidence
that the public used the data to choose health care providers, its publication did apparently affect the providers themselves.
None of the hospitals was happy about the publication of the data and, at the time of publication, poorly rated hospitals expressed
concern about its potential effect on their reputations. Follow-up six months later showed that all the hospitals had put quality
improvement initiatives in place; but those poorly rated units for which data had been published had introduced more initiatives
than the ones for which it was not published.
It therefore seems that publication of data serves as a spur to service providers, encouraging performance improvement initiatives.
The challenge is to find ways to link this positive effect with increases in public trust, since this will not follow automatically.
13
“ Attempts to introduce a quasi-scientific approach to performance
measures face difficulty when measures are thought to be arbitrary,
meaningless, reflecting the easy to measure… or ignoring the nature
and realities of work ”
Professor Barbara Townley
University of Edinburgh Management School
The management of metrics: performance measures in the public sector
The research, undertaken with particular reference to the Criminal Justice System (CJS), had two primary objectives. These were
to examine the concept of rationality as understood in organisational analysis literature, and to consider its relevance for an
examination of perspectives on the introduction and use of performance measures.
Earlier research on the introduction of performance measures in public sector organisations has shown a contrast between
aspirations that performance measures evoke and problems encountered in their development and implementation. This experience
was highlighted by the House of Commons Public Administration Select Committee when it concluded of the target setting regime:
“It does not appear to be particularly effective at motivating people. Few of our witnesses claimed that in themselves, targets
were inspirational and… some saw them as obstacles to professional satisfaction and improved performance.” (2003, 32).
It concluded however: “the measurement culture cannot and should not be abolished”.
Engagement with the concept of rationality is an attempt to address the paradox between aspiration and implementation,
by identifying the different dimensions of rationality that sustain its use and appeal. Performance measures are not a neutral
template, but carry a number of implicit assumptions about rationality. A number of functioning rationalities inform individuals’
actions and their responses to performance measures. An important element of understanding the responses to performance
measures is to appreciate these various dimensions of rationality and the interactions between them.
From the initial readings of the organisational analysis literature, three different forms of rationality are identified, dependent on
whether they are disembedded, embedded or embodied. From this three ‘disembedded’ rationalities are identified: economic,
bureaucratic and technocratic. These are variations on the attempt to introduce a universal concept of rationality: the neutrally
objective ‘view from nowhere’. Three ‘embedded’ rationalities, institutional, contextual and situated, illustrate how rationality, to be
operational, is dependent on being embedded within a certain set of assumptions or context. An embodied rationality recognises
the importance of the body in organisational functioning, and how knowledge ‘through’ the body can have an impact on activities.
Finally two areas that conventional understandings of rationality have difficulties dealing with are considered: the irrational and the
collective. The final type of rationality identified is that of practical reason.
Participants’ understandings of their work contexts and experience of performance measures were analysed for what they
reflected of these dimensions of rationality. The following is an illustration of issues highlighted by each dimension.
The influence of an economic rationality in the CJS is ambiguous. Economic justification for actions and awareness of costs do
not generally inform the discourse of criminal justice. However, economic rationality is pervasive as principal agency relationships
organise the structure of the CJS and contracts govern the relationship between the executive and the prison and court services,
local authorities, voluntary organisations and others. Targets are ubiquitous throughout the CJS and introduce a heightened
awareness of transaction costs. There is evidence that performance measurement, by increasing awareness of the costs of some
activities, has brought an element of negotiation or bargaining into relationships. The ubiquity of organisational targets also introduces
a mentality of ownership and responsibility which, when it comes to shared targets, may cause problems in their operation.
Bureaucratic rationality, the exercise of ‘rule through the written word,’ informs the processes of codification, formalisation,
standardisation and centralisation that influence the role of measures in the CJS. Performance measurement requires some
basic agreement on what is to be measured and appropriate definitions. Some of the difficulties encountered in ‘writing the world’
in the CJS are reflected in, for example:
? What constitutes measurement of crime levels;
? What is encompassed by the criminal justice system and the extent of its jurisdiction;
? Problems of imposing a standardised national measurement framework on a CJS that displays many local variations.
14
A technocratic rationality introduces instrumental problem solving with the application of scientific theory and technique or
specialised knowledge. However, there is a failure to define, beyond the most general terms, what ‘the problem’ of the CJS is.
Rather there are a range of perceived problems and issues. Despite this, performance measures are presented, often in a
quasi-scientific way, as a solution, to a range of issues. For example, the impact of the work of one agency on another, often
exacerbated through conflicting targets, is taken to imply a technical or mechanical system. In fact ‘system’ is used in range
of different ways each having different implications for how the nature of interventions that may be made.
Measures are important because they support an evidence based approach to policy and a focus on outcomes. A process of
‘learning to measure’ was reported by many of those involved. However, attempts to introduce a quasi-scientific approach to
performance measures face difficulty when measures are thought to be arbitrary, meaningless, reflecting the easy to measure,
giving simplistic views of activity or ignoring the nature and realities of work, as well as the question of devising measures that
address quality as well as process issues. One of the biggest issues is control. Measures were perceived as out of the control of
those measured by them, particularly with regard to the CJS’s three outcome measures of fear of crime, satisfaction and confidence.
An institutional rationality emphasises the influence of broad institutional logics that inform what is appropriate or ‘rational’ in a
particular sphere of life. A number of ‘rationalised myths’ inform the rationale for performance measures in the Criminal Justice
System. We can identify three. First, the importance of government being able to deliver on outcomes generates expectations and
political pressure for demonstrated change and results, raising the problem of long term versus short term time frames. This was
perceived by respondents as fostering a tendency to initiatives. The second is the role of performance measures as indicators of
responsible and accountable government. These pressures were identified as generating the need for ‘good news’ stories and
‘good measures’. Finally, as crime has ridden higher in the political agenda, the ‘management’ of crime and allaying fears of it have
become more urgent political priorities, with responses often making heavy demands on the CJS. The media is a powerful force
in stimulating concerns with crime and the demand that ‘something be done’. It has made those managing organisations very
aware of the political significance of measures that are taken. The institutional concerns of senior managers about the reporting
of performance measures are often in conflict with the operational concerns of operating staff, exacerbating tensions and
communication difficulties.
A contextual rationality shows how beliefs and actions are rational because they are sustained by the community and context that
legitimises them. Strong organisational, agency and professional identities exist among the police, prosecution, lawyers, judges
and other groups, linked to views about their purpose and role in the CJS; these are very influential in informing actions and
behaviour. Challenges to these identities come from an agenda that emphasises the need to ‘manage’ the system. For some,
performance measures are an attempt to change ‘culture’ and introduce thinking that sees beyond the confines of particular roles.
An increased managerial agenda affects all CJS agencies.
A situated rationality is a spatially located rationality of making an organisation work. The factor that emerged more than any
other was the immense importance of understanding the nature of work organisation and work flows within the CJS. Often,
in the immediate work context, performance measures are seen as a hindrance, an irrelevance, or detrimental to getting the
job done. Performance measures increased the internal divisions between front line and more senior staff, with a perception of
management not knowing the details of work organisation. The complexity of the processes involved highlights the importance
of local knowledge.
A situated rationality also indicates the importance of informal, day to day interactions, threatened in some circumstances by
a performance measurement programme that introduces a blame culture. Performance measures are not seen as core if they
cannot speak to the immediate context, which in a lot of cases amounts to ‘keeping the show on the road’. No amount of
exhortation or pleas for ‘ownership’ will change this as the ultimate test of relevance. There are issues that require attention,
remedy and change in the CJS, but these are more likely to be resolved through local experimentation, co-location,
problem-solving, and task-based initiatives.
Rationality as an embodied phenomenon considers the role of the tacit and the emotional. The tacit and embodied knowledge
of ‘knowing what works’ in the CJS, absorbed through years of experience and by definition difficult to articulate, is an important
basis of legitimacy, competence and effectiveness. A large degree of tacit knowledge is often lost through perceived ‘false
economies’ that cut support staff and place too much faith in technology as a solution to problems.
The irrational or the absurd is reflected in individuals working to targets or performance measures but not understanding why, and
by a ‘measurement mentality’ that collects information rather than knowledge. Observable in most experience with performance
measurement, the CJS is no exception. A more important element is the importance of recognising that the different CJS
agencies interact in a very complex manner. Given this complexity, there are difficulties with assumptions of the CJS acting
either in a linear manner or as a self-regulating system that returns to equilibrium. Wisdom in these circumstances would
suggest that measures should act as indicators of a ‘direction of travel’, rather than as a tightly connected cascade of targets.
There is a strong belief in the official literature that performance measures act as a stimulant to communication and cross-agency
collaboration, enhancing a collective or communicative rationality. There was strong support for the need for communication
across, and co-operation between, agencies. Although measures may influence communication, at a policy level they do not
function to stimulate collective discussion as to what is required of the CJS or what policies should be. Measures rarely stimulate
debate as to what constitutes the public good.
15
Responses in the CJS are informed by a range of concerns that reflect practical reason. These include being informed by human
factors such as lessening emotional distress, helping the vulnerable, protection from fear, and helping people who are caught up
by crime. Others reflect the social value of CJS and what it is to live in a ‘just’ society and are directed towards maintaining the
ethos of CJS, protecting the accused, and preserving the integrity of CJS institutions. These all have important implications for
perceptions of, and responses to, change within the CJS.
Publications
The major output from the research is an academic monograph, Reason’s Neglect: Rationality and Organization, under review by
Oxford University Press, work on which is in progress. Other publications focus on three substantive areas: book chapters and
journal articles on the concept of rationality found in organisational analysis and the ways of conceptualising organisational studies
(Townley, 2005a; Townley, 2006a; Townley 2006b; Townley forthcoming); publications on performance measurement (Townley
2005b; Townley and Doyle 2006); and conference papers presented on the Scottish Criminal Justice System (Townley 2005c;
Townley 2005d).
Townley, B. Reason’s Neglect: Rationality and Organization. In progress.
Townley, B. 2005a. ‘Organization Studies and Academic Boundaries: The Rationale of the Subject’ Human Relations, 57, 7, 893-909.
Townley, B. 2005b. ‘Critical Views of Performance Measurement’ Encyclopaedia of Social Measurement. Elsevier. Vol. 1. 565-571.
Townley, B. 2006a. ‘La Place du Sujet dans la Theorie Organisationelle’ in Après Foucault. A Hachuel & E Pezet (Eds.) Quebec,
Canada: University of Laval Press.
Townley, B. 2006b. ‘Neglecting Reason’ Organization. The Interdisciplinary Journal of Theory, Organization and Society.
Townley, B. ‘Controlling Foucault’ Organization. The Interdisciplinary Journal of Theory, Organization and Society. Forthcoming.
Townley, B. & Doyle, R. 2006. ‘Performance Measures’ in The Blackwell Encyclopaedia of Sociology. George Ritzer (Ed.).
Oxford: Blackwell.
Conference papers
Townley, B. 2005c. ‘Performance Measures and the Criminal Justice System: Constructing the Object’ Paper presented at the
Governing by Numbers Workshop, Edinburgh, May.
Townley, B. 2005d. ‘Performance Measures and the Criminal Justice System: Representation and Re-presentation’ Paper presented
at the 4th International Conference on Performance Measurement and Management. Performance Measurement Association.
Edinburgh, July.
Conference Organisation
As part of the International Visitor element of the Fellowship, a specialist inter-disciplinary workshop was organised entitled
Governing by Numbers and held in Edinburgh, May 27th to 28th. The workshop of 25 invited participants was interdisciplinary,
drawing on academics from accounting, education, management, public sector management and the social studies of science,
to discuss the role of numbers and measurement in organisational and public governance.
Co-organised the stream on ‘Changing Institutions, Instituting Change: Heterogeneity in Institutional Theories of Organisation’ at
EGOS (European Group of Organization Studies) Ljubljana, July 2005. A paper, ‘Reason and Rationalization and its implication for
Institutional Theory’, was presented at the plenary session on ‘Looking to the frontlines of methods and theory’ based on work
undertaken on rationality, and rationalisation in institutional theory.
Convened the AIM stream at the 4th International Conference on Performance Measurement and Management, Edinburgh, July
as a showcase for the research of AIM Public Service Fellows. Four of the Public Service Fellows were able to present their research
in this stream. The PMA is a major conference on performance measurement attended by both academics and practitioners.
16
“ We should not exaggerate the particular questions raised by
measurement in public services since all measurement is complex ”
Professor Mary O’Mahony,
National Institute of Economic and Social Research (NIESR)
Metrics for service delivery
Can measurement techniques employed for private market services be used, with adjustments, to measure performance in
non-market sectors such as health and education? That was the main question addressed by this research, which was concerned
with measuring public sector output and productivity in an international comparative context. Specific research compared
performance in the UK and US education sectors, together with an overview of performance in the EU health sector.
The starting point was to consider how methods of measurement in the private sector could be translated to measuring output
and productivity in public services. In the absence of prices, quantity indicators such as numbers of pupils educated, or operations
performed, can be employed to measure output growth. Outputs require quality adjustment using information on characteristics
of the services produced, and these can in turn be based on outcomes.
Conceptually, the main difference between measuring output between the two sectors is that the information required for
measurement is embedded in market prices in the private sector, whereas direct measures of the marginal contribution of
services to outcomes are required in the public sector. We should not, however, exaggerate the particular questions raised
by measurement in public services since all measurement is complex. An important consideration in the practical application
of methods is whether outcomes might be influenced by extraneous factors, or innovations originating in other sectors.
A range of methods can be used to take account of such attribution issues.
The education research combined information on school and college enrolments with estimates of the returns to education from
micro survey data to produce quality adjusted output growth rates in the UK and US. This showed considerably higher output
growth in the UK than the US since 1979, with the largest difference in the 1990s. A benchmark of relative levels of labour
productivity, derived by comparing numbers of undergraduate equivalent students, suggested equal productivity levels in the
two countries at the end of the 1990s.
The research therefore concluded that the UK education sector had experienced positive and relatively
high growth rates in the two decades since 1979. This can be explained largely by the UK catching up to
US levels through a combination of expansion in higher education and significant quality improvement.
Since the UK has now caught up with the US, it is likely that further productivity gains will be minimal.
European health care sectors differ in terms of systems of provision. There are varying degrees of private and
public provision and funding mechanisms (private or social insurance or tax) although no European system
approaches the private-provision private-funding model of the US. A range of macroeconomic indicators
was examined to ascertain whether performance was related to inputs used and systems of provision.
The review concluded that there appears to be little relationship between expenditure and outcomes at
the macro level. Microeconomic studies that looked at specific diseases were also reviewed. Although
these suggested some link between the way systems were organised and outcomes, few general
conclusions emerged.
Specific
research
compared
performance
in the UK and
US education
sectors,
together with
an overview
of performance
in the EU
health sector.
17
This review pointed to the need for a better way of measuring the output of the sector to assess performance. Drawing on the
conceptual approach outlined above, characteristics of services produced by the health sector, including health outcomes and
reductions in waiting times, could be measured.
Research on measuring performance would benefit enormously from greater availability of micro data that allowed measurement of
characteristics and control for outside influences. Record data that followed patients from first diagnosis to post treatment outcomes,
for example, or pupil record data that tracked educational attainment and subsequent earnings, would be particularly useful.
Performance measures based on characteristics valued by consumers can potentially be very useful for policy-makers.
At present there is too much attention paid to measures based on indicators with arbitrary weights. Such measures are
too easy to manipulate and there is a need for an underlying framework to combine performance indicators.
Papers resulting from the research include:
Mary O’Mahony and Philip Stevens (2004) Outcome Based Measures in International Comparisons of Productivity in Public Service
Provision: A review, will appear in an edited Volume of papers to be published by Cambridge University Press, Summer 2006.
Mary O’Mahony, Philip Stevens and Lucy Stokes (2005), Measuring Performance in the EU Health Sector: What do we know
and what would we like to know?, mimeo, NIESR.
Mary O’Mahony and Philip Stevens (2005), Output and Productivity in the Education Sector: Comparisons for the UK and US,
mimeo, NIESR.
Papers can be downloaded from the following site:http://www.niesr.ac.uk/research/pubprod.htm
18
“ Continually changing priorities are unlikely to enable police forces
to make investments in infrastructure, personnel and training that
would lead to sustainable long-term improvements in performance ”
Dr. Paul M Collier,
Aston Business School
Managing police performance
Accountabilities, performance management and control
The research was concerned with the AIM theme of ‘excellence in public service delivery’. While the study focused on how
managers might enhance the performance and productivity of police forces, it may have wider application to public sector
performance measurement generally.
A major objective of the research design was to gain a better understanding of the drivers, constraints and consequences of police
performance management and how they might be influenced. The research sought to investigate the processes being used to
manage and improve performance in four police forces. While the initial object of inquiry was the performance management (PM)
process the National Intelligence Model (NIM) plays an important part in performance management, so the study encompassed
both PM and NIM processes and the interactions between them.
The method of research was case study through in-depth research in the four forces. Each case study comprised a series of
interviews with chief police officers and other senior officers located both at headquarters and basic command unit (BCU) levels.
Lower ranks and police staff were also interviewed. In each force, access was provided to meetings in relation to both PM
and NIM. Documentation was examined and the researcher spent 18 hours observing three shifts in one force. This included
attendance at shift briefings and discussions with control room staff receiving calls from the public and despatching officers to
incidents. This case study work was supported by desk-based research of publicly available information and interviews outside
the four case study forces.
Findings
Changes in performance indicators identified under Statutory Instruments reflect changing political priorities over the 12 years
from 1992-2004. They portray a shift in focus since the introduction of Best Value towards input and process indicators, and
a shift since 2004 in favour of outputs (detections) and outcomes (surveys of public satisfaction).
Despite the current ascendancy of Public Service Agreements (PSAs), National Policing Plans, the Police Performance Assessment
Framework and Statutory Performance Indicators, life cycles of earlier initiatives introduced by government have been short;
changes made through each successive initiative have not been incremental, unidirectional or unambiguous. Senior police
managers can therefore be excused if they take performance initiatives lightly, perhaps wondering how long it will be before
the latest one is consigned to history and replaced by some other.
Continually changing priorities are unlikely to enable police forces to make investments in infrastructure, personnel and training
that would lead to sustainable long-term improvements in performance. Short-term re-focusing of what is currently considered
by government to be important is likely to lead to equally short-term initiatives by police forces to achieve quick results. What
is needed is a consistent set of priorities, embedded in a three year policing plan that actually stays in place for three years.
PM and NIM are both governmental technologies but have different genealogies. PM is rooted in the New Public Management
concept, PSA targets and relative performance improvement; NIM is rooted in police professionalism based on understanding
crime and criminal behaviour through crime series and patterns, hotspots, and prolific offenders. The research makes explicit
the distinction between the managerial accountability of chief police officers to government (through PM) and the professional
accountability of BCU command teams to local populations (through NIM).
19
The threat of intervention by the Home Office’s Police Standards Unit (PSU) was a real concern for chief police officers and BCU
command teams in three of the four forces studied. This threat focused the corporate mind on national targets and the need to
avoid being on the PSUs ‘radar’. On the other hand, NIM is about local priorities and governance of the police by themselves.
Most high volume crime such as domestic burglary, vehicle crime, street robbery and violent offences – the major concern of
government – fits readily into both national and local priorities. However, anti-social behaviour and some other types of crime
have not tended to be equally important.
The research found that different forms of coupling reflect the relative importance attributed to professionalism and managerialism
in individual forces. It also found that professionalism and managerialism in forces are not mutually exclusive. Managerialism is
seen as having a political focus reflected in compliance with rules and regulations, a PM regime concerned only with measurable
productivity and reducing crime. Professionalism is seen as having a provider focus, concerned as much with reducing the fear
of crime as with reducing crime itself. This emphasises both results and the methods by which they are achieved, addressing
satisfaction with the quality of service. However, professionalism is not possible without legitimation and accountability for
quantitative performance, while managerialism without professionalism may lead to poor methods, poor public satisfaction, or both.
Implications for policy-makers
The research found there was a very real focus on improving performance in all the four forces. However, it identified a danger
that, in some forces, the NIM process may become dominated by the PM process and lose its inherent value of being more
operationally focussed. Both are modes of management control aimed at changing behaviour, but in different ways. There are
strengths in each method and points of overlap, but there are aspects of each which need to be emphasised. In the practitioner
report, 30 recommendations for best practice in PM and NIM were identified.
An important role for senior police managers and policy-makers is to allocate resources according to need and balance competing
requirements. The most significant problems in managing this balance are the lack of any link between resources and
performance, and failure to identify opportunity costs when shifting resources between activities. Linking resources with
performance, an important issue for all forces, is likely to re-emerge as a national priority.
Presentation of results
The research results were presented to the 3rd National Police Performance Management conference in London on 27-28
January 2005, chaired by the researcher and attended by 150 senior police officers and senior policy-makers from the Home Office.
A non-technical report containing the research findings and recommendations was produced and disseminated to the four
participating forces and to the Home Office. Findings have subsequently been discussed with HM Inspector of Constabulary
for Scotland as part of their development of performance management.
Ongoing work
A series of papers resulting from the research is presently being written and presented at conferences. These are:
? Development of performance management identifying the policy-led changes in what is measured, how it is measured
and targets between 1992-2004;
? The intelligence-led approach to policing and how knowledge is shared, identifying the serious limitations of knowledge
management in policing;
? The benchmarking of police performance, encompassing the strengths and weaknesses of relative improvement and
continuous improvement processes;
? A Foucauldian analysis of sovereignty, discipline and governmentality as it applies to performance management in policing;
? PM and NIM as competing modes of control: the development of concepts of loose coupling and isomorphism as explanations
for heterogeneity in organisational fields;
? Organisational, behavioural and social consequences of accounting and accountability for crime and detections.
20
“ As progress is made in measuring NHS productivity in terms of
reduced mortality and improvements in quality of life, questions
about the value to attach to such gains will arise ”
Professor Cam Donaldson,
University of Newcastle upon Tyne
Health care priority setting
The overall theme of the Fellowship, Health Care Priority Setting, focused on three areas of work. The first involved updating a
book entitled ‘Economics of Health Care Financing: the Visible Hand’, first published in 1993. This provides the economic rationale
for extensive government intervention in health care and a summary of health care reform initiatives over the past 20 years.
The updated second edition, published this year by Palgrave/Macmillan, involved collaboration with Karen Gerard, Senior Lecturer
in Health Economics at the Universities of Southampton and Oxford.
This starting point provided the background for the second and third specific research programmes. The second was in health care
priority setting in National Health Service (NHS) commissioning organisations, mainly primary care trusts. Such organisations are
faced with more claims on resources than they have available to expend. In such a context it would be natural to expect that
economics-based frameworks, established to aid decision makers in making choices, would be used. However, this is rarely
the case. The initial aims of the research were to establish current practice in priority setting among NHS commissioners within
the Northumberland and Tyne & Wear Strategic Health Authority area, and the potential for using more systematic frameworks.
Angela Bate, a Research Associate, conducted 22 interviews with NHS executives and these are currently being analysed.
The third area of work related to how to establish a monetary value for health gains produced by the NHS. Such gains are
commonly measured in terms of quality adjusted life years (QALYs). Existing life years and survival gains are adjusted according
to the degree of impairment in which these years are spent. Concern about the monetary value of QALYs has existed in the
health economics literature for some time and, more recently, such concern has arisen in UK health policy. As single health care
interventions are evaluated by the National Institute for Health and Clinical Excellence (NICE) in terms of their incremental cost
per QALY gained, it is important for NICE to know what value to attach to such QALY gains before making recommendations
to the rest of the NHS as to whether to adopt such interventions. Similarly, as progress is made in measuring NHS productivity
in terms of reduced mortality and improvements in quality of life, questions about the value to attach to such gains will arise.
In the longer-term, a willingness–to-pay based value (WTP) of a QALY could be estimated through survey research. More
immediately, it is possible to ‘model’ a WTP-based value of a QALY from pre-existing values of prevented fatalities (VPFs) and
values of non-fatal injuries prevented, as used in transport safety policy. An aim of the Fellowship was to design a research project
for conducting survey research, and to design and conduct a research project on modelling a value of a QALY from the VPF.
Working with economists at the Universities of Newcastle upon Tyne, East Anglia and Aberdeen two applications were submitted
and have been awarded by the Department of Health’s National Coordinating Centre for Research Methodology (NCCRM).
The award holder is principal investigator.
The implications of such work are profound. WTP-based values of a QALY, if thought to be plausible, are likely to play an important
part in deciding what guidance to recommend to the NHS on the adoption of technologies which are welfare-enhancing but often
net resource consuming. Initial estimates of the value of a QALY from modelling work range fall in the range £20,000-£64,000
depending on underlying assumptions.
Items in the above research agenda are strongly linked in four ways. Firstly, the work on reviewing health care reforms provides
an overview of the structures of health care systems within which the other two areas of work fit. Secondly, while many of the
priorities for health commissioning bodies might seem to be set at national level through NICE and National Service Frameworks,
it is important for recommendations of such national bodies to be weighed against ‘local’ initiatives which may provide more value
for money. Only through having a systematic framework in place at local level can successful or unsuccessful implementation of
a national recommendation be defended.
21
Thirdly, health care encompasses much more than the set of technologies considered by bodies such as NICE. This makes the
case for developing priority-setting frameworks at the local level, and means that NICE’s attempts to place a monetary value on
a QALY may aid local decision making. Fourthly, all of the research requires multiple methods and close collaboration with local
and national health care managers. Dialogue created through two workshops and discussions with national policy-makers has
helped improve the quality of the proposed research and should ensure effective dissemination of the results.
To take the research further, the period of the Fellowship was also used to write and submit four applications for further funding
in addition to those funded by NCCRM. They are as follows:
? A Primary Care Researcher Development Award, for Angela Bate, requesting £181,000 from the Department of Health’s
National Coordinating Centre for Research Capacity Building;
? A Knowledge Transfer Partnership (£148,566 funded by the ESRC and Newcastle Primary Care Trust);
? A research project (submitted, in collaboration with Dr. Craig Mitton, an AIM International Visiting Fellow, to the Canadian
Institutes for Health Research to the tune of Can$1,037,000);
? A research seminar series on ‘Managing scarcity in the National Health Service: building on theory, learning from practice’
(requesting £13,045 from different sources, including £8-10,000 from ESRC).
The first two of the above applications have been successful, with decisions pending on the others. These projects are focussed
on working closely with commissioning entities to both promote the use of more systematic economics-based frameworks and
observe the barriers and facilitators to this. In addition to nine refereed journal articles, the main project outputs have been as follows:
Books
Mitton C and Donaldson C. The Priority Setting Toolkit: a Practical Guide to the Use of Economics in Health Care Decision Making.
BMJ Books, London, 2004.
Donaldson C and Gerard K (with Mitton C, Jan S and Wiseman V). Economics of Health Care Financing: The Visible Hand
(2nd edition). Palgrave Macmillan, London, 2005.
Chapters forthcoming in book produced by AIM Public Services Fellows
Donaldson C, Bate A, Mitton C, Peacock S and Ruta D. Priority Setting in the Public Sector: Turning Economics
into a Management Process.
Baker R, Mason H, Donaldson C, Jones-Lee M. Valuing Public Sector Outputs.
22
“ Some single purpose bodies enable greater democratic performance
through the use of new deliberative forms of public participation, but
they can be isolated from the political process due to their managerially-
oriented boards and governance ”
Professor Chris Skelcher,
INLOGOV, University of Birmingham
Governance and performance
Forms of public service governance have expanded significantly in the past decade. Appointed boards, multi-organisational
partnerships, companies, and various other corporate forms are now found alongside more traditional elected bodies. The policy
rationale is that these new types of organisation will improve performance. But how, and in what ways, are unclear, as are the
wider consequences of moving away from control of public services by elected politicians.
The Fellowship examined the relationships between governance modes and public service performance. It involved a literature
review, small scale case studies (Scottish Water, Los Angeles Community College District and Unitary School District,
Thinktank Trust (England), Viking Ship Museum (Denmark), Herefordshire and Worcestershire Connexions (England), and
Newark Development Corporation (USA)).
A literature review produced little academic analysis of this issue; neither did an examination of UK Government commissioned
evaluations of major policy initiatives. A group of US academics has pursued theory building and empirical testing under the banner
‘governance and performance’. This work is developing a sophisticated, intellectually coherent, and substantial body of literature
yet – despite its title – the governance and performance research stream concentrates on management rather than governance.
Analysis of our case studies led to the conclusion that public service performance needs to be understood in relation to three
realms: organisational, democratic and system.
Performance in the UK context is understood largely in organisational terms, relating to factors such as efficiency, quality, outputs
and outcomes. Democratic performance concerns the anchorage of an institution in the wider governmental and political process,
while systemic performance involves the implications of forms of governance for the overall operation of the governmental system.
The implications for policy-makers relate particularly to a growing recourse to single purpose bodies set up to deliver specific
services or programmes. These allow a strong focus for the delivery of public policy objectives, thus potentially enhancing
organisational performance. Some enable greater democratic performance through the use of new deliberative forms of public
participation but, equally, they can be isolated from the political process due to their managerially-oriented boards and governance.
Single purpose bodies can enhance system performance where they are designed to enable collaboration between different
entities, but may deliver contrary outcomes as a result of fragmenting power and authority in a polycentric system.
Many of the bodies studied were hybrids, such as public private partnerships, which combine the governance arrangements and
cultural characteristics of more than one organisation or sector. Analysis of the case studies resulted in the identification of five
archetypes: the club, polity, agency, firm, and collegiate. The study concluded that the ability to deliver organisational, democratic
or system performance is affected by the characteristics of each archetype and the coherence and/or fault lines between the
archetypes in any hybrid.
23
There is a substantial literature on organisational performance, but little on democratic performance. Methodologies for assessing
democratic performance were developed during the project. This was assisted by a commission from the Home Office to produce
a paper on ways in which public organisations could assess their fitness for purpose in engaging with the Government’s civil
renewal agenda.
The AIM initiative was designed to foster collective activity between Fellows, as well as individual activity. The key elements of
this collective endeavour included: bi-monthly meetings to present and discuss work from the Fellowships; submission of a book
proposal; and a dissemination conference.
Dissemination from this specific award involved presentations to twelve conferences and seminars in the UK and abroad, and
several events for policy-makers. The AIM International Visiting Fellows played a key role at these events, and at a conference
hosted for the British Academy of Management Interorganisational Relations Special Interest Group. There have been four
published or accepted academic outputs, two outputs specifically for non-academic audiences, and four further articles for
publication are in draft or preparation. Four papers given by invited speakers at the international colloquium hosted by the
award holder have been published or accepted. Conference and working papers from the Fellowship are available athttp://www.inlogov.bham.ac.uk/research/aimfellowship.htm
Fellowship activities contributed to non-academic research users through a scoping paper to the Home Office on civil renewal,
advice to the Audit Commission on partnership governance, and a seminar at the House of Commons. The Public Service Fellows
collectively contributed to the design of the ESRC Programme Public Services: quality, performance and delivery, which was being
established concurrently by Professor Hood, University of Oxford.
The Fellowship has generated continuing streams of work, arising especially from collaboration with the AIM International
Visiting Fellows. These include a bid to the European Science Foundation Collaborative Research Programme in Social Science;
collaboration on a PhD summer school and bid to Marie Curie Research Institute; further consolidation of relationships with the
University of Southern California on neighbourhood governance in metropolitan areas, and invitations to
join conference organising panels, address conferences and author papers. Very constructive collaboration
with one of the AIM International Visiting Fellows, Professor Klijn, Erasmus University, has resulted in his
appointment as Visiting Professor at the award holder’s university to continue joint activities on PhD student
capacity building, research and publication.
Performance in
the UK context
is understood
largely in
organisational
terms, relating
to factors such
as efficiency,
quality, outputs
and outcomes.
24
“ The demonstrator work is developing into a London-wide initiative
to better target primary health care interventions ”
Professor Paul Longley,
University College, London
Geodemographics and local service delivery
Geographical Information Systems (GIS) add valuable context to decision making. Geodemographic modelling of the social,
economic and demographic conditions of small areas has been used successfully in business for a quarter of a century; yet its
potential remains under-exploited in public service delivery in spite of evidence that it prove useful in areas such as health care
and policing. This is partly because many of the data now routinely collected by public services remain unused. It is also because
synergies with new private sector data sources are not achieved, while traditions for representation and analysis in government
are different to those in business. Important commercial, bureaucratic and ethical impediments also need to be resolved.
The objective of my Fellowship was to address each of these issues, through the development of contact networks initiated by
recent projects with private and public sector organisations. A primary motivation was to present the benefits of pooling private
and public sector data. This was achieved through a data integration demonstrator project in the King’s Cross area of London. In
addition, the prospects for encapsulation of such data with GIS, and the wider implications for quantitative social science, were
assessed through collaboration with leading US GIS software and GI data integration institutions, supported by
a Visiting Overseas Research Fellow from ESRI Inc, the world’s most successful GIS vendor.
The Fellowship led to the production of an expanded edition of a research-led textbook combining generic quantitative spatial
science treatments with a wide-ranging treatment of geographic information (GI) management and policy. This is the most
accessible, authoritative and comprehensive statement of how GI technologies and geodemographics can be used to support
decision-making.
Additional specific objectives of the Fellowship were to:
? Examine the prospects for adapting best management practices in the use of geodemographic data to public sector
GIS applications;
? Identify synergies and common areas of interest in public-private sector data pooling, and establish acceptable information
sharing protocols;
? Create the King’s Cross data sharing demonstrator project, assessing how it might be used to improve performance and
productivity in health care, policing and retailing;
? Evaluate the demonstrator project and user interest in it;
? Establish a framework for greater engagement between academia and public and private sector users of GIS.
The US has long recognised the importance of dissemination of public, and to some extent private, data. This has only recently
been emulated in the UK through such initiatives as the new Neighbourhood Statistics Service (www.neighbourhood.gov.uk
and the UK Market Research Society (www.geodemographics.org.uk).
ESRI has a world lead in the supply of encapsulated business and government data products, notably through its Business Analyst
product, which offers decision-makers in both public and private sectors access to an easy-to-use, self-contained package of data
to help make strategic, tactical and operational decisions about key geographic issues. Such decisions might involve designing
inward investment or economic development programmes, evaluating sites for building shops, libraries or health care facilities, or
improving the penetration of marketing campaigns. In these important respects, public service planning in the US is moving away
from a ‘one size fits all’ approach.
25
In this context, the design of the King’s Cross demonstrator project entailed both a top-down and bottom-up approach to area
profiling. Working relationships were developed with public service organisations including the Camden NHS Primary Care Trust
and the local police to unlock data that, while collected locally, were not being used to improve public service delivery. In addition,
local area profiling was enriched by datasets with national coverage from Experian Ltd, the UK’s largest data warehouse.
Involvement of an overseas public service adviser from a leading US software house led to a research methodology entailing
technology transfer at two levels: through enhanced development of the London demonstrator project, and through development
of a contact network of European researchers and AIM Fellows. These activities sought to develop an extensive local, national
and international contact network of experts concerned with dissemination and sharing of data.
In a technical sense, this network was designed to evaluate the potential development of geographic information portals. In a
substantive sense, however, it was intended to galvanise interest from public and private sector participants in examining best
practices in using geodemographics in the private sector, and addressing impediments to their transfer to the public sector. Together,
these activities were designed to enable new ways of working between public sector organisations that are prepared to collaborate
and pool data under agreed protocols, and to empower decision makers working within devolved organisational structures.
The research was, in large part, concerned with encouraging local public service organisations to think about ways of acquiring
and developing sustainable GIS and data sharing infrastructures. Much academic activity in this area ultimately peters out into
‘soft consultancy’ or repays only long term investment of effort. The timeliness of this research, however, generated early
paybacks in the form of research contracts to develop partnerships fostered by the AIM research. It has led to two knowledge
transfer partnerships, with Camden and Southwark Primary Care Trusts, and the demonstrator work is developing into a
London-wide initiative to better target primary health care interventions; the King’s Cross demonstrator project continues
to engage end-users. It has also contributed to the award of a £3.9m Centre of Excellence in Teaching and Learning by HEFCE.
The medium of a written report is unsuitable for describing the project in appropriate detail. This extract from the narration of a 3-D
demonstrator shown to delegates at last year’s ESRI user conference in San Diego, however, may convey some of the potential:
“For the second part of the ArcGlobe demo, let’s zoom in on London, England, and take a look at how we can use 3D symbols
and geoprocessing within our 3D virtual city.
“We start one step at a time, building up our virtual city using data from the Ordnance Survey of Great Britain. We add the
River Thames to our aerial photography… We can then extrude all of our building footprints to their correct heights based
on an attribute.
“We can then zoom in for a closer perspective view along the river. The 3D Analyst also allows us to import 3D models such
as VRML, OpenFlight, 3Ds, and SkethUp, so I can start to add in more detailed geometrics of features like the London Eye
and the old County Hall… and then we can fly along the river to the Cannon Street bridge if we want…
“Now it is important to understand that ArtGlobe is not just about visualisation but is, in fact, a complementary 3D GIS application
to your 2D ArcMap environment. It is about integrating data, analysing data and solving problems.
“For example, I can add in a raster grid showing the results of a survey on fear of crime throughout the city. We can then zoom
in to the Lincoln Fields neighbourhood and start to explore what can be done to reduce fear in this area.
“This leads us right into geoprocessing and a model that was created to analyse in 3D the natural surveillance of the area.
By using the viewshed tools we can calculate how well all areas can be seen. The theory of this model is to identify areas
with poor natural surveillance which will increase fear of crime and then look at street lighting and closed circuit TVs to alter
this landscape…”
26
“ The term ‘managing change’ is a misnomer. It implies that practitioners
should have greater ability to control change than they typically experience.
But the modest possibilities for maximising control offer a productive focus
for improving the capacity to manage public service change ”
Professor Mike Wallace,
University of Bath
Managing complex and programmatic change
Theme and Developmental Activities
This Fellowship addressed the question: How might an initial conception of complex educational change and an associated practical
planning framework, derived inductively from ESRC-funded research, be refined to extend their applicability to other changes in different
public service settings? The purpose was to contribute towards a stronger basis for future research, and practical guidance on developing
generic capacity to manage complex and programmatic change within and between administrative system levels. Opportunities were
taken to collaborate with academic experts based in the UK, USA and Canada, to consult senior practitioners from the education and
health services in England, and to establish dialogue between academics and senior practitioners. Major activities were:
? A focused critical literature review of practically oriented texts employing chaos and complexity theory deductively as a basis
for prescribing practice in public service change;
? Small-scale case study research assessing the application of the initial conception and planning framework to contrasting
changes in different national contexts (the emergence of the US ‘hospitalist movement’, an increasingly complex change where
a hospital-based physician coordinates treatment for acutely ill patients, and the programmatic ‘Kindergarten-Senior 4 Agenda’
for school improvement in Manitoba, Canada);
? Two invitation seminars with senior practitioners from the UK education and health services to explore the practical applicability
of the framework;
? Consultation with experts through two ‘international expert seminars’ bringing together leading North American academics in
education (Professors William Firestone, Rutgers University, Benjamin Levin, Deputy Minister of Education in Ontario, Karen
Seashore, University of Minnesota) and health management (Professors Ann Casebeer, University of Calgary, Jean-Louis Denis,
University of Montreal, Eugene Schneller, Arizona State University, Paul Thomas, University of Manitoba). They worked with
UK academics and senior UK practitioners from the education and health services;
? A visit with Eric Hoyle, Professor Emeritus at the University of Bristol, to consult Professor James March, the organisation
theorist, of Stanford University about conceptualising the contribution of ambiguity to change;
? A think-tank exercise at the Prime Minister’s Delivery Unit involving PMDU members and UK academics to exchange ideas
on managing change that might have practical implications for the PMDU;
? Three workshops on engaging critically with literature, a contribution to AIM’s management research capacity-building.
Conception of the Change Process in Summary
Many changes are planned interventions designed to improve service provision, either incrementally or through more extensive
reform. Others are planned responses to emerging, unplanned changes. The complexity of change ranges from a simple shift
to a multiplicity of radically new linked practices. Complex change implies an overall entity containing multiple parts. Many such
changes are centrally planned as complex innovations. Others are locally emergent. They may evolve from disparate planned
changes affecting individual service organisations to complex sets of formalised practices adopted across a public service system.
Programmatic change comprises a package of mutually reinforcing individual changes. Its scope, and therefore its complexity, may
range from a single service sector to a unified strategy spanning all sectors in all services. Each of its constituent parts is of some
complexity and interacts with other changes in the programme. An individual change that becomes complex may be emergent.
But a programmatic change is likely to entail sufficient initial planning for a package of desired changes to be put forward.
Minimally, proposals for a change agenda may be articulated, promoting emergence through consultation and diverse local
development. At the other extreme a centrally-specified profile of innovations may be proposed, incorporating a comprehensive
strategy for their phasing and implementation.
27
Coping with constraints that ambiguity imposes on the manageability of change
Ambiguity implies uncertainty, where people waver between alternative interpretations or are equivocal about judging what to do.
Change increases endemic ambiguity due to the learning required to implement a new or altered practice. Complex and
programmatic change implies a substantial shift, creating uncertainty about what this practice will be and how to implement
it across public service systems. Sources of ambiguity that change exacerbates include:
? Diverse and diffuse goals – where some are incompatible, they cannot all be pursued with equal vigour, or they cannot
be put into operation in terms of measured performance;
? Limits to rationality – cognitive, since all possible outcomes of a change decision cannot be predetermined; logical where
individuals’ choice to pursue a goal prevent it being achieved when others also choose to pursue it; interpretive where people
come to competing views of the same events or change proposals; control where practitioners use influence to resist or
subvert the achievement of change goals;
? Irresolvable dilemmas – where action oriented towards one pole brings negative consequences, building pressure for action
towards the opposite pole: as in trying to promote fast-pace change and its longer-term sustainability.
Ambiguity creates conditions where change efforts are likely to generate unintended consequences. Those that are negative from
the perspective of people responsible for managing change may undermine the achievement of their goals. These consequences
also tend to stimulate ameliorative policy-making, bringing more change with its additional uncertainties. The term ‘managing
change’ is therefore a misnomer. It implies that practitioners should have greater ability to control change than they typically
experience. But the modest possibilities for maximising control inside these limits offer a productive focus for improving the
capacity to manage public service change.
A framework for analysing and planning complex and programmatic change
As a heuristic analytical or planning tool, thinking about the management of a change or programme may be informed by referring
to three aspects:
1 Change management themes, headed by ‘orchestration’;
2 Characteristics of complex and programmatic change with management implications;
3 Sequential stages in the evolution of the change process.
Aspect 1 – the overarching change management theme of orchestration implies coordinated activity within set parameters which
is expressed by a network of senior leaders at different administrative levels to instigate, organise, oversee and consolidate
complex and programmatic change across a multi-organisational system. Orchestrators take responsibility for any change
alongside other changes and the rest of their ongoing work, which for programmatic change implies coping with implications of
each change for others in the package. Steering change through orchestration is evolutionary, often unobtrusive, and includes
addressing an array of related tasks. Orchestration frames three subordinate themes:
? Flexible planning and coordination – ongoing creation, monitoring and adjustment of plans, reflecting the dilemma of
retaining short-term flexibility through step-by-step planning while retaining coherence through longer-term planning cycles;
? Culture building and communication – promoting acquiescence or support for change or programme goals. Communication
means giving consistent messages and gathering feedback to assist coordination and to pre-empt resistance;
? Differentiated support – identifying diverse needs and organising responsive activities. It extends to backing other
orchestrators across a service system.
Aspect 2 – as the complexity of a change or programme increases, so does the range and interrelationship of its constituent parts.
This complexity features up to five characteristics with management implications. Such change tends to be:
? Large-scale – involving multiple stakeholders with diverse specialist knowledge and interests who may come to incompatible
interpretations of the change;
? Componential – comprising multiple, sequential, loosely coupled or closely linked tasks;
? Systemic – spanning multiple administrative system levels, requiring influence to stimulate shifts in practice across and within
each level;
? Differentially impacting – requiring variation in learning, generating diverse responses, and resulting in varying awareness
of the totality of the change;
? Context dependent – reflecting contingent factors including resources, the legacy of past changes, and the political
environment governing emergence or planned interventions.
28
Aspect 3 – the change process comprises sequential stages, whose boundaries may be blurred:
? Initiation – leading up to some decision to proceed with an individual change or a programme, varying with the degree
of emergence or prior planning;
? Implementation – attempting to put change into practice, sometimes in phases;
? Institutionalisation – building the change into normal practice, which may vary from instigators’ vision.
Other outcomes are possible, as where implementation is abandoned.
Implications for Policy-makers
The framework has potential as a source of practical guidance. It may be used by policy-makers and senior practitioners at
different levels of public service systems as a heuristic tool, examining one or more aspects to think through how to manage the
change process as effectively as possible within the limits imposed by ambiguity. It suggests that fruitful directions for developing
generic complex and programmatic change capacity include:
? Extending systemic awareness to inform decision-making through gathering qualitative information, alongside performance
data, about the impact of initiatives on the coping capacity of practitioners;
? Identifying and supporting those in a position to act as change orchestrators at different administrative levels;
? Promoting link roles for boundary-spanning across administrative levels and between public services;
? Focusing training support on how to cope effectively with ambiguity and the limited manageability of change;
? Complementary risk-taking through combining ‘low-trust’ target and measurement with ‘high trust’ open-ended encouragement
to experiment, within limits;
? Fostering local innovativeness for incremental improvement amongst service and administrative organisations which is sensitive
to locally contingent circumstances.
A research priority is to inform change capacity-building by exploring, across a wider range of contexts, how contemporary public
service change may be managed effectively in the face of the ambiguity it generates, and how this capacity may be developed
and sustained. Recommendations for a research agenda generated during the first international expert seminar informed the focus
of a successful research proposal within this priority. The ESRC Research Grants Board has made an award of £318,000 for a
project entitled ‘Developing Organisation Leaders as Change Agents in the Public Services’, which runs from January 2006 to June
2008. Mike Wallace leads the project team, which includes Professors Rosemary Deem (University of Bristol), Mike Reed and
Jonathon Morris (Cardiff Business School). The study will investigate major forms of external leadership development support
sought and experienced by present and aspiring leaders of service organisations in education and health in England, with special
reference to provision offered by national leadership development bodies. The study will examine how leadership capacity
development to acculturate organisation leaders as change agents in three publicly funded service sectors (secondary schools,
hospitals, primary care trusts) and one part-public funded sector (higher education) relates both to the UK government strategy
for public service modernisation and personalisation and to fostering independent change and improvement.
Output Publications
Wallace, M (2004) Orchestrating Complex Educational Change: Local Reorganisation of Schools in England
Journal of Educational Change 5, 1: 57-78.
Hoyle, E and Wallace, M (2005) Educational Leadership: Ambiguity, Professionals and Managerialism
London: Sage (in press).
Wallace, M, Fertig, M and Schneller, E (eds) (2006) Managing Change in the Public Services Oxford:
Blackwell (in preparation).
(Chapter forthcoming in a book to be produced by AIM Public Service Fellows) Wallace, M: Orchestrating
Complex and Programmatic Change in Public Service Reform.
Many changes
are planned
interventions
designed to
improve service
provision,
either
incrementally
or through
more extensive
reform.
29
30
AIM Public Service Fellows – contacts
Professor George Boyne
Cardiff Business School
Cardiff University
Aberconway Building
Colum Drive
Cardiff
CF10 3EU
[email protected]
www.cf.ac.uk
Dr. Paul Collier
Aston Business School
Aston University
Aston Triangle
Birmingham
B4 7ET
[email protected]
www.abs.aston.as.uk
Professor Cam Donaldson
School of Population & Health Sciences
University of Newcastle upon Tyne
William Leech Building
Newcastle Upon Tyne
NE1 7RU
[email protected]
www.ncl.ac.uk
Professor Jean Hartley
Lead Public Service Fellow
Institute of Governance and Public Management
Warwick Business School
Warwick University
Coventry
CV4 7AL
[email protected]
www.wbs.ac.uk
Professor Paul Longley
Department of Geography
University College London
22 Bedford Way
London
WC1H 0AP
[email protected]
www.geog.ucl.ac.uk/
Professor Mary O’Mahony
Birmingham Business School
University of Birmingham
Edgbaston
Birmingham
B15 2TT
[email protected]
www.bham.ac.uk
Professor Mike Pidd
Lancaster University Management School
Bailrigg
Lancaster
LA1 4YX
[email protected]
www.lancs.ac.uk
Professor Chris Skelcher
INLOGOV
University of Birmingham
Edgbaston
Birmingham
B15 2TT
[email protected]
www.inlogov.bham.ac.uk
Professor Barbara Townley
University of Edinburgh Management School
7 Bristo Square
Edinburgh
EH8 9AL
[email protected]
www.ems.ed.ac.uk/
Dr. Richard Walker
Cardiff Business School
Cardiff University
Aberconway Building
Colum Drive
Cardiff
CF10 3EU
[email protected]
www.cf.ac.uk/cplan/
Professor Mike Wallace
Department of Education
University of Bath
Bath
BA2 7AY
[email protected]
www.bath.ac.uk/education
For further information about the
AIM initiative and the Public Service
Fellowships please visit the AIM website:
www.aimresearch.org
or email [email protected]
31
About AIM
The Advanced Institute of Management Research
(AIM) develops UK-based world-class management
research. AIM seeks to identify ways to enhance
the competitiveness of the UK economy and its
infrastructure through research into management
and organisational performance in both the private
and public sectors.
AIM consists of:
? Over 150 AIM Fellows and Scholars of which 11 are Public Service Fellows
– all leading academics in their fields…
? Working in cooperation with leading international academics and specialists
as well as UK policy-makers and business leaders…
? Undertaking a wide range of collaborative research projects on management…
? Disseminating ideas and shared learning through publications, reports,
workshops and events…
? Fostering new ways of working more effectively with managers
and policy-makers…
? To enhance UK competitiveness and productivity.
AIM’s Objectives
Our mission is to significantly increase the contribution of and future capacity
for world class UK management research.
Our more specific objectives are to:
? Conduct research that will identify actions to enhance the UK’s international
competitiveness;
? Raise the quality and international standing of UK research on management;
? Expand the size and capacity of the active UK research base on management;
? Engage with practitioners and other users of research within and beyond the
UK as co-producers of knowledge about management.
32
Notes
For all general enquiries please contact:
Advanced Institute of Management Research (AIM)
6-16 Huntsworth Mews
London NW1 6DD
Tel: +44 (0) 870 734 3000
Fax: +44 (0) 870 734 3001
Email: [email protected]
Web: www.aimresearch.org
The Advanced Institute of Management Research (AIM) was founded in October
2002. It is a multi council initiative of the UK’s Economic and Social Research Council
(ESRC) and Engineering and Physical Sciences Research Council (EPSRC) – with
activities at over 30 institutions in the UK and overseas, and offices at London
Business School.
ISBN 0-9546885-8-9
© Professor G Boyne, Dr. P Collier, Professor C Donaldson, Professor J Hartley, Professor P Longley, Professor M O’Mahony,
Professor M Pidd, Professor C Skelcher, Professor B Townley, Dr. R Walker and Professor M Wallace 2005

doc_867857509.pdf