Evaluating Change

sunandaC

Sunanda K. Chavan
1. Conduct on-going evaluation research - Evaluation is essential for promoting effective training. Research suggests that many training programs do not fulfill their promise. Only through evaluation can poor programs be improved and effective ones retained. By evaluation, we mean a process that focuses on continuous improvement rather than just a "pass-fail" test in which individuals associated with a program win or lose credibility.

When an evaluation suggests that a program falls short in achieving its goals, it should not be used to punish an individual or group. Rather, it should be used as a guide for improving the training that is offered. Evaluation should be linked to learning and the continual pursuit of quality. By making evaluation an integral part of the process, training programs will gradually become more effective.

What Are the Guidelines Worth?

Not all training programs in social and emotional competence follow these guidelines. How much money currently does training that does not follow these guidelines lose? The estimated figure is between 5.6 and 16.8 billion dollars. One arrives at this estimate by starting with the commonly quoted figure of $50 billion spent on training each year. It is then assumed that the average cost per worker for 1 week of training is $1500. Dividing this figure into the $50 billion total gives an estimate of the total number of workers trained, which is 33 million workers.

It is next assumed that only a quarter of these workers receive training related to emotional competence (The number probably is higher, but to be on the conservative side in this estimate). The rest receive technical and cognitive training. Thus, it is estimated that adopting the guidelines would improve training for about 8 million workers.

It is next factored in the difference in effect size between training programs that follow the guidelines and those that don’t. The data from a recent study of training programs in one large corporation was used, which found that programs adhering to most of the guidelines improved the impact of training over those that didn’t follow the guidelines. To be conservative, it is then assumed that only about half the workers who now are trained (the lower half of the distribution) would do better by that amount if the guidelines were adopted. Thus, it is estimated that if the guidelines were adopted uniformly, about 4 million workers would show an improvement in training impact.


GRAPH 1: EI over the Age Span

It is possible for people of all ages to become more socially and emotionally competent. However, the principles for developing this type of competence differ greatly from those that have guided much training and development practice in the past. Developing emotional competence requires that one unlearns old habits of thought, feeling, and action that are deeply ingrained, and grow new ones. Such a process takes motivation, effort, time, support, and sustained practice, as the guidelines presented in this article make clear. The guidelines also suggest that the preparation and transfer-and-maintenance phases of the training process are especially important. Yet too often these phases are neglected in practice.


Organizations increasingly are providing training and development that is explicitly labeled as "Emotional Intelligence" or "Emotional Competence" training. However, the guidelines presented here apply to any development effort in which personal and social learning is a goal. This would include most management and executive development efforts as well as training in supervisory skills, diversity, teamwork, leadership, conflict management, stress management, sales, customer relations, etc.

Ideally, efforts to develop emotional competence would include all the elements we have identified here, but one realizes that it often will not be practical to do so. Fortunately, the effect of adhering to the guidelines is multiplicative and synergistic: the more guidelines that trainers can follow the greater and more lasting will be their impact.
 
1. Conduct on-going evaluation research - Evaluation is essential for promoting effective training. Research suggests that many training programs do not fulfill their promise. Only through evaluation can poor programs be improved and effective ones retained. By evaluation, we mean a process that focuses on continuous improvement rather than just a "pass-fail" test in which individuals associated with a program win or lose credibility.

When an evaluation suggests that a program falls short in achieving its goals, it should not be used to punish an individual or group. Rather, it should be used as a guide for improving the training that is offered. Evaluation should be linked to learning and the continual pursuit of quality. By making evaluation an integral part of the process, training programs will gradually become more effective.

What Are the Guidelines Worth?

Not all training programs in social and emotional competence follow these guidelines. How much money currently does training that does not follow these guidelines lose? The estimated figure is between 5.6 and 16.8 billion dollars. One arrives at this estimate by starting with the commonly quoted figure of $50 billion spent on training each year. It is then assumed that the average cost per worker for 1 week of training is $1500. Dividing this figure into the $50 billion total gives an estimate of the total number of workers trained, which is 33 million workers.

It is next assumed that only a quarter of these workers receive training related to emotional competence (The number probably is higher, but to be on the conservative side in this estimate). The rest receive technical and cognitive training. Thus, it is estimated that adopting the guidelines would improve training for about 8 million workers.

It is next factored in the difference in effect size between training programs that follow the guidelines and those that don’t. The data from a recent study of training programs in one large corporation was used, which found that programs adhering to most of the guidelines improved the impact of training over those that didn’t follow the guidelines. To be conservative, it is then assumed that only about half the workers who now are trained (the lower half of the distribution) would do better by that amount if the guidelines were adopted. Thus, it is estimated that if the guidelines were adopted uniformly, about 4 million workers would show an improvement in training impact.


GRAPH 1: EI over the Age Span

It is possible for people of all ages to become more socially and emotionally competent. However, the principles for developing this type of competence differ greatly from those that have guided much training and development practice in the past. Developing emotional competence requires that one unlearns old habits of thought, feeling, and action that are deeply ingrained, and grow new ones. Such a process takes motivation, effort, time, support, and sustained practice, as the guidelines presented in this article make clear. The guidelines also suggest that the preparation and transfer-and-maintenance phases of the training process are especially important. Yet too often these phases are neglected in practice.


Organizations increasingly are providing training and development that is explicitly labeled as "Emotional Intelligence" or "Emotional Competence" training. However, the guidelines presented here apply to any development effort in which personal and social learning is a goal. This would include most management and executive development efforts as well as training in supervisory skills, diversity, teamwork, leadership, conflict management, stress management, sales, customer relations, etc.

Ideally, efforts to develop emotional competence would include all the elements we have identified here, but one realizes that it often will not be practical to do so. Fortunately, the effect of adhering to the guidelines is multiplicative and synergistic: the more guidelines that trainers can follow the greater and more lasting will be their impact.

Hey sunanda, its a very nice topic and you explained it very effectively. I am also downloading a document which contains limitations of evaluation, different methods for evaluation of changes. So please check and let me know your feedback.
 

Attachments

Back
Top