Description
In algebra, the discriminant[1] of a polynomial is a function of its coefficients which gives information about the nature of its roots. For example, the discriminant of the quadratic polynomial
Management Research on Moral Competency Inventory Validation: Content, Construct, Convergent and Discriminate Approaches
Purpose: Our research fulfills three purposes: 1) introduce practitioners to the appropriate use of measures of unethical behavior; 2) evaluate the use of integrity related assessments for use in personnel selection; and 3) determine the validity of the MCI instrument using standard validation procedures. Methodology: Content, construct, convergent and discriminant approaches are applied to establish the relative validity of the assessment tool. Findings: The results of the MCI purport to align with one's moral values and behaviors. We establish face validity of the MCI measure, but fail to establish: 1) an appropriate simple factor structure, 2) convergent validity, 3) discriminant validity, and 4) support for the lack of impact of demographic factors on the purported measure of Moral intelligence. Research limitations: An acceptable but constrained (working students) sample was used in the validation. Practical implications: Researchers and practitioners should be familiar with psychometric principles to ensure the use of valid tools in a predictive and defensible manner. New measures can be developed, but should be validated before used for developmental or personnel decision making purposes. Originality/value: 1) The lack of validity associated with the MCI instrument is established; 2) Researchers and practitioners are exposed to considerations in the appropriate use of measures of unethical behavior, and 3) exposed to several previously validated integrity related assessments for use in personnel decision making.
2
Introduction With the recent corporate scandals involving Tyco, Enron, and Worldcom, the moral standards of leaders have come under heavy scrutiny. Managers, human resources practitioners, and individuals involved in personnel selection decisions all have a stake in preventing their organization from becoming victims of unethical behavior, be it white-collar or other. Given these concerns, measurement of candidate integrity and responsibility as individual differences variables are well represented in the industrial/organizational psychological literature (Wanek, 1999). These measures have been used to predict a wide range of ethically related employee behavior such as honesty, violent tendencies, turnover, theft likelihood, white-collar crime and other counterproductive employment behaviors (Miner and Capps, 1996, Wanek, 1999). Understanding the moral and ethical principles underlying the decisions of successful leaders has captured the interest of business people for sometime (Bass and Stodgill, 1990). However, agreeing on, and measuring purported moral principles can be fraught with difficulty. First, what are moral principles? Lennick and Kiel (2005) define moral principles as values that cut across all cultural boundaries. Lennick and Kiel argue that people overwhelmingly prefer to follow leaders who exhibit integrity, integrity being a universal human (extant across time and cultures) moral principle. Other moral principles they claim and attempt to measure in their Moral Competency Index (MCI) are responsibility, compassion, and forgiveness. The results of the MCI purport to align with one's moral values and behaviors. While the extant industrial/organizational psychology literature supports the naming conventions associated with the above constructs, no evidence of measurement validity for the MCI is presented in their
3
book, Moral Intelligence (2005). Our research aims to fulfill three purposes: 1) to introduce practitioners to the appropriate use of measures of unethical behavior (integrity related assessments); 2) to evaluate the use of integrity related assessments for use in personnel selection; and 3) to determine the validity of the MCI instrument using standard validation procedures. Moral Intelligence Moral intelligence is presented as the ability to methodically apply universal moral principles to one's ethics, objectives, and dealings (Lennick and Kiel, 2005). One premise to such a definition is that morality is a priori, meaning individuals have an innate tendency to act with integrity, responsibility, compassion, and forgiveness. The MCI attempts to measure the dynamic nature (it aims to measure if actions and beliefs are aligned) of one's moral intelligence. Lennick and Kiel use the metaphor of a moral compass to measure alignment. The moral compass is one's innermost beliefs and values that guide all thoughts and actions, but no objective measure of the moral compass is provided. Furthermore, according to Lennick and Kiel, a moral decision process is not contingent on demographic factors such as gender, race, nationality, or religious practice, which runs against assumptions in the current literature (Martin, Rao and Sloan, 2008; Ford and Richardson, 1994). Rather, from the compass perspective, judgment with respect to good deeds and bad is rooted in universal principles and cross all cultural barriers. It is asserted that everyone supports values from all of the philosophies mentioned above but weights vary between individuals and weights change as one progresses in age (Lennick and Kiel, 2005). Other similar research that has sought to understand and identify moral standards by conducting in-depth interviews with managers (Lennick and Kiel, 2005; Bird and Waters, 1987).
4
The Bird and Waters study identified seven moral standards as opposed to the four measures and ten attributes of the Lennick and Kiel instrument. Many models have been developed to understand the precursors to ethical decision making, with personality traits being seen as the mitigating factors in ethical decision-making, in contrast to the assertions underpinning the MCI. (Bartels, 1967; Hunt and Vitell, 1986). Still other research looked to develop frameworks on explaining the process for ethical decision-making (Ferrell and Gresham, 1985; Robin and Reidenback, 1988). Lennick and Kiel present their approach as being more interested in helping people understand how well they are aligned with their respective moral compass at a given moment in time. Accordingly, the purpose of the MCI is to help people identify attributes within a moral framework that can be further developed. Thus, the research conducted by Lennick and Kiel purports to offer a tool that an individual can use to measure their current moral alignment and determine what areas are in need of improvement such that one is better aligned to their own moral compass, whereas previous research attempted to better understand the precedent factors to ethical decision-making or to model decision-making processes within an ethical context. While the MCI does not aim to provide people with personnel selection or employee development tools, it is presented in the format of a leadership development opportunity, one that might be misconstrued as offering organizations a facile tool for identifying potential leaders. Given the .41 correlation between supervisory ratings of overall job performance and personality based measures of integrity in the Ones and Viswesvaran (2001) metaanalysis, there may be grounds for an empirical test of Lennick and Kiel's assertion. Any tool that purports to measure psychological constructs of integrity and responsibility is should be subject to an empirical validation. Lennick and Kiel do not provide an empirical validation of the MCI.
5
Integrity Integrity tests grew out of the lack of validity in polygraph tests in the measurement of honesty (Miner and Capps, 1996) as a predictor of workplace behavior. They have been used for pre-employment selection tests and are associated with workplace behaviors such as: honesty, violent tendencies, turnover, theft likelihood, white-collar crime and other counterproductive employment behaviors (Wanek, 1999). There are two types of integrity measures: overt and personality-based (Sackett, Burris, and Callahan, 1989). Overt measures indicate the specific type of unethical behavior in which the test taker may have participated in. In this sense they exhibit high face validity and are called "clear-purpose tests". Personality-based measures do not specify the unethical behavior, but assess personality traits that may be associated with the likelihood of participation in unethical conduct. The MCI seems to exhibit the qualities of a personality-based measure of integrity. Wanek (1999) reports the professional consensus on integrity measures in the statement: "To summarize what we know, evidence supports the conclusions that professionally developed, commercially available integrity tests are reliable, valid predictors of productive and counterproductive measures of job performance" (P.190). Integrity and Personnel Selection The practical use of integrity measures are clear in human resources selection practices, such as in selecting salespeople or accountants. In a metaanalysis, Ones and Viswesvaran (2001) explored the impact of criterion focused occupational personality scales (COPS), with an emphasis on construct and incremental validity in integrity measures, and measures of counterproductive behavior. They focused on these measures related to personnel selection as many American businesses use integrity measures as part of their hiring process. Based on 665
6
validity coefficients, they investigated integrity test validities across three different categories: job performance, general counterproductive work behavior, and theft. They found that after correcting for range restriction and unreliability in the criterion, validity coefficients of .29 were found for broad counterproductive behaviors, and .41 between supervisory ratings of overall job performance and personality based measures of integrity. In sum, the metaanalysis justifies the use of integrity measures in personnel selection. Hiring companies reap the benefit of selecting employees that are less prone to participate in counterproductive behavior. Integrity Related Measures used to Predict White-Collar Crime Given the MCI integrity and responsibility measurement claim, we reviewed Collins and Schmidt's (1993) comparison of integrity-related differences between federal prison inmates convicted for white-collar crimes to white-collar managers. In their study they used subscales of the California Personality Inventory (specifically the responsibility, stability, and self-control scales), a biodata scale, and a PDI Employment Inventory (a personality measure predictive of productive or counterproductive employee behavior). They found strong to moderate correlations between the 365 criminals and 344 managers on measures of responsibility (.40). It is difficult to assess how prevalent white-collar crime is in business today due to a lack of effective governance and compliance agencies. There also may be key psychological differences between those who participate in white-collar crimes and those caught participating in white-collar crimes, thus the Collins and Schmidt study may be dealing with a highly attenuated sample). Drawing from this research, the current study examines the relationship between the MCI and measures of integrity, and responsibility in an independent sample of students.
7
Moral Intelligence and Integrity Given the above descriptions of moral intelligence and its emphasis on interpersonal integrity, it became clear that there may be some commonalities between the MCI and integrity related measures. We have discussed the utility of integrity measures above, and note that prior researchers concluded that situational variables are superior to individual variables in valid measures of integrity (Lucas & Friedrich, 2005). Much like the research mentioned, the constructs of the MCI are all situational variables, and may be less susceptible to social desirability bias as overt measures. The first four of the competencies in the MCI (Acting consistently with principles, values and, beliefs, telling the truth, standing up for what is right, keeping promises) claim to measure integrity. Though Lennick and Kiel purport not to use personality traits to measure moral intelligence or alignment, the commonality between the items and constructs bears examination. Moral Intelligence and Responsibility According to Lennick and Kiel, responsibility is measured in competencies five through seven (Taking responsibility for personal choices, Admitting mistakes and failures, Embracing responsibility for serving others). As with integrity, Lennick and Kiel appear to attempt to measure constructs that are valid across time and culture. Other research has supported responsibility as paramount in importance by quantifying financial results between irresponsible corporations, and those who take responsibility for the communities in which they operate (Cava and Mayer, 2007). Bird and Waters (1987) found that where managers discussed moral issues, seven distinct moral standards were found, two of which included 'responsibility' in context. Thus, the MCI seeks to assess an individual's responsibility as it applies in a leadership context and corporate environment.
8
Current Study The current study evaluates the validity of the MCI from multiple measurement perspectives. Specifically, we first addressed the content validity of the MCI. Content validity refers to the operationalization of the MCI against the integrity and responsibility content domain. Given previous validations of the International Personality Item Pool (IPIP) domain measures, we used them as detailed descriptions of the content domain. Second, we examined the construct validity of the MCI by establishing its simple factor structure. We specifically were looking for concurrence between the theoretical domains postulated by Lennick and Kiel and actual construct loadings through principal axis factoring. Principal axis factoring seeks the least number of factors (constructs) which can account for the common variance (correlation) in a set of items. Third, we aimed to establish both the convergent and discriminant validity of the MCI by using previously measures of theoretically related measures, specifically measures of integrity and responsibility. In convergent validity, we examine the degree to which the operationalization is similar to (converges on/correlates with) other operationalizations that it theoretically should be similar to. Discriminant validity provides information regarding the lack of correlation between theoretically different constructs. Finally, we explored the claim that the constructs the MCI purports to measure are independent from demographic factors such as gender, race, nationality, or religious practice, by looking for significant differences in participant demographics in scale responses. Methodology Participants Participants were students from a medium-sized western United States university. The
9
sample consisted of 171 participants from both graduate and undergraduate classes. Data collection was part of an opportunity for extra credit in all of the classes during the Winter and Spring quarters of 2006/2007. Participation was voluntary and the instruments were administered via an online survey package at three different points during the quarter to ensure students did not suffer fatigue. The participants were from Business Administration disciplines. The age ranged from 21 to 50, the majority being between 21 and 25 (58%). Descriptive statistics for participants are presented in Table 1 (see below). Table 1 about here
Instruments Moral Competency Inventory Measurement Instrument Lennick's and Kiel's contention is that leadership potential is highly correlated with moral intelligence. The MCI sets out to measure ten competencies within a moral framework. The competencies are: 1) Acting consistently with principles, values and, beliefs 2) Telling the truth 3) Standing up for what is right 4) Keeping promises 5) Taking responsibility for personal choices 6) Admitting mistakes and failures 7) Embracing responsibility for serving others 8) Actively caring about others 9) Ability to let go of one's own mistakes
10
10) Ability to let go of others' mistakes. A five point Likert-like scale (1=Never, 2=Infrequently, 3=Sometimes, 4=In most situations, and 5=In all situations) remains constant through the forty question instrument. Additive scores of 90 to 100 are considered very high, 80 to 89 high, 60 to 79 moderate, 40 to 59 low, 20 to 39 very low. The score is indicative of one's actions with respect to one's beliefs and is labeled as an "alignment score" (Lennick and Kiel, 2005). A lower score in a specific competency is indicative of area in need of improvement. The alignment takes into account three factors: moral intelligence, moral competence, and emotional competence (Lennick and Kiel, 2005). Moral competence complements moral intelligence in that it is the social skill set needed (read "courage") that allows one to act on principles. Underlying moral competence is emotional competence, which provides one with the steadfastness needed to handle morally difficult situations and emotions within ourselves, and with others (Lennick and Kiel, 2005). As of our writing there were no measures associated with the moral or emotional competence constructs, nor psychometrics (validity, reliability) associated with the MCI. As noted by Lennick and Kiel, they conducted research through case study, and by interviewing CEO's and other senior managers. Currently, they are in the process of quantitative data collection and analysis in efforts to validate the MCI. The majority of research regarding integrity measures has been conducted by test publishers (Camara and Schneider, 1994) for the purpose of validating and publishing commercial tests. Accordingly, academic research can be prohibitively expensive. As such, we used public domain individual differences instruments available from the International Personality Item Pool (IPIP, 2006) for this study. The psychometric properties of the IPIP scales
11
have been assessed and validated by Goldberg and colleagues (Goldberg. L, 1999, Goldberg, Johnson, Eber, Hogan, Ashton, Cloninger, and Gough, 2006). In our study the IPIP proxy scales for the Personality Attributes Scale (PAS), the California Psychological Inventory (CPI), and the Values in Action (VIA) scale. All IPIP scales are five point Likert scales, with one signifying "very inaccurate", and five signifying "very accurate". Integrity Related Individual Difference Measures To assess integrity we used the following scales from the IPIP CPI proxy; Responsibility (Cronbach's = 0.66) which evaluates an individual's level of "conscientiousness, honesty, dependability, and practicality" (Kaplan and Saccuzzo, 2001), and the Values in Action Integrity/Honesty/Authenticity (Cronbach's = .72) IPIP proxy. Social Desirability To control for social desirability (a person's idealized projection of themselves: Derlega, Winstead, and Jones, 2005), we assessed impression management of our respondents by administering the IPIP PAS proxy (Cronbach's alpha, 0.82). Results Basic descriptive statistics and reliability estimates (Cronbach's alphas) for all of the MCI scales were established for the sample and are reported in Table 2. Most yielded acceptable alpha levels (see table 2). Table 2 about here
To extract factors and better understand the structure of the MCI, we used Principle Axis Factoring procedure for extraction with Direct Oblimin rotation. The first factor accounted for
12
38% of variance (Table 3) The factor analysis established eight viable factors, not the ten posited (please see Table 4). . Table 3 about here
Table 4 about here
To establish the relationship between the MCI and related integrity measures, bivariate correlations were run. These were followed by partial correlations with impression management partialed out to control for socially desirable response bias. Interestingly, predominately larger significant correlations were found between the integrity, responsibility, and MCI scales in the partialed correlations. This finding suggests that, while our instructions asked participants to use candor in responding to the questionnaire, impression management had an impact on participants' responses. We also included a measure of self esteem to establish the discriminant validity of the measures (Table 5). Table 5 about here
Finally to investigate the claim that moral intelligence is not impacted by demographic variables, we performed a univariate analysis of variance (Lunney, 1970) on the scales with gender as are fixed factor. While the sample size in this study was relatively small , gender did significantly impact scale responses for the "Keeping Promises" scale, as well as the "Actively Caring" scale. Interestingly the univariate analysis of variance for the total MCI score approached significance (p. = .06), suggesting potential problems with the assertion that other demographic characteristics do not impact the construct of moral intelligence (see table 6). Table 6 about here
13
Discussion Clearly understanding concepts of validity and reliability in the use of any assessment tool irrespective of its intended end-use (for personnel selection or development) remains critical for practitioners. In the case of the moral competency index, it does not seem too difficult to conceive of circumstances where a manager or human resources practitioner would attempt to do the organization a favor by utilizing such a tool for selection purposes. It is important to recognize established psychological, educational and business testing conventions, as well as federal guidelines in the use of tests for selection purposes. If invalid measures such as the MCI are used for leadership potential, scores may be used as potential evidence, leading to promotions, training, or other benefits that may lead to personnel decision making. This creates invalid and potentially discriminatory selection practices, leading to flawed selections, or discrimination law suits. Accordingly we present a brief summary of our findings and the concepts and minimum requirements for using assessments in development identification. Content Validity Given the similarity of materials presented in the items of the MCI and other valid measures of integrity and responsibility, the content seems to be appropriate given the domains the MCI attempts to measure. Several of the questions in the MCI suffer from item writing problems (for example: double-barreled questions. "I can clearly state the principles, values, and beliefs that guide my actions", Lennick and Kiel, 2003) but overall the content seems to be adequately measured from a content validity perspective. Construct Validity We examined the construct validity of the MCI by establishing its simple factor structure. Based on the results of our Principal Axis Factor analysis, the constructs the MCI attempts to
14
measure do not seem to exist outside of the philosophical domains the authors cite. Of course, given the types of items they use, there are some similarities, but the factor loadings suggest that if the authors aim to measure their described constructs for multiple purposes, the scales will have to be refined. Convergent/Discriminant Validity Finally, we aimed to establish both the convergent and discriminant validity of the MCI using previous measures of theoretically related measures, specifically measures of integrity and responsibility. In convergent validity, we examine the degree to which the operationalization is similar to (converges on) other operationalizations that it theoretically should be similar to. Using the commonly held r = .85 cutoff for an assessment of conceptual agreement, we do not find strong evidence to support the use of the MCI as a measure of either integrity and responsibility. Given the lack of significant correlations between self-esteem and the MCI scales we have some evidence for discriminant validity. However given the preponderance of evidence from the above analyses, this does not offer sufficient support to utilize the tool in any developmental or selection circumstance. Accordingly, the measure should advise individuals who might use it that it exists solely for philosophical/entertainment purposes as there does not seem to be any sufficient relationship between it and any other established measure that attempts to measure what it purports to. Conclusion In summary, some do not feel that Lennick and Kiel have captured anything of significance with their book (Landry, 2005). We had the opportunity to establish the psychometric validity of the MCI in a diverse and moderately sized sample. A larger, more
15
diverse and balanced sample would have been desirable. At the same time, the theory argues that demographic factors do not impact ones moral competence, and the diversity of our sample offered an empirical test of the claim. If Lennick and Kiel choose to revise the MCI in light of this or other research (according to the website - http://www.moralcompass.com/ongoingresearch.php - ongoing research is being conducted) they should either revise the theory of moral intelligence to reflect the discrepancies established in our research, or consider some of the aforementioned work in the field of psychology (see literature review above). While the Landry review looks at the overall content of the text, a more stringent review of the psychometrics associated with the MCI measure is presented here. The question as to what the MCI aims to measure becomes paramount in the analyses of the data. The field of industrial organizational psychology has examined the relationship between individual differences, leadership potential, and honesty, integrity and responsibility for close to 60 years, but it seems that this material was not present in the development of the concepts purportedly measured by the MCI. From a strictly validity-based perspective, Landry's perspective is quantified here. Practical applications in the fields of leadership, management, organizational behavior, and human resources are plentiful. As such practitioners should become familiar with preestablished valid tools that measure psychological constructs in a predictive and defensible manner. This is not to suggest that new measures cannot be developed that can add a great deal of benefit to any field in which leadership potential or measures of moral, ethical, honest and integrity based differences will be related to performance. However, we suggest that practitioners maintain their awareness of psychometric principles and a basic understanding of validity at a minimum before they use any measure for developmental or personnel selection purposes. With the use of psychometrically valid tools, the scientist is better able to model
16
behavior and the practitioner is better able to meet the challenges of their work. We encourage future members of both parties to closely examine popular assessment tools to ensure they offer the utility they purport to.
17
References Bartels, R. (1967) "A Model for Ethics in Marketing", Journal of Marketing, Vol. 31, No. 1 Bass, B., and Stogdill, R. (1990) Bass & Stogdill's Handbook of Leadership. 3rd edition, Free Press Bird, F., and Waters, J. (2003) "The nature of managerial moral standards", Journal of Business Camara WJ, Schneider DL. (1994). Integrity tests: Facts and unresolved issues. American Psychologist, 49,112-119. Cava, A. and Mayer, D. (2007)" Integrative Social Contract Theory and Urban Prosperity Initiatives", Journal of Business Ethics, Volume 72, Number 3 Collins, J. M., and Schmidt, F. L. (1993) "Personality, Integrity, and White-Collar Crime: a Construct", Personnel Psychology, 46 (2). Derlega, V. J., Winstead, B. A., and Jones, W. H. (2005) Personality: Contemporary Theory and Research (3rd ed.), Thomson. Ford, R.C. and Richardson, W.D. (1994) "Ethical Decision Making: A Review of the Empirical Literature", Journal of Business Ethics, 13 (3). Ferrell, O., and Gresham, L. (1985) "A Contingency Framework for Understanding Ethical Decision Making in Marketing", Journal of Marketing, Vol. 49, No. 3 Goldberg, L. (1999) "A Broad-Bandwidth, Public-Domain, Personality Inventory Measuring the Lower-Level Facets of Several Five-Factor Models", In I. Mervielde, I. Deary, F. De Fruyt, and F. Ostendorf (Eds.) Personality Psychology in Europe, Vol. 7. (pp. 7-28). Tilburg, The Netherlands: Tilburg University Press.
18
Goldberg, L., Johnson, J., Eber, H., Hogan, R., Ashton, M., Cloninger, C., and Gough, H. (2006) "The International Personality Item Pool and the Future of Public-Domain Personality Measures", Journal of Research in Personality, 40, 84-96. Hunt. S., and Vitell, S. (1986) "A General Theory of Marketing Ethics", Journal of Macromarketing, Vol. 6, No. 1, 5-16 International Personality Item Pool Contributors (2006) Individual Differences Scales, Administration Instructions Retrieved September, 2004 from http://ipip.ori.org/ipip/ Kaplan, R. M., and Saccuzzo, D. P. (2001) Psychological Testing: Principles, Applications and Issues (5th ed.), Belmont, California: Wadsworth/Thomson Learning. Kline, P. (1986) Handbook of Test Construction: Introduction to Psychometric Design. London, Methuen Landry, J.T. (2005) Moral Intelligence: Enhancing Business Performance and Leadership Success-Book Review.Harvard Business Review. Boston: May 2005. Vol. 83, Iss. 5; p. 30 Lennick, D., and Kiel, F., (2005) Moral Intelligence: Enhancing Business Performance and Leadership Success. Wharton School Publishing Lucas, G. M., and Friedrich, J. (2005) "Individual Differences in Workplace Deviance and Integrity as Predictors of Academic Dishonesty", Ethics and Behavior, 15 (1). Lunney, G. H. (1970) "Using Analysis of Variance with a Dichotomous Dependent Variable: An Empirical Study", Journal of Educational Measurement, 7 (4). Martin, D.E., Rao, A., Sloan, L. R. (In Press) Plagiarism, Integrity, and Workplace Deviance: A Criterion Study Ethics and Behavior
19
Miner, J. and Capps, M. (1996) How Honesty Testing Works, Westport, CT, Quorum Books. Ones, D. S., and Viswesvaran, C. (2001) "Integrity Tests and other Criterion Focused Occupational Personality Scales (COPS) used in Personnel Selection" , International Journal of Selection and Assessment, 9 (1/2). Robin, D. A. and R. E. Reidenback. (1988). "Social Responsibility Ethics and Marketing Strategy", Journal of Marketing. 51 (Jan): 44-58 Sackett, P. R., Burris, L. R., and Callahan, C. (1989) "Integrity Testing for Personal Selection: An Update", Personnel Psychology, 42 (3). Wanek, J. E. (1999) "Integrity and Honesty Testing: What do we know? How do we use it?", International Journal of Selection and Assessment, 7 (4).
20
Appendix A: Instruments
The 40 item Moral Competency Inventory is displayed in Table 4 (Page 14): Rotated Component Matrix for Moral Competency Inventory All instruments from the International Personality Item Pool (IPIP, 2004) were given the following administration instructions (IPIP, 2004):
On the following pages, there are phrases describing people's behaviors. Please use the rating scale below to describe how accurately each statement describes you. Describe yourself as you generally are now, not as you wish to be in the future. Describe yourself as you honestly see yourself, in relation to other people you know of the same sex as you are, and roughly your same age. So that you can describe yourself in an honest manner, your responses will be kept in absolute confidence. Please read each statement carefully, and then fill in the bubble that corresponds to the number on the scale.
Scale items are either positively keyed (+) or negatively keyed (-).
Response Options
1: Very Inaccurate 2: Moderately Inaccurate 3: Neither Inaccurate nor Accurate 4: Moderately Accurate 5: Very Accurate
21
Integrity/Honesty/Authenticity (9 items; Alpha = .72) 1. Am trusted to keep secrets. (+) 2. Keep my promises. (+) 3. Believe that honesty is the basis for trust. (+) 4. Can be trusted to keep my promises. (+) 5. Am true to my own values. (+) 6. Lie to get myself out of trouble. (-) 7. Am hard to understand. (-) 8. Feel like an imposter. (-) 9. Like to exaggerate my troubles. (-) Impression Management (20 items; Alpha = .82) 1. Would never take things that aren't mine. (+) 2. Would never cheat on my taxes. (+) 3. Believe there is never an excuse for lying. (+) 4. Always admit it when I make a mistake. (+) 5. Rarely talk about sex. (+) 6. Return extra change when a cashier makes a mistake. (+) 7. Try to follow the rules. (+) 8. Easily resist temptations. (+) 9. Tell the truth. (+) 10. Rarely overindulge. (+) 11. Have sometimes had to tell a lie. (-) 12. Use swear words. (-)
22
13. Use flattery to get ahead. (-) 14. Am not always what I appear to be. (-) 15. Break rules. (-) 16. Cheat to get ahead. (-) 17. Don't always practice what I preach. (-) 18. Misuse power. (-) 19. Get back at others. (-) 20. Am likely to show off if I get the chance. (-) Responsibility (10 items: Alpha =.66) 1. Return extra change when a cashier makes a mistake. (+) 2. Try to forgive and forget. (+) 3. Like to be of service to others. (+) 4. Act according to my conscience. (+) 5. Anticipate the needs of others. (+) 6. Take others' interests into account. (+) 7. Am polite to strangers. (+) 8. Am able to cooperate with others. (+) 9. Appreciate people who wait on me. (+) 10. Try not to think about the needy. (-)
23
Tables Table 1: Descriptive Statistics for Participants Frequency Valid Percent Gender 118 69.0 31.0 Female Male 53 Educational Standing Undergraduate 153 89.5 10.5 Graduate 18 Ethnicity 1.9 3 African 4.5 African 7 49.7 American 77 Asian 21.9 Caucasian 34 .6 Middle 1 15.5 Eastern 24 5.8 Latino Other 9 Age 4.1 7 57.9 20.5 18-20 99 9.4 2.3 3.5 21-25 26-30 35 1.8 .6 31-35 16 46 36-40 31 41-45 46-50 50+
24
Table 2: Descriptive Statistics and Cronbach's Alpha for Moral Competency Inventory Cronbach's Alpha Mean SD
Acting consistently with principles, values and, beliefs .84 16.45 2.73 Telling the truth 16.72 .72 2.33 Standing up for what is right 14.65 .69 2.68 Keeping promises 16.56 .65 2.36 Taking responsibility for personal choices .73 16.36 2.36 Admitting mistakes and failures .75 16.33 2.53 Embracing responsibility for others .74 14.95 2.91 Actively caring about others .81 15.74 3.03 Ability to let go of one's own mistakes .66 15.07 2.57 Letting go of others mistakes .74 14.83 2.88 Table 3: Total Variance Explained for Moral Competency Inventory factor analysis Factor New Name Consistent social conviction Active care of others Taking ethical action Obligation to the truth Self forgivingness Owning ones mistakes Accepting own mistakes Accepting others mistakes Having others trust Initial Eigenvalues % of Total Variance 15.368 38.421 2.336 5.840 1.789 4.472 1.678 4.194 1.523 3.808 1.278 3.194 1.112 2.780 1.027 2.568 Cumulative % 38.421 44.261 48.732 52.927 56.734 59.928 62.708 65.276
1 2 3 4 5 6 7 8
25
Table 4: Rotated Component Matrix for Moral Competency Inventory (
Factor Item My co-workers would say that my behavior is very consistent with my beliefs and values. My friends would say that my behavior is very consistent with my beliefs and values. I can clearly state the principles, values, and beliefs that guide my actions. My co-workers would say that I am the kind of person who stands up for my convictions My co-workers would say that I take ownership of my decisions My co-workers think of me as an honest person. I spend a significant amount of my time providing resources and removing obstacles for my co-workers Because I care about my co-workers, I actively support their efforts to accomplish important personal goals. I discuss my mistakes with co-workers to encourage tolerance for risk. I pay attention to the development needs of my co-workers. My first response when I meet new people is to be genuinely interested in them. My leadership approach is to lead by serving others. I will generally confront someone if I see them doing something that isn't right. If I knew my company was engaging in unethical or illegal behavior, I would report it, even if it could have an adverse effect on my career. When a situation may prevent me from keeping a promise, I consult with those involved to renegotiate the agreement. When things go wrong, I do not blame others or circumstances. If I believe that my boss is doing something that isn't right, I will challenge him or her. When I agree to do something, I always follow through. I tell the truth unless there is an overriding moral reason to withhold it. When faced with an important decision, I consciously assess whether the decision I wish to make is aligned with my most deeply held principles, values, and beliefs. Even when I have made a serious mistake in my life, I am able to forgive myself and move ahead. I resist the urge to dwell on my mistakes When I forgive someone, I find that it benefits me as much as it does them. I am able to "forgive and forget," even when someone has made a serious mistake. I own up to my own mistakes and failures. When I make a decision that turns out to be a mistake, I admit it. My colleagues would say that I go out of my way to help them. I am willing to accept the consequences of my mistakes. I am able to deliver negative feedback in a respectful way When I make a mistake, I take responsibility for correcting the situation. I use my mistakes as an opportunity to improve my performance. My co-workers would say that I have a realistic attitude about my mistakes and failures. I accept that other people will make mistakes My co-workers would say that I am a compassionate person. Even when people make mistakes, I continue to trust them. My friends know they can depend on me to be truthful to them.When someone asks me to keep a confidence, I do so. I truly care about the people I work with as people - not just as the "human capital" needed to produce results. My friends and co-workers know they can depend on me to keep my word. I appreciate the positive aspects of my past mistakes, realizing that they were valuable lessons on my way to success. 1 .634 .610 .574 .465 .422 .375 .801 .672 .666 .619 .557 .409 .348 .685 .457 .412 .352 .756. 452 .424 .695 .587 .576 .487 -.901 -.567 -.481 -.434 -.427 -.370 -.624 -.504 -.410.393 -.337 -.316 -.398 -.341 2 3 4 5 6 7 8
Extraction Method: Principal Axis Factoring, Rotation Method: Oblimin with Kaiser Normalization. Rotation converged in 22 iterations.
26
Table 5: Bivariate and Partial (Impression Management) Correlations between Moral Competency Index and Integrity Measures
Integrity Integrity Acting Consistently and Values Telling Truth Standing for Right Keeping Promises Taking Responsibility Admitting Mistakes .435(**) .000 .307(**) .008 .352(**) .002 .345(**) .003 .287(*) .013 .327(**) .004 74 .262(*) .024 .193 .099 .245(*) .036 .253(*) .030 .390(**) .001 .493 .000 .384 .001 .381 .001 .418 .000 .331 .006 .321 .008 66 .248 .041 .175 .154 .275 .023 .244 .045 .418 .000 Responsibility (Partial r) .356(**) .002 .438(**) .000 .322(**) .005 .380(**) .001 .287(*) .013 .397(**) .000 74 .462(**) .000 .468(**) .000 .300(**) .009 .441(**) .000 .510(**) .000 Responsibility Self Esteem (Partial r) .362 .002 .473 .000 .294 .015 .441 .000 .301 .013 .361 .002 66 .417 .000 .432 .000 .340 .005 .403 .001 .500 .000 .058 .613 -.083 .469 .005 .965 .043 .712 -.011 .924 -.030 .797 78 .030 .791 .038 .743 .025 .825 -.096 .404 -.001 .993
Embracing Responsibility for Others Actively Caring Owning Own Mistakes Letting Go Others Mistakes MCI Total
Table 6: Univariate ANOVA: Gender and Significantly Different MCI Scales Dependent Variable MCI Keeping Promises MCI Actively Caring MCI Total Type III Sum of df Squares 30.142 34.417 285.239 1 1 1 F p. Partial ?2 Male Mean 15.80 14.52 74.94 Female Mean 17.13 15.94 79.04
7.952 .006 .095 4.764 .032 .059 3.535 .064 .044
27
doc_709603075.docx
In algebra, the discriminant[1] of a polynomial is a function of its coefficients which gives information about the nature of its roots. For example, the discriminant of the quadratic polynomial
Management Research on Moral Competency Inventory Validation: Content, Construct, Convergent and Discriminate Approaches
Purpose: Our research fulfills three purposes: 1) introduce practitioners to the appropriate use of measures of unethical behavior; 2) evaluate the use of integrity related assessments for use in personnel selection; and 3) determine the validity of the MCI instrument using standard validation procedures. Methodology: Content, construct, convergent and discriminant approaches are applied to establish the relative validity of the assessment tool. Findings: The results of the MCI purport to align with one's moral values and behaviors. We establish face validity of the MCI measure, but fail to establish: 1) an appropriate simple factor structure, 2) convergent validity, 3) discriminant validity, and 4) support for the lack of impact of demographic factors on the purported measure of Moral intelligence. Research limitations: An acceptable but constrained (working students) sample was used in the validation. Practical implications: Researchers and practitioners should be familiar with psychometric principles to ensure the use of valid tools in a predictive and defensible manner. New measures can be developed, but should be validated before used for developmental or personnel decision making purposes. Originality/value: 1) The lack of validity associated with the MCI instrument is established; 2) Researchers and practitioners are exposed to considerations in the appropriate use of measures of unethical behavior, and 3) exposed to several previously validated integrity related assessments for use in personnel decision making.
2
Introduction With the recent corporate scandals involving Tyco, Enron, and Worldcom, the moral standards of leaders have come under heavy scrutiny. Managers, human resources practitioners, and individuals involved in personnel selection decisions all have a stake in preventing their organization from becoming victims of unethical behavior, be it white-collar or other. Given these concerns, measurement of candidate integrity and responsibility as individual differences variables are well represented in the industrial/organizational psychological literature (Wanek, 1999). These measures have been used to predict a wide range of ethically related employee behavior such as honesty, violent tendencies, turnover, theft likelihood, white-collar crime and other counterproductive employment behaviors (Miner and Capps, 1996, Wanek, 1999). Understanding the moral and ethical principles underlying the decisions of successful leaders has captured the interest of business people for sometime (Bass and Stodgill, 1990). However, agreeing on, and measuring purported moral principles can be fraught with difficulty. First, what are moral principles? Lennick and Kiel (2005) define moral principles as values that cut across all cultural boundaries. Lennick and Kiel argue that people overwhelmingly prefer to follow leaders who exhibit integrity, integrity being a universal human (extant across time and cultures) moral principle. Other moral principles they claim and attempt to measure in their Moral Competency Index (MCI) are responsibility, compassion, and forgiveness. The results of the MCI purport to align with one's moral values and behaviors. While the extant industrial/organizational psychology literature supports the naming conventions associated with the above constructs, no evidence of measurement validity for the MCI is presented in their
3
book, Moral Intelligence (2005). Our research aims to fulfill three purposes: 1) to introduce practitioners to the appropriate use of measures of unethical behavior (integrity related assessments); 2) to evaluate the use of integrity related assessments for use in personnel selection; and 3) to determine the validity of the MCI instrument using standard validation procedures. Moral Intelligence Moral intelligence is presented as the ability to methodically apply universal moral principles to one's ethics, objectives, and dealings (Lennick and Kiel, 2005). One premise to such a definition is that morality is a priori, meaning individuals have an innate tendency to act with integrity, responsibility, compassion, and forgiveness. The MCI attempts to measure the dynamic nature (it aims to measure if actions and beliefs are aligned) of one's moral intelligence. Lennick and Kiel use the metaphor of a moral compass to measure alignment. The moral compass is one's innermost beliefs and values that guide all thoughts and actions, but no objective measure of the moral compass is provided. Furthermore, according to Lennick and Kiel, a moral decision process is not contingent on demographic factors such as gender, race, nationality, or religious practice, which runs against assumptions in the current literature (Martin, Rao and Sloan, 2008; Ford and Richardson, 1994). Rather, from the compass perspective, judgment with respect to good deeds and bad is rooted in universal principles and cross all cultural barriers. It is asserted that everyone supports values from all of the philosophies mentioned above but weights vary between individuals and weights change as one progresses in age (Lennick and Kiel, 2005). Other similar research that has sought to understand and identify moral standards by conducting in-depth interviews with managers (Lennick and Kiel, 2005; Bird and Waters, 1987).
4
The Bird and Waters study identified seven moral standards as opposed to the four measures and ten attributes of the Lennick and Kiel instrument. Many models have been developed to understand the precursors to ethical decision making, with personality traits being seen as the mitigating factors in ethical decision-making, in contrast to the assertions underpinning the MCI. (Bartels, 1967; Hunt and Vitell, 1986). Still other research looked to develop frameworks on explaining the process for ethical decision-making (Ferrell and Gresham, 1985; Robin and Reidenback, 1988). Lennick and Kiel present their approach as being more interested in helping people understand how well they are aligned with their respective moral compass at a given moment in time. Accordingly, the purpose of the MCI is to help people identify attributes within a moral framework that can be further developed. Thus, the research conducted by Lennick and Kiel purports to offer a tool that an individual can use to measure their current moral alignment and determine what areas are in need of improvement such that one is better aligned to their own moral compass, whereas previous research attempted to better understand the precedent factors to ethical decision-making or to model decision-making processes within an ethical context. While the MCI does not aim to provide people with personnel selection or employee development tools, it is presented in the format of a leadership development opportunity, one that might be misconstrued as offering organizations a facile tool for identifying potential leaders. Given the .41 correlation between supervisory ratings of overall job performance and personality based measures of integrity in the Ones and Viswesvaran (2001) metaanalysis, there may be grounds for an empirical test of Lennick and Kiel's assertion. Any tool that purports to measure psychological constructs of integrity and responsibility is should be subject to an empirical validation. Lennick and Kiel do not provide an empirical validation of the MCI.
5
Integrity Integrity tests grew out of the lack of validity in polygraph tests in the measurement of honesty (Miner and Capps, 1996) as a predictor of workplace behavior. They have been used for pre-employment selection tests and are associated with workplace behaviors such as: honesty, violent tendencies, turnover, theft likelihood, white-collar crime and other counterproductive employment behaviors (Wanek, 1999). There are two types of integrity measures: overt and personality-based (Sackett, Burris, and Callahan, 1989). Overt measures indicate the specific type of unethical behavior in which the test taker may have participated in. In this sense they exhibit high face validity and are called "clear-purpose tests". Personality-based measures do not specify the unethical behavior, but assess personality traits that may be associated with the likelihood of participation in unethical conduct. The MCI seems to exhibit the qualities of a personality-based measure of integrity. Wanek (1999) reports the professional consensus on integrity measures in the statement: "To summarize what we know, evidence supports the conclusions that professionally developed, commercially available integrity tests are reliable, valid predictors of productive and counterproductive measures of job performance" (P.190). Integrity and Personnel Selection The practical use of integrity measures are clear in human resources selection practices, such as in selecting salespeople or accountants. In a metaanalysis, Ones and Viswesvaran (2001) explored the impact of criterion focused occupational personality scales (COPS), with an emphasis on construct and incremental validity in integrity measures, and measures of counterproductive behavior. They focused on these measures related to personnel selection as many American businesses use integrity measures as part of their hiring process. Based on 665
6
validity coefficients, they investigated integrity test validities across three different categories: job performance, general counterproductive work behavior, and theft. They found that after correcting for range restriction and unreliability in the criterion, validity coefficients of .29 were found for broad counterproductive behaviors, and .41 between supervisory ratings of overall job performance and personality based measures of integrity. In sum, the metaanalysis justifies the use of integrity measures in personnel selection. Hiring companies reap the benefit of selecting employees that are less prone to participate in counterproductive behavior. Integrity Related Measures used to Predict White-Collar Crime Given the MCI integrity and responsibility measurement claim, we reviewed Collins and Schmidt's (1993) comparison of integrity-related differences between federal prison inmates convicted for white-collar crimes to white-collar managers. In their study they used subscales of the California Personality Inventory (specifically the responsibility, stability, and self-control scales), a biodata scale, and a PDI Employment Inventory (a personality measure predictive of productive or counterproductive employee behavior). They found strong to moderate correlations between the 365 criminals and 344 managers on measures of responsibility (.40). It is difficult to assess how prevalent white-collar crime is in business today due to a lack of effective governance and compliance agencies. There also may be key psychological differences between those who participate in white-collar crimes and those caught participating in white-collar crimes, thus the Collins and Schmidt study may be dealing with a highly attenuated sample). Drawing from this research, the current study examines the relationship between the MCI and measures of integrity, and responsibility in an independent sample of students.
7
Moral Intelligence and Integrity Given the above descriptions of moral intelligence and its emphasis on interpersonal integrity, it became clear that there may be some commonalities between the MCI and integrity related measures. We have discussed the utility of integrity measures above, and note that prior researchers concluded that situational variables are superior to individual variables in valid measures of integrity (Lucas & Friedrich, 2005). Much like the research mentioned, the constructs of the MCI are all situational variables, and may be less susceptible to social desirability bias as overt measures. The first four of the competencies in the MCI (Acting consistently with principles, values and, beliefs, telling the truth, standing up for what is right, keeping promises) claim to measure integrity. Though Lennick and Kiel purport not to use personality traits to measure moral intelligence or alignment, the commonality between the items and constructs bears examination. Moral Intelligence and Responsibility According to Lennick and Kiel, responsibility is measured in competencies five through seven (Taking responsibility for personal choices, Admitting mistakes and failures, Embracing responsibility for serving others). As with integrity, Lennick and Kiel appear to attempt to measure constructs that are valid across time and culture. Other research has supported responsibility as paramount in importance by quantifying financial results between irresponsible corporations, and those who take responsibility for the communities in which they operate (Cava and Mayer, 2007). Bird and Waters (1987) found that where managers discussed moral issues, seven distinct moral standards were found, two of which included 'responsibility' in context. Thus, the MCI seeks to assess an individual's responsibility as it applies in a leadership context and corporate environment.
8
Current Study The current study evaluates the validity of the MCI from multiple measurement perspectives. Specifically, we first addressed the content validity of the MCI. Content validity refers to the operationalization of the MCI against the integrity and responsibility content domain. Given previous validations of the International Personality Item Pool (IPIP) domain measures, we used them as detailed descriptions of the content domain. Second, we examined the construct validity of the MCI by establishing its simple factor structure. We specifically were looking for concurrence between the theoretical domains postulated by Lennick and Kiel and actual construct loadings through principal axis factoring. Principal axis factoring seeks the least number of factors (constructs) which can account for the common variance (correlation) in a set of items. Third, we aimed to establish both the convergent and discriminant validity of the MCI by using previously measures of theoretically related measures, specifically measures of integrity and responsibility. In convergent validity, we examine the degree to which the operationalization is similar to (converges on/correlates with) other operationalizations that it theoretically should be similar to. Discriminant validity provides information regarding the lack of correlation between theoretically different constructs. Finally, we explored the claim that the constructs the MCI purports to measure are independent from demographic factors such as gender, race, nationality, or religious practice, by looking for significant differences in participant demographics in scale responses. Methodology Participants Participants were students from a medium-sized western United States university. The
9
sample consisted of 171 participants from both graduate and undergraduate classes. Data collection was part of an opportunity for extra credit in all of the classes during the Winter and Spring quarters of 2006/2007. Participation was voluntary and the instruments were administered via an online survey package at three different points during the quarter to ensure students did not suffer fatigue. The participants were from Business Administration disciplines. The age ranged from 21 to 50, the majority being between 21 and 25 (58%). Descriptive statistics for participants are presented in Table 1 (see below). Table 1 about here
Instruments Moral Competency Inventory Measurement Instrument Lennick's and Kiel's contention is that leadership potential is highly correlated with moral intelligence. The MCI sets out to measure ten competencies within a moral framework. The competencies are: 1) Acting consistently with principles, values and, beliefs 2) Telling the truth 3) Standing up for what is right 4) Keeping promises 5) Taking responsibility for personal choices 6) Admitting mistakes and failures 7) Embracing responsibility for serving others 8) Actively caring about others 9) Ability to let go of one's own mistakes
10
10) Ability to let go of others' mistakes. A five point Likert-like scale (1=Never, 2=Infrequently, 3=Sometimes, 4=In most situations, and 5=In all situations) remains constant through the forty question instrument. Additive scores of 90 to 100 are considered very high, 80 to 89 high, 60 to 79 moderate, 40 to 59 low, 20 to 39 very low. The score is indicative of one's actions with respect to one's beliefs and is labeled as an "alignment score" (Lennick and Kiel, 2005). A lower score in a specific competency is indicative of area in need of improvement. The alignment takes into account three factors: moral intelligence, moral competence, and emotional competence (Lennick and Kiel, 2005). Moral competence complements moral intelligence in that it is the social skill set needed (read "courage") that allows one to act on principles. Underlying moral competence is emotional competence, which provides one with the steadfastness needed to handle morally difficult situations and emotions within ourselves, and with others (Lennick and Kiel, 2005). As of our writing there were no measures associated with the moral or emotional competence constructs, nor psychometrics (validity, reliability) associated with the MCI. As noted by Lennick and Kiel, they conducted research through case study, and by interviewing CEO's and other senior managers. Currently, they are in the process of quantitative data collection and analysis in efforts to validate the MCI. The majority of research regarding integrity measures has been conducted by test publishers (Camara and Schneider, 1994) for the purpose of validating and publishing commercial tests. Accordingly, academic research can be prohibitively expensive. As such, we used public domain individual differences instruments available from the International Personality Item Pool (IPIP, 2006) for this study. The psychometric properties of the IPIP scales
11
have been assessed and validated by Goldberg and colleagues (Goldberg. L, 1999, Goldberg, Johnson, Eber, Hogan, Ashton, Cloninger, and Gough, 2006). In our study the IPIP proxy scales for the Personality Attributes Scale (PAS), the California Psychological Inventory (CPI), and the Values in Action (VIA) scale. All IPIP scales are five point Likert scales, with one signifying "very inaccurate", and five signifying "very accurate". Integrity Related Individual Difference Measures To assess integrity we used the following scales from the IPIP CPI proxy; Responsibility (Cronbach's = 0.66) which evaluates an individual's level of "conscientiousness, honesty, dependability, and practicality" (Kaplan and Saccuzzo, 2001), and the Values in Action Integrity/Honesty/Authenticity (Cronbach's = .72) IPIP proxy. Social Desirability To control for social desirability (a person's idealized projection of themselves: Derlega, Winstead, and Jones, 2005), we assessed impression management of our respondents by administering the IPIP PAS proxy (Cronbach's alpha, 0.82). Results Basic descriptive statistics and reliability estimates (Cronbach's alphas) for all of the MCI scales were established for the sample and are reported in Table 2. Most yielded acceptable alpha levels (see table 2). Table 2 about here
To extract factors and better understand the structure of the MCI, we used Principle Axis Factoring procedure for extraction with Direct Oblimin rotation. The first factor accounted for
12
38% of variance (Table 3) The factor analysis established eight viable factors, not the ten posited (please see Table 4). . Table 3 about here
Table 4 about here
To establish the relationship between the MCI and related integrity measures, bivariate correlations were run. These were followed by partial correlations with impression management partialed out to control for socially desirable response bias. Interestingly, predominately larger significant correlations were found between the integrity, responsibility, and MCI scales in the partialed correlations. This finding suggests that, while our instructions asked participants to use candor in responding to the questionnaire, impression management had an impact on participants' responses. We also included a measure of self esteem to establish the discriminant validity of the measures (Table 5). Table 5 about here
Finally to investigate the claim that moral intelligence is not impacted by demographic variables, we performed a univariate analysis of variance (Lunney, 1970) on the scales with gender as are fixed factor. While the sample size in this study was relatively small , gender did significantly impact scale responses for the "Keeping Promises" scale, as well as the "Actively Caring" scale. Interestingly the univariate analysis of variance for the total MCI score approached significance (p. = .06), suggesting potential problems with the assertion that other demographic characteristics do not impact the construct of moral intelligence (see table 6). Table 6 about here
13
Discussion Clearly understanding concepts of validity and reliability in the use of any assessment tool irrespective of its intended end-use (for personnel selection or development) remains critical for practitioners. In the case of the moral competency index, it does not seem too difficult to conceive of circumstances where a manager or human resources practitioner would attempt to do the organization a favor by utilizing such a tool for selection purposes. It is important to recognize established psychological, educational and business testing conventions, as well as federal guidelines in the use of tests for selection purposes. If invalid measures such as the MCI are used for leadership potential, scores may be used as potential evidence, leading to promotions, training, or other benefits that may lead to personnel decision making. This creates invalid and potentially discriminatory selection practices, leading to flawed selections, or discrimination law suits. Accordingly we present a brief summary of our findings and the concepts and minimum requirements for using assessments in development identification. Content Validity Given the similarity of materials presented in the items of the MCI and other valid measures of integrity and responsibility, the content seems to be appropriate given the domains the MCI attempts to measure. Several of the questions in the MCI suffer from item writing problems (for example: double-barreled questions. "I can clearly state the principles, values, and beliefs that guide my actions", Lennick and Kiel, 2003) but overall the content seems to be adequately measured from a content validity perspective. Construct Validity We examined the construct validity of the MCI by establishing its simple factor structure. Based on the results of our Principal Axis Factor analysis, the constructs the MCI attempts to
14
measure do not seem to exist outside of the philosophical domains the authors cite. Of course, given the types of items they use, there are some similarities, but the factor loadings suggest that if the authors aim to measure their described constructs for multiple purposes, the scales will have to be refined. Convergent/Discriminant Validity Finally, we aimed to establish both the convergent and discriminant validity of the MCI using previous measures of theoretically related measures, specifically measures of integrity and responsibility. In convergent validity, we examine the degree to which the operationalization is similar to (converges on) other operationalizations that it theoretically should be similar to. Using the commonly held r = .85 cutoff for an assessment of conceptual agreement, we do not find strong evidence to support the use of the MCI as a measure of either integrity and responsibility. Given the lack of significant correlations between self-esteem and the MCI scales we have some evidence for discriminant validity. However given the preponderance of evidence from the above analyses, this does not offer sufficient support to utilize the tool in any developmental or selection circumstance. Accordingly, the measure should advise individuals who might use it that it exists solely for philosophical/entertainment purposes as there does not seem to be any sufficient relationship between it and any other established measure that attempts to measure what it purports to. Conclusion In summary, some do not feel that Lennick and Kiel have captured anything of significance with their book (Landry, 2005). We had the opportunity to establish the psychometric validity of the MCI in a diverse and moderately sized sample. A larger, more
15
diverse and balanced sample would have been desirable. At the same time, the theory argues that demographic factors do not impact ones moral competence, and the diversity of our sample offered an empirical test of the claim. If Lennick and Kiel choose to revise the MCI in light of this or other research (according to the website - http://www.moralcompass.com/ongoingresearch.php - ongoing research is being conducted) they should either revise the theory of moral intelligence to reflect the discrepancies established in our research, or consider some of the aforementioned work in the field of psychology (see literature review above). While the Landry review looks at the overall content of the text, a more stringent review of the psychometrics associated with the MCI measure is presented here. The question as to what the MCI aims to measure becomes paramount in the analyses of the data. The field of industrial organizational psychology has examined the relationship between individual differences, leadership potential, and honesty, integrity and responsibility for close to 60 years, but it seems that this material was not present in the development of the concepts purportedly measured by the MCI. From a strictly validity-based perspective, Landry's perspective is quantified here. Practical applications in the fields of leadership, management, organizational behavior, and human resources are plentiful. As such practitioners should become familiar with preestablished valid tools that measure psychological constructs in a predictive and defensible manner. This is not to suggest that new measures cannot be developed that can add a great deal of benefit to any field in which leadership potential or measures of moral, ethical, honest and integrity based differences will be related to performance. However, we suggest that practitioners maintain their awareness of psychometric principles and a basic understanding of validity at a minimum before they use any measure for developmental or personnel selection purposes. With the use of psychometrically valid tools, the scientist is better able to model
16
behavior and the practitioner is better able to meet the challenges of their work. We encourage future members of both parties to closely examine popular assessment tools to ensure they offer the utility they purport to.
17
References Bartels, R. (1967) "A Model for Ethics in Marketing", Journal of Marketing, Vol. 31, No. 1 Bass, B., and Stogdill, R. (1990) Bass & Stogdill's Handbook of Leadership. 3rd edition, Free Press Bird, F., and Waters, J. (2003) "The nature of managerial moral standards", Journal of Business Camara WJ, Schneider DL. (1994). Integrity tests: Facts and unresolved issues. American Psychologist, 49,112-119. Cava, A. and Mayer, D. (2007)" Integrative Social Contract Theory and Urban Prosperity Initiatives", Journal of Business Ethics, Volume 72, Number 3 Collins, J. M., and Schmidt, F. L. (1993) "Personality, Integrity, and White-Collar Crime: a Construct", Personnel Psychology, 46 (2). Derlega, V. J., Winstead, B. A., and Jones, W. H. (2005) Personality: Contemporary Theory and Research (3rd ed.), Thomson. Ford, R.C. and Richardson, W.D. (1994) "Ethical Decision Making: A Review of the Empirical Literature", Journal of Business Ethics, 13 (3). Ferrell, O., and Gresham, L. (1985) "A Contingency Framework for Understanding Ethical Decision Making in Marketing", Journal of Marketing, Vol. 49, No. 3 Goldberg, L. (1999) "A Broad-Bandwidth, Public-Domain, Personality Inventory Measuring the Lower-Level Facets of Several Five-Factor Models", In I. Mervielde, I. Deary, F. De Fruyt, and F. Ostendorf (Eds.) Personality Psychology in Europe, Vol. 7. (pp. 7-28). Tilburg, The Netherlands: Tilburg University Press.
18
Goldberg, L., Johnson, J., Eber, H., Hogan, R., Ashton, M., Cloninger, C., and Gough, H. (2006) "The International Personality Item Pool and the Future of Public-Domain Personality Measures", Journal of Research in Personality, 40, 84-96. Hunt. S., and Vitell, S. (1986) "A General Theory of Marketing Ethics", Journal of Macromarketing, Vol. 6, No. 1, 5-16 International Personality Item Pool Contributors (2006) Individual Differences Scales, Administration Instructions Retrieved September, 2004 from http://ipip.ori.org/ipip/ Kaplan, R. M., and Saccuzzo, D. P. (2001) Psychological Testing: Principles, Applications and Issues (5th ed.), Belmont, California: Wadsworth/Thomson Learning. Kline, P. (1986) Handbook of Test Construction: Introduction to Psychometric Design. London, Methuen Landry, J.T. (2005) Moral Intelligence: Enhancing Business Performance and Leadership Success-Book Review.Harvard Business Review. Boston: May 2005. Vol. 83, Iss. 5; p. 30 Lennick, D., and Kiel, F., (2005) Moral Intelligence: Enhancing Business Performance and Leadership Success. Wharton School Publishing Lucas, G. M., and Friedrich, J. (2005) "Individual Differences in Workplace Deviance and Integrity as Predictors of Academic Dishonesty", Ethics and Behavior, 15 (1). Lunney, G. H. (1970) "Using Analysis of Variance with a Dichotomous Dependent Variable: An Empirical Study", Journal of Educational Measurement, 7 (4). Martin, D.E., Rao, A., Sloan, L. R. (In Press) Plagiarism, Integrity, and Workplace Deviance: A Criterion Study Ethics and Behavior
19
Miner, J. and Capps, M. (1996) How Honesty Testing Works, Westport, CT, Quorum Books. Ones, D. S., and Viswesvaran, C. (2001) "Integrity Tests and other Criterion Focused Occupational Personality Scales (COPS) used in Personnel Selection" , International Journal of Selection and Assessment, 9 (1/2). Robin, D. A. and R. E. Reidenback. (1988). "Social Responsibility Ethics and Marketing Strategy", Journal of Marketing. 51 (Jan): 44-58 Sackett, P. R., Burris, L. R., and Callahan, C. (1989) "Integrity Testing for Personal Selection: An Update", Personnel Psychology, 42 (3). Wanek, J. E. (1999) "Integrity and Honesty Testing: What do we know? How do we use it?", International Journal of Selection and Assessment, 7 (4).
20
Appendix A: Instruments
The 40 item Moral Competency Inventory is displayed in Table 4 (Page 14): Rotated Component Matrix for Moral Competency Inventory All instruments from the International Personality Item Pool (IPIP, 2004) were given the following administration instructions (IPIP, 2004):
On the following pages, there are phrases describing people's behaviors. Please use the rating scale below to describe how accurately each statement describes you. Describe yourself as you generally are now, not as you wish to be in the future. Describe yourself as you honestly see yourself, in relation to other people you know of the same sex as you are, and roughly your same age. So that you can describe yourself in an honest manner, your responses will be kept in absolute confidence. Please read each statement carefully, and then fill in the bubble that corresponds to the number on the scale.
Scale items are either positively keyed (+) or negatively keyed (-).
Response Options
1: Very Inaccurate 2: Moderately Inaccurate 3: Neither Inaccurate nor Accurate 4: Moderately Accurate 5: Very Accurate
21
Integrity/Honesty/Authenticity (9 items; Alpha = .72) 1. Am trusted to keep secrets. (+) 2. Keep my promises. (+) 3. Believe that honesty is the basis for trust. (+) 4. Can be trusted to keep my promises. (+) 5. Am true to my own values. (+) 6. Lie to get myself out of trouble. (-) 7. Am hard to understand. (-) 8. Feel like an imposter. (-) 9. Like to exaggerate my troubles. (-) Impression Management (20 items; Alpha = .82) 1. Would never take things that aren't mine. (+) 2. Would never cheat on my taxes. (+) 3. Believe there is never an excuse for lying. (+) 4. Always admit it when I make a mistake. (+) 5. Rarely talk about sex. (+) 6. Return extra change when a cashier makes a mistake. (+) 7. Try to follow the rules. (+) 8. Easily resist temptations. (+) 9. Tell the truth. (+) 10. Rarely overindulge. (+) 11. Have sometimes had to tell a lie. (-) 12. Use swear words. (-)
22
13. Use flattery to get ahead. (-) 14. Am not always what I appear to be. (-) 15. Break rules. (-) 16. Cheat to get ahead. (-) 17. Don't always practice what I preach. (-) 18. Misuse power. (-) 19. Get back at others. (-) 20. Am likely to show off if I get the chance. (-) Responsibility (10 items: Alpha =.66) 1. Return extra change when a cashier makes a mistake. (+) 2. Try to forgive and forget. (+) 3. Like to be of service to others. (+) 4. Act according to my conscience. (+) 5. Anticipate the needs of others. (+) 6. Take others' interests into account. (+) 7. Am polite to strangers. (+) 8. Am able to cooperate with others. (+) 9. Appreciate people who wait on me. (+) 10. Try not to think about the needy. (-)
23
Tables Table 1: Descriptive Statistics for Participants Frequency Valid Percent Gender 118 69.0 31.0 Female Male 53 Educational Standing Undergraduate 153 89.5 10.5 Graduate 18 Ethnicity 1.9 3 African 4.5 African 7 49.7 American 77 Asian 21.9 Caucasian 34 .6 Middle 1 15.5 Eastern 24 5.8 Latino Other 9 Age 4.1 7 57.9 20.5 18-20 99 9.4 2.3 3.5 21-25 26-30 35 1.8 .6 31-35 16 46 36-40 31 41-45 46-50 50+
24
Table 2: Descriptive Statistics and Cronbach's Alpha for Moral Competency Inventory Cronbach's Alpha Mean SD
Acting consistently with principles, values and, beliefs .84 16.45 2.73 Telling the truth 16.72 .72 2.33 Standing up for what is right 14.65 .69 2.68 Keeping promises 16.56 .65 2.36 Taking responsibility for personal choices .73 16.36 2.36 Admitting mistakes and failures .75 16.33 2.53 Embracing responsibility for others .74 14.95 2.91 Actively caring about others .81 15.74 3.03 Ability to let go of one's own mistakes .66 15.07 2.57 Letting go of others mistakes .74 14.83 2.88 Table 3: Total Variance Explained for Moral Competency Inventory factor analysis Factor New Name Consistent social conviction Active care of others Taking ethical action Obligation to the truth Self forgivingness Owning ones mistakes Accepting own mistakes Accepting others mistakes Having others trust Initial Eigenvalues % of Total Variance 15.368 38.421 2.336 5.840 1.789 4.472 1.678 4.194 1.523 3.808 1.278 3.194 1.112 2.780 1.027 2.568 Cumulative % 38.421 44.261 48.732 52.927 56.734 59.928 62.708 65.276
1 2 3 4 5 6 7 8
25
Table 4: Rotated Component Matrix for Moral Competency Inventory (
Factor Item My co-workers would say that my behavior is very consistent with my beliefs and values. My friends would say that my behavior is very consistent with my beliefs and values. I can clearly state the principles, values, and beliefs that guide my actions. My co-workers would say that I am the kind of person who stands up for my convictions My co-workers would say that I take ownership of my decisions My co-workers think of me as an honest person. I spend a significant amount of my time providing resources and removing obstacles for my co-workers Because I care about my co-workers, I actively support their efforts to accomplish important personal goals. I discuss my mistakes with co-workers to encourage tolerance for risk. I pay attention to the development needs of my co-workers. My first response when I meet new people is to be genuinely interested in them. My leadership approach is to lead by serving others. I will generally confront someone if I see them doing something that isn't right. If I knew my company was engaging in unethical or illegal behavior, I would report it, even if it could have an adverse effect on my career. When a situation may prevent me from keeping a promise, I consult with those involved to renegotiate the agreement. When things go wrong, I do not blame others or circumstances. If I believe that my boss is doing something that isn't right, I will challenge him or her. When I agree to do something, I always follow through. I tell the truth unless there is an overriding moral reason to withhold it. When faced with an important decision, I consciously assess whether the decision I wish to make is aligned with my most deeply held principles, values, and beliefs. Even when I have made a serious mistake in my life, I am able to forgive myself and move ahead. I resist the urge to dwell on my mistakes When I forgive someone, I find that it benefits me as much as it does them. I am able to "forgive and forget," even when someone has made a serious mistake. I own up to my own mistakes and failures. When I make a decision that turns out to be a mistake, I admit it. My colleagues would say that I go out of my way to help them. I am willing to accept the consequences of my mistakes. I am able to deliver negative feedback in a respectful way When I make a mistake, I take responsibility for correcting the situation. I use my mistakes as an opportunity to improve my performance. My co-workers would say that I have a realistic attitude about my mistakes and failures. I accept that other people will make mistakes My co-workers would say that I am a compassionate person. Even when people make mistakes, I continue to trust them. My friends know they can depend on me to be truthful to them.When someone asks me to keep a confidence, I do so. I truly care about the people I work with as people - not just as the "human capital" needed to produce results. My friends and co-workers know they can depend on me to keep my word. I appreciate the positive aspects of my past mistakes, realizing that they were valuable lessons on my way to success. 1 .634 .610 .574 .465 .422 .375 .801 .672 .666 .619 .557 .409 .348 .685 .457 .412 .352 .756. 452 .424 .695 .587 .576 .487 -.901 -.567 -.481 -.434 -.427 -.370 -.624 -.504 -.410.393 -.337 -.316 -.398 -.341 2 3 4 5 6 7 8
Extraction Method: Principal Axis Factoring, Rotation Method: Oblimin with Kaiser Normalization. Rotation converged in 22 iterations.
26
Table 5: Bivariate and Partial (Impression Management) Correlations between Moral Competency Index and Integrity Measures
Integrity Integrity Acting Consistently and Values Telling Truth Standing for Right Keeping Promises Taking Responsibility Admitting Mistakes .435(**) .000 .307(**) .008 .352(**) .002 .345(**) .003 .287(*) .013 .327(**) .004 74 .262(*) .024 .193 .099 .245(*) .036 .253(*) .030 .390(**) .001 .493 .000 .384 .001 .381 .001 .418 .000 .331 .006 .321 .008 66 .248 .041 .175 .154 .275 .023 .244 .045 .418 .000 Responsibility (Partial r) .356(**) .002 .438(**) .000 .322(**) .005 .380(**) .001 .287(*) .013 .397(**) .000 74 .462(**) .000 .468(**) .000 .300(**) .009 .441(**) .000 .510(**) .000 Responsibility Self Esteem (Partial r) .362 .002 .473 .000 .294 .015 .441 .000 .301 .013 .361 .002 66 .417 .000 .432 .000 .340 .005 .403 .001 .500 .000 .058 .613 -.083 .469 .005 .965 .043 .712 -.011 .924 -.030 .797 78 .030 .791 .038 .743 .025 .825 -.096 .404 -.001 .993
Embracing Responsibility for Others Actively Caring Owning Own Mistakes Letting Go Others Mistakes MCI Total
Table 6: Univariate ANOVA: Gender and Significantly Different MCI Scales Dependent Variable MCI Keeping Promises MCI Actively Caring MCI Total Type III Sum of df Squares 30.142 34.417 285.239 1 1 1 F p. Partial ?2 Male Mean 15.80 14.52 74.94 Female Mean 17.13 15.94 79.04
7.952 .006 .095 4.764 .032 .059 3.535 .064 .044
27
doc_709603075.docx