Description
Factor loading plot, Factor matrix, Kaiser-Meyer-Olkin (KMO), Priori Determination, Scree plot, varimax, oblique rotation alongwith SPSS
Factor Analysis
Factor Analysis
?
?
Factor analysis involves procedures primarily used for data reduction and summarization. Factor analysis is an interdependence technique in that relationships are examined without making the distinction between dependent and independent variables. Factor analysis is used to identify
?
underlying dimensions, or factors, that explain the correlations among a set of variables.
Factor Analysis Model
Mathematically, each variable is expressed as a linear combination of underlying factors.
The factor model may be represented as:
Xi = Ai 1F1 + Ai 2F2 + Ai 3F3 + . . . + AimFm + ViUi
where
Xi Aij F Vi Ui m
= = = = = =
i th variable
coefficient of variable i on common factor j common factor standardized regression coefficient of variable i on unique factor i the unique factor for variable i number of common factors
Factor Analysis Model
The unique factors are uncorrelated with each other and with the common factors. The common factors themselves can be expressed as linear combinations of the observed variables. Fi = Wi1X1 + Wi2X2 + Wi3X3 + . . . + WikXk where Fi Wi k = = =
i th factor
weight or coefficient number of variables
Factor Analysis Model
?
Select weights so that the first factor explains the largest portion of the total variance. Then select a second set of weights so that the second factor accounts for most of the residual variance, subject to being uncorrelated with the first factor. This same principle could be applied to selecting additional weights for the additional factors.
?
?
Statistics Associated with Factor Analysis
?
Bartlett's test of sphericity is a test statistic used to examine the hypothesis that the variables are uncorrelated in the population. In other words, the population correlation matrix is an identity matrix; each variable correlates perfectly with itself (r = 1) but has no correlation with the other variables (r = 0).
Statistics Associated with Factor Analysis
? ? ?
?
?
Communality. Communality is the amount of variance explained by the common factors. Eigenvalue. The eigenvalue represents the total variance explained by each factor. Factor loadings. Factor loadings are simple correlations between the variables and the factors. Factor loading plot. A factor loading plot is a plot of the original variables using the factor loadings as coordinates. Factor matrix. A factor matrix contains the factor loadings of all the variables on all the factors extracted.
Statistics Associated with Factor Analysis
?
Factor scores. Factor scores are composite scores estimated for each respondent on the derived factors. Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy. High values (between 0.5 and 1.0) indicate factor analysis is appropriate. Values below 0.5 imply that factor analysis may not be appropriate. Percentage of variance. The percentage of the total variance attributed to each factor.
?
?
?
Residuals are the differences between the observed correlations and factor matrix.
Scree plot. A scree plot is a plot of the Eigenvalues against the number of factors in order of extraction.
?
Conducting Factor Analysis
Table 1
RESPONDENT NUMBER 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 V1 7.00 1.00 6.00 4.00 1.00 6.00 5.00 6.00 3.00 2.00 6.00 2.00 7.00 4.00 1.00 6.00 5.00 7.00 2.00 3.00 1.00 5.00 2.00 4.00 6.00 3.00 4.00 3.00 4.00 2.00 V2 3.00 3.00 2.00 5.00 2.00 3.00 3.00 4.00 4.00 6.00 4.00 3.00 2.00 6.00 3.00 4.00 3.00 3.00 4.00 5.00 3.00 4.00 2.00 6.00 5.00 5.00 4.00 7.00 6.00 3.00 V3 6.00 2.00 7.00 4.00 2.00 6.00 6.00 7.00 2.00 2.00 7.00 1.00 6.00 4.00 2.00 6.00 6.00 7.00 3.00 3.00 2.00 5.00 1.00 4.00 4.00 4.00 7.00 2.00 3.00 2.00 V4 4.00 4.00 4.00 6.00 3.00 4.00 3.00 4.00 3.00 6.00 3.00 4.00 4.00 5.00 2.00 3.00 3.00 4.00 3.00 6.00 3.00 4.00 5.00 6.00 2.00 6.00 2.00 6.00 7.00 4.00 V5 2.00 5.00 1.00 2.00 6.00 2.00 4.00 1.00 6.00 7.00 2.00 5.00 1.00 3.00 6.00 3.00 3.00 1.00 6.00 4.00 5.00 2.00 4.00 4.00 1.00 4.00 2.00 4.00 2.00 7.00 V6 4.00 4.00 3.00 5.00 2.00 4.00 3.00 4.00 3.00 6.00 3.00 4.00 3.00 6.00 4.00 4.00 4.00 4.00 3.00 6.00 3.00 4.00 4.00 7.00 4.00 7.00 5.00 3.00 7.00 2.00
Conducting Factor Analysis
Fig 1 Problem formulation Construction of the Correlation Matrix Method of Factor Analysis Determination of Number of Factors Determination of Model Fit
Conducting Factor Analysis
Formulate the Problem
? ?
?
The objectives of factor analysis should be identified. The variables to be included in the factor analysis should be specified based on past research, theory, and judgment of the researcher. It is important that the variables be appropriately measured on an interval or ratio scale. An appropriate sample size should be used. As a rough guideline, there should be at least four or five times as many observations (sample size) as there are variables.
Correlation Matrix
Table 2
Variables V1 V2 V3 V4 V5 V6
V1 1.000 -0.530 0.873 -0.086 -0.858 0.004
V2 1.000 -0.155 0.572 0.020 0.640
V3
V4
V5
V6
1.000 -0.248 -0.778 -0.018
1.000 -0.007 0.640
1.000 -0.136
1.000
Conducting Factor Analysis
?
Determine the Method of Factor Analysis
In principal components analysis, the total variance in the data is considered. It is recommended when the primary concern is to determine the minimum number of factors that will account for maximum variance in the data. The factors are called principal components. In common factor analysis, the factors are estimated based only on the common variance. This method is appropriate when the primary concern is to identify the underlying dimensions
?
Results of Principal Components Analysis
Table 3
Communalities
Variables V1 V2 V3 V4 V5 V6 Initial 1.000 1.000 1.000 1.000 1.000 1.000 Extraction 0.926 0.723 0.894 0.739 0.878 0.790
Initial Eigen values
Factor 1 2 3 4 5 6 Eigen value 2.731 2.218 0.442 0.341 0.183 0.085 % of variance 45.520 36.969 7.360 5.688 3.044 1.420 Cumulat. % 45.520 82.488 89.848 95.536 98.580 100.000
Results of Principal Components Analysis
Table 3 cont.
Extraction Sums of Squared Loadings
Factor 1 2 Eigen value 2.731 2.218 % of variance 45.520 36.969 Cumulat. % 45.520 82.488
Factor Matrix
Variables V1 V2 V3 V4 V5 V6
Factor 1 0.928 -0.301 0.936 -0.342 -0.869 -0.177
Factor 2 0.253 0.795 0.131 0.789 -0.351 0.871
Rotation Sums of Squared Loadings
Factor Eigenvalue % of variance 1 2.688 44.802 2 2.261 37.687 Cumulat. % 44.802 82.488
Results of Principal Components Analysis
Table 3 cont.
Rotated Factor Matrix
Variables V1 V2 V3 V4 V5 V6 Factor 1 0.962 -0.057 0.934 -0.098 -0.933 0.083 Factor 2 -0.027 0.848 -0.146 0.845 -0.084 0.885
Factor Score Coefficient Matrix
Variables V1 V2 V3 V4 V5 V6 Factor 1 0.358 -0.001 0.345 -0.017 -0.350 0.052 Factor 2 0.011 0.375 -0.043 0.377 -0.059 0.395
Conducting Factor Analysis
?
Determine the Number of Factors
A Priori Determination. Sometimes, because of
prior knowledge, the researcher knows how many factors to expect and thus can specify the number of factors to be extracted beforehand. Determination Based on Eigenvalues. In this approach, only factors with Eigenvalues greater than 1.0 are retained. An Eigenvalue represents the amount of variance associated with the factor. Hence, only factors with a variance greater than 1.0 are included.
?
Conducting Factor Analysis
?
Determine the Number of Factors
Determination Based on Scree Plot. A scree plot is a plot of the Eigenvalues against the number of factors in order of extraction. Experimental evidence indicates that the point at which the scree begins denotes the true number of factors. Generally, the number of factors determined by a scree plot will be one or a few more than that determined by the Eigenvalue criterion. Determination Based on Percentage of Variance. In this approach the number of factors extracted is determined so that the cumulative percentage of variance extracted by the factors reaches a satisfactory level. It is recommended that the factors extracted should account for at least 60% of the variance.
?
Scree Plot
Fig 2
3.0 2.5 Eigenvalue 2.0 1.5 1.0 0.5 0.0 1
2
3 4 5 Component Number
6
Conducting Factor Analysis
Determine the Number of Factors
?
Determination Based on Split-Half Reliability. The sample is split in half and factor analysis is performed on each half. Only factors with high correspondence of factor loadings across the two subsamples are retained. Determination Based on Significance Tests. It is possible to determine the statistical significance of the separate Eigenvalues and retain only those factors that are statistically significant.
?
Conducting Factor Analysis
Rotate Factors
?
?
?
Although the initial or unrotated factor matrix indicates the relationship between the factors and individual variables, it seldom results in factors that can be interpreted, because the factors are correlated with many variables. Therefore, through rotation the factor matrix is transformed into a simpler one that is easier to interpret. In rotating the factors, we would like each factor to have significant, loadings or coefficients for only some of the variables. Likewise, we would like each variable to have significant loadings with only a few factors, if possible with only one. The rotation is called orthogonal rotation if the axes are maintained at right angles.
Conducting Factor Analysis
Rotate Factors
?
?
The most commonly used method for rotation is the varimax procedure. This is an orthogonal method of rotation that results in factors that are uncorrelated. The rotation is called oblique rotation when the axes are not maintained at right angles, and the factors are correlated.
Conducting Factor Analysis
Interpret Factors
?
A factor can then be interpreted in terms of the variables that load high on it. Another useful aid in interpretation is to plot the variables, using the factor loadings as coordinates. Variables at the end of an axis are those that have high loadings on only that factor, and hence describe the factor.
?
Factor Loading Plot
Fig 3
Rotated Component Matrix Variable V1 Component 1 2 0.962 -2.66E-02
Component Plot in Rotated Space Component 1 1.0 0.5 0.0 -0.5 V4
?? ?
V2
V6 Component 2
V2
V3 V4 V5
-5.72E-02
0.934 -9.83E-02 -0.933
0.848
-0.146 0.854 -8.40E-02
? V5
? ? V3
V1
V6
8.337E-02 0.885
-1.0 1.0 0.5 0.0 -0.5 -1.0
Conducting Factor Analysis
Select Surrogate Variables
?
By examining the factor matrix, one could select for each factor the variable with the highest loading on that factor. That variable could then be used as a surrogate variable for the associated factor.
SPSS Windows
To select this procedures using SPSS for Windows click: Analyze>Data Reduction>Factor …
doc_484776917.ppt
Factor loading plot, Factor matrix, Kaiser-Meyer-Olkin (KMO), Priori Determination, Scree plot, varimax, oblique rotation alongwith SPSS
Factor Analysis
Factor Analysis
?
?
Factor analysis involves procedures primarily used for data reduction and summarization. Factor analysis is an interdependence technique in that relationships are examined without making the distinction between dependent and independent variables. Factor analysis is used to identify
?
underlying dimensions, or factors, that explain the correlations among a set of variables.
Factor Analysis Model
Mathematically, each variable is expressed as a linear combination of underlying factors.
The factor model may be represented as:
Xi = Ai 1F1 + Ai 2F2 + Ai 3F3 + . . . + AimFm + ViUi
where
Xi Aij F Vi Ui m
= = = = = =
i th variable
coefficient of variable i on common factor j common factor standardized regression coefficient of variable i on unique factor i the unique factor for variable i number of common factors
Factor Analysis Model
The unique factors are uncorrelated with each other and with the common factors. The common factors themselves can be expressed as linear combinations of the observed variables. Fi = Wi1X1 + Wi2X2 + Wi3X3 + . . . + WikXk where Fi Wi k = = =
i th factor
weight or coefficient number of variables
Factor Analysis Model
?
Select weights so that the first factor explains the largest portion of the total variance. Then select a second set of weights so that the second factor accounts for most of the residual variance, subject to being uncorrelated with the first factor. This same principle could be applied to selecting additional weights for the additional factors.
?
?
Statistics Associated with Factor Analysis
?
Bartlett's test of sphericity is a test statistic used to examine the hypothesis that the variables are uncorrelated in the population. In other words, the population correlation matrix is an identity matrix; each variable correlates perfectly with itself (r = 1) but has no correlation with the other variables (r = 0).
Statistics Associated with Factor Analysis
? ? ?
?
?
Communality. Communality is the amount of variance explained by the common factors. Eigenvalue. The eigenvalue represents the total variance explained by each factor. Factor loadings. Factor loadings are simple correlations between the variables and the factors. Factor loading plot. A factor loading plot is a plot of the original variables using the factor loadings as coordinates. Factor matrix. A factor matrix contains the factor loadings of all the variables on all the factors extracted.
Statistics Associated with Factor Analysis
?
Factor scores. Factor scores are composite scores estimated for each respondent on the derived factors. Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy. High values (between 0.5 and 1.0) indicate factor analysis is appropriate. Values below 0.5 imply that factor analysis may not be appropriate. Percentage of variance. The percentage of the total variance attributed to each factor.
?
?
?
Residuals are the differences between the observed correlations and factor matrix.
Scree plot. A scree plot is a plot of the Eigenvalues against the number of factors in order of extraction.
?
Conducting Factor Analysis
Table 1
RESPONDENT NUMBER 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 V1 7.00 1.00 6.00 4.00 1.00 6.00 5.00 6.00 3.00 2.00 6.00 2.00 7.00 4.00 1.00 6.00 5.00 7.00 2.00 3.00 1.00 5.00 2.00 4.00 6.00 3.00 4.00 3.00 4.00 2.00 V2 3.00 3.00 2.00 5.00 2.00 3.00 3.00 4.00 4.00 6.00 4.00 3.00 2.00 6.00 3.00 4.00 3.00 3.00 4.00 5.00 3.00 4.00 2.00 6.00 5.00 5.00 4.00 7.00 6.00 3.00 V3 6.00 2.00 7.00 4.00 2.00 6.00 6.00 7.00 2.00 2.00 7.00 1.00 6.00 4.00 2.00 6.00 6.00 7.00 3.00 3.00 2.00 5.00 1.00 4.00 4.00 4.00 7.00 2.00 3.00 2.00 V4 4.00 4.00 4.00 6.00 3.00 4.00 3.00 4.00 3.00 6.00 3.00 4.00 4.00 5.00 2.00 3.00 3.00 4.00 3.00 6.00 3.00 4.00 5.00 6.00 2.00 6.00 2.00 6.00 7.00 4.00 V5 2.00 5.00 1.00 2.00 6.00 2.00 4.00 1.00 6.00 7.00 2.00 5.00 1.00 3.00 6.00 3.00 3.00 1.00 6.00 4.00 5.00 2.00 4.00 4.00 1.00 4.00 2.00 4.00 2.00 7.00 V6 4.00 4.00 3.00 5.00 2.00 4.00 3.00 4.00 3.00 6.00 3.00 4.00 3.00 6.00 4.00 4.00 4.00 4.00 3.00 6.00 3.00 4.00 4.00 7.00 4.00 7.00 5.00 3.00 7.00 2.00
Conducting Factor Analysis
Fig 1 Problem formulation Construction of the Correlation Matrix Method of Factor Analysis Determination of Number of Factors Determination of Model Fit
Conducting Factor Analysis
Formulate the Problem
? ?
?
The objectives of factor analysis should be identified. The variables to be included in the factor analysis should be specified based on past research, theory, and judgment of the researcher. It is important that the variables be appropriately measured on an interval or ratio scale. An appropriate sample size should be used. As a rough guideline, there should be at least four or five times as many observations (sample size) as there are variables.
Correlation Matrix
Table 2
Variables V1 V2 V3 V4 V5 V6
V1 1.000 -0.530 0.873 -0.086 -0.858 0.004
V2 1.000 -0.155 0.572 0.020 0.640
V3
V4
V5
V6
1.000 -0.248 -0.778 -0.018
1.000 -0.007 0.640
1.000 -0.136
1.000
Conducting Factor Analysis
?
Determine the Method of Factor Analysis
In principal components analysis, the total variance in the data is considered. It is recommended when the primary concern is to determine the minimum number of factors that will account for maximum variance in the data. The factors are called principal components. In common factor analysis, the factors are estimated based only on the common variance. This method is appropriate when the primary concern is to identify the underlying dimensions
?
Results of Principal Components Analysis
Table 3
Communalities
Variables V1 V2 V3 V4 V5 V6 Initial 1.000 1.000 1.000 1.000 1.000 1.000 Extraction 0.926 0.723 0.894 0.739 0.878 0.790
Initial Eigen values
Factor 1 2 3 4 5 6 Eigen value 2.731 2.218 0.442 0.341 0.183 0.085 % of variance 45.520 36.969 7.360 5.688 3.044 1.420 Cumulat. % 45.520 82.488 89.848 95.536 98.580 100.000
Results of Principal Components Analysis
Table 3 cont.
Extraction Sums of Squared Loadings
Factor 1 2 Eigen value 2.731 2.218 % of variance 45.520 36.969 Cumulat. % 45.520 82.488
Factor Matrix
Variables V1 V2 V3 V4 V5 V6
Factor 1 0.928 -0.301 0.936 -0.342 -0.869 -0.177
Factor 2 0.253 0.795 0.131 0.789 -0.351 0.871
Rotation Sums of Squared Loadings
Factor Eigenvalue % of variance 1 2.688 44.802 2 2.261 37.687 Cumulat. % 44.802 82.488
Results of Principal Components Analysis
Table 3 cont.
Rotated Factor Matrix
Variables V1 V2 V3 V4 V5 V6 Factor 1 0.962 -0.057 0.934 -0.098 -0.933 0.083 Factor 2 -0.027 0.848 -0.146 0.845 -0.084 0.885
Factor Score Coefficient Matrix
Variables V1 V2 V3 V4 V5 V6 Factor 1 0.358 -0.001 0.345 -0.017 -0.350 0.052 Factor 2 0.011 0.375 -0.043 0.377 -0.059 0.395
Conducting Factor Analysis
?
Determine the Number of Factors
A Priori Determination. Sometimes, because of
prior knowledge, the researcher knows how many factors to expect and thus can specify the number of factors to be extracted beforehand. Determination Based on Eigenvalues. In this approach, only factors with Eigenvalues greater than 1.0 are retained. An Eigenvalue represents the amount of variance associated with the factor. Hence, only factors with a variance greater than 1.0 are included.
?
Conducting Factor Analysis
?
Determine the Number of Factors
Determination Based on Scree Plot. A scree plot is a plot of the Eigenvalues against the number of factors in order of extraction. Experimental evidence indicates that the point at which the scree begins denotes the true number of factors. Generally, the number of factors determined by a scree plot will be one or a few more than that determined by the Eigenvalue criterion. Determination Based on Percentage of Variance. In this approach the number of factors extracted is determined so that the cumulative percentage of variance extracted by the factors reaches a satisfactory level. It is recommended that the factors extracted should account for at least 60% of the variance.
?
Scree Plot
Fig 2
3.0 2.5 Eigenvalue 2.0 1.5 1.0 0.5 0.0 1
2
3 4 5 Component Number
6
Conducting Factor Analysis
Determine the Number of Factors
?
Determination Based on Split-Half Reliability. The sample is split in half and factor analysis is performed on each half. Only factors with high correspondence of factor loadings across the two subsamples are retained. Determination Based on Significance Tests. It is possible to determine the statistical significance of the separate Eigenvalues and retain only those factors that are statistically significant.
?
Conducting Factor Analysis
Rotate Factors
?
?
?
Although the initial or unrotated factor matrix indicates the relationship between the factors and individual variables, it seldom results in factors that can be interpreted, because the factors are correlated with many variables. Therefore, through rotation the factor matrix is transformed into a simpler one that is easier to interpret. In rotating the factors, we would like each factor to have significant, loadings or coefficients for only some of the variables. Likewise, we would like each variable to have significant loadings with only a few factors, if possible with only one. The rotation is called orthogonal rotation if the axes are maintained at right angles.
Conducting Factor Analysis
Rotate Factors
?
?
The most commonly used method for rotation is the varimax procedure. This is an orthogonal method of rotation that results in factors that are uncorrelated. The rotation is called oblique rotation when the axes are not maintained at right angles, and the factors are correlated.
Conducting Factor Analysis
Interpret Factors
?
A factor can then be interpreted in terms of the variables that load high on it. Another useful aid in interpretation is to plot the variables, using the factor loadings as coordinates. Variables at the end of an axis are those that have high loadings on only that factor, and hence describe the factor.
?
Factor Loading Plot
Fig 3
Rotated Component Matrix Variable V1 Component 1 2 0.962 -2.66E-02
Component Plot in Rotated Space Component 1 1.0 0.5 0.0 -0.5 V4
?? ?
V2
V6 Component 2
V2
V3 V4 V5
-5.72E-02
0.934 -9.83E-02 -0.933
0.848
-0.146 0.854 -8.40E-02
? V5
? ? V3
V1
V6
8.337E-02 0.885
-1.0 1.0 0.5 0.0 -0.5 -1.0
Conducting Factor Analysis
Select Surrogate Variables
?
By examining the factor matrix, one could select for each factor the variable with the highest loading on that factor. That variable could then be used as a surrogate variable for the associated factor.
SPSS Windows
To select this procedures using SPSS for Windows click: Analyze>Data Reduction>Factor …
doc_484776917.ppt