User friendly science Package of R Programming Language: A Veritable Tool for Reliability Estimate of Non-cognitive Scale
Keywords:
Ordinal Alpha, Cronbach Alpha, MacDonald Omega, Guttman Lambda,, Revelle Beta, GLB, Userfriendlyscience Package, R programming LanguageAbstract
Having quality instruments is essential in ensuring data integrity. Indiscriminately application and over-dependency on Cronbach alpha index for multiple measured items (ordinal scale) and usage of SPSS software, which produce spurious estimation, have been a subject of technical debates in the literature. This debate toes the path of fulfilling stringent underlying assumptions of Cronbach alpha, such as uni-dimensionality, tau- equivalent, etc. However, modern approaches like ordinal alpha, Omega coefficient, GLB, Guttman Lambda, and Revelle Beta have been suggested with precise estimates and confidence intervals via R programming language. Thus, this paper examined the performance of alternative approaches to Cronbach alpha and documented practical step by step of establishing it. Non-experimental design of scale development research was adopted, and a multi-stage sampling procedure was used to sample N = 883 subjects that participated in the study. Findings showed that the instrument is multidimensional, in which Cronbach alpha is not apt for its estimation. Also, other forms of reliability methods produced better and more precise estimates, though their performance differs among themselves. The authors concluded that estimation of Cronbach Alpha using SPSS when the instrument is ordinal is absolutely not sufficient. Therefore, it is recommended that researchers explore and shift their paradigm from traditional reliability estimates through SPSS to modern approaches using an R programming language.
Downloads
References
Hoque, M.E., Three domains of learning: Cognitive, affective and psychomotor. The Journal of EFL Education and Research, 2016. 2(2): p. 45-52.
Sijtsma, K., On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika, 2009. 74(1): p. 107-120.DOI: https://doi.org/10.1007/s11336-008-9101-0.
Yang, Y. and S.B. Green, Coefficient alpha: A reliability coefficient for the 21st century? Journal of psychoeducational assessment, 2011. 29(4): p. 377-392.DOI: https://doi.org/10.1177/0734282911406668.
Cronbach, L.J., Coefficient alpha and the internal structure of tests. psychometrika, 1951. 16(3): p. 297- 334.DOI: https://doi.org/10.1007/BF02310555.
Gadermann, A.M., M. Guhn, and B.D. Zumbo, Estimating ordinal reliability for Likert-type and ordinal item response data: A conceptual, empirical, and practical guide. Practical Assessment, Research, and Evaluation, 2012. 17(1): p. 3.DOI: https://doi.org/10.1177/0013164406288165.
McDonald, R.P., Test theory: A unified treatment. 1st Edition ed. 2013: psychology press.DOI: https://doi.org/10.4324/9781410601087-18.
Revelle, W., Hierarchical cluster analysis and the internal structure of tests. Multivariate Behavioral Research, 1979. 14(1): p. 57-74.DOI: https://doi.org/10.1207/s15327906mbr1401_4.
Guttman, L., A basis for analyzing test-retest reliability. Psychometrika, 1945. 10(4): p. 255-282.DOI: https://doi.org/10.1007/BF02288892.
Jackson, P.H. and C.C. Agunwamba, Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: I: Algebraic lower bounds. Psychometrika, 1977. 42(4): p. 567- 578.DOI: https://doi.org/10.1007/BF02295979.
Cortina, J.M., What is coefficient alpha? An examination of theory and applications. Journal of applied psychology, 1993. 78(1): p. 98.DOI: https://doi.org/10.1037/0021-9010.78.1.98.
Zumbo, B.D., A.M. Gadermann, and C. Zeisser, Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of modern applied statistical methods, 2007. 6(1): p. 4.DOI: https://doi.org/10.22237/jmasm/1177992180.
Cronbach, L.J. and R.J. Shavelson, My current thoughts on coefficient alpha and successor procedures. Educational and psychological measurement, 2004. 64(3): p. 391-418.DOI: https://doi.org/10.1177/0013164404266386.
Dunn, T.J., T. Baguley, and V. Brunsden, From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British journal of psychology, 2014. 105(3): p. 399-412.DOI: https://doi.org/10.1111/bjop.12046.
Green, S.B. and Y. Yang, Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika, 2009. 74(1): p. 155-167.DOI: https://doi.org/10.1007/s11336-008-9099-3.
Raykov, T., Scale construction and development using structural equation modeling. 2012: p. 472-492.
Raykov, T. and G.A. Marcoulides, A direct latent variable modeling based method for point and interval estimation of coefficient alpha. Educational and Psychological Measurement, 2015. 75(1): p. 146- 156.DOI: https://doi.org/10.1177/0013164414526039.
Zinbarg, R.E., et al., Cronbach’s α, Revelle’s β, and Macdonald’s ωH: their relations with each other and two alternative conceptualizations of reliability. Psychometrika. 2005. 70: p. 123–133.DOI: https://doi.org/10.1007/s11336-003-0974-7.
Liu, Y., A.D. Wu, and B.D. Zumbo, The impact of outliers on Cronbach’s coefficient alpha estimate of reliability: Ordinal/rating scale item responses. Educational and Psychological Measurement, 2010. 70(1): p. 5-21.DOI: https://doi.org/10.1177/0013164409344548.
Lewis, C., Classical test theory. In C. R. Rao and S. Sinharay (Eds.), Handbook of Statistics. 2007. 26: p. 29-43.DOI: https://doi.org/10.1016/S0169-7161(06)26002-4.
Ten Berge, J.M.F. and G. Sočan, The greatest lower bound to the reliability of a test and the hypothesis of unidimensionality. Psychometrika, 2004. 69(4): p. 613-625.DOI: https://doi.org/10.1007/BF02289858.
Chakraborty, R., Estimation of greatest lower bound reliability of academic delay of Gratification scale. IOSR Journal of Research & Method in Education, 2017. 7(2): p. 75-79.DOI: https://doi.org/10.9790/7388-0702017579.
Revelle, W., ICLUST: A cluster analytic approach to exploratory and confirmatory scale construction. Behavior Research Methods & Instrumentation, 1978. 10(5): p. 739-742.DOI: https://doi.org/10.3758/BF03205389.
Cooksey, R.W. and G.N. Soutar, Coefficient beta and hierarchical item clustering: An analytical procedure for establishing and displaying the dimensionality and homogeneity of summated scales. Organizational Research Methods, 2006. 9(1): p. 78-98.DOI: https://doi.org/10.1177/1094428105283939.
May, J., G. Hughes, and A.D. Lunn, Reliability estimation from appropriate testing of plant protection software. Software Engineering Journal, 1995. 10(6): p. 206-218.DOI: https://doi.org/10.1049/sej.1995.0026.
Peters, G.J., The Alpha and the Omega of Scale Reliability and Validity: why and how to Abandon Cronbach’ s Alpha. European Health Psychologist, 2014: p. 576-576.
Graham, J.M., Congeneric and (essentially) tau-equivalent estimates of score reliability: What they are and how to use them. Educational and psychological measurement, 2006. 66(6): p. 930-944.
Revelle, W. and R.E. Zinbarg, Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma.Psychometrika, 2009. 74(1): p. 145-154.DOI: https://doi.org/10.1007/s11336-008-9102-z.
Downloads
Published
Issue
Section
License
Copyright (c) 2022 Author
This work is licensed under a Creative Commons Attribution 4.0 International License.
You are free to:
- Share — copy and redistribute the material in any medium or format for any purpose, even commercially.
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
- The licensor cannot revoke these freedoms as long as you follow the license terms.
Under the following terms:
- Attribution — You must give appropriate credit , provide a link to the license, and indicate if changes were made . You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
Notices:
You do not have to comply with the license for elements of the material in the public domain or where your use is permitted by an applicable exception or limitation .
No warranties are given. The license may not give you all of the permissions necessary for your intended use. For example, other rights such as publicity, privacy, or moral rights may limit how you use the material.
