Evaluation of a Higher Order Thinking Skills Test for the Topic of Cell Division and Subtopic of Gametogenesis (UKBATG)
Keywords:
Validity, reliability, difficulty index, discrimination index, biologyAbstract
This quantitative study aimed to evaluate the quality of a Higher Order Thinking Skills (HOTS) Test instrument for the topic of Cell Division and the subtopic of Gametogenesis (UKBATG). UKBATG contains contextual question and problem- solving question which is 25 items in the form of multiple-choice and 6 subjective items that can measure students' Higher Order Thinking Skills (HOTS). The process of validity was conducted by five experts in the field of Biology and HOTS and two experts in the field of Malay language and English language. The reliability of UKBATG was determined by calculating the KR-20 coefficient, Cronbach’s alpha coefficient, and inter-rater reliability (IRR) by determining the intra-class correlation coefficient (ICC). The quality of UKBATG was also determined by calculating the difficulty index, p and discrimination index, D for each item. Findings showed that all UKBATG items have a good validity exceeding 70% agreement among experts. In terms of reliability, it was found that the KR-20 coefficient was 0.774, with Cronbach's alpha of 0.844 and the ICC coefficient in the range of 0.635 to 0.841. The values of p and D indicated that the UKBATG items were at a moderate level and were accepted as good items. In conclusion, this study successfully developed an instrument that is good in terms of validity and reliability and also has good item quality. The implication is that the UKBATG can be used to measure students’ HOTS and increase the number of HOTS test instruments, especially in Biology.
Downloads
References
Don, Y., et al., Educational leadership competencies and Malaysia education development plan 2013-2025. Humanities and Social Sciences Review, 2015. 4(3): p. 615-625.
Macinko, J., B. Starfield, and L. Shi, The contribution of primary care systems to health outcomes within Organization for Economic Cooperation and Development (OECD) countries, 1970–1998. Health services research, 2003. 38(3): p. 831-865.DOI: https://doi.org/10.1111/1475-6773.00149.
Mensah, C.N., et al., Technological innovation and green growth in the Organization for Economic Cooperation and Development economies. Journal of Cleaner Production, 2019. 240: p. 118204.DOI: https://doi.org/10.1016/j.jclepro.2019.118204.
Hamdi, S., I.A. Suganda, and N. Hayati, Developing higher-order thinking skill (HOTS) test instrument using Lombok local cultures as contexts for junior secondary school mathematics. REiD (Research and Evaluation in Education), 2018. 4(2): p. 126-135.DOI: https://doi.org/10.21831/reid.v4i2.22089.
Adnan, M., et al., Construction of the Form One Mathematics Higher Level Thinking Skills item for the topic of Fractions. Malaysian Journal of Science and Mathematics Education, 2018. 8(1): p. 46-
DOI: https://doi.org/10.37134/jpsmm.vol8.1.4.2018.
Lestari, N.A., et al. A Preliminary Study of Environmental Learning to Improve Students’ Higher Order Thinking Skills in Physics. IOP Publishing.DOI: https://doi.org/10.1088/1742- 6596/1805/1/012033.
Raflee, S.S.M. and L. Halim, The Effectiveness of Critical Thinking in Improving Skills in KBAT Problem Solving. Jurnal Pendidikan Sains dan Matematik Malaysia, 2021. 11(1): p. 60-76.
Abd Ghani, A., The teaching of indigenous Orang Asli language in Peninsular Malaysia. Procedia- Social and Behavioral Sciences, 2015. 208: p. 253-262.DOI: https://doi.org/10.1016/j.sbspro.2015.11.201.
Anderson, L.W. and D.R. Krathwohl, A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives, p. 323-331. 2001: Longman.
Heong, Y.M., et al., The level of marzano higher order thinking skills among technical education students. International Journal of Social Science and Humanity, 2011. 1(2): p. 121.DOI: https://doi.org/10.7763/IJSSH.2011.V1.20.
Kiong, T.T., et al., Thinking skills for secondary school students in Malaysia. Journal of Research, Policy & Practice of Teachers and Teacher Education, 2012. 2(2): p. 12-23.
Merta Dhewa, K., et al., The development of Higher Order Thinking Skill (Hots) instrument assessment in physics study. IOSR Journal of Research & Method in Education (IOSR-JRME), 2017. 7(1): p. 26-32.DOI: https://doi.org/10.9790/7388-0701052632.
Hadi, S., et al., The difficulties of high school students in solving higher-order thinking skills problems. Problems of Education in the 21st Century, 2018. 76(4): p. 520.DOI: https://doi.org/10.33225/pec/18.76.520.
Peng, C.F. and Z.H. Hamad, Higher Order of Thinking Skills in Teaching and Learning Malay Language through Questioning Technique. Jurnal Pendidikan Bahasa Melayu, 2018. 8(2): p. 1-12.
Ramdiah, S., et al., Understanding, planning, and implementation of HOTS by senior high school biology teachers in Banjarmasin-Indonesia. International Journal of Instruction, 2019. 12(1): p. 425- 440.
Ichsan, I.Z., et al., HOTS-AEP: Higher Order Thinking Skills from Elementary to Master Students in Environmental Learning. European Journal of Educational Research, 2019. 8(4): p. 935-942.DOI: https://doi.org/10.12973/eu-jer.8.4.935.
Sanip, F.A. and C.N. Che Ahmad, Awareness of metacognitive strategies and high -level thinking skills among Biology students. Journal of Educational Research, 2014. 15.
Chinedu, C.C., O.S. Olabiyi, and Y. Kamin, Strategies for improving higher order thinking skills in teaching and learning of design and technology education. 7(2). 2015.
Saido, G.M., et al., Higher order thinking skills among secondary school students in science learning. MOJES: Malaysian Online Journal of Educational Sciences, 3(3), 13-20. 2015.
Innocenti, B., et al., Development and validation of a wear model to predict polyethylene wear in a total knee arthroplasty: a finite element analysis. Lubricants, 2014. 2(4): p. 193-205.DOI: https://doi.org/10.3390/lubricants2040193.
Abidin, M.Z.Z. and K. Osman, The level of knowledge, understanding, skills and implementation of science teachers on high -level thinking skills (HLTS). Journal of Advanced Research in Social and Behavioral Sciences, 8 (1), 97-113. 2017.
Peng, C.F. and Z.H. Hamad, Higher Order of Thinking Skills in Teaching and Learning Malay Language through Questioning Technique. Journal of Malay Language Education, 2018. 8(2): p. 1- 12.
Kassim, N. and E. Zakaria, Integration of high -level thinking skills in mathematics teaching and learning: An analysis of teacher needs. Journal of Mathematics Education, 3 (1), 1–12. 2015.
Saido, G.A.M., et al., Teaching strategies for promoting higher order thinking skills: A case of secondary science teachers. MOJEM: Malaysian Online Journal of Educational Management, 2017. 3(4): p. 16-30.
Retnawati, H., et al., Teachers’ knowledge about higher-order thinking skills and its learning strategy. Problems of Education in the 21st Century, 2018. 76(2): p. 215.DOI: https://doi.org/10.33225/pec/18.76.215.
Tandas, J.B., Development of a Valid and Reliable Summative Test in Plane Trigonometry. Journal of Science and Mathematics Letters, 2021. 9(1): p. 46-59.
Rahman, S.M.A. and S. Ya’acob, Gamified learning to improve higher order thinking among Malaysian students. Open International Journal of Informatics (OIJI), 2018: p. 9-21.
Chun, T.C. and M.N.L.Y.B. Abdullah, The teaching of higher order thinking skills (HOTS) in Malaysian schools: Policy and practices. MOJEM: Malaysian Online Journal of Educational Management, 2019. 7(3): p. 1-18.DOI: https://doi.org/10.22452/mojem.vol7no3.1.
Nor, M., et al., Application of high -level thinking skills (HLTS) in the Design and Technology (RBT) curriculum of primary schools. International journal of education and training, 2017. 3(2): p. 1-7.
Seman, S.C., W.M.W. Yusoff, and R. Embong, Teachers challenges in teaching and learning for higher order thinking skills (HOTS) in primary school. International Journal of Asian Social Science, 2017. 7(7): p. 534-545.DOI: https://doi.org/10.18488/journal.1.2017.77.534.545.
Subiantoro, A.W. and D.F. Treagust, Development and validation of an instrument for assessing high-school students’ perceptions of socio-scientific issues-based learning in biology. Learning Environments Research, 2021. 24(2): p. 223-237.DOI: https://doi.org/10.1007/s10984-020-09332- z.
Aisyah, A., et al. Eliciting Elements of Higher Order Thinking Skills in the Higher Secondary Examination Question Structure in Japan and Malaysia. Springer.DOI: https://doi.org/10.1007/978- 981-13-0203-9_42.
Agarwal, S., et al., Guidelines for reporting of health interventions using mobile phones: mobile health (mHealth) evidence reporting and assessment (mERA) checklist. bmj, 2016. 352.DOI: https://doi.org/10.1136/bmj.i1174.
Sulaiman, T., A.F.M. Ayub, and S. Sulaiman, Curriculum change in English language curriculum advocates higher order thinking skills and standards-based assessments in Malaysian primary schools. Mediterranean Journal of Social Sciences, 2015. 6(2): p. 494-494.DOI: https://doi.org/10.5901/mjss.2015.v6n2p494.
Morkel, C. and M. Ramasobama, Measuring the effect of evaluation capacity building initiatives in Africa: A review. African Evaluation Journal, 2017. 5(1).DOI: https://doi.org/10.4102/aej.v5i1.187.
Tekkaya, C., Ö. Özkan, and S. Sungur, Biology concepts perceived as difficult by Turkish high school students. Hacettepe üniversitesi eğitim fakültesi dergisi, 2001. 21(21).
Atilla, C. and imer, What makes biology learning difficult and effective: Students’ views.Educational research and reviews, 2012. 7(3): p. 61-71.
Ozcan, T., et al., Identifiying and comparing the degree of difficulties biology subjects by adjusting it is reasons in elemantary and secondary education. Procedia-Social and Behavioral Sciences, 2014. 116: p. 113-122.DOI: https://doi.org/10.1016/j.sbspro.2014.01.177.
Haris, N. and K. Osman, The effectiveness of a virtual field trip (VFT) module in learning biology. Turkish Online Journal of Distance Education, 2015. 16(3): p. 102-117.DOI: https://doi.org/10.17718/tojde.13063.
Gungor, S.N. and M. Ozkan, Evaluation of the concepts and subjects in biology perceived to be difficult to learn and teach by the pre-service teachers registered in the pedagogical formation program. European Journal of Educational Research, 2017. 6(4): p. 495-508.DOI: https://doi.org/10.12973/eu-jer.6.4.495.
Fauzi, A. and M. Mitalistiani, High school biology topics that perceived difficult by undergraduate students. Didaktika Biologi: Jurnal Penelitian Pendidikan Biologi, 2018. 2(2): p. 73-84.DOI: https://doi.org/10.32502/dikbio.v2i2.1242.
Salleh, W.N.W.M., C.N.C. Ahmad, and E. Setyaningsih, Difficult topics in Biology from the view point of students and teachers based on KBSM implementation. EDUCATUM Journal of Science, Mathematics and Technology, 2021. 8(1): p. 49-56.
Harrison, R.G., A textbook of human embryology. Academic Medicine, 1964. 39(9): p. 868.
Anwar, A.H., N.Y. Rustaman, and W. Purwianingsih. Development of three-tier diagnostic test instruments for detecting students’ conception. IOP Publishing.DOI: https://doi.org/10.1088/1742- 6596/1318/1/012064.
Kazeni, M.M.M., Development and Validation Test of the Integrated Science Process Skills for the Further Education and Training Learner's, University of Pretoria, South Africa. Retrieved June, 2005. 10: p. 2019.
Caruth, G.D., Demystifying mixed methods research design: A review of the literature. Online Submission, 2013. 3(2): p. 112-122.DOI: https://doi.org/10.13054/mije.13.35.3.2.
Cohen, R.J., M.E. Swerdlik, and S.M. Phillips, Psychological testing and assessment: An introduction to tests and measurement 7th edition. 1996: Mayfield Publishing Co.
Anastasi, A. and S. Urbina, Psychological testing 7th edition 1997: Prentice Hall/Pearson Education.
Popham, W.J. and T.R. Husek, IMPLICATIONS OF CRITERION‐REFERENCED MEASUREMENT 1, 2. Journal of Educational Measurement, 1969. 6(1): p. 1-9.
Haladyna, T.M., Writing Test Items to Evaluate Higher Order Thinking. 1997: ERIC.
Morrison, S. and K.W. Free, Writing multiple-choice test items that promote and measure critical thinking. Journal of Nursing Education, 2001. 40(1): p. 17-24.DOI: https://doi.org/10.3928/0148- 4834-20010101-06.
Popham, W.J., Classroom assessment: What teachers need to know. 1999: ERIC.
Naglieri, J.A., et al., Psychological testing on the Internet: new problems, old issues. American Psychologist, 2004. 59(3): p. 150.DOI: https://doi.org/10.1037/0003-066X.59.3.150.
Kryklywy, J., Kaplan, RM & Saccuzzo, DP (2009) Psychological Testing Principles, Applications, and Issues. –abbreviated paperback edition printed especially for this course.(Belmont, CA.: Wadsworth).(Note: the words Psychology 2080 A/B appear on the cover of this book.).
Czerwinski, F., A.C. Richardson, and L.B. Oddershede, Quantifying noise in optical tweezers by allan variance. Optics express, 2009. 17(15): p. 13255-13269.DOI: https://doi.org/10.1364/OE.17.013255.
Nunnally, J.C., Psychometric Theory 2nd edition (New York: McGraw). 1978.
Moore, G.C. and I. Benbasat, Development of an instrument to measure the perceptions of adopting an information technology innovation. Information systems research, 1991. 2(3): p. 192-222.DOI: https://doi.org/10.1287/isre.2.3.192.
Wilson, F.R., W. Pan, and D.A. Schumsky, Recalculation of the critical values for Lawshe’s content validity ratio. Measurement and evaluation in counseling and development, 2012. 45(3): p. 197-210.DOI: https://doi.org/10.1177/0748175612440286.
Cohen, L., L. Manion, and K. Morrison, Research methods in education, 6th edition. . 2002: routledge.DOI: https://doi.org/10.4324/9780203224342.
Popham, W.J., Modern educational measurement: Practical guidelines for educational leaders, 3rd edition. 2000: Pearson College Division.DOI: https://doi.org/10.1111/j.1745-3984.1969.tb00654.x.
Aiken, L.R., Psychological testing and assessment, 9th edition. . 2009: Pearson Education India.
Cohen, R.J. and R. Jay, Exercises in psychological testing and assessment. Vol. 2. 2005: McGraw- Hill New York, NY, USA:.DOI: https://doi.org/10.1037/1040 3590.17.3.375.
Dorset, D.L., X-ray diffraction: a practical approach. Microscopy and microanalysis, 1998. 4(5): p. 513-515.DOI: https://doi.org/10.1017/S143192769800049X.
Loewenthal, K.M., An Introduction to Psychological Tests and Scales (2nd Ed.). University of London: Psychology Press. 2001.
Onyefulu, C., Assessment practices of teachers in selected primary and secondary schools in Jamaica. Open Access Library Journal, 2018. 5(12): p. 1-25.DOI: https://doi.org/10.4236/oalib.1105038.
Kerlinger, F.N., H.B. Lee, and D. Bhanthumnavin, Foundations of behavioral research: The most sustainable popular textbook by Kerlinger & Lee (2000). Journal of Social Development, 2000. 13:p. 131-144.
Brasselet*, E., et al., Light-induced nonlinear rotations of nematic liquid crystal droplets trapped in laser tweezers. Molecular Crystals and Liquid Crystals, 2009. 512(1): p. 143-1989.DOI: https://doi.org/10.1080/15421400903050780.
Lewis, A. and D. Smith, Defining higher order thinking. Theory into practice, 1993. 32(3): p. 131-137.DOI: https://doi.org/10.1080/00405849309543588.
Aziz, N.F.A., H. Ahmad, and I.M. Nashir, Validation of technical and vocational teachers’ competency evaluation instrument using the rasch model. Jurnal Pendidikan Sains Dan Matematik Malaysia, 2019. 9(1): p. 18-25.DOI: https://doi.org/10.37134/jpsmm.vol9.1.3.2019.
Downloads
Published
Issue
Section
License
Copyright (c) 2022 Author
This work is licensed under a Creative Commons Attribution 4.0 International License.
You are free to:
- Share — copy and redistribute the material in any medium or format for any purpose, even commercially.
- Adapt — remix, transform, and build upon the material for any purpose, even commercially.
- The licensor cannot revoke these freedoms as long as you follow the license terms.
Under the following terms:
- Attribution — You must give appropriate credit , provide a link to the license, and indicate if changes were made . You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
- No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.
Notices:
You do not have to comply with the license for elements of the material in the public domain or where your use is permitted by an applicable exception or limitation .
No warranties are given. The license may not give you all of the permissions necessary for your intended use. For example, other rights such as publicity, privacy, or moral rights may limit how you use the material.
