The meta-analysis of the concept map-based tests validity

Document Type : Original Article




The purpose of the present research was to perform a meta-analysis on the research that was conducted to evaluate the validity of the concept map-based tests. The sampling methods of the present research were purposive and snowball. After removing the irrelevant research, selected 29 research as sample. This research consisted of the dissertations, articles, and projects that were printed or performed in the different journals, valid universities and institutions in and out of Iran. Of these research obtained 153 effect sizes. After checking the entrance and exit criterions and removing digressive data, remained 143 effect sizes that analyzed by CMA software version 2. The meta analysis results showed that concept map-based tests have possessed mix effect size ./486 that according to the Cohen criterion, it is a large effect size in the correlational research. The results of the present meta-analysis showed the concept maps-based tests possess an appropriate validity and can be used for evaluation aims in the various domains.


ایزانلو، بلال و حبیبی، مجتبی (1390). کاربرد فراتحلیل در تحقیقات علوم اجتماعی و رفتاری: مروری بر مزایا، تنگناها و روش‌شناسی. تحقیقات علوم رفتاری، 9 (1)، 70 – 82.
رعایی، فرزانه؛ دلاور، علی و فرخی، نورعلی (1389). فراتحلیل تحقیقات انجام شده در حوزه اضطراب و افسردگی. فصلنامه اندازه‌گیری تربیتی، 1 (4)، 1 – 25.
سیف، علی‌اکبر (1389). روان‌شناسی پرورشی نوین: روان‌شناسی یادگیری و آموزش، ویراست ششم. تهران: نشر دوران.
عبادالله‌وند، کوثر (1395). بررسی روایی و پایایی آزمون‌های تکمیل مبتنی بر نقشه مفهومی در ماده درسی علوم تجربی پایه ششم. پایان‌نامه‌ کارشناسی ارشد، دانشگاه شهید مدنی آذربایجان.
مصرآبادی، جواد (1390). بررسی روایی و پایایی شیوه‌های نمره‌گذاری آزمون‌های مبتنی بر نقشه مفهومی. فصلنامه اندازه‌گیری تربیتی، 7 (2)، 9 – 32.
مصرآبادی، جواد (1388). امکان‌سنجی به کارگیری روش‌های ارزشیابی مبتنی بر نقشه مفهومی در زیست‌شناسی دوره متوسطه. وزارت آموزش‌وپرورش، سازمان پژوهش و برنامه‌ریزی درسی، مؤسسه پژوهشی برنامه‌ریزی درسی و نوآوری‌های آموزشی.
هومن، حیدرعلی (1390). شناخت روش علمی در علوم رفتاری. تهران: سازمان مطالعه و تدوین کتب علوم انسانی دانشگاه‌ها (سمت)، مرکز تحقیق و توسعه علوم انسانی.
Arneson, B. T. (2005). On the role of concept mapping assessment in todayʼs constructivist classroom. Doctoral dissertation, The University of Texas at Austin.
Borenstein, M., Hedges, L, V., Higgins, J, P, T., Rothstein, H, R. (2009). Introductionto Meta-Analysis. John Wiley & Sons, Ltd, UK.
Chang, K.-E.; Sung, Y.-T.; Chang, R.-B. & Lin, S.-C. (2005). A New Assessment for Computer-based Concept Mapping. Educational Technology & Society, 8 (3), 138-148.
Ciprian Sas, I. (1999). The multiple-choice concept map (MCCM): an interactive computer-based assessment method. Doctoral dissertation, University of Nevada, Las Vegas.
Clariana, R.B. (2009). A comparison of pair-wise, list-wise, and clustering approaches for eliciting structural knowledge. International Journal of Instructional Media, 36 (3), 287-302.
Cline, B. E.; Brewster, C. C. & Fell, R.D. (2010). A rule-based system for automatically evaluating student concept maps. Expert Systems with Applications. 37, 2282–2291.
Conradty, C. & Bogner, F. X. (2012) Knowledge presented in concept maps: correlations with conventional cognitive knowledge tests. Educational Studies, 38 (3), 341-354
Coul, R.; Clariana, R. B. & Salehi, R. (2005). Comparing several human and computer-based methods for scoring concept maps and essays. Journal of Educational Computing Research, 32 (3), 227-239.
Cravalho, P. F. (2010). Learning Statistics using Concept Maps: Effects on Anxiety and Performance. Master dissertation, San Jose State University.
Ekin, B.; Ulcinar Sagir, S. & Saltan, F. (2016). The Comparison on Evaluation of Concept Map and Structured Grid with Multiple-Choice Test. Participatory Educational Research (PER), Special Issue 2016-II; 100-111.
 Eroglu, M. G. & Kelecioglu, H. K. (2011). An analysis on the validity and reliability of concept map and structural communication grid scores. Journal of Education, 40, 210 - 220.
Hung, C. H. & Lin, C. U. (2015). Using concept mapping to evaluate knowledge structure in problem-based learning. BMC Medical Education, 15, 212. DOI 10.1186/s12909-015-0496-x.
İngeç, S. K. (2009). Analysing Concept Maps as an Assessment Tool in Teaching Physics and Comparison with the Achievement Tests. International Journal of Science Education, 31 (14), 1897-1915.
Jonsson, A. & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2, 130–144.
Kassab, S. E. & Hussain, S. (2010). Concept mapping assessment in a problem-based medical curriculum. Journal of Medical Teacher. 32, 926 – 931.
Kaya, O. N. & Kilic, Z. (2004). Student-Centered Reliability, Concurrent Validity and Instructional Sensitivity in Scoring of Students' Concept Mapsin a University Science Laboratory. Poster presented at 18th International Conference on Chemical, İstanbul, Turkey.
 Klein, D. C. D.; Chung, K. W. K.; Osmundson, E. & Herl, H. E. (2002). The validity of knowledge mapping as a measure of elementary student's scientific understanding. CRESST/University of California, Los Angeles.
 Ley, S. L.; Krabbe, H. & Fischer, H. E. (2012). Convergent validity: concept maps and test for student's diagnosis in physics. Fifth International Conference on Concept Mapping Valletta, Malta
Liping, Z.; Yan, W.; Boqing, D. & Zengyao, Z. (2009). The comparison study of Chinese and American secondary school students’ knowledge structure - an experimental research based on concept map assessment technique. Front. Educ China, 4 (2), 286–297. DOI 10.1007/s11516-009-0015-0.
 Lim, Y. K. (2008). The effect of concept mapping with different levels of generativity and learners self-regulated learning skills on knowledge acquisition and representation. Doctoral dissertation, The Pennsylvania State University.
Liu, J. (2013). The Assessment Agent System: design, development, and evaluation. Education Tech Research Dev. 61,197–215. DOI 10.1007/s11423-013-9286-5.
 Liu, X. & Hinchey, M. (1996). The internal consistency of a concept mapping scoring scheme and its effect on prediction validity. International Journal of Science Education, 18 (8), 921 - 937.
Liu, T. C., Lin, Y. C., & Tsai, C. C. (2009). Identifying senior high school student'smisconceptions about statistical correlation, and their possible causes: an exploratory study using concept mapping with interviews. International Journal of Science & Mathematics Education. 7, 791 – 820.
 McClure, J. R., Sonak, B., & Suen, H. K. (1999). Concept map assessment of classroom learning: Reliability, validity, and logistical practically. Journal of Research in Science Teaching, 36 (4), 475–492.
Nicoll, G.; Francisco, J. & Nakhleh, M. (2001). A three-tier system for assessing concept map links: a methodological study. International Journal of Science Education, 8, 863 – 875.
Novak, J. D. & Gowin, B. (1984). Learning how to learn. New York: Cambridge University Press.
Plummer, K. J. (2008). Analysis of the psychometric properties of two different concept-map assessment tasks. Doctoral dissertation, Brigham Young University.
Reiska, P.; Soika, K.; Mollits, A.; Rannikmae, M. & Soobard, R. (2015). Using Concept Mapping Method for Assessing Students’ Scientific Literacy. Procedia - Social & Behavioral Sciences, 177, 352 – 357.
Rice, D. C.; Ryan, J. M. & Samson, S. M. (1998). Using concept maps to assess student learning in the sciences classroom: Must different methods compete? Journal of
Research in Science Teaching, 35 (10), 1103-1127.
Ruiz-Primo, M. A. (2000). On the Use of Concept Maps as an Assessment Tool in Science: What We Have Learned so Far. Revista Electrónica de Investigación Educativa, 2 (1), 2000-2030.
Ruiz-Primo, M. A.; Schultz, S. E.; Li, M. & Shavelson, R. J. (2001). Comparison of the reliability and validity of scores from two concept-mapping techniques. Journal of Research in Science Education, 38 (2), 260–278.
Ruiz-Primo, M. A. & Shavelson, R. J. (1996). Problems and issues in use of concept maps in science assessment. Journal of Research in Science Teaching, 33, 569-600.
Rye, J. & Rubba, P. A. (2002). Scoring Concept Maps: An Expert Map-Based Scheme Weighted for Relationships. Journal of School Science & Mathematics, 102 (1).
Sasson, E.; Ravid, G. & Pliskin, N. (2015). Improving similarity measures of relatedness proximity: Toward augmented concept maps. Journal of Informetrics, 9, 618 – 628.
Schau, C.; Mattern, N.; Zeilik, M.; Teague, K. W.  & Weber, R. J. (2001). Select-and-Fill-in Concept Map Scores as a Measure of Students' Connected Understanding of Science. Educational & Psychological Measurement, 61, 136.
Schwendimann, B. A. (2015). Concept maps as versatile tools to integrate complex ideas: From kindergarten to higher and professional education. Knowledge Management & E-Learning: Special Issue on Novakian Concept Mapping in University & Professional Education, 7 (1), 73-99.
Stoddart, T, Abrams, R., Gasper, E. & Canaday, D. (2000). Concept maps as assessment in science inquiry learning - a report of methodology. International Journal of Science Education, 22, 1221 – 1246.
Strautmane, M. (2012). Concept map-based knowledge assessment tasks and their scoring criteria: an overview. Concept Maps: Theory, Methodology, Technology Proc. of the Fifth Int. Conference on Concept Mapping Valletta, Malta 2012.
Tsai, C. C.; Lin, S. S. J. & Yuan, S. M. (2001). Students’ use of web-based concept map testing and strategies for learning. Journal of Computer Assisted Learning, 17, 72 – 84.
Tzeng, J. Y. (2014). Mapping for depth and variety: using a “Six W’s”scaffold to facilitate concept mapping for different history concepts with different degrees of freedom. Educational Studies, 40 (3), 253-276, DOI: 10.1080/03055698.2014.889595.
Vodovozov, V. & Raud, Z. (2015). Concept Maps for Teaching, Learning, and Assessment in Electronics. Hindawi Publishing Corporation Education Research International, Volume 2015, Article ID 849678, 9 pages.
Watson, M. K.; Pelkey, J.; Noyes, C. & Rodgers, M. (2016). Assessing impacts of a learning-cycle-based module on students' conceptual sustainability knowledge using concept maps and surveys. Journal of Cleaner Production, 133, 544 – 556.
West, D. C.; Park, J. K.; Pomeroy, J. R. & Sandoval, J. (2002). Concept mapping assessment in medical education: A comparison of two scoring systems. Medical Education, 36, 820-826.
Won, M.; Krabbe, H.; Siv Ling Ley, S. L.; Treagust, D. F. & Fisher, H. E. (2017). Science Teachers’Use of a Concept Map Marking Guide as a Formative Assessment Tool for the Concept of Energy. Educational Assessment, 22 (2), 95 - 110.
Wu, P. H.; Hwang, G. J.; Milrad, M.; Ke, H. R. & Huang, Y.M. (2012). An innovative concept map approach for improving students’ learning performance with an instant feedback mechanism. British Journal of Educational Technology, 2, 217 – 232.
Yin, Y. & Shavelson, R. J. (2008). Application of generalizability theory to concept map assessment research. Applied Measurement in Education, 21, 273-291.
Yin, Y.; Vanides, J.; Ruiz-Primo, M. A.; Ayala, C. C. & Shavelson, R. J. (2005). Comparison of Two Concept-Mapping Techniques: Implications for Scoring, Interpretation, and Use. Journal of Research in Science Teaching, 42 (2), 166–184