Measuring conceptual understanding in randomised controlled trials: can comparative judgement help?
An impediment to conducting high-quality quantitative research studies in education is the paucity of valid measures of learning gains. Studies often seek to investigate students’ deep, conceptual understanding yet many measures assess only surface, procedural understanding. One reason is that the d...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Default Article |
| Published: |
2019
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/2134/37721 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| id |
rr-article-9367286 |
|---|---|
| record_format |
Figshare |
| spelling |
rr-article-93672862019-06-01T00:00:00Z Measuring conceptual understanding in randomised controlled trials: can comparative judgement help? Ian Jones (1384110) Marie-Josee Bisson (7157723) Camilla Gilmore (1256451) Matthew Inglis (1384290) Other education not elsewhere classified Assessment Comparative judgement Quantitative research methods Mathematics education An impediment to conducting high-quality quantitative research studies in education is the paucity of valid measures of learning gains. Studies often seek to investigate students’ deep, conceptual understanding yet many measures assess only surface, procedural understanding. One reason is that the development of validated measures of conceptual understanding is resource intensive, time consuming, and success is not guaranteed. We evaluated a novel and efficient technique, based on comparative judgement, for assessing conceptual understanding. We applied the technique to a randomised controlled trial in which students were taught simple algebra based on either the Grid Algebra or the MiGen software package. The participants were Year 5 students (N = 188) drawn from four primary schools who had not encountered algebra previously. An instrument from the literature (Concepts in Secondary Mathematics and Science: Algebra Scale), and a novel comparative judgement assessment were administered following the intervention. Students in the Grid Algebra condition outperformed those in the MiGen condition on both post-test measures. The comparative judgement technique performed similarly to the standard instrument but was far more efficient to design and implement. The technique can, in principle, be quickly applied to any target concept of interest. We conclude that comparative judgement is a valid, reliable and practical tool that could help to increase both the quantity and quality of quantitative research in education. 2019-06-01T00:00:00Z Text Journal contribution 2134/37721 https://figshare.com/articles/journal_contribution/Measuring_conceptual_understanding_in_randomised_controlled_trials_can_comparative_judgement_help_/9367286 CC BY-NC-ND 4.0 |
| institution |
Loughborough University |
| collection |
Figshare |
| topic |
Other education not elsewhere classified Assessment Comparative judgement Quantitative research methods Mathematics education |
| spellingShingle |
Other education not elsewhere classified Assessment Comparative judgement Quantitative research methods Mathematics education Ian Jones Marie-Josee Bisson Camilla Gilmore Matthew Inglis Measuring conceptual understanding in randomised controlled trials: can comparative judgement help? |
| description |
An impediment to conducting high-quality quantitative research studies in education is the paucity of valid measures of learning gains. Studies often seek to investigate students’ deep, conceptual understanding yet many measures assess only surface, procedural understanding. One reason is that the development of validated measures of conceptual understanding is resource intensive, time consuming, and success is not guaranteed. We evaluated a novel and efficient technique, based on comparative judgement, for assessing conceptual understanding. We applied the technique to a randomised controlled trial in which students were taught simple algebra based on either the Grid Algebra or the MiGen software package. The participants were Year 5 students (N = 188) drawn from four primary schools who had not encountered algebra previously. An instrument from the literature (Concepts in Secondary Mathematics and Science: Algebra Scale), and a novel comparative judgement assessment were administered following the intervention. Students in the Grid Algebra condition outperformed those in the MiGen condition on both post-test measures. The comparative judgement technique performed similarly to the standard instrument but was far more efficient to design and implement. The technique can, in principle, be quickly applied to any target concept of interest. We conclude that comparative judgement is a valid, reliable and practical tool that could help to increase both the quantity and quality of quantitative research in education. |
| format |
Default Article |
| author |
Ian Jones Marie-Josee Bisson Camilla Gilmore Matthew Inglis |
| author_facet |
Ian Jones Marie-Josee Bisson Camilla Gilmore Matthew Inglis |
| author_sort |
Ian Jones (1384110) |
| title |
Measuring conceptual understanding in randomised controlled trials: can comparative judgement help? |
| title_short |
Measuring conceptual understanding in randomised controlled trials: can comparative judgement help? |
| title_full |
Measuring conceptual understanding in randomised controlled trials: can comparative judgement help? |
| title_fullStr |
Measuring conceptual understanding in randomised controlled trials: can comparative judgement help? |
| title_full_unstemmed |
Measuring conceptual understanding in randomised controlled trials: can comparative judgement help? |
| title_sort |
measuring conceptual understanding in randomised controlled trials: can comparative judgement help? |
| publishDate |
2019 |
| url |
https://hdl.handle.net/2134/37721 |
| _version_ |
1799634339358572544 |