About the Study
This study examined the use of Motivation Math in 3rd, 4th, and 5th grade classrooms in multiple states. Motivation Math is an educational program developed by Mentoring Minds with a focus on critical thinking and problem solving.
SEG Measurement, an independent research firm, conducted a study of the effectiveness of the use of Motivation Math in mathematics classes from January through June 2013. The primary goal of the study was to determine the extent to which Motivation Math users achieved more in mathematics than students who did not use Motivation Math materials. In addition, the study collected information regarding use and feedback regarding the materials.
About Motivation Math
Motivation Math is a supplemental resource, aligned to the Common Core Standards, which complements and supports any mathematics curriculum. It provides focused practice with the depth and complexity needed to prepare students for the rigor of standardized testing. The Motivation Math Student Edition contains questions reflective of the Common Core Standards and is designed to improve students’ problem-solving capabilities. Critical thinking is incorporated to extend mathematical reasoning and concept development. The Motivation Math Teacher Edition is packed with information and instructional strategies that promote sound mathematics instruction. Each unit focuses on a specific Common Core content standard and helps teachers plan, deliver, and assess effective instruction for the standard while incorporating appropriate standards for mathematical practice.
Study Design and Participants
Over 300 students were included in the study across grades 3, 4, and 5. The study incorporated a pretest and posttest design with Treatment and Control Groups. The Treatment Group was composed of students in classes that used Motivation Math. The Control Group was composed of students in comparable schools that did not use Motivation Math. Students across grades 3, 4, and 5 were included in each of the study groups.
Mathematics skill growth of students using Motivation Math (Treatment Group) was compared to students who did not use Motivation Math (Control Group). The Stanford 10 Mathematics Total Test was used as the pre and post measure, and was administered in January and June 2013 to all of the students. The Stanford 10 Mathematics Test includes two sections with a total of 50 multiple choice items. The test results are vertically scaled allowing for comparison within and across grades.
Students who completed both the pretest and the posttest were included in the analyses. The comparability of the pretest ability of the two groups was evaluated to ensure the posttest data could be compared. Nearly 240 students had both pretest and posttests as summarized in Table 1.
The teachers included in this study ranged in experience from first year teaching to more than 10 years of experience, with the majority having between 5 and 9 years of experience teaching mathematics. Schools participating in the study were located in rural, urban, and suburban areas across five states. The majority of the classes contained between 21 and 30 students.
Analysis and Results
After ensuring that the Treatment and Control groups were initially comparable in ability, an Analysis of Covariance was used to evaluate the posttest performance between the Treatment Group and the Control Group. The June 2013 Stanford 10 Scaled Total Math score was the dependent variable, the Study Group was the fixed factor, and the January 2013 Stanford 10 Math Scaled score served as the covariate. Using the pretest score as the covariate allows for a comparison of posttest scores with adjustment for any initial difference in ability between the two study groups.Students who used Motivation Math were found to have a higher mean posttest ability level than the students who did not use Motivation Math (F = 5.437, p < 0.05). Students in the Treatment Group grew 17.84 points compared to the Control Group growing 10.14 points.
Students who used Motivation Math grew significantly more in mathematics skills in just 6 months than students who did not use Motivation Math.The effect size of the difference found in this study was .17. For a student at the 50th percentile, this reflects an increase of 7 percentile points, or an increase the to 57th percentile between January and June 2013.
Throughout the study and at the conclusion of the study, the teachers in the Treatment Group provided feedback regarding their usage of the Motivation Math materials. On average, the teachers used Motivation Math with their students about one and one half hours each week. Not every teacher used Motivation Math each week due to school breaks.
The most commonly used components used within the Student Edition were the Introduction, Partner Practice, Independent Practice, Assessments, and Critical Thinking. The least used components were the Journal, Motivation Station, and Parent Activities. The most commonly used Teacher Edition components were the Standards, Getting Started, Vocabulary, Suggested Instructional Activities, Interventions, and the Assessment Answer Keys.
“The questions were very relevant to what we were learning in class. I also liked the types of questioning and the higher level thinking that was required.”
– Grade 5 Teacher
The least commonly used Teacher Edition components were the Literature Connection, Performance Task Assessments, and Reflection Tasks. The teachers who used Motivation Math reported how well they felt Motivation Math helped improve their students’ skills as summarized in Table 3.
The teachers indicated that the Motivation Math materials were aligned to the Common Core. In addition, 100 percent of the teachers were satisfied with Motivation Math’s coverage of knowledge gaps for each Common Core standard.
Over 70 percent of the teachers indicated that their formative assessment improved as a result of suggestions in the Motivation Math Teacher Edition. Twenty nine percent of the teachers indicated they increased the amount of formative assessments in their classroom as a result of Motivation Math activities.
“I really like the way that the book is aligned with the standards that we are teaching within the classroom.”– Grade 4 Teacher
Conclusion and Next Steps
Students who used Motivation Math showed greater gains in mathematics skills than students who did not use Motivation Math.
Teachers who used Motivation Math as a supplemental tool found it to be effective and would recommend the resource to others.
A study will be conducted during the 2013 – 2014 school year to evaluate the effectiveness of a full year of Motivation Math usage on students’ mathematics skills. Given the amount of growth found in student achievement in this half-year study, it is anticipated that Motivation Math will be found to produce even greater gains after a full year of use.