During Spring 2020, Cignition conducted a randomized controlled trial (RCT) to assess their online tutoring solution using an approach that combined online 1:1 tutoring with a fractions game, called "Fogstone Isle." The game provided students with additional learning opportunities and tutors with the information they could use to plan tutoring sessions. The RCT investigated whether students who participate in online tutoring and a related mathematical game learn more about fractions than students who only have access to the game.
There were 144 5th-grade students that participated in the study. The students were enrolled in four school sites, all serving low-income students with low prior mathematics achievement. After taking the same diagnostic pre-test, students in each research site were paired based on pre-test scores. Each pair was randomized: one student was assigned to the treatment condition and the other to the control condition. In the treatment condition, students received tutoring and were also assigned to play the game. Students were offered 10 weeks of tutoring, twice a week, and each session was approximately 25 minutes long. In the control condition, students were able to play the game only and play was optional. In both conditions, students continued to attend their existing classrooms and received ongoing instruction on fractions.
Students who participated in Cignition one-to-one tutoring and played the "Fogstone Isle" game gained more knowledge of fractions than students who did not receive tutoring. Students who received one-to-one tutoring from Cignition scored, on average, 12 percentage points higher on the post-math assessment compared with students who only had the option to play the "Fogstone Isle" game (see Figure 1). The 12 percentage point difference between tutored participants and nonparticipants translates to an effect size of 0.46 standard deviations (Hedge's g).
Researchers found that students who received online tutoring and played the related "Fogstone Isle" game learned more. By way of meaningfulness, an effect size of 0.46 is commonly interpreted as corresponding to an additional year of instruction (Lipsey et al., 2012). If the effect size in this approach could be maintained for a whole year, students would gain as much as they would in an additional year of math instruction. As many students are a year behind their peers in math in 5th grade, this is a meaningful effect; it could allow students with weak prior knowledge to meet grade-level expectations.
During Spring 2022, Cignition conducted a randomized controlled trial (RCT) to assess their online group tutoring solution (4:1 student-tutor ratio). The RCT investigated whether students who receive group tutoring score higher on the fraction math assessment than students who do not receive group tutoring. Secondarily, we wanted to understand students' math confidence and enjoyment of math.
The study included 157 5th-grade students who were enrolled in eleven schools in one rural/suburban school district in the United States. All the students in the study were performing below grade level in mathematics and were recommended for tutoring by their teachers. Students were randomly assigned to either the treatment or control condition. In the treatment condition, students received 30-minute tutoring sessions 4 days a week, averaging 26 sessions. The control condition did not receive tutoring.
Students who participated in Cignition group tutoring gained more knowledge of fractions than students who did not receive tutoring. Students who received group tutoring from Cignition scored, on average, 9 percentage points higher on the post-math assessment than students who did not receive group tutoring (see Figure 2). The analysis supports 99% confidence that Cignition tutoring had a positive effect on student learning.
The 9 percentage point difference between tutored participants and nonparticipants translates to an effect size of 0.44 standard deviations (Hedge's g).
To interpret an effect size, Lipsey et al. (2012) recommends using the expected amount of mathematics learning from a full year of instruction as a benchmark. For 5th-grade students, the effect size benchmark is 0.41. Hence the interpretation of an effect size of 0.44 is that if low-performing students were provided a full year of Cignition group, instead of only 9 weeks, they would gain about as much in that single year as normally expected in two years of instruction. This is based on the assumption that a 0.44 effect size would be maintained if Cigniton tutoring lasted the entire year.
Furthermore, students who participated in Cignition group tutoring reported high levels of math confidence and enjoyment of math (see figure 3). Students also reported positive relationships with their tutor, with little variation by tutor. Students who participated in Cignition group tutoring reported a 4.25 average rating on questions related to their relationship with their tutor, which places them between "agreement" or "strong agreement" on average.
The findings from this study provide evidence of the effectiveness of Cignition's 4:1 math tutoring offering in increasing students' math knowledge. In addition to increased math achievement, as evidenced by the post-tutoring math assessment, students who participated in Cignition group tutoring expressed more confidence in and enjoyment of math.
The degree of effect size on math knowledge found in this study of group tutoring (4:1) was similar to the effect previously measured in a study of Cignition's individual tutoring (1:1), which suggests that Cignition's group tutoring is as effective as our 1:1 offering. As a result of the similar effect sizes between 1:1 and 4:1 tutoring, there can be a higher potential to reach more students through math tutoring in a more cost-effective way.
Impact Your Students today
Empower your students to achieve the success they’re capable of and match them with the right tutor - inside or outside of the classroom!