Denver Public Schools ('22-23)
Working with Cignition has been first-rate. Their willingness to ensure alignment with our schools and ambition for change and success gives them a competitive advantage over other online tutoring programs.
Across the District, students had an absolute gain of 13.0% growth in their diagnostic scores1 and a Standards Mastery score2 of 67.5%.
For students who met our initial* three key metrics (70% attendance, 80% engagement, and 50 contact hours), their absolute gain was 19.7%, and their Standards Mastery score was 78.0%.
The number of students who demonstrated growth on ten or more standards was 51.6% higher in 2022-23 than in 2021-22 at DPS.3
*DPS requested 50 hours of instruction, but adjustments were made later because of logistics.
Cignition's Collaborative Journey with Denver Public Schools
The 2022-23 school year marked the third instance of Cignition partnering with Denver Public Schools to offer math tutoring to select students. The collaboration began with a Spring 2020-21 pilot program, followed by full programs in 2021-22 and 2022-23. Each year boasted over 1000 participants, with this year serving 1044 students spanning grades 3 through 9.
Schools were chosen based on five primary equity metrics:
- EL (English Learners) population of 25% or more
- Students experiencing homelessness at 10% of the school population or more
- FRL (Free and Reduced Lunch) students at 25% or more
- SPED (Special Education) students at 10% or more
- A focus on specific Ethnicity groups: African American population at 15% or more, and Hispanic population at 25% or more
Schools were ranked by the number of these criteria they met. Priority was given to those satisfying all five, with a descending order of preference thereafter.
In pinpointing participants, we emphasized students negatively impacted by the pandemic: students of color, EL (English Learners), and SPED (Special Education) students. Figure 1 showcases the efficacy of our student selection, with the exception of SPED students. Upon collaboration with DPS leadership, it was revealed that many of the participating schools chose for their SPED students to receive individualized intervention services instead of Cignition tutoring.
Program Standardization and Efficacy: 2022 - 2023 Highlights
Compared to the 2021-22 school year, this year's Denver program was markedly more standardized. Working with the Denver Grade Level Tutoring (GLT) team, Cignition curated a uniform program design, building on the successes of the preceding year. Although some schools maintained autonomy, student selection, group tutoring size standardization, and regularized scheduling were recurring elements.
The data demonstrates the effectiveness of those guidelines in program design. Overwhelmingly, sessions were scheduled during the school day. Only 4 of 263 groups (1.5%) were scheduled outside of regular school hours. Those four groups took place on campus before school started. Tutoring groups comprised three or four students (an average of 3.57 students per session), homogeneously grouped by math proficiency. Groups met consistently, meeting on the same days and times each week. Each group was assigned a lead tutor, who conducted all of the sessions unless they were unable to attend, in which case a substitute was employed. The lead tutors attended 90.9% of their assigned sessions. These changes led to higher levels of student engagement and improved student outcomes, as shown in Figure 2.
Data Informed Instruction
Last year, Cignition, in collaboration with the Denver GLT team, set 5 goals for improving the 2022-23 student outcomes. Four of those goals revolved around improving our use of student data to provide more targeted instruction. Those goals were:
- Teachers will be able to select standards to be covered and the time frame for those sessions to align our efforts with their classroom more closely.
- Assessments will measure the learning gains in reference to the specific standards that were covered in the student’s group.
- Assessments will provide a standards analysis so teachers and tutors can help identify misconceptions and address individual student needs.
- An improved feedback tool to accurately communicate student interim progress to their instructors.
All five goals were reached by the start of the 2022-23 school year. Teachers had the ability to pinpoint specific standards for tutors to concentrate on for each individual group. This enabled tutors to tailor their focus according to the data from classroom assessments and insights from our pre-diagnostic evaluation. Weekly student progress reports now included outcomes at the standards level, allowing teachers to make adjustments to the instruction as needed.
These changes gave us a new tool to measure student outcomes: Standards Mastery Percentage (also referred to as Standards Progress.) This assessment involved students solving representative problems and explaining their thought processes. Furthermore, tutors directly observed students' methods to arrive at their answers, ensuring a comprehensive understanding of the student's approach. From these observations, students were measured on a three-point Likert scale:
- Complete Understanding (1.0 point)
- Partial Understanding (0.5 points)
- No Understanding (0.0 points)
The program's effectiveness was measured by comparing student engagement metrics against our Standards Progress. The hypothesis was that as students were more engaged, they would demonstrate progress by mastering a larger percentage of the standards they worked on. Our key engagement metrics were attendance (70% was our standard), overall engagement (80%, as measured daily by the tutors, was our standard), and contact hours (25 hours was our standard.) In Figure 3, the Standards Progress score of students who met none, one, two, or all three standards was averaged. As expected, more engaged students mastered a higher percentage of the standards they engaged.
In the second year of the program, I can see a deeper conceptual understanding in the kids.
To learn more about what sets Cignition apart and the impact it has had on Meg and students at Place Bridge Academy, please click the video below.
As we head into our fourth year in partnership with Denver Public Schools, we continue to look for ways to improve student outcomes. Our goals for the upcoming year include:
- Report Standards Progress in a more detailed manner to communicate student progress to teachers and parents more effectively.
- Make Standards Progress reporting more prominent and tied to assigned standards clusters on the data dashboard.
- Individualize daily tutor comments for each student to more effectively communicate student progress.
We look forward to continuing our partnership with Denver Public Schools for the 2023-2024 school year.
For a detailed data summary, compiled of items such as Student Engagement Metrics, Standards Progress, and much more, click the button below and download the complete data report.
1 Absolute Gain score is calculated by subtracting the percentage correct on a diagnostic test administered near the end of the series of sessions from the percentage correct on an identical diagnostic administered near the beginning of the series of sessions. (Absolute Gain = Post-Diagnostic Score - Pre-Diagnostic Score)
2 Standards Mastery score is calculated from the student progress measured by the tutor in each session. Students are given a formative assessment from the standard addressed in that session. In addition to assessing whether the answer is correct, the tutor also requires the student to explain how they arrived at the answer, taking into consideration the process demonstrated in obtaining the answer. Student progress is measured with a three-point Likert scale”
- Complete Understanding (100%)
- Partial Understanding (50%)
- No Understanding (0%)
The Standards Mastery score is the average of these scores. In essence, it is the percentage of standards attempted by the student that were actually mastered.
3 Cignition’s goal for each student is to see them demonstrate conceptual mastery of each concept attempted. Using formative assessments, tutors measure student progress and use that data to inform instruction. As described in note 2, student progress is measured and recorded each session by tutors. The measurement “demonstrated growth” indicates that the student moved from “No Understanding” to “Partial Understanding” or “Complete Understanding”.
Impact Your Students Today
Cignition's pool of expert math tutors are ready to help your students succeed, inside and outside of school time. Learn more about incorporating Cignition tutors into your classroom and after school programs