Overview
In a face-to-face course, an instructor often promotes student retention and success by assessing students’ engagement or participation in the course. Are students showing up to class? Are they taking notes? Participating in discussions? Are assignments being turned in on time? Have students read the material? Do they look confused? Are they awake? While some of these cues are observed behaviors, many can also be measured in an online course.
Learning Analytics (LA) can be used by instructors to analyze students’ online behavior including their engagement with course content, the instructor, and their peers. LA can be utilized to make informed decisions about the instructional materials and content to include in an online course. It can also be used to provide improved feedback to students by analyzing engagement rates and assignment data (Caspari-Sadeghi, 2022). New Analytics in Canvas features course-wide and individual student analytics reports for published courses. These reports can provide a snapshot of how and when the system is being used, when submissions are taking place relative to set due dates, and what student achievement looks like in terms of scores.
In Practice
New Analytics can be accessed from the Course Home Page by clicking the “New Analytics” button in the Sidebar. By default, the average course grade percentage for all students will be displayed at the top. Four main sections are included to filter the analytics: Course Grade, Weekly Online Activity, Students, and Reports.
COURSE GRADE | The “Course Grade” tab provides both the average and the point distribution for each activity. By default, all assignment types (assignments, discussions, quizzes) will be displayed. These can be unchecked to display analytics for a specific assignment type. For each assignment, analytics are provided on the average score, lowest and highest scores, number of missing assignments, and number of late assignments. These analytics can be shown as a graph or a data table then filtered by section, by individual students, or by assignment. This tab also has a “Message Students Who” feature that permits the instructor to send a group message to all students who are missing a given assignment, those who submitted the assignment late, or those who scored within a particular range.
An instructor may wish to consider the activities that deviate from a normal distribution. Using the filters, the course average can be compared with individual assignments, course sections, or individual students. Was the assignment too challenging or not challenging enough? Were the objectives, content, and assessment all in alignment? When considering the data, be wary of late assignments for which you have not yet entered a grade of zero. The distribution only features scores that have been entered and could be misinterpreted for that reason.
WEEKLY ONLINE ACTIVITY | The “Weekly Online Activity” tab reports the number of page views and participation (posting to a discussion, submitting an assignment or quiz, joining a conference, etc.) by date and by page. The weekly analytics can be shown as a graph or a data table then filtered by section or by individual students. This tab also has a “Message Students Who” feature that permits the instructor to send a group message to all students who are viewed or didn’t view a specific resource or who participated or didn’t participate on a specific assignment.
An instructor may look for and consider trends such as greater activity as a due date approaches. An instructor may also notice a week where activity is significantly lower and wish to add an element to motivate students to interact with the content or fellow classmates. Unlike previous versions of Analytics in Canvas, activity on a mobile device is now recorded and included in the data. When looking at the data, take into consideration the time students may have spent with content or activities taken outside of the system (for example, downloading and reading a PDF or working on a project using an external program or website) which may not be included in the analytics.
STUDENTS | The “Students” tab gives analytics for individual students. The data for each individual student includes current course grade, percentage of assignments turned in on or before the due date, the date of last participation, date of last page view, number of page views, and number of participations. Data can be filtered by section or by individual students. This tab also has a “Message Students Who” feature that permits the instructor to send a group message to all students who are missing a given assignment, those who submitted the assignment late, or those who scored within a particular range.
An instructor may consider this data when determining which students are low scoring versus high scoring and why. Is it because not enough time is being spent in the course? Is it because many assignments are being turned in late? Or is there another reason (for example, a disconnect between the content and objectives) that may not be reflected in this data?
REPORTS | The “Reports” tab allows the instructor to run a report that generates a CSV file with near real-time data. A report can be run for missing assignments, late assignments, excused assignments, class roster, or course activity. Missing assignments, late assignments, and excused assignments reports can be filtered by assignment, by student name, or by section name. Course activity reports can be filtered by date, by student name, or by section name.
Instructors may use these reports when looking to download course analytics rather than view them in the graph or table view. When analyzing report data, it is important to note that data may be delayed by 24 hours. Data for a Course Activity report may be delayed by 40 hours and will only show data for the past 14 days.
For more information on when to monitor analytics, see the Canvas Analytics Checklist.
References
Caspari-Sadeghi, S. (2022). Applying learning analytics in online environments: Measuring learners’ engagement unobtrusively. Frontiers in Education, 7. https://doi.org/10.3389/feduc.2022.840947