Blank Space (small)
(text and background only visible when logged in)

As enrollment increases, teaching assistants (TAs) need help prioritizing responding to students’ posts in on-line forums. In a graduate level Artificial Intelligence forum, three instructors rank the urgency of posts which is compared to the ratings of 13 course TAs; correlation with the instructors’ scores have an r=0.55, with a TA inter-rater reliability of 35%. However, when TAs used a codebook containing seven dimensions created by the instructors to define urgency levels, correlation increased to r=0.73 and reliability to 53%. The instructor rankings are also compared to cognitive presence ratings from the Community of Inquiry framework. Cognitive presence correlates with urgency with r=-0.68. These results suggest that recommendation agents that prioritize posts based on cognitive presence or the urgency codebook may be beneficial for TAs.

Reference

In Proceedings of the Tenth ACM Conference on Learning@ Scale (pp. 406-410).

Irish, I., Chatterjee, S., Jivani, S., Jia, X., Lee, J., Arriaga, R., & Starner, T. (2023, July).