A Help Site Story

When the pandemic hit, we scrambled to create an effective approach to providing individual tutoring online. The process also resulted in some new insights about how to build community in CS1, and thoughts on the challenge of increasing diversity in our field.

Note that this is the second in a⁽¹⁾ series of essays describing my course systems and infrastructure.

I had planned to publish this years ago. At this point it also functions as a bit of a retro diary of the early years of the COVID-19 pandemic. As a told someone recently, I essentially went into fight-or-flight emergency education mode in early 2020 and didn’t emerge for at least a year. That was a tough period to live through, but it feels like enough time has passed to make it safe to revisit.

March 2020. It was the week before spring break when I started to make adjustments to my class to accommodate the approaching pandemic.

Students left for break a few days later, uncertain if they would return. In all their wisdom, our administration decided to wait until the middle of the break to infom students that campus was closing and we would transition to virtual instruction for the remainder of the spring semester. As a result, many students had to add an extra round-trip back and forth to campus to retrieve their belongings before settling at home. Nothing like some extra unnecessary travel during a global health crisis.

In fairness, there was a lot of difficult decision making going on at that time, accompanied by some fairly questionable messaging. But it was clear to me and at least a few of my colleagues which way things were headed. Suzanna and I canceled a planned trip to Seattle to meet a new niece. And I suddenly had a week free to prepare for the rest of what promised to be a challenging semester.

I anticipated that one of the biggest challenges to transitioning to remote instruction would be finding ways to support students.

At the time, my CS1 course normally ran in-person drop-in tutoring hours 7 days a week for long stretches—usually at least eight hours starting midday and running into the early evening. Tutoring hours were held in a basement room in our computer science building. Siebel 0403 is an ugly windowless room tucked into a far corner of the basement⁽³⁾. But I fought long and hard to obtain exclusive access it for my course. It seats forty students—5 times more than the space we were using previously. And it was all ours. My amazing course staff transformed it into a welcoming space for students who needed help. Just come by: No appointment needed. Find a seat, open your laptop, and a staff member will be by shortly.

Students and staff both get a lot of different benefits out of a drop-in tutoring space. For staff, it helps create a sense of community and facilitates them learning from and supporting each other. If one gets stuck on a problem, they can easily ask a colleague for help. Usually we’ll have a few more experienced staff around prepared to help with trickier questions. For new staff, working side by side helps them learn to maintain a positive and professional demeanor when working with students, who can understandably become frustrated as they struggle with their code. We’ve all been there.

Obviously students come to our tutoring hours to get help—and do. But they also benefit from overhearing as staff help other nearby students, and I suspect they absorb a fair amount of knowledge that way as well. While they are there they also interact with other students, which can help create social connections. And there is definitely some sense of comfort and togetherness in shared struggle, knowing that you are not the only one having a tough time with the assignment. Other discussions may involve sharing tips or bits of advice. Tutoring hours represent a safe space for this kind of thing, since it occurs under the watchful eye of the course staff, who can help guide students away from interactions that might violate course collaboration policies.

But there are some things that even great technology can’t replicate. Moving lectures and quizzes online was going to have a negative impact on the course, for sure. But there are so many different ways that in-person tutoring creates a positive environment for students and staff. This is why I fought so hard to get us that space in the first place. Losing our tutoring space was going to hurt. So I decided to try and address that first.

Now comes the part where I describe the technical solution that I created. And you say: “Wait, you just told me that technical solutions couldn’t replicate the sense of community created by an in-person tutoring space!” You’re not wrong. But a crisis is looming, and we’re doing triage, prioritizing work on things that we hope will have the greatest impact.

I decided to focus on two things. First, making sure that students could get help easily. That’s the ultimate point of our tutoring space, all of the positive wraparound effects notwithstanding.

My second goal was preserving the ability of staff to operate office hours fairly and efficiently. We focus a lot on this when we train staff, and specifically on avoiding what I call help queue meltdown.

Help queue meltdown occurs when a positive feedback loop forms between the number of students waiting for help and the amount of time staff spend helping each student. As students spend longer waiting for help, they naturally expect more help once they reach the front of the line. But as staff spend more time helping each student, the line slows down, and students spend more time waiting. And then expect more help. And off we go!

This can get very ugly, very quickly. Reports of multi-hour wait times are not unusual for some of our larger courses—either because they don’t effectively manage this dynamic, or because they just don’t have enough staff to do so.

The only way to solve this problem is for staff to recognize and steer away from this situation. First, having enough staff on hand is a hard requirement. Once the student to staff ratio gets too high, no amount of situational awareness will help.

This also avoids another huge problem with office hour queue meltdown. Not only is the line moving more and more slowly. But as student help sessions get longer, staff efficiency also drops. Because pretty soon you are sitting there watching the student follow your advice, rather than rotating on to the next problem.

We train our staff to avoid this dynamic. Check in, help the student get unstuck, provide some guidance about what to do next. Don’t forget to include some encouragement and emotional support. Then keep moving. This also promotes active waiting, where students are not just sitting there passively waiting for help from staff, but continuing to keep their feet moving—either incorporating advice from a previous course staff member, or working in anticipation that their next help session is not going to consist of a staff member solving all of their problems.

Training staff to work this way is challenging! Students implore them not to leave. And it’s satisfying to stick around and see your advice work out, or just take a breather before moving on to the next vexing question. But if staff can rotate effectively, we can keep wait times down, keep staff efficiency up, and help all students make forward progress, together. It’s hard, but it works.

As it turns out, having a tutoring space also helps avoid help queue meltdown. When sharing physical space, students and staff have a shared sense of how busy things are. For staff working a crowded room, many students waiting gives their work a sense of urgency, and helps them not linger too long with any one person. Students waiting for help in a crowded room naturally expect less help than when more staff are available. Both of these contextual calibrations help avoid help queue meltdown.

As you would expect, tutoring hours can get quite busy before deadlines. We encourage staff to prioritize students differently at that point. It’s appropriate for them to focus more on students who have started on time and are nearly done, and resist the urge to try to save students who waited until the last minute—and probably aren’t going to finish regardless of how much help they receive.

So our online help system had two primary design requirements. First, allow students to easily receive individual help. Second, encourage staff to work efficiently and avoid help queue meltdown—even while lacking the normal situational awareness that arises naturally in a physical space.

Over Spring Break 2020, I rapidly cobbled together a prototype of the help site that we’ve continued to use to this day. It’s a browser-based tool that uses the Jitsi as a videoconferencing backend. The first version was a standalone web application built using a React frontend, a node.js backend, and a mixture of a web API and websockets for real-time communication. Subsequent versions are architected similarly but have been integrated directly into our course website, which has undergone its own complete transformation since Fall 2020.

I was lucky that I had acquired some familiarity with React by using React Native on an ill-fated attempt to create a cross-platform smartphone app—which, it turns out, was also intended to help us manage tutoring hours. You never know when the technical skills that you acquire will come in handy. (Not that I didn’t think that React wouldn’t be useful someday. It’s a common tool used to build interactive websites.)

I decided to use Jitsi over other options like Zoom for several reasons. First, I couldn’t quickly figure out Zoom’s API, specifically whether it was possible to embed a call into a webpage. This was clearly possible using Jitsi and there was documentation explaining exactly how to do so. Jitsi is also free, open source, and something that we have at times hosted on-site⁽⁵⁾. At the time Zoom was also being trotted out as the solution to all kinds of videoconferencing use cases it didn’t seem well-suited for, so I wanted to explore an alternative in case I needed it for something else.

Note that at this point (circa 2023) we’ve moved away from self-hosting Jitsi and are running our help site on top of their freely-available service. We don’t generate that much load, and this seems to help with connectivity issues, particularly when working with students who are far away or behind strange networking setups.

Embedding the video conferencing window into another page allows our help site to provide additional contextual information surrounding the active call that helps us avoid help queue meltdown. Students are shown their position in line, both to help them prepare to receive help and give them a sense of how many other students are waiting. Staff are provided with more detailed information, including maximum current waiting time, a history of interactions with the student that they are working with, and a count-up timer that measures the amount of time that they have spent with the current student. When not joined to a call, staff can view the entire queue and the status of each request.

All of this information is intended to act as hints and nudges to staff, not as hard constraints. We don’t shut down calls after a fixed amount of time, and staff are free to choose any students from the queue to work with next and not required to select the student who has been waiting the longest. Staff working the help site are frequently communicating via a back channel, allowing them to exchange notes or vector additional assistance to students that might need help from a more experienced staff member.

Overall this system has worked well since we deployed it three years ago. There have been bugs. One caused students to disappear from the queue during portions of the Fall 2020 semester. Worse yet, it would only affect students who had been waiting for a while. This was understandably super frustrating for students. I should have dropped everything and fixed it. At the time I was hesitant to work on it, partly because I was worried about further breaking a system that was in continuous use and seemed fairly stable. But also because I was exhausted. Still, I should have taken these reports more seriously and addressed them sooner. If you’re a student and this affected you, I apologize.

We did fix this for the Spring 2021 semester and past that point the system has worked more reliably. There was a second bug where students would appear to still need help but didn’t, but unlike the Fall 2020 bug, this was an error in a better direction. It was eventually fixed, and again by the stupid programmer who introduced it. (Me.) And we ran into an additional set of issues when Jitsi’s own participant counting feature broke, which trigged some odd behavior on our end. That has also been fixed.

The help site has proven a popular and valuable way for students to receive assistance. 80% of Spring 2021 students used the help site at least once, and 131 staff responded to at least one help request. In total staff ran 11,138 help sessions providing a total of 1,078 hours of assistance, or an average of 10 hours per day over the semester. The median help session length was 4 minutes, reflecting the fact that the tool is a great way to get quick questions answered. But 10% were longer than 14 minutes, demonstrating that staff can still spend longer supporting students when appropriate. Keeping calls short to avoid queue meltdown is only critical when the line is actually long, and during quieter moments staff are free to hang out with students as they work through their question or problem.

Working together, my help site staff have done a great job at avoiding queue meltdown. In Spring 2021, the median wait time was under a minute, and only 10% of waits were longer than 38 minutes. Lines can still get long around deadlines, something that we’ve taken other steps to address in later semesters.

At this point we’ve also resumed in-person shoulder-to-shoulder tutoring hours as well. My department made a very smart move⁽⁶⁾. Using the pandemic as an opportunity to rethink aspects of how we were using space, we decided to replace several underutilized computer labs with a deparmental tutoring center that is now shared by many of our courses.

Returning to our campus “meatspace”⁽⁷⁾ has allowed us to recapture the benefits associated with in-person tutoring: the sense of community for both students and staff, the opportunities for overhearing and students helping each other, and just the buzzy positive energy of a room full of people engaged in a frequently difficult but ultimately worthwhile struggle. Some staff and students are really enjoying the chance to work side-by-side again.

I also have no qualms about providing unlimited support and assistance. I point this out because I’ve heard anecdotally about other CS1 courses that don’t provide continuous support because they claim that students will become reliant on course staff and unable to work independently. We see this happen in my course, but only to a tiny handful of students each semester⁽⁸⁾. Definitely not enough to outweigh the benefits of removing barriers and encouraging students ask small questions and get minor doubts cleared up before they become serious confusions.

I do worry about courses that see the high rates of support seeking that may indicate support dependency. When a ton of students have a question about the same thing, my reaction is usually to go back to our online textbook and shore up that topic. So one observation is that overuse of staff resources can be a sign of poor materials. We have certainly observed certain types of help-seeking activity decline as we’ve developed stronger primary materials—even as students continue to demonstrate their understanding of the same concepts on assessments. Support dependency can also arise from an unprepared student population, poor course structure, or unrealistic course expectations. Regardless of the cause, it usually seems like a hint that something’s wrong.

One of the more interesting tradeoffs between online and in-person⁽⁹⁾ tutoring is how the way that students receive help intersects with identify formation within computer science.

We frequently focus on ensuring that a course provides a positive and healthy community for students and staff. But, while a course community is great for students that feel comfortable participating in it, not every student does, and doing so shouldn’t be a requirement for getting help. My staff and I are 100% committed to providing a positive and supportive atmosphere for everyone who wants to start their journey in computer science. We stress this over and over with staff—that their main role is to provide emotional support, not technical assistance. We’ve also made strides in improving gender and other types of diversity among my staff.

But the unfortunate reality is that students from diverse or underrepresented groups aren’t going to find many people like them at our tutoring center. Computer science can also have a certain dorky vibe to it that might not agree with everyone—even students that do at least outwardly resemble the group you’ll find in our tutoring center on any given afternoon. And then there’s all those students who are just introverted—and who might find a packed room of students terrifying or exhausting.

I’m not sure how this looks at a curricular level. Students do need to learn to work together. And we can’t hide the demographics and culture of computer science from them forever. So I’m not sure of the right way to balance individual skill building versus experience in teamwork and managing group dynamics. I do have the sense that we typically have students working together too much too soon, before they develop the individual confidence required to hold their own in a group—particularly among peers who might doubt their abilities. But this is a topic for another time.

At the same time, there is awareness of the negative dynamics that can arise between beginners and non-beginners and between over- and under-represented groups in introductory courses. A common “solution” is to hustle the more experienced students out of CS1—which is not achievable nor always appropriate. I’m also not sure it works. Some of the most obnoxiously overconfident students are ones who really need to take my course, since their attitude is a front erected to hide internal anxiety and confusion.

But if some students are turned away from learning computer science because of the community, maybe part of the solution is just not require immediate participation in the community to learn computer science? This does not mean that we give up trying to create a more inclusive community. And it also doesn’t mean that we don’t support these students in other ways. It just means that maybe we continue to provide private one-on-one tutoring, instead of requiring that they pack shoulder-to-shoulder into a shared space. Maybe learning computer science shouldn’t require immediately becoming a computer scientist.