Divide and Blunder

To continue a discussion of large courses, let’s examine one reason why they aren’t great: the counterproductive tendency to divide them into smaller pieces. (This is second in a three-part series: 1, 2.)

To recap the first post in this series—large courses should be better:

• They need to be better, because most courses students take are large, even if most courses offered are small. Large courses tend to be early and important, covering foundational knowledge that later courses rely on, and are also frequently where students who are struggling to adapt to college fail.

• They can be better, by aggregating resources to develop excellent content and materials that support all students.

But large courses are not currently as great as they should be, and one of the primary culprits is small course mythology. Many large courses fail to reach their full potential due to misguided efforts to make them appear smaller by spliting them into smaller pieces—an approach I call divide and blunder.

Imagine you launch a technology startup. You write and deploy some code. That allows you to support a certain number of users. But more users arrive, and your existing approach is no longer adequate to keep up with demand. What do you do?

And yet, this divide and blunder technique is precisely how many institutions deal with courses that get large: break them into multiple independent sections that may not share assessments, staff, systems, or even syllabi. The only thing uniting such independent sections may be a one-paragraph description in the course catalog. Everything else is left to the discretion of the instructor running each section.

Divide and blunder both fails to leverage advantages inherent to large courses, while creating knowledge consistency problems that negatively affect students and later courses.

Let’s start by examining an important implementation detail—the degree to which separate sections are functionally independent. This has a big impact on the amount of damage done by divide and blunder. There are two things to consider:

1. How many separate sections are created?
2. How independent are they from each other?

Our focus is really on the degree to which separate sections are operating independently and without coordination, but as we’ll see, dividing courses into smaller and smaller pieces tends to drive independence. We’ll use the example of a 1000-student course, and work from the reasonable end of the divide and blunder spectrum to the unreasonable end. This will also demonstrate how this kind of subdivision may happen—possibly making some amount of sense at first, but being increasingly hard to justify as we go on.

In the beginning, we just needed more space. Once courses get large, it can be hard to find a single room that fits everyone. If the course is taught in a lecture format, this can be the initial trigger for the creation of separate sections.

So imagine we divide our 1000-student course into two 500-student sections, each taught by the same instructor, but with lectures at different times. Everything else about the course is shared—the syllabus, assessments, staff, and so on. Note that this division can help make the course more available to students by reducing scheduling conflicts.

At this end of the spectrum things are not that bad. Yes, you’re wasting instructor time by presenting the same content twice. Having done this myself, it can be hard to keep multiple lectures in sync, even when the same person is delivering them. Questions or corrections from students may not propagate, particularly from later lectures to earlier ones. Useful discussions can cause one meeting to get behind relative to the others. As anyone who has tried this knows, it’s not unusual to have one section that asks a lot more questions than others, probably due to students making each other more comfortable interacting with the instructor.

Here’s where the trouble starts. Imagine we want to split this 1000-student course into 10 100-student sections—or even just 5 200-student sections. We quickly reach the point where it’s not feasible for a single instructor to provide all the lecture instruction. For example, a typical lecture-based course at Illinois might have three hours of instruction per week. Two separate sections requires six hours—which is repetitive, but not terrible. But five separate sections requires 15 hours, and ten requires 30 hours. Even the most performative chalkface droner is probably not going to go for that, and that much repetitive presentation leaves scant time for all the other things an instructor is responsible for.

Inevitably, once we start subdividing large courses into smaller sections, we quickly reach the point where we need to add instructors. That has a high potential to cause trouble, because academic freedom has entered the chat.

Let’s consider how this heads sideways. Alice has been teaching a course for a few years, but it’s grown large enough that her department decides to assign Bob as a co-instructor. Alice provides Bob with her slides, and Bob tries to adhere as closely as possible to Alice’s content and delivery in his own sections.

Unfortunately, this is not what always happens—because there’s often nothing guaranteeing Bob stays in sync with Alice, and lots of reasons they might diverge. Maybe Alice provides Bob with her slides. Or maybe she considers them her intellectual property and either refuses, or allows Bob to use them only accompanied by large and humiliating “Written by Alice” watermarks⁽¹⁾.

Even if Alice does share her slides with Bob, Bob may not want to use them unmodified. He may notice mistakes, or disagree with her approach to explaining certain things, or not understand certain examples, or prefer to sequence certain material differently, and so on. If Alice is teaching faculty and Bob is research faculty, he may approach the task with an innate sense of superiority, armed with the knowledge that of course he can do better at the craft Alice has spent years perfecting. Or maybe Bob just doesn’t like Alice, and decides that’s a good enough reason to teach the material differently. Who knows.

There’s a lot of university culture pushing Alice and Bob in separate directions, and little holding them together. Alice and Bob are both creative and innovative faculty, right? Shouldn’t they each develop their own approach to teaching the material? Isn’t this kind of independence what academic freedom is intended to protect? (No.) And if Alice (or Bob) decided to let Bob (or Alice) handle all the classroom instruction while contributing to the course in other ways, wouldn’t their student evaluations suffer? Would students even know that they were teaching the course if they didn’t see them lecturing at the front of the room? In the face of these forces driving divergence, there’s rarely anyone championing cooperation.

Even if we could get Alice and Bob to use identical slides, that alone doesn’t ensure consistency of classroom instruction. Because their verbal explanations will differ, and they’ll move at different speeds at different points, and they’ll answer questions differently, and they’ll make different mistakes—all the same sources of variation as an individual instructor lecturing multiple times, compounded by the fact that multiple instructors don’t share one brain. And forcing instructors to adopt another person’s mental model may actually reduce their individual effectiveness, compared with allowing them to explain something in a way that makes more sense to them.

When what’s taught in multiple sections of “the same” course starts to diverge—when separate becomes independent—this is the small hole that threatens to blow our entire consistency dam apart. Because it all cascades from here. Once the material starts to diverge, expect the assessments to follow—and they should, because assessments should align with how the material was taught. At that point it’s likely that the course staff will also splinter, since staff from one section won’t be well-prepared to answer questions about material taught in another, or be aware of what’s being tested and emphasized across the entire course.

Now we’re well on our journey to the other end of the divide and blunder spectrum—where multiple sections of a single course are functionally independent, and share little except that one-paragraph catalog description. At that point we’ve abandoned most if not all of the benefits inherent to large courses, and are creating a consistency mess for later courses to deal with.

Just to give you a sense of how far you can go in this direction, let me share one real-world example. I have heard of a large introductory computer science course that, in full thrall to small course mythology, is taught entirely in independent 40-student sections by dozens of non-cooperating instructors. These sections are not required to share anything: not syllabi, not slides, not textbooks, not staff, not assessments—nothing. Some sections require that students purchase expensive online textbooks, others don’t, so even the cost to students varies. Every instructor is freelancing, and new instructors are free to start from scratch.

Splitting large courses into smaller independent pieces suffers from both of the serious problems we identified in the software development analogy above. The parallels may or may not be obvious, but let’s examine them to drive the point home.

Previously we noted that the resources generated by large courses support the creation of excellent materials. A 15-week course with weekly assessments needs 15 assignments, regardless of enrollment. A course with 1000 students should have 10 times the resources to create and maintain these assignments than a course with 100 students. When independent sections maintain independent materials, divide and blunder has surrendered this key benefit of large courses. The resulting materials are mediocre, stale, or both.

When you don’t hire excellent faculty, you don’t get excellent materials. And when independent sections don’t share materials, even the better materials—assuming you can identify them—only reach a subset of students. Put more bluntly: the independent sections created by divide and blunder put subsets of students in the hands of mediocre or even poor instructors using mediocre or even poor materials. Rather than the alternative, where excellent instructors create excellent materials that benefit every student.

Equally if not more damning are the consistency problems created by divide and blunder. Returning briefly to our software development analogy. Imagine you’re an external developer trying to use an API provided by our hypothetical technology startup. Now imagine that every API call may be handled by one of several independent implementations chosen at random. Now imagine that the only information shared between the independent development teams is a single paragraph describing how the API should work—no detailed documentation, no tests, nothing like that. Now imagine trying to get anything useful done using that interface.

Note that in this discussion I’m talking about accommodating students who “did well” in the previous course, not any student regardless of their previous performance. But another facet of the inconsistency created by independent sections is that “did well” loses its meaning. For example: the course description says to cover recursion. In one section the instructor waives their arms briefly over a few slides and includes a few multiple-choice questions on the exam. In another section multiple weeks are spent on the topic with students given ample opportunities to demonstrate their understanding. Two students took “the same” course. Both earned a good grade. But did they both learn recursion?

When addressed, these inconsistencies slow down later courses, which now have to reteach large amounts of material that students have already paid once to learn. A bit of review is appropriate, but too much means too little space to cover new material. A well-run follow-on course will also calibrate on the least-prepared “successful” incoming students, meaning that a single poorly-taught upstream independent section can ruin the entire group.

When left unaddressed, knowledge inconsistencies confuse and demoralize students. Both students above think they understand recursion. But once they are challenges to apply that knowledge later, one is set up for success, the other for failure. And this may be a cue for the struggling student to think: “I’m just not good at this”, rather than realizing that they just weren’t taught that topic well.

Note that I’m certainly not arguing that consistency will address all knowledge gaps affecting later courses. We still need effective communication across course sequences to ensure that later courses understand what is and isn’t taught in prerequisites. But it’s just much easier for a later course when all incoming students have been taught the same way and share similar misunderstandings.

At this point I wouldn’t blame you for hoping there’s a silver lining here. Couldn’t multiple independent sections represent laboratories of education—experimenting with different approaches to determine which is the most effective?

In theory: maybe. In practice: never. There are just too many variables to control and, when multiple independent sections don’t share assessments, no way to evaluate the results. As much as instructors might bristle at the idea of being forced to share assessments—and despite the negative connotations that have gathered around the term “standardized testing”—that’s really the only way to even start to figure out what’s going on.

Registration priority then interacts with differences between independent sections that make any kind of analysis even more difficult. As an example: assume that Alice has a reputation for having high standards and expecting students to work hard, while Bob covers less material and grades more leniently. Also assume that the students who register earlier tend to be more well-prepared—which is a reasonable assumption given that they tend to be more senior, majoring in the subject, or participants in honors programs.

Given typical student tendencies, it’s easy to imagine most of the well-prepared students registering for Bob’s section and the less-prepared for Alice’s. Maybe this sounds cynical—but my sense is that the majority of today’s student will take the easy A over “risking” their GPA in order to actually learn something. They want that cushion that Bob’s section provides. We can imagine how student performance looks at the end of the term. So now, who’s the better instructor? Bob, who started with stronger students and demanded little of them? Or Alice, who started with weaker students and challenged them? And this can also be turned around: for example, in the case where Bob is research faculty moonlighting in the course, with unrealistic expectation and disorganized and confusing materials; and Alice is teaching faculty offering a properly-calibrated and well-run section.

Is it possible to tease out these kinds of differences to determine which section or instructor is most effective? With enough time, data and effort: sure. But this just isn’t being done regularly today. And even if you could determine which section was more effective, identifying why would be equally challenging, given that independent sections usually diverge in multiple ways.

What happens today is simpler and sadder: independent sections freely create significant consistency problems. Either the next course in the sequence corrects these disparities, or students who are left behind give up. Divide and blunder indeed.

Building better large courses requires rejecting small-course mythology and embracing what can make large courses great: the availability of resources to hire great instructors and the potential for the great materials they create to scale infinitely. In the final installment of this series, I’ll share ideas about how to design large courses so that they not only should be great, but can be.