Repl.it during Spring 2020

Carleton is on a trimester system. My spring term normally starts at the very end of March. So unlike most schools that had to suddenly pivot mid-semester to going online, we were able to do our entire spring term online from the start. We also started our term a week later than usual, in part to give us time to prepare. Many of the decisions I made were done very quickly, and with less research than I would have liked. Nonetheless, I was still pretty happy under the circumstances with how the technology stack worked out.

People reading this blog entry are presumably more interested in the tools that I used rather than the curriculum itself, but a bit of context helps nonetheless. During a normal offering of this course during a 10 week term, students spend the first 2.5 weeks learning how to do functional programming in Scheme. They then learn how to program in C, and do about 1.5 weeks doing various targeted exercises in C to bring them up to speed. Finally, they spend the remainder of the term building a Scheme interpreter in C.

In the past, I used DrRacket for Scheme programming. It has generally worked pretty well for the course. I like that it is easy to use, works well across multiple operating systems, and contains a solid debugging system. I had moved away from the Racket dialect of the language in particular, however, as it did a number of optimizations that sometimes resulted in hard-to-explain results from the context of building a compatible interpreter. So I was using the r5rs dialect.

For C programming, I was using the clang compiler, with Valgrind to help catch memory errors.

I made the choice to go with Repl.it very quickly, during a brief departmental discussion after we learned that we'd be going online for the spring. My critical use case was that I wanted people to be able to remotely collaborate on code. I hoped that I would be able to preserve some aspect of pair programming, though I had no idea if it would succeed. I also hoped that I would be able to collaborate myself with students during office hours, so that I could help them diagnose problems. Repl.it offers "multiplayer" capability, as it refers to it, which allows multiple people to see a shared code editor and a terminal window. Just like a Google doc, multiple collaborators can simultaneously edit and see each other's changes.

Given how remotely we'd all be working, I was also worried about challenges in getting students computers set up with the right software. And I knew that not all of them had access to a computer in which they could install anything, and some would be using tablets (or even phones, yikes). I hoped to find an environment that would run completely in a browser, which Repl.it does.

The other factor driving my choice of Repl.it was that it was free. The pricing model for Repl.it at the time I chose it was that the free version gives you public repls, but not private ones. This is problematic, as I didn't want students' work to all be publicly viewable. However, Repl.it offered a free upgrade to their Hacker plan via the GitHub student developer pack, which is free for any student with a .edu email address. This upgrade allowed students to make their repls private. So, the set of instructions I gave my students looked something like this:

1. Sign up for GitHub
2. Sign up for GitHub Student Developer Pack
3. Sign up for Repl.it, making sure to use the GitHub account from step 1 above to get the free upgrade.

This mostly worked fine, as far as I know. One frustrating aspect was that you can't configure Repl.it to make repls private by default; every time a student creates a new repl, they need to hit a toggle switch to do so. Undoubtedly, some students forgot to do so. I didn't check, as I decided that in the emergency situation I was in, close enough was good enough.

I also had one student who couldn't get the GitHub Student Developer Pack quickly because of some historical mixup involving his GitHub account. He had some kind of problem previously with his school-related .edu account involving two-factor authentication, which left his GitHub account in a state where the automatic approval for the Developer Pack didn't work. I shrugged and told him to keep his repls public until they approved it. I assume that it eventually got approved, but I never went back to check.

I've been a bit behind the curve in using autograding for my assignments. I've been slowly moving in that direction, but going online this spring rapidly forced me there. Since communication would be more difficult between me and my students, I wanted them to be able to tell in a more automated fashion whether or not their code was running correctly. So naturally, I first looked to autograding features within Repl.it.

Repl.it offers this as part of Repl.it Classroom. It's pretty slick and seems to hold some promise. However, I decided against using it for two reasons:

1. Repl.it Classroom doesn't support mutiplayer, which was the key reason I chose Repl.it in the first place.
2. The interface for autograding required that I use a GUI approach for entering the tests. I've got a lot of tests for my code for grading, which I've built up over years, and I use them anew every time I reteach the class. I wanted something scriptable so that I wouldn't have to redo lots of data entry every time. It didn't look like Repl.it Classroom supported that well.

Next, I considered using autograding with GitHub Classroom. I had already chosen to use GitHub Classroom anyway to handle distribution of starter code and work submission, so it would be nice if I could use it for the autograding as well. GitHub Classroom had just announced a new autograding feature, so I took a look. It looks quite nice, but it also suffered from problem #2 above. However, experimenting with it helped me realize that GitHub Classroom autograding was really just a fancy interface for GitHub actions. So I instead went to GitHub actions instead to run the tests, which worked great. I configured the repos that automated tests would run whenever students pushed to GitHub, and then students could see whether or not the tests passed. Unfortunately, this didn't connect automatically back to a gradebook or anything like that, so it wasn't really autograding but connecting the automated results back to grading was pretty fast (especially with student graders to help me.)

As far as I can tell, Repl.it has two different Scheme environments installed and ready to go: one is obviously there; the other is more buried. Of course, I ended up with the more buried one, as I'll explain.

The obvious one is visible right when you create a new repl. Repl.it gives you the choice as to what language you want, and Scheme is one of the options. When you do this, you get a repl built around BiwaScheme, which is a Scheme interpreter written in JavaScript. It's pretty snazzy; BiwaScheme runs Scheme code right in your browser, in a virtual terminal window. I can see why Repl.it chose to place it up front. I really hoped to use BiwaScheme for my class, but decided against it because it didn't have (what I considered to be) usable tracing or debugging. There is a tracer, but it would be very difficult for a student learning Scheme for the first time to use. A further challenge is that the tracer doesn't seem to be accessible from Repl.it directly. And finally, with the limited time I had, it was unclear to me how I could set up an automated testing framework at the command line around BiwaScheme. Presumably I could have with Node.js, but I was pretty concerned about sinking a bunch of time into and discovering it wouldn't really work.

The other Scheme environment that's hidden away on Repl.it is Guile, which is maintained by the GNU Project. Guile seems to be intended for embedding Scheme code within larger C programs, but it works wonderfully as a stand-alone Scheme programming environment. Repl.it doesn't advertise anywhere that it is installed, but I discovered it when playing around with a Repl.it bash repl. A bash repl within Repl.it gives you a bare-bones Ubuntu Linux virtual machine in the terminal window. It is somewhat limited in what is installed, but Guile was simply there as an executable. I hadn't used Guile previously, but it did what I needed it to do. It allows relatively simple tracing. Not as good as DrRacket's visual debugging, but good enough. It runs on any command line, so tests are easily scriptable around it. And it follows standards. So, I went with instructing my students to create bash repls, and to use the Guile environment within.

With the decision made to use a bash repl, working with C was easy. Repl.it supported both clang, gdb, and valgrind at the command line. This meant that my students could use both Scheme and C in the same repl, just as they would if they were working on their own computer. It was pretty awesome.

Repl.it promotes that it has strong integration with GitHub. Unfortunately, I wasn't able to make that work with my setup. Repl.it does have some really nice functionality where you can set up an OAuth-based link between Repl.it and GitHub. Once you've granted permission on GitHub for Repl.it to talk to it, Repl.it can see your entire list of public/private repos on GitHub and allow you to select one for cloning. Repl.it will set up a new repl for you based on this repo. To commit and push, you can then use GUI buttons that Repl.it provides. On the whole, it works nice, and I wish I were able to use it.

The problem I had was with regards to authentication. Since I was using GitHub Classroom, it meant that all of my students' git repos were being stored on GitHub, as private repos, in a GitHub organization that I had created. GitHub Classroom works out all the permissions wonderfully so that the repos stay private, so that students can see all of their own repos, and they can't see anyone else's. However, the Repl.it/GitHub combo didn't allow (as far as I could tell) this fine-grained level of access. I tried pretty hard, but I couldn't find any way of granting Repl.it access to any repos in the class organization without granting every student access to all of them. I definitely didn't want my students to have access to everyone else's repos, so I had to rule out this approach.

So instead, I did it the old-fashioned way. Since the students were working at a bash command-line anyway, they had access to git at the command-line as well. So they issued classic command-line git commands to clone, commit, and push to their repos. On the whole, this mostly worked pretty well. There was one particularly frustrating issue I'll mention in the Annoyances section near the end.

So, with the above all figured out, I was all set to go… and remarkably enough, the approach worked really well!

On the whole I was amazed at how successfully the students were able to collaborate, and how positive most of them found the experience. Except in cases where the students had poor local internet connections, they were all able to pair-program using the Repl.it multiplayer mode (and chatted by voice with Zoom, or Facetime, or whatever they wanted). Likewise, they could then share their session with me in Zoom office hours by sending me a multiplayer link, and I could similarly collaborate with them. Many of them want to stick with this approach even if we're back in person, rather than go back to the old shared-keyboard version.

The part that was most surprising to me was that I think I was more effective at helping them debug this way than I am in person. That's because in person I would never take the keyboard from a student, but in a shared Repl.it session it feels natural to say "can I type something?" and demonstrate. Again, I can't quite explain it but typing on my own keyboard in a live session that I'm sharing with the student seems collaborative, whereas typing on a student's keyboard seems invasive. This freed me up to be able to do much better demonstrations of debugging processes in office hours that I've never pulled off in person.

There were any number of irritating glitches that we faced, that we regularly had to work around. Early in the term, I feared that I wouldn't be able to repeat this approach because of them. On the whole, however, students were supportive about working around the challenges, and they seemed to mostly agree that the advantages were worth suffering through the difficulties. That said, here's a list of the irritants we faced with this approach.

Repl.it was pretty fantastic for us. It allowed me to very rapidly set up an online version of my class, which was fabulous. And it appears that it just might be better for pair programming than a traditional shared keyboard. We'll see!