Creating a PDF Resource to Support Writing Handoffs
PAPER Education
2023
PAPER Education provides online tutoring to over 3 million students across North America. Students are able to get help with a number of subjects, either from the live chat, where they can talk directly with a tutor via internet relay chat, or from the Review Center, where they can upload work to have a tutor review it asynchronously. The services provided in chat are different from those in the Review Center, but many students, and many tutors from other disciplines, have trouble understanding the difference.
In late 2022 and early 2023, as PAPER was experiencing rapid growth, there was an increase in mistransfers to writing tutors, when students actually needed either the Review Center, or a subject-specific tutor.
My goal: reduce tutor confusion around transfers, and provide a robust reference tool they can use to manage transfers quickly and easily.
Together with another full time tutor, I created a PDF guide for distribution through the company Learning Management System which could provide this support.
Contents:
My roles:
-
Research
-
Writing
-
Design
-
Project planning
-
Testing
Design Process
Plan and Explore
With a Gantt chart and project plan together, I used guerilla methods to assess tutors' attitudes towards transfers.
Following the release of an updated training module aimed at reducing unnecessary student handoffs, the company Slack was full of message threads from tutors frustrated and confused over student handoffs. Having experienced quite a number of these mismanaged handoffs ourselves, we wanted to get a better idea of how other tutors felt.
We first tried to distribute surveys via Slack. However, these were quickly buried by other threads, and received very few responses. Without clearance to distribute a survey via email, this method was closed to us.
Instead, I conducted a guerilla study by combing through public Slack messages for the month of February 2023, and logging relevant threads and messages in a spreadsheet. I then went through the spreadsheet and open-coded the data. Open coding is one of my favourite methods for drawing quantitative insights from qualitative data, as it is flexible and adaptive. I came up with two main classifications: tutor emotions towards handoffs, and tutor attitudes towards handoffs.
-
Emotions: How tutors felt subjectively about transfers, whether optimistic, neutral, or pessimistic.
-
Comprehension of policy: Whether or not tutors show an understanding of the recently updated policies.
Quantitative Insights
While we had expected tutors' attitudes to be more negative than positive, tutors’ attitudes were in fact quite evenly split between positive, negative, and neutral emotions.
Furthermore, almost a third of tutor comments showed a misunderstood the policy surrounding handoffs, a factor likely influencing those negative emotions.
Qualitative Insights
As a STEM tutor, I don't really have any confidance tutoring anything humanities related. I always just tell the student, that I can transfer them to a tutor who can help a lot more than I could. Let me deal with the fun numbers please
Some tutors become anxious when faced with a topic outside of their concept map. This leads them to hand off a student as quickly as possible, without gathering adequate information.
...I received an elementary math student in my classroom, and I tried really hard to work with them to avoid transferring them because I thought I could answer their questions. I quickly realized I wasn't as cut out for math as I thought. I think I ended up frustrating them more and they self-transferred. I feel awful about it, but I know that there's going to be a curve getting the hang of the new transfer guidelines...
Some tutors like helping students beyond their concept map, but are anxious about “failing” students.
I’m a little unsure if I’m allowed to do that given the new rules, I thought we had to upload it here
Many tutors are simply anxious about handoffs of students outside of their subject areas affecting their performance scores.
Our data revealed three main requirements for the final resource:
-
A step-by-step guide: a detailed flowchart could help ease the anxiety of the most nervous tutors by offering them a detailed process to follow.
-
An outcomes-based guide: this kind of resource could help tutors who want to try working in topics outside of their concept map, only to find themselves out of their depth.
-
Quality Assurance involvement: Any final resource will need to have approval from the QA team, as this team is responsible for tutor performance scores. This is the only way to reassure the large number of tutors who have shown strong anxiety over performance scores in the wake of the training.
Empathise
Using our research findings, I created three personas to help guide our work.
Design
Based on our research findings and personas, we created three separate versions of the transfer flowchart.
-
The detailed flowchart will support the anxious and detail-oriented Dexters.
-
A sample comment bank will provide additional support to these high-needs tutors.​
-
-
A higher-level linear flow will help the relaxed and confident Sathas.
-
An simple outcomes list will give the competent and experienced Brunos a quick reference to lean on.
I used FigJam to design and refine the three versions of the resource, as well as the bank of sample comments. FigJam provides simple, easy-to-use flowchart tools that were perfect for this project.
The rough version of the chart shown to tutors at the focus group.
Test
We presented the draft resource to a group of full time tutors from a mix of Humanities and STEM backgrounds.
In a Google Meet video call, I walked the participants through the resource step by step. Then, we opened the floor to questions. Feedback was overwhelmingly positive. Almost all participants said they would have used something like this early in their role, and some—particularly STEM tutors—stated they would still make use of this type of document. Enthusiasm was so strong that tutors began contributing comments to add to the sample comment bank.
Tutors had lots of ideas they wanted to add to this sample comment bank.
One insight we made special note of was that tutors wanted the text of the resource to be as large as possible. This supported our decision to split the chart over two pages.
We released the flowchart to two new STEM tutors to get their feedback.
Both tutors were pleased with the resource and made use of it in their work. One tutor commented that the flowchart was hard to read in places due to colour blindness. We immediately made adjustments to ensure the chart was clear for all tutors.
We ported the resource to a company PDF template and submitted it to the Teaching and Learning and Quality Assurance teams for review.
Teaching and Learning responded with some detailed feedback. Much of the feedback had to do with style, capitalization, and consistency issues. They recommended we move the flowchart to a single page, but thanks to our focus group we were able to explain why we had kept the resource divided.
Reflection
The Good
The project moved swiftly and on-schedule until the very last day.
Based on our research, we have no doubt this resource will be useful to, and appreciated by, tutors.
Hurdles
The resource was delayed at the last minute by staffing changes.
The day we were expecting to receive feedback from the QA team, we received news that the particular people we were dealing with were no longer with the company. The remaining QA team was left with an increased workload, and no longer had time to review our resource. As this was determined to be a key element to the success of the resource, we chose to delay release until the QA team is able to review the document.
As of the latest update to this page, we are still waiting.
Lessons
Guerilla research is worth the effort.
Despite not being able to conduct the survey we had originally envisioned, we were able to collect very helpful data through alternative means. The use of Slack comments as a data source proved to be more enlightening than I had expected, and I will likely use similar guerilla methods in the future.
"The best laid schemes o' mice an men gang aft a-gley." - Robert Burns
This project was well-timed, well-planned, and well-executed—but there are always elements beyond control. Sometimes there's nothing else to do but find a way around.
Results
The first few pages of the finished resource.
In the span of just a few weeks of scattered work, we were able to produce a useful resource that tutors confirmed would be useful to them in their daily work. While our planning and execution were exemplary, there are always elements that remain beyond our control. We will continue pushing to have this resource released.
In the end, we have produced something we both feel proud of, and which we strongly believe will help reduce improper transfers of students seeking help with their writing, improving studetns' experience of the PAPER platform overall.