Measuring Learning in an Online World
by Laura Keehan, Executive Director, Assessment | April 23, 2020
In our (not so) new (anymore) world of online learning, you’ve probably quickly learned that more traditional, classroom-based assessment no longer really applies. That said, I bet that’s not the only way you were evaluating student understanding before―and it certainly can’t be now! Just as in your regular classroom, assessment doesn’t have to equal test.
Many school districts and universities are adopting pass/fail grading schemas in order to best accommodate potential opportunity differences within their school communities. Some institutions are holding to grades, but are adjusting requirements to account for variability in availability of technology, connectivity, and even time and available assistance across their at-home learners.
Regardless of what you, as an educator, will be required to hand in at the end of this semester, you’ll no doubt still want to measure learning, and ensure that meaningful work is still taking place! Of course, keeping the emotional health and well-being of everyone involved should be first in mind, but here are some tips that might streamline your evaluation process.
Use the Tools you Already Have
Many of the Learning Management Systems (LMS) you have been using all year allow for both more traditional assessment, as well as some interesting alternatives―all online. Blackboard, Canvas, Moodle, and other LMS offer ways to administer more standard assessment to assigned students.
- Multiple choice and short answer question forms
Think about what kind of content/objectives you’re trying to measure, as well as the potential accessibility issues of your class. If this seems like a fit, this could be a way to learn about student understanding in a more traditional way.
- Long-form essay or open-ended items
These don’t need to be administered through an existing LMS, as clear direction and response could be sent via email or posted via a class/school website. These methods will allow for more evaluation/analyze/solve type activities, and (again, as in your traditional classroom) can give you a deeper understanding of how students are thinking and problem solving.
- Worksheet/PDF of multiple types of questions
Provide a traditional “worksheet,” and have students either print or write over the PDF and send their work back to you, or even take a picture with a cell phone and email or upload to a shared site. Again, depending on available resources on both sides, this could be a great solution for those with limited tech availability.
Use the New Tools You’ve Been Given
While it can be completely nerve wracking and frustrating, once everyone becomes a bit more comfortable with some of the technology we’ve all started using during this time, they can actually serve to enhance measurement opportunities, as opposed to stifling them!
Start out a bit more simply. Many virtual meeting platforms (Zoom, etc.) offer hand signals, or “wave/thumbs up” notifications for attendees to share on the screen during a synchronous session. A quick check of understanding or engagement could be asking for “a quick thumbs up!” in your full-class view.
Many platforms also offer polling during meetings. Polling is a controlled, usually multiple-choice question which you pose to room attendees. Responses can be anonymous, which can encourage participation and honesty. Used appropriately, polls can help you gauge not only that learning is going on, but also give a general sense of the “room,” as it were. A reflective question posed after a 20 minute discussion on students’ grasp of the topic can provide you with valuable insight on how instruction is proceeding, and potential adjustments moving forward.
- Chat Function
Depending on the age group you are working with, the chat can be useful for assessing understanding. It could be as simple as posing a formative question in the middle of a lecture and asking for a typed response, or a more complex query in which you ask learners to reply just to you, and you evaluate responses later.
- Small-Group Sessions
Some students might feel intimidated to participate in discussions online with a larger group. For courses in which participation in discussion is required, creating weekly or biweekly breakout/smaller group sessions in a seminar-type format could help their comfort level and drive instruction. While clearly more time consuming on the instructor end, this could replace your grading time spent on other types of assessment, and be more fruitful a measure to boot!
Consider Remote Proctoring
Many companies offer remote proctoring of a multitude of different types of exams. While the exact offering varies somewhat between products, students will either download software onto their computer or log into a secure browser that locks down the rest of their computer, except for the window/tab on which the assessment is being administered. In addition, sometimes the student will have to scan the room with a webcam to show their location, ensuring there are no others in the room to aid in answering questions.
Pros of utilizing remote proctoring services:
Remote proctoring allows for a more secure environment for students to take tests remotely, online
- Cheating deterrent
Are sufficiently effective in serving as a deterrent for those students who would cheat
- Protection for future exams
Can be an aid to institutions as far as test form/item security goes for future test takers and administrations, in that copying/screenshots are not allowed.
In sum, there is no such thing as the “cheat-proof” test, but these programs definitely make it harder. That said, security concerns in showing student living quarters, storage of personal data (ID verification, facial recognition and/or biometric data, and storage of video footage of students taken during the test), cost to the institution, technology equity, and potential accessibility issues are all concerns that must be considered when thinking about these programs for your institution.
Get Creative and Invent Some New Tools!
I’m always impressed by the creative solutions that come out of chaos. Alternative learning calls for alternative assessment. Here are some ways to get creative with what you are given, but still keep learning and measurement on track!
- Involve the students in the process
Have students share a project or give a presentation to your online class, or submit a demonstration to you on video for individual review/grading. Still pictures of creative work can also serve as submissions for more performance- or project-based learning.
- Use voice and video recordings
The use of voice and/or video recordings can be beneficial going both ways. While students might send you work in audio or video form, you could also provide feedback to students in the same way for evaluation/grading. This also adds another much-needed touchpoint for faculty and students these days. Sometimes hearing your voice evaluate their work can make such a difference and connection!
- Explore self-assessment options
Use Google Drive, a class or school website, or even email to aid in journaling and reflective learning. Have a question of the day/week for students to respond to, self-assess something they worked on this week, or just share a current thought or feeling.
Alternative Grading Solutions and Considerations
Grading can undoubtedly be challenging at this time. As with all assessment, be clear and upfront around expectations and scoring. Be sure to follow any and all guidelines provided by your institution, but for a few additional ideas, grading can be:
Written responses to writing, online class participation, or other forms of question
- Audio or video recordings
A quick audio clip or video with an encouraging word or constructive criticism might just be the connection your student needs from you today!
Grading also must be:
Your learners should never wonder what is required of them. Clear and detailed scoring rubrics (including what is required for a top-score response) for open-ended questions, deadlines, and content requirements (eg., three examples to back up a thesis, show all computation work, etc.) need to be presented to students ahead of time so they know what is required of them.
Is this test open or closed book? Can/must the students use supplemental materials such as calculators or tables? Will there be timing considerations for all learners? As with all assessment, be clear upfront around expectations and scoring for students - not only around administration rules, but grading, as well.
Learning Happens When Students Connect New Information to What They Already Know
Understand and appreciate that you already have a wealth of “data” on your students with which you can measure learning, including any observation or interaction you’ve had with them! Think about which learning objectives are most important to measure, how they can now be measured, and what tasks YOU can complete to be sure everyone is on the same page. Don’t restrict yourself to usual, traditional assessment methods, and think outside the box a bit.
All that said, do keep in mind that this process is now occurring in brains busy with excess stress and overwhelmed with extraneous information. While standards have to be met, when creating instruction and assessment, be mindful of where students and teachers are at this moment. All in all, relax. You know your students and are the best for them right now!
Download "Tips for Measuring Learning in an Online World" for steps you can take today to foster communication and community with your students.
For more educator resources to use in the digital classroom, read more articles on our blog.
As the assessment specialist on the Learning Science team, Laura helps content teams implement best practices in item writing, field testing, and test development. She works with psychometricians to evaluate the performance of our assessment content, and helps content owners revise their items and test forms based on psychometric analyses. She also consults on various research and evaluation projects. Her 17 years of experience at Kaplan includes work with most K12, precollege, graduate, and licensure exams.