ADVENTURES in Online Testing (Volume 5, Issue 2)
by: Christopher Dalessandri, President and CEO, OWL Testing Software
If you are like me, you may be finding your web alert somewhat inundated by articles keying on the words “testing” and “learning.” Anecdotal evidence supporting the case for testing as a studying tool has been around for years. However, the subject appears to be gaining momentum as a multitude of studies have been published showing a correlation between testing and learning. Some of these studies look at testing and learning in general and give strong evidence that students who practice retrieving information learn more efficiently than those students who employ elaborate study strategies like memorizing, chapter outlining, and concept diagramming. Other studies focus specifically on the use of computers in learning assessment. These studies point to the computer as a very effective tool for practicing information retrieval.
Although we are not researchers at OWL Testing Software, we do interact with a wide variety of learning institutions on a daily basis. Over the past 11 years, we have learned a lot from our customers about the learning strategies they employ in their virtual and physical classrooms. Most new customers come to us seeking a more efficient way to facilitate a single high-stakes test – either for program placement (such as SPEAK®) or end-of-program summative assessment (like SOPI). Almost without exception, OWL customers quickly recognize the potential applications for OWL in their daily learning environment. Customers report that they use OWL Testing Software to facilitate regular quizzes and practice activities (PALS) to improve student learning and facilitate formative assessment. Given OWL's integral role in all aspects of assessment, it seems appropriate to examine the case for testing as a learning tool and attempt to synthesize its relevance to OWL's online testing community.
Testing by any other name... In some studies, that other name is 'information retrieval.' At Purdue University, Jeffrey D. Karpicke, an assistant professor of psychological sciences who studies learning and memory, recently conducted a study showing a strong correlation between testing, or information retrieval, and learning.
“...learning is fundamentally about retrieving, and our research shows that practicing retrieval while you study is crucial to learning. Self-testing enriches and improves the learning process, and there needs to be more focus on using retrieval as a learning strategy." 1
A second study refers to this as the 'testing effect' and examines possible reasons why information retrieval is more effective than the traditional ways of studying. This study points out that retrieving information can take many forms, “...whether in a pop quiz, flashcards or even a classroom game of Jeopardy”2
Getting it right... One of the most important aspects of testing as a learning tool is immediate feedback. In fact, it can be argued that it is crucial for the test taker to immediately know whether he has given the correct answer. If incorrect, he also should have an immediate opportunity for self-correction. It is extremely important that learners do not practice the wrong answer. This means that if we provide a variety of methods and repetitions for students to practice recall with immediate feedback, learners will learn more, and learn faster. This can be achieved by giving practice tests or other activities, such as homework or games, that give the same opportunity to practice information retrieval as traditional testing.
The Advantages of Computerized Testing... Computerized testing is uniquely suited to provide repetition, constant availability and immediate feedback to the learner. While some class time can be devoted to practicing recall, it is important for students to have access to practice recall outside of the classroom.
A second benefit for today’s students is that they find the computer interface innate. These digital natives are more comfortable completing practice activities and taking tests on a computer than they are with traditional oral exams or paper and pencil activities. According to an article in Language Testing, “Analysis of the results indicated that the test takers had a highly positive attitude towards digitally delivered test prompts compared to the face-to-face tests of speaking proficiency. In addition, examinees found computer-based tests less threatening and more comfortable than face-to-face interview tests.”3
Computers can make each interaction unique by presenting scenarios in different formats and in a randomized fashion.
Computers also more effectively measure a student's success and report progress. According to Purdue's Karpicke, “most students are not good at judging the success of their study habits.”1 Taking practice tests online allows the computer to give the student definitive progress results. With OWL, students can even maintain an electronic portfolio of their work.
The OWL PALS program is well suited for this type of learning. PALS allows students infinite opportunity to not just show what they know—but also to learn as they go. The activity items are limited in number to encourage rapid learning. Students receive immediate feedback. To increase their likelihood of success, they are encouraged to move through the activities from the easier receptive skills to the more difficult productive skills.4
Areas of concern... There appear to be two major areas of concern with implementing these findings. The first issue is how much to test. In many cases, today's students are over-tested. This means that students develop testing fatigue and waste the most valuable learning resource they have -- instructional time with a teacher. With the advent of standardized testing as a panacea for all that ails public education, many students spend entire months completing standardized tests instead of learning. If you add in the time devoted to preparation for these standardized tests, students may be loosing 6-8 weeks (or more) of instructional time.
The second concern involves the inappropriate application of computers. Computers should be used for what they are good for and should NOT be used for what they are not particularly good at. Despite the notoriety of computers like IBM's “Watson,” who beat human contestants on the popular game show Jeopardy!, computers are not yet capable of replacing human raters for analyzing complex speech or writing.
While computers are not good at analyzing writing or speech, computers can make it easier for those human raters to do their job and for students to access their valuable feedback. OWL users find it particularly beneficial that the software puts assessment, evaluation and feedback all in one online tool.
Computers can provide a better way... Educators who see the potential benefits of online testing solely as the mass marking of exams are missing the true rewards. Computers are great for making testing fun by testing recall in a game format. With software, like OWL, computers can better simulate real life situations by integrating multimedia. Networked computers are excellent as a means to remove time and distance constraints from testing. Online testing offers an opportunity to integrate technology into the formative assessment process, raise the student's comfort level, and ultimately contribute to overall learning success.
As both the reports from our customers and the findings of these recent studies point out; part of what we have to do if we want our students to achieve the best results is rethink our definition of 'testing.' Practicing information retrieval is important to efficient and effective learning and can be accomplished in many ways other than a traditional tests. Computers can play an important role in practicing information retrieval by creating games, self-correcting activities, as well as testing. While implementing such findings in today's rigid and cost-constrictive learning environment proves to be quite complex, most would agree that helping students learn more efficiently is a worthy endeavor. In the end, computers may provide a better way to achieving that worthwhile objective.
1. AMY PATTERSON NEUBERT, “Research finds practicing retrieval is best tool for learning” Purdue University News Service, Jan. 20, 2011.http://www.purdue.edu /newsroom/research/2011/110120KarpickeScience.html
2. ANNE MCILROY, “Tests get high marks as a learning too” Globe and Mail, Dec. 03, 2010.http://www.theglobeandmail.com/news/technology/science/tests-get-high-marks-as-alearning-tool/article1824878/page1/
3. “Evidence about computerized testing / digital natives more comfortable” Language Testing, Vol. 5 |Issue 1, Cover date: July 2009, page(s): 37-48 (Print ISSN:1833-4105)
4. “PPS Launches District-Wide Foreign Language Competition.” Pennsylvania Language Forum, Spring2010, 80 (1), 80-81.
RADDOUANE CHIHEB, RDOUAN FAIZI, ABDELLATIF ELAFIA, “Using Objective Online Testing Tools to Assess Students' Learning; Potentials and Limitations”Journal of Theoretical and Applied InformationTechnology. http://www.jatit.org/volumes/researchpapers/Vol24No1/8Vol24No1.pdf