Adaptive testing has met with some resistance, but it really works to your advantage.
Adaptive Testing Works for You
Adaptive testing has met with some resistance, but it really works to your advantage.
How can something so right seem so wrong? That might
sound like the title of a country western song, but it’s
no tune you’ll be hearing those of us at Microsoft
sing any time soon. Instead, it sums up the negative reaction
we’ve been met with in some quarters to an innovation
that only works to your advantage.
That innovation is computerized adaptive testing (CAT),
which Microsoft adopted last fall to deliver many of its
MCP exams. CAT has a proud pedigree going back for centuries
(although not always with the computer, of course). CAT’s
biggest advantage is its greater efficiency. It’s
possible for a CAT exam to assess a person’s knowledge
in 60 percent of the time required for more traditional
tests. For the typical MCP exam, that means a test that
used to take up to an hour and three-quarters can now
be completed in an hour or less. The exams also offer
greater security, because no one candidate sees all the
questions.
CAT works by tailoring each test to the individual exam
taker. Examinees all start with an easy-to-moderate question.
If they answer the question correctly, they get a more
difficult follow-up question. If they answer that question
correctly, the difficulty of subsequent questions likewise
increases. Conversely, if the second question is answered
incorrectly, the following questions will be easier. This
process continues only until the CAT determines the candidate’s
ability.
While most test takers appreciate CAT’s benefits,
a few have expressed frustration. “These questions
are too easy. Everyone will pass,” say some. “The
test was too hard and too short. If I got the longer test
that my colleague took, I could have passed,” say
others. Ironically, both comments are off-base; the percentage
of test-takers achieving a passing score hasn’t changed.
Consider Albert, who gets just 15 questions that are
all very hard for him. When Albert answers his first or
second question incorrectly, the CAT offers him an easier
question. After he fails increasingly easy questions,
the computer determines that Albert isn’t going to
pass and therefore ends the test.
Or, consider Barbara. She gets the same number of questions
(15), but they seem so easy that she thinks everyone will
pass. However, Barbara is answering her personalized set
of questions so well that the computer can’t find
a question that she can’t answer correctly, and it
quickly determines that she’s MCP material.
Finally, there’s Charlie. Charlie gets many more
questions than the others—maybe 25 or more. Is the
CAT discriminating against him? No. Charlie’s first
few answers—some right, some wrong—give him
a border-line score. The computer needs more questions
to determine which side of that border he belongs on.
Some of you have complained that the test doesn’t
provide feedback to help improve future performance. That’s
because CAT—while accurate at what it’s designed
to do—isn’t designed to provide such feedback.
Because it uses fewer questions, it would produce misleading
feedback results.
If you still have questions about CAT, we’re still
listening. To get in touch with us and to learn more about
CAT—including the chance to take a sample test—go
to the Microsoft Web site at www.microsoft.com/trainingandservices/
default.asp?PageID=mcp&PageCall=tesinn&SubSite=examinfo.
You can write to us with comments at [email protected].
About the Author
Cyndy Fitzgerald, Ph.D., is the Manager of the Psychometrics and Research Certification and Skills Assessment Group at Microsoft.