Test models for IT certification exams: industry overview

August 14, 2008 at 12:11 pm | Posted in Microsoft, Performance-Based Testing | 3 Comments
Tags: 70-620, braindumps, exam integrity, Simulations, Transcender practice exams

A BRIEF HISTORY OF CERTIFICATION TEST TECHNIQUES
Troy McMillan

There is ongoing discussion throughout the IT certification community concerning the value of certification testing. Most of this discussion boils down to a small number of questions:

• Can standard multiple-choice certification tests accurately describe an individual’s technical knowledge?
• Do the limitations of the testing model(s) fail to measure hands-on knowledge?
• Isn’t it too easy to cheat with a multiple-choice model?

As the debate has raged, the major IT vendors ( Cisco, IBM, CompTIA, Microsoft, etc.) have tried various approaches to inspire confidence in certification testing as a valid tool. These are some of the item types and test formats we have seen in the past in an attempt to accomplish this:

Adaptive testing: Designed to seek out your weaknesses like a shark smelling blood. These types of test had the ability to ask you additional questions in a topic area if you missed a question on that topic. This was the format used by A+ software and hardware tests in the late 90s, but is not widely used today by any vendor. Regardless, frightened students still ask me from time to time, “Is the test adaptive?” [Editor’s note: current Microsoft exam catalogs note that adaptive testing technology could be used.]

The infamous “Choose all that apply”: I guess this is not really an item type, just a variant of multiple-choice where you are not told how many answers to pick. However, it’s a tactic that also strikes fear in the heart of test takers, especially those who are not as prepared as they should be. The typical student response: “Why can’t they tell me how many answers I need to pick?” With this item type, adding one wrong response can invalidate the parts of the answer that might have been correct. You may or may not see these on vendor exams today.

Case studies: These were lauded for their ability to require the test taker to sift through reams of data and pick out the relevant details. This format required a higher level of knowledge than multiple-choice, but critics complained that test-takers could still get the answers ahead of time from illegal braindump Web sites or criminal test-crackers.

Simulations: These item types were the first real attempts to test hands-on ability. They require the test-taker to “do something” (create a user, add to a group, change a policy setting) in a simulated software or hardware environment, and be graded on whether the task was accomplished. Cisco commonly uses simulations; Microsoft has rolled out higher-level exams with simulations in the past year or two. More about simulations in a minute.

Emulations: This is the coming wave in item types. While a simulation is a somewhat artificial environment for testing (possibly created with an Adobe Flash file), an emulation item connects the test taker to a virtual machine running a partially or fully functional operating system. See George’s post for more detailed news about the push to develop emulations for Microsoft certification exams.

Now, regarding simulations: Microsoft’s recent betas have piloted their simulation items. Proving how complicated these things really are, problems were reported at many test centers (including those encountered by our team). On a positive note, the problem seems to be solved in most areas. New technology growing pains are only to be expected. And if these simulations up the value of the certification as a whole, I think it’s a worthwhile inconvenience.

We liked the simulations that we saw. We thought they were straightforward representations of the technology, and fell within the exam objectives. They will certainly enhance the validity of the exam and help weed out the rote memorizers from the knowledgeable IT technicians.

For those who depend on Transcender training products to prepare for their exams, I’d like to announce some very good news: we are developing simulation item types for an upcoming revision of 70-620 : TS: Configuring Microsoft Windows Vista Client. The new Transcender simulations will operate very much like the simulations we saw in the Microsoft beta exams, and will cover the topic areas and functions that test takers might encounter on the exam. (I’d love to give you a release date, but I can’t yet, so keep checking back.)

The 70-620 practice test with simulations will be available in download and CD versions. Existing customers who purchased the previous version will be able to access the new version as soon as it is available.

So for all of you who have wondered about Microsoft’s new exam simulations, rest assured that the Transcender 70-620 practice test will have simulations soon, and will, as always, help you prepare for your exam experience.