Does Certification Prove You’re #1?

August 11, 2008 at 8:56 pm | Posted in Performance-Based Testing, Vendor news | 2 Comments
Tags: , , ,

It’s that time of year when the football talk begins at the office. Recently a friend told me that his team was ranked #1 in the nation. What I want to know is, how is this possible? The team has not played a single game yet, so that team has not proven itself worthy to be #1. I feel the same way about tech job candidates. When applying for a job, you have to have a lot more going on than your tech recruiter’s hype. You have to prove that you are the #1 candidate. A certification is great, but the certification has to prove that you know your stuff.

If you have taken any certifications for Microsoft Office, you might be familiar with the Microsoft Office Specialists exams. These exams incorporate “Live-in-the-Application” technology. Test candidates are graded solely on their ability to perform tasks within Microsoft Word, Excel or Powerpoint. If you know how to do the tasks, you swim and pass. If not, you sink and fail. This is a great idea for testing – but it’s only implemented for the Office exams. It wouldn’t be feasible to give a candidate, say, a server farm to manage in order to administer a Windows Server 2008 exam.

Recently the friendly folks at Microsoft started to “kick it up a notch” with some of their exams by including simulation questions. The purpose of sims is, obviously, to test whether a candidate can actually perform the tasks required for certification. Simulations are fine from the HR manager’s perspective: they help weed out the 100% Book Learners from the experienced workers. But simulations have some downsides from the test vendor’s perspective. A simulation question is pretty expensive to build, pretty difficult to grade, and usually has only one way that the task can be performed. This is a problem if you work with scripts or the command line and believe that the mouse is just another communist plot the government warned you about. In reality, there may be more than one way to perform the task being simulated on a certification exam, but you have to perform it the way the test builders planned for it to be graded. It’s even possible that you could poke around the limited functionality of the simulation and guess the right answer just by looking at the available functions.

There’s got to be a better way to test candidates. This is the 21st century. I do not have a flying car – in fact I still drive a 1973 VW beetle – but I should be able to perform a task correctly in more than one way on a certification test.

Let us all welcome the emulation questions. Emulation questions are similar to simulation questions in that both types of questions allow you to perform tasks on a server or workstation. Unlike a simulation, emulations allow you to actually perform the tasks on a server or workstation in a virtual machine, not a limited sim with only one path. You can perform the tasks any way you want. You can use the GUI interface or be a real pro and use the command line. [Blogmistress’ note: Prejudicial, but I’ll let it stand.] It does not matter how you perform the task as long as you get the correct result. You are graded by the final state of the task.

If you had an opportunity to take the 70-113 pilot test, then you got a sneak peak at the new emulations. As has been posted in various Microsoft blogs, you’ll get a section of multiple choice questions and a section of labs with specific tasks. For the labs, you connect to virtual machines with servers and workstations to perform various tasks. This is no simulation, it is the real thing! There is no better representation of someone’s skills. Either you can do it or you can’t.

So here is a quick chart to illustrate the three types of performance-based testing (PBT):

Question Types

Tasks are

performed …



In a recreated subset of the application with very limited functionality

Cisco routing/switching sims in CCNA and CCNP exams

Microsoft’s implementation of sims in Windows 2003 Server exams (70-290, 70-291, 70-293, 70-294, and 70-431)


By interacting with the live application(s) on a live system

Microsoft Office Specialist Exams


By interacting with the application in a virtual environment

Upcoming 70-113 exam

In closing, if you want to be the #1 candidate, you must prove it. I feel like Microsoft is improving their tests to incorporate real life situations, because not all situations you face in the IT world can be represented by multiple choice questions. I predict that the days of the “paper certifications” are long gone. I also predict that the New England Patriots will not go 19-0. The Atlanta Falcons will win more games than the NY Jets, even with Brett Favre.

George Monsalvatge


RSS feed for comments on this post. TrackBack URI

  1. Nice to know about new style of MICROSOFT exams, it would be beneficiary for genuine professionals. And would provide a way to prove their skills.

  2. Hi! I was surfing and found your blog post… nice! I love your blog. :) Cheers! Sandra. R.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at
Entries and comments feeds.

%d bloggers like this: