Just a reminder that the Call for Papers for the International Testing Commission Conference (to be held July 19-21, 2010 in Hong Kong) closes on December 1.  I had the opportunity to attend the ITC Conference in 2007 and would suggest it to anyone interested in increasing their understanding of issues related to the globalization of assessment. There are multiple considerations that emerge when a testing program is implemented internationally with challenges ranging from the practical to the psychometric to the legal. As the need for assessment programs that be used in multinational contexts grows, so does the necessity for assessment professionals to understand the complexities that come along with the territory.

Reid Klion

Yesterday was the day for almost 700,000 Korean high school students to take the nearly 9-hour long College Scholastic Ability Test. Given only once a year, it is critical to gaining admission to a Korean university. Coupled with the fact that more than 80% of Korean high school seniors attend university, this is a critical event. High school students spend months preparing, and their families often go to churches and temples to pray for a good outcome. To help manage the logistics, work started an hour late yesterday to minimize traffic jams, and aircraft were diverted from flying near test sites to minimize noise disruptions. In past years, the power company has even placed crews on standby in case the power grid would somehow become disrupted and impact testing.

While we often think of testing as a Western phenomenon, standardized tests have been used in Asia for over 2000 years. Indeed, the concept of using tests to hire civil servants was first implemented in the West in the late 1800s when the British began following testing practices they first observed in China.

Reid Klion

The Merit Systems Protection Board, a federal agency charged with ascertaining that other federal agencies carry out personnel-related activities in compliance with the law, recently released a report on the use of job simulations in assessing job candidates.  While written specifically for the federal context, it is a good source of information to anyone interested in the topic.

For those who may not be familiar, a job simulation presents a realistic work situation and assesses how an applicant responds to it. When you think about it, the set of tasks that might be considered job simulations is rather broad to include things like having welders demonstrate their skills on standard tasks, having pilots demonstrate their skills in a flight simulator, or a store manager asking applicants how they might respond to specific customer inquiries. While they may differ in terms of fidelity to actual work tasks, job simulations often provide good information about a job candidate’s skill level with a high degree of face validity.  The MSPB report discusses the different types job simulations, their relative benefits and weakness, as well as how to create and conduct them. The document also goes on to detail how simulations might be more broadly used by federal government. Some very useful information at the right price...

Reid Klion

In light of all the current excitement in the world of professional sports with the World Series finishing up this week, the NFL heating up, and the NBA season starting, a frequent topic of discussion is player salaries. This seems particularly salient given that the New York Yankee’s payroll for 2009 tops 200 million dollars.

When I hear these numbers, it brings to mind a Sports Illustrated article from earlier this year documenting the poor job that most athletes do in managing their money. While many of us cast a bit of an envious eye on the salaries that some top athletes make (for example, Tiger Woods was estimated to have made $110 million last year), the statistics on athletes who wind up broke despite multi-million dollar paydays is staggering. Some interesting statistics:

• Within 5 years of retirement, 60% of NBA players are broke.
• Within two years of retirement, 78% of NFL players have gone bankrupt or are under financial stress.
• Divorce rates for athletes range from 60-80%. Emotional issues aside, this also tends to have a major negative financial impact, especially if there are children involved.

The article outlines a number of contributing factors. Some of the most common are naiveté about financial affairs (certainly not uncommon for most folks in their early 20s) poor decisions about whom to trust (especially when it comes to managing millions of dollars), and a desire for tangible investments in things like businesses and real estate that seem easier to understand (though are almost always riskier) than stocks and bonds.

Reid Klion

While virtually all the proctored testing that pan does is in the HR space, there is a huge amount of proctored testing that takes place within an education context--think PSAT, SAT,  MCAT, GRE, LSAT and the like. A recent NYT piece focuses on the problem of proctors who fail to do their jobs or even manage to be disruptive during SAT testing sessions. The article indicates that SAT proctors are not specifically trained for the task (which surprised me a bit since our policy is to train all proctors on each program they supervise).  And reviewing some of the reported issues that occur during paper and pencil testing, it is clear that computer-based test administration (as we use in our testing centers) can ensure a much more standardized experience in terms of test timing, the right test forms going to the right candidate, and the like.

One bit of advice provided by an SAT test coach (yes, this is apparently a whole industry in and of itself) is that if a candidate notices a problem during test administration, he or she should either voice a complaint immediately and ask that it be addressed or “forget about it” (as opposed to simply grumbling about it after the fact). Here, it is important that test takers assume some responsibility in advocating for themselves by pointing out problems if they become apparent. The coach also goes on to say that “you have to expect that there will be distractions. If you’re seated next to a kid who is muttering under his breath, or sneezing, or grunting, just deal with it. Move on.”

Reid Klion

We are often concerned about the role of self-presentation during hiring situations. For example, much attention has been devoted to understanding hown “faking good” may impact personality assessments, the potential role of cheating on exams, and the fact that people tend to exaggerate their qualifications during interview situations. That being the case, at least the folks who engage in these behaviors are making an effort to present themselves in a positive light (and, as have some like Bob Hogan argue, the ability and willingness to present oneself positively is often a critical job qualification in and of itself).

In contrast, consider those individuals who fail to put their best foot forward when applying for a job (and the reaction this tends to elicit from hiring personnel). For example, virtually all of us can recount instances of job candidates who have multiple misspellings on their résumés, manifest a complete lack of phone etiquette, and the like.  To this end, a recent CNN article took an informal survey of hiring managers on some of the grossly inappropriate behaviors they have seen during interviews. These include job candidates answering cell phones during an interview, acting bizarrely, and being completely uninformed about the company or what it does. While it has been well-established that non-structured interviews are poor predictors of future job performance, it is also clear that certain interview behaviors are too salient and extreme to be ignored even by the most heavily research-based interviewer.

Reid Klion

Awareness and prevention of sexual harassment in the workplace has been a major concern for the past several years. In 2008, the EEOC received almost 15,000 complaints of sexual harassment with employers having to pay nearly $50 million in resultant benefits.

There have been growing concerns about the role of e-mail and text messaging in this, and a recent SHRM article outlines some of these issues. The piece makes a couple of excellent points about how social networking sites can lead to a blurring of the line between the social and work domains (where behavior acceptable in one’s personal life may not be appropriate for the workplace) and how the informal and spontaneous nature of texting may lead to types of communication that might not take place if a different medium were used.

The challenge for HR professionals lies in providing appropriate workplace training to educate employees about the potential negative implications of their behavior as well as of the appropriate use of company resources.  Employees should also likely be reminded that their workplace electronic communications typically can be accessed in the case of a complaint. For example, records of inappropriate text messages between the ex-mayor of Detroit and his chief of staff were one set of factors that lead to his eventual resignation. The bottom line is that employees need to be aware of policies regarding sexual harassment and that they still pertain whether an interaction is electronic or in person.

Reid Klion

Thomas Friedman, well-known author and columnist, recently wrote  about the workers he labels the “untouchables.” He argues that those individuals who have the imagination and drive to “invent smarter ways to do old jobs, energy-saving ways to provide new services, new ways to attract old customers or new ways to combine existing technologies” are the people who are critical to organizations that want to be successful in today’s economic environment. As a result, these employees will always be of great value. In contrast, average workers who can only carry out routine work are becoming increasingly endangered because many of these tasks can be automated or outsourced. Friedman further points out how our educational system needs to find ways to increase the number of students with high level problem solving and analytic skills and have a spirit of creativity, innovation, and entrepreneurship. However, until that day (and if it ever comes), it is incumbent upon employers to carefully select and build a workforce comprised of individuals who can truly create value for their organization.

Reid Klion

The program for the Association of Test Publisher’s Innovations in Testing Conference (to be held February 7-10, 2010 in Orlando) is now available. As usual, there appear to be a large number of interesting presentations on the docket. Some that initially caught my eye focused on topics including a survey about the future of testing, detecting cheating, and ATP’s new revised guidelines on pre-employment integrity testing. Also, registration is open with a $75 discount to folks who do so before November 16.

Reid Klion

I receive regular updates from ATA which is the largest testing company in China (and also happens to trade on the NASDAQ as ATAI). Last month, they administered the National Judicial Examination in China to more than 350,00 candidates. An individual must pass this exam to work in Chinese legal system and is a requirement for lawyers, prosecutors, and judges. Due to the high stakes nature of this exam, there were major concerns about cheating. (Cheating on exams in China has been an issue for over 2000 years with reports dating back to at least the Han Dynasty. There are a number of cultural factors which contribute to this which I hope to find the time to blog about fairly soon.)

In addition to taking candidates’ temperatures to make sure no one was ill, many test centers were also outfitted with handheld devices to detect the presence of wireless earphones well as equipment to block mobile phone signals. Additionally, 8 fixed monitoring stations and 2 mobile signal detection vehicles (labeled “Signal Detection”) were used to identify “suspicious” signals in the proximity of testing centers in  Beijing. In at least one instance, a test taker was discovered to have a wrist watch which could send and receive data. 

Reid Klion