pan blog

Driving, Cell Phones and Psychologists

Reid Klion — Fri, 05 Mar 2010 14:11:06 GMT

When we think of psychologists, we often first think of either clinicians or people who create and use tests. However, while often not as visible, it is important to remember that psychology is filled with people who study all sorts of things related to behavior. One area of psychological research that I have found interesting to follow as of late is on the relationship between multi-tasking and attention, and, more specifically, perhaps because I see it all the time getting back and forth to work, cell phone use and driving. As might be expected, the topic which has garnered quite a bit of press coverage in recent months.

In one of my favorite studies, researchers at Western Washington University had a clown ride a unicycle around a sculpture (see picture on p. 6) on the main campus quad. Then, they interviewed students who walked by and asked if they had seen anything unusual. Rather shockingly, only 25% of the phone-using passersby (vs. over half of people who were doing other things like listening to music or simply walking) remembered seeing the clown. More specifically to the question of driving, recent research indicates that when drivers text, they avert their eyes from the road for nearly 5 seconds. Similarly, other studies using driving simulators have found that the risk of a crash increased by a factor of eight when drivers are texting. I think that the takeaway lesson is quite clear that using a cellphone while driving needlessly puts everyone on the road in danger.

Reid Klion

More on Credit Checks and Hiring

Reid Klion — Wed, 03 Mar 2010 19:11:48 GMT

As a follow-up to the posting a few weeks about credit checks, there are at least 5 states that are challenging the use of employer credit checks for selection. The legislators involved claim that their actions are spurred by the perception that employers are using credit checks inappropriately. If any of these laws are passed, each one would need to be examined carefully to understand its implications, and this would need to be done on a state-by-state basis.

However, it is important to remember that credit checks are a legally defensible selection method if creditworthiness is indeed a job requirement. For example, applicants for high risk positions, such as intelligence work, are often not hired if they have substantial levels of debt because this is seen as increasing their risk for bribery. However, employers do need to take a second look when they use credit data as an indirect predictor of other behavior such as trustworthiness. While there are a few studies that indicate some relationship between these two factors, there are several well-developed psychometric assessments that have been designed to measure integrity and trustworthiness directly. Additionally, unlike credit data which is often prone to adverse impact, there is a substantial body of literature indicating that integrity tests are not. Here, like in any assessment situation, it is important to first have a clear understanding of what the requirements for a position are and then finding the assessment strategy that is most likely to meet the organization’s goals which limiting unnecessary risks.

Reid Klion

Certification Body Wins Lawsuit Against Cheater

Reid Klion — Mon, 22 Feb 2010 16:23:19 GMT

National Council of Examiners for Engineering and Surveying (NCEES) won a civil lawsuit against a civil engineering candidate who attempted to take one of the organization’s primary exams with wireless a/v transmitter, video camera, receiver, pocket video recorder, and battery packs all on her person (see picture) with the intent to steal the content of the exam. She apparently had done this previously in 2005 but was caught by an alert proctor in the most recent incident. The federal court awarded NCEES a judgment of over a million dollars for loss of intellectual property. This figure was determined by estimating the cost that is involved in developing the items that had to be discarded. This individual was previously convicted of fraud in a criminal court for charges related to the incident.

I suspect that judgments of this type and size are rare but they do highlight the costs that can be involved in the development of sophisticated testing programs.

Reid Klion

Credit and Background Checks in Hiring

Reid Klion — Mon, 15 Feb 2010 22:22:58 GMT

Here is an update on the EEOC’s E-RACE (Eradicating Racism and Colorism from Employment) Program. Ongoing for the past several years, E-RACE focuses on race and color discrimination in the workplace with a particular focus on employment screening methods (such as background and credit checks) that are prone to have adverse impact. Here, it is important to recall that adverse impact occurs whenever a seemingly neutral “test” unintentionally results significantly lower selection rates for protected groups of persons. (A classic example is that if the physical ability requirements are extreme, many women will be disqualified from certain labor jobs.)

Since credit and background checks are quite susceptible to having adverse impact, businesses must be sure that their use is job related (just as they must for any assessment method that has adverse impact). Not only is this consistent with the law, it also prevents the needless disqualification of job applicants who otherwise may become excellent employees.

Reid Klion

IPAC Call for Papers

Reid Klion — Wed, 03 Feb 2010 21:45:28 GMT

The call for proposals for the 2010 IPAC Conference is now open. Formerly known IPMAAC, IPAC will meet on July 18-21 in Newport Beach, California at the Hyatt Regency. Session formats include symposia, panel discussions, paper presentations, tutorials, and workshops. Presentation proposals will be accepted until March 5th.

pan has long been active in IPMAAC and now IPAC and is proud to have been an conference sponsor for many years. IPAC is an organization of applied HR selection and assessment professionals, with members who work for both public and private-sector organizations. If you aren’t familiar with the organization, I might suggest you take a look a IPAC website and consider attending or even submitting to the conference.

Reid Klion

ATP Conference Coming Up

Reid Klion — Mon, 25 Jan 2010 16:13:37 GMT

The Association of Test Publishers Conference is coming up on February 7-10 in Orlando. The ATP Conference is a good opportunity to see what is new in the world of testing and meet many of people who are active in the industry. The conference program has tracks on promoting state-of-the-art assessment, integrating technology into assessment, and innovations in test security. pan will also have a booth in the exhibit hall, and a number of us will be attending including Doug Cole, Jocelyn Courtney (who recently became Secretary of the ATP I-O Division), and Jenni Ginsburg. If you will be there and would like to arrange a time to meet with one of us, please drop me a note. We look forward to seeing you in Orlando in a couple of weeks!

Reid Klion

On Writing Items...

Reid Klion — Tue, 19 Jan 2010 16:58:27 GMT

Writing items for tests and surveys is often more difficult than it appears. There are professional guidelines for item writing but it is also an art that might be best learned through experience and observing how items perform when they are actually used.

How an item is written can have a substantial impact upon how subjects respond to it and inadvertently skew the results. As an example, I received what was labeled a survey in the mail the other day. Here are a couple of items:

-Do you believe that Barack Obama's nominees for federal courts should be immediately and unquestionably approved for their lifetime appointments by the U.S. Senate?

-Do you believe the best way to increase the quality and effectiveness of public education in the U.S. is to rapidly expand federal funding while eliminating performance standards and accountability?

To be honest, I suspect that the underling intent of the survey is to motivate political supporters (as opposed to actually being an opinion poll) because it was accompanied by a fund raising request. And regardless of one’s political leanings, I suspect that it would be difficult to find anyone who would endorse either of these statements as written. However, it illustrates how an item is constructed can impact the results. In this case, I think that it is fair to conclude this was the intent. Problems do occur, though, when the goal is gather information in an unbiased way but poor item writing gets in the way.

Reid Klion

Trust but Verify

Reid Klion — Thu, 14 Jan 2010 16:16:00 GMT

One of the major reasons to use an assessment is to ascertain if a candidate is actually capable of doing the job. While there other ways to accomplish this, such as reviewing references and resumes, these approaches can fall short. Unfortunately, employers often do a poor job of actually checking references and many of us don’t verify a candidate’s educational history. (There haven been some well-documented cases of people in high profile positions who don’t have the background they claimed. For example, there is the former director of admissions at MIT didn’t have any of the degrees listed on her resume and George O’Leary who was the head football coach at Notre Dame for 5 days until it was discovered that both his claims to have played college football and having a graduate degree were false.)

Another challenge in credentials verification is the apparent growth of degree granting universities that may not have particularly stringent academic standards. There is growing concern about U.S. military personnel getting degrees (almost wholly paid for by the government) from propriety, for profit institutions. While the students who enroll in these programs do so with good intentions, they often discover their degrees are not well-regarded by employers because of the nature of the schools. In a similar vein, there is a physics professor at the University of Illinois who has essentially been on a crusade against diploma mills, businesses often based outside the United States where one can essentially purchase a degree without fulfilling any requirements at all. When it comes to vetting a job candidate’s credentials, it is important the employers have a way of verifying that an individual actually has the qualifications to do the job well. Whether it be with looking into a candidate’s credentials or using an assessment, this is critical information to have before making a hiring decision.

Reid Klion

IRS to Test Tax Preparers

Reid Klion — Mon, 11 Jan 2010 15:17:46 GMT

The Internal Revenue Service recently announced that it intends to develop a testing program for tax preparers. While CPAs and enrolled agents (non-attorneys who may represent clients before the IRS) have had testing and licensure programs for a number of years, the estimated 900,000 to 1.2 million individuals who prepare tax returns for a fee can do so without any sort of credential. The logic behind this new IRS initiative is to protect the public from incompetent or unscrupulous practitioners. Apparently, this move is supported by some of the large tax preparations firms who view this as a way to force ill-prepared practitioners out of the field.

There are some who argue that licensure programs should not exist because they represent a restraint of trade and that the market should be the sole determinant of who can offer professional services. Indeed, there may be situations where the development of a licensure program may be a bit misguided. (For example, there is a battle in Virginia over the licensure of yoga schools.) That being said, tax preparation very much seems a situation where some form of regulation seems reasonable to protect the public. Similarly, until the recent mortgage crisis, many states did not have a licensure program for mortgage brokers. To that end, pan was pleased to assist the Indiana Secretary of State in implementing a testing program for loan brokerage professionals.

Reid Klion

Looking Back: Job Creation in the Last Decade

Reid Klion — Tue, 05 Jan 2010 21:00:47 GMT

While yesterday's post looked at where the jobs are anticipated to be in the coming decade, it is also helpful to see what happened over the past 10 years in terms of employment. Overall, the picture was not rosy. Indeed, the past decade saw no net job creation. In a similar vein, many households are making less money now (when adjusted for inflation) than they were 10 years ago. While the pain this has created is real, it also should be noted that some of this is due to measurement issues as the late 1990s were times marked by economic growth largely fueled by an investment bubble that subsequently burst while we are currently working through some economic issues that haven’t been seen since prior to the Second World War. As a result, the past decade is bracketed between a high point and what is an historic low.

All that being said, one can only hope that the early signs of improvement in the job picture continue to emerge. Additionally, the trend toward workers returning to school for additional training as well as the emergence of specialized training programs can only lead toward increasing the overall value of the pool of human capital that is available.

Reid Klion