pan blog

Throwing the Baby Out With the Bathwater

Reid Klion — Mon, 30 Aug 2010 15:07:38 GMT

Chicago has a long history of difficulty with its public safety testing programs. For example, there was a Supreme Court ruling last year that allowed litigation to continue in relation to a fire department assessment that was administered in 1995. Now, there are reports that Chicago may do away with its police testing program to save money, avoid the potential for litigation, and boost minority hiring.

It is difficult to understand how this is a good business decision. Not using a test certainly does not ensure against litigation because any process used to evaluate a job candidate is legally considered to be a “test” and subject to the same legal considerations as a formal psychometric assessment program. Additionally, other cities have managed to develop public safety recruitment and selection programs that have yielded a diverse police force. Finally, and perhaps most importantly, a well-designed assessment program is the fairest and most effective means of selecting job candidates. Removing the objective data from the decision making process and simply depending upon the subjective review of job applications is likely to result in more difficulties and expense in the long run than would developing a validated testing program when the costs of a poorly selected police force are considered. Additionally, not using a formal testing program would make Chicago unique amongst major city police departments.

Reid Klion

Reconisdering Vocational Interest in Selection

Reid Klion — Fri, 20 Aug 2010 18:15:40 GMT

I heard a presentation the other day by Dan Putka of HumRRO about the role vocational interest in job selection. To this point, the received wisdom has been that vocational interest has little value in predicting job performance after the Big Five personality factors and cognitive ability are taken into account. While counter-intuitive, this finding has been replicated repeatedly in the literature and has become quite well-accepted. Putka and his colleagues hypothesized that part of this may be due to the fact that most measures of vocational preference were developed to measure occupational interest (e.g., what are the jobs that match my interests) and not designed for selection (e.g., what jobs do I want to do). For example, while some may enjoy fixing things around the house, they may not necessarily be interested in an occupation that involves this.

Based upon this assumption, they created a job interest inventory for selection that targets the sort of work an individual would be interested in having. Using in a very large scale study with the military, they found that job interest was more predictive of both intent to stay in a position and job knowledge than were either personality factors or cognitive ability. They hypothesize that while initial performance may be better predicted by other factors, whether an individual will have a long-term emotional investment in a job (as evidenced by the desire to stay in the job and learn more about it over time) are better predicted by interest. The article is currently in press at the Journal of Applied Psychology.

Reid Klion

Pass the Test, Get a Loan

Reid Klion — Tue, 10 Aug 2010 17:27:18 GMT

There was a recent article in Businessweek about the use of assessments to help banks in developing countries to identify those individuals who are likely to be good candidates for loans. The assessment looks at traits such as integrity, motivation, and intelligence. It is reported to work as well as (or better than) traditional methods of assessing creditworthiness with claims it can reduce default rates up to 40%.

I have heard of other organizations that have developed assessments for similar purposes. There is little reason to think such an approach would not have some measure of efficacy, especially in developing nations where most individuals have little to no exposure to formalized banking which makes credit history difficult to obtain. As such, psychometric assessment data may be one of the few pieces of objective information available upon which to base a loan decision. While testing loan applicants in the U.S. would likely run afoul of a number of a legal, business, and cultural barriers, it is still an interesting concept to ponder as well as a rather unique example of the innovative use of assessments.

Reid Klion

E-mail, Multi-tasking, and Loss of Productivity

Reid Klion — Fri, 06 Aug 2010 15:13:30 GMT

In the past, I have blogged about research by experimental psychologists into multi-tasking and how people tend to think they are better at it than they actually are. A major driver of multi-tasking in the workplace is often e-mail, and this was recently highlighted in an article entitled “E-mail is making you stupid.”

Most of us work in environments where we receive scores if not hundreds of e-mails and instant messages each day. While much of the information they contain is often of marginal relevance and almost never critical to what we are trying to accomplish at the moment, the frequent re-direction of attention (and attendant loss of focus on the task at hand ) is quite costly in terms of productivity. Some large organizations estimate that they lose up to $1 billion annually in productivity because of this. There is also a tendency for some workers to become so accustomed to checking their e-mail and instant messages very frequently that they become anxious when they try to cut back.

Some remedies include eliminating alerts that announce new messages, only checking e-mail a handful of times each day at regular intervals, and using either face-to-face communications or phone calls more frequently. The latter also frequently has the benefit of building closer relationships with colleagues and clients which often brings additional value to the organization.

Reid Klion

A New Perspective on "Faking" and Personality Assessments

Reid Klion — Tue, 03 Aug 2010 21:30:02 GMT

The issue of “faking” or impression management on personality inventories is a fascinating topic. It was recognized as a potential concern in the 1930s, and has both led to nearly 1000 publications and (in the words of Robert Hogan) “consumed the best minds of the discipline.” Despite this attention, the field has not reached any definitive conclusions other than the recognition that personality assessments can be highly effective tools for personnel selection when used in the right context even though some candidates very likely exaggerate their positive attributes while minimizing the negative.

A recent paper by Bernd Marcus, a German social psychologist, recently cast this in a new light. Marcus argues that we need to understand the applicant-employer dynamic from the applicant’s perspective. Some of the points he makes are that both the applicant and employer are engaged in goal-directed behavior. The applicant wants to induce the employer to hire him or her while the employer’s task is two-fold: to evaluate the candidate but also encourage the applicant to take the job if offered. Essentially, both parties are in the process of negotiating a long term relationship (perhaps not dissimilar to dating). As a result, it is no surprise both will be on their best behavior. In this vein, just as applicants are unlikely to disclose much negative about themselves, this is also the case for the employer. For example, how often does a potential employer openly share things like the reason why the position is being filled, concerns expressed by incumbents about the heavy workload associated with the job, or disclose other negative factors about the organization or position in question?

Reid Klion

Needing to Base Hiring Decisions on Something

Reid Klion — Tue, 27 Jul 2010 16:49:43 GMT

While those of us who work with assessments are well-aware of their limitations, research data have consistent indicated that well-validated personality and cognitive assessments are almost always the best predictors we have of future occupational success. This has been established repeatedly in the professional literature for the past 15 years.

While assessments are obviously not foolproof, the critical underlying issue is that we need to make hiring decisions based upon something. And in most cases, that best something is an assessment. In making this statement, we need to consider what the alternatives are. This issue was recently highlighted for me in a recent series of articles about GPAs which employers often consider when hiring recent graduates.

For example, there are now high schools that name up to 30 valedictorians. Traditionally reserved for the student with the highest GPA, it is now seen an honor to be shared by all “top” students. Apparently, many schools give extra credit for Advanced Placement classes so a 4.0 might be posted as a 4.5. Also, since some schools will cap GPAs at an arbitrary level, say 4.3, the result can be several students who all appear to have achieved a perfect record. All that being said, while there may be sound reasons for naming multiple valedictorians, it obviously no longer represents the academic distinction it once did.

In a related development, some law schools are now increasing their students’ GPAs retroactively. For example, a B- may become a B overnight, even though the student may have graduated some time ago. The rationale is that some law schools grade on a curve (e.g., only permitting a fixed percentage of 4.0s to be assigned) whereas others do not or do so less stringently. The more stringent schools, being concerned that potential employers may not be aware of this, have taken the step of increasing all their students' GPA by .33--even if they graduated five years ago. The point is not to argue the propriety of a school’s adopting this policy. Rather, it highlights the fact that when compared to other data that we have available, a well-validated assessment program often provides the fairest source of data about a potential employee’s suitability for a particular position within an organization.

Reid Klion

Colleges and Cheating

Reid Klion — Wed, 07 Jul 2010 14:48:26 GMT

While most of us tend to focus on testing in HR settings, we sometimes forget that the people who probably administer the most tests work in education, both at the K-12 and university level. While likely not a surprise, cheating can be a major concern in these contexts. For example, the University of Central Florida developed a specialized testing center to reduce the incidence of cheating. Amongst the measures used is an array of cameras monitoring student behavior and video monitors recessed into desks so that any efforts to take pictures of the test will become evident. Even gum chewing is prohibited for fear that it could be used to disguise a student’s efforts to speak into a recording device or phone.

While certainly interesting from a test security perspective, one also wonders about the message this sends to student about how they are viewed by the university. This point was made salient to me last week when I was on a college tour with my daughter at a couple of institutions which have honor codes. For example, students at Harvey Mudd College take their exams without supervision in the setting of their choice. As the walking-backwards-while talking student tour guide explained, professors simply distribute exams to students with instructions about how long they have to take the test and what reference materials are permitted. While the setting prevented my asking many probing questions, the faculty is apparently content with this approach as it has been in place for some time. “Take-home” testing is also the norm at other institutions such as CalTech which uses a similar honor code model.

Reid Klion

Bad Hiring Practices: An Illustration of What Not To Do

Reid Klion — Mon, 21 Jun 2010 20:12:41 GMT

The EEOC just recently announced that it was suing FAPS, a New Jersey company that processes newly imported cars for sale in the United States. While we won’t know about the truthfulness of the charges until the legal process is completed, the allegations against FAPS provide examples of the sorts of things that employers clearly need to avoid.

FAPS was found to have a statistically smaller percentage of African American employees in entry level positions than would be expected given the demography of the area (northern New Jersey). First, it is alleged that the organization relied almost exclusively on word-of-mouth recruiting which led to limited number of African Americans applying for jobs. This recruitment model would likely be seen as having adverse impact because it in and of itself is not deliberately discriminatory but did result in decreased numbers of African Americans applying for work. (This also illustrates how the concept of adverse impact applies to the entire selection process, not simply assessments). However, it is also further alleged that when African Americans did come to apply, they were told that no jobs were available ever though they were. This would be considered to be discriminatory because the intent is deliberate. Making matters worse, the EEOC also claims that the company asked questions about physical impairments on the application form which is a violation of the Americans with Disabilities Act.

Reid Klion

Lie Detection and Pre-Employment Selection

Reid Klion — Wed, 16 Jun 2010 15:01:06 GMT

The National Security Administration (NSA) recently posted a video attempting to quell applicant anxiety about taking a polygraph test as part of its pre-employment selection process. Whether this has the desired impact or not remains to be seen. However, the use of lie detectors for job selection has a long and interesting history.

The "father of polygraph" was a Harvard psychologist named William Marston (who later went on to fame and fortune as creator of the “Wonderwoman” comic book character). It was thought that the truth of a candidate’s responses could be determined by measuring changes in blood pressure, breathing, and other physiological indices. Unfortunately, data supporting the validity of lie detectors has never been strong, and a federal law passed in 1988 largely prevented the use of polygraphs for pre-employment screening other than by the government in the case of national security, defense or law enforcement. (It is also interesting to note that the development of “integrity” or work attitude tests in the late 1980s was spurred by this law and the desire to find more effective ways to select good employees.) While the polygraphs’ proponents acknowledge its limitation, they support its use continued as an interrogation aid in encouraging individuals to reveal information that they may not have otherwise.

Reid Klion

Multi-Tasking: A Myth?

Reid Klion — Mon, 14 Jun 2010 17:37:35 GMT

Our days at work are often filled with a series of tasks that complete for our attention including e-mail, phone calls, cell phones, and instant messaging. Not surprisingly, recent data indicate that people working at computers change tpiawindows or check their e-mail almost 37 times an hour. Indeed, there are growing concerns about how this may impact brain functioning as well as increase work stress levels.

A related issue is whether this is a productive way to work. Fortunately, psychologists have started turning their attention to this question. Researchers at Stanford University compared individuals who were heavy vs. light multi-taskers. From the outset, my expectations were that people who multi-task often would become quite adept at working in this fashion. To the contrary, it was found that frequent multi-taskers were more easily distracted by irrelevant stimuli and actually did worse at multi-tasking than were their peers who did it less often.

If you are curious about your own abilities to attend and multi-task, here are samples of the tools used in the study to assess attention and ability to shift between tasks.

Reid Klion