There was a recent NYT Op-Ed by a guy who is authoring a book on the standardized testing industry and his exploits as a “test grader.” For those not familiar with this aspect of the testing industry (or who don’t have school-aged kids), the standardized tests administered by schools often include “open ended” questions where students need to provide a written response.

In order to scoring, armies of temporary workers are hired to do the work. The author recounts some of the goings-on as well as his less-than-conscientious behavior in this work. His conclusion is that “scoring should be done only by professionals who have made a commitment to education—rather than people like me.”

This brings a couple of thoughts to mind. The first is that one wonders if a work attitude assessment should be part of the hiring criteria for the essay graders. In this, as in all jobs, we want people who will carry out their work in a responsible and conscientiousness manner. Additionally, it also highlights why there is significant interest in the use of automated essay scoring systems where computers grade the written responses. There are now several years of research on these systems, and they tend to work well. The obvious benefits are that compared to human raters, they are not susceptible to either fatigue or poor work ethic. And unlike humans, they are 100% reliable because they will grade the same essay the same way every time.

Reid Klion

One of the services that pan provides is a network of over 600 proctored testing centers. We offer these to clients who require the supervised delivery of high-stakes assessments. However, outside of our Operations Team, most of us have little regular contact with the candidates who use our testing centers.  

I was jolted from this earlier today when I had to make quick stop at my local bank branch. When I pulled up to the office building where it is located, I saw several people anxiously smoking and pacing in front of the building. I quickly realized that across the lobby from the bank was a proctored testing center operated by another assessment company. When I understood the situation and that these folks were about to take a test, I was brought back to the recognition that taking a high stakes test can be very stressful given the potential outcome—perhaps not qualifying for a coveted job or obtaining the license required to enter a profession.

While testing is something that we do every day, we can sometimes become a bit detached from the actual experience of the candidate. As such, when we ask someone to take a test, we need to be aware of the anxiety that this can create and the importance of using good test administration practices such as providing appropriate information about the test beforehand and maintaining a quiet, comfortable, and professional  environment. This is especially important to bear in mind when assessments are not administered by trained staff in a formal proctored testing center but are supervised by individuals who have other duties and are carried out done in a location where other activities are also conducted.

Reid Klion

I recently finished Richard Nisbett’s fascinating book, Intelligence and How to Get It.  Subtitled, Why School and Cultures Count, Nisbett (a prominent cognitive psychologist at the University of Michigan) argues that intelligence is quite malleable and that it has a more limited genetic component than many believe. Arguing against a strong hereditarian model of intelligence (and contrary to Herrnstein and Murray’s controversial 1994 book, The Bell Curve), his stance is that while there are some genetic components to intelligence, environmental and cultural factors have the greatest influence (as well as being the domains that can be influenced and shaped). Overall, he provides a good review of the nature vs. nurture debate on intelligence (and rather clearly comes down on the nurture side).

As one who stresses environmental factors in the development of intelligence, Nisbett also makes some research-based suggestions about how to raise a child’s IQ. Included are:
•    Exercise during pregnancy
•    Talk to children using complex vocabulary and include them in adult conversations.
•    Teach kids to categorize and compare objects in their environment
•    Foster curiosity
•    Encourage regular exercise and activity
•    Limit family stress and instability
•    Teach and reward self-control and delaying gratification
•    Praise children for hard work and efforts which are under their control.
•    Have high expectations and model the behaviors you would like to see in your kids

Reid Klion

The modern psychological study of personality can be traced back to Sigmund Freud and his student, Carl Jung. Jung later broke with Freud and developed his own theoretical system which focused upon concepts such as the collective unconscious and the role of archetypes. While the study of personality has obviously evolved very substantially in the 100 or years from the time that Freud and Jung started their work, their influence can be seen today. For example, the Meyers-Briggs Type Indicator and other assessments of psychological type are based upon Jung's work.

It also should be noted while Freud and Jung are largely viewed as historical footnotes for most of us interested in workplace personality, both continue to have very ardent followers in other circles. The New York Times Magazine has a fascinating (but long) article about the efforts of Jung's family (he died in 1961) to prevent the publication of book that he wrote while in the throes of a personal crisis. Filled with mythology, fantasy, calligraphy, and paintings, Jung's family always considered it to be too personal to be made public. The article describes the history of the "Red Book" and how it has recently come to be published. 

Reid Klion

I had the pleasure of attending the joint IPMA-HR/IPAC meeting this past week in Nashville. The conference was composed of both public sector HR professionals and assessment-oriented psychologists. The mix led to some interesting discussions where the HR folks could hear about the nitty-gritty involved in testing and the psychologists learn more about the day-to-day challenges faced in managing public sector workforces in challenging economic times.

Many of the presentations are posted. As always, Bob Hogan gave an entertaining and scholarly talk on the critical importance of leadership in organizational success. Bryan Baldwin (whose blog HR Tests is certainly worth a look) also gave a great talk on the potential role and challenges associated with using social networking websites in recruiting.  (Bryan didn’t post his presentation but his recent article can be found in the Assessment Council News on page 6.)

Reid Klion

John Callender (Biodata Online) and Jack Smith (Kingwood Group), two respected testing and assessment experts and long-time pan content partners, will  discuss how the current economic climate impacts recruitment and assessment strategies.  Some of the topics to be covered include how investing in assessments makes economic sense and the potential benefits and consequences of shifting to a lower cost assessment program. Please join us!

Thursday, September 17 @ 1PM ET     Register

Reid Klion

There was a WSJ article a couple weeks back on the growing trend for universities to consider using personality assessments to assist in admissions decisions. In addition to the ETS Personal Potential Index  I blogged about a while back, there are a number of other tools in development. For example, ACT is developing an assessment to identify students who are at high risk for dropping out while the College Board has been working at Michigan State University to develop a biodata predictor for university success.

Interest in these instruments appears to be spurred by the seeming unassailable belief that there is more to predicting academic success than GPA and standardized tests scores. However, there are also concerns about cheating and students learning how to “game” such non-cognitive assessments. It should be noted that colleges have always attempted to discern a potential student’s strengths and weaknesses outside the academic realm through personal statements and admissions essays. Proponents of these new non-cognitive assessments argue they are carrying on this same tradition but doing so in a way that standardizes how these non-academic factors are evaluated.

Reid Klion

While it appears that the rate of job loss has slowed in the US, it still managed to creep up to 9.7% in August. However, others argue that the actual unemployment rate is closer to 17% when one takes into account people who have been reduced to part-time hours, stopped looking for work, and the like.  

As might be expected, unemployment also takes an emotional toll. According to research carried out at Rutgers University, over half the people currently looking for work are jobless for the first time in over 5 years. Not surprisingly, many of the unemployed describe stress, sadness, and strains in family relationships.

Another trend related to the economic slowdown is that older workers are understandably less inclined to retire due to losses in their retirement accounts. Economists point out that this has a de-stabilizing effect on the economy in that in good times, people are much more inclined to take an early retirement (when there is the need for additional workers) but more likely to hold onto their jobs when times are tough (like now when there are more than enough workers available).

On the other hand, there is a fairly broad consensus that unemployment is starting to bottom out. While there are somewhat different economic factors in play, Canada actually added jobs to its economy in August. Additionally, it appears the US federal government still needs to hire more than 270,000 workers over the next few years in “mission critical” posts in health care, homeland security, and law enforcement, according to the Partnership for Public Service.

Reid Klion

We have always had a number of clients who are interested in using 360 degree feedback instruments. Also known as multi-rater feedback, these tools gather information from several people who interact with an employee on a regular basis such as managers, peers, or subordinates. Each tool varies slightly in its focus, but all use questions that are behaviorally-based and focus on work-related behaviors and job performance analysis. Jocelyn blogged last week on some tips and suggestions to consider when using 360º feedback tools.

The major benefit of 360 degree feedback instruments is that they gather information from a wide array of individuals who interact with the subject of the assessment. A 360 degree feedback tool can either be used alone or as one component of a broader development or coaching program. While controversial, some organizations also use multi-rater feedback instruments as part of their employee performance review process. However, it is important that you inform raters that their feedback may be used for evaluative purposes (such as for an employee performance review).if you chose to use 360 degree feedback instruments in this way.

Finally, organizations can use a 360 degree feedback tool as a teamwork inventory to provide insight into each team member’s strengths and weaknesses and how they function together as a team. This information can inform the way team members interact with one another as well as help them to pinpoint and understand potential areas of conflict.

IPAC (formerly known as IPMAAC) will be holding its annual conference in conjunction with IPMA-HR on September 12-16 in Nashville. IPAC is comprised of individuals who are interested in public sector selection and assessment. As it has for the past several years, pan is pleased to be a conference sponsor. Additionally, Nate Studebaker will present on his work with the Texas Department of Family and Protective Services that helped to streamline their assessment process while I will participate in panel discussion on technology trends in assessment. We hope to see you there!

Reid Klion