ETS recently rolled out a product called the Personal Potential Index. Intended for use by graduate school applicants, it is a standardized rating form with 24 items pertaining to factors such as creativity, communication skills, and teamwork that is completed by a set of student-nominated “evaluators” (typically professors). The intent is to provide graduate program admission committees with information beyond GPA, GRE scores, and the traditionally glowing letters of recommendation. (Exaggerated letters of recommendation are not a new problem; there is a classic American Psychologist article from 1966 on the topic rather brilliantly entitled “Mine Eyes Have Seen a Host of Angels.”)

The ETS tool is premised upon the recognition that non-cognitive factors likely contribute to success in graduate school (as well as virtually everywhere else in life) and part of a larger ETS project to find ways of assessing them. There are concerns about using self-report inventories in a graduate admissions context due to “faking” (though such instruments have been found to function effectively in pre-employment situations). As a result, attention is being directed to rating scales as well as biodata and situational judgment tests.

The Personal Potential Index appears to be a work in progress. There are plans for developing local norms as well as procedures to adjust for differences in individual rater severity.  Additionally, only further research will be able to determine the extent to which this rating approach actually adds any incremental validity (and all of this is predicated upon the ability to collect enough real-life data). A concern that comes to my mind is the degree to which most professors actually have enough observational data to accurately rate their undergraduate students’ “non-academic” behaviors, especially in the case of institutions with large, often anonymous, classes. Additionally, one also wonders about the extent to which raters, knowing the implications of their evaluations, will engage in the same rampant exaggeration that is well-recognized in letters of recommendation.

Reid Klion