Online personality profiling is growing fast, and a few mouse clicks can shepherd tens of thousands of candidates through the mysteries of the psychometrics process. But, what is the cost? And what are the common mistakes to avoid?
Personality profiling, once almost exclusively the realm of psychologists, has become widespread to inform hiring decisions, a development dramatically accelerated by the internet. Global revenues for the most widely used questionnaires can easily top $500 million per year.
The commercial potential for online questionnaires has proved to be a magnet for many whose credentials are more entrepreneurial rather than psychological or psychometric. However would Arthur Daley have resisted that temptation? After all, how difficult can it be to come up with a bunch of questions? And do these enthusiastic users really have the depth of knowledge and understanding required to reap the benefits, while also avoiding the pitfalls?
When you purchase a car, a holiday or an insurance policy, you need to look into the quality of the goods and the nature of the deal. If you do not know how to do that, you mitigate the risk by seeking advice from someone who does. When you put your faith in a personality questionnaire, the risk extends to the people you are going to assess, the decisions that you make on the back of that assessment, and the organisation that employs you to do it. So, what pitfalls stand in the way of realising the benefits of a personality assessment?
Pitfall 1: Not knowing what makes a good personality test
Knowing what distinguishes good test instruments from bad is the essential first step. It requires a grasp of the theory and technology; the concepts of reliability; and validity research as a minimum. It is helpful to know where to find the critical reviews, but these too can be challenging. They also assume some prior understanding of personality and its implications for employment and career development. Knowing the reputations of suppliers could help, but that too is more easily said than done. These basics offer at least some protection against improbable claims and commercial hype.
Pitfall 2: Tripping over the numbers
Numbers can give a spurious air of certainty, but personality can never be absolutely measured. It is important to appreciate the status of test results and how much weight should be given to them when making decisions. Test scores always have a margin of error and can be expressed in a confusing number of ways, eg as percentiles, or as one of a variety of standard scale scores.
These are important differences that determine what you can do with the numbers. You cannot add percentiles, for example, or sensibly compare scores on two different tests, and aggregating scores can play havoc with the process.
Pitfall 3: Interpretation drift
Language is inherently ambiguous. Terms like “conscientious”, “agreeable” and “anxious”, which frequently appear in personality reports, are complex and highly nuanced. Each has numerous synonyms and can imply different things to different people.
Without any structure to anchor your thinking, interpretation easily creeps past concept boundaries and into unintended semantic territory. This is an incremental chain of inferences that can meander far from the original intention – and back again.
Pitfall 4: Knowing what personality characteristics to look for
If you do not know what you are looking for, no test is going to find it for you. Most personality questionnaires are “panoramic” in their scope; they address the complete spectrum of personality. If you have not already prioritised the characteristics that you are looking for, you are in danger of being swamped by an Aladdin’s cave of options and of selecting on the basis of personal preferences, rather than job requirements.
For all the convenience and accessibility of online testing, you still have to prepare properly in order to make the exercise worthwhile. Some form of job analysis is necessary to consider which competencies are required. You then need to consider which aspects of personality would likely contribute or interfere with performance.
Pitfall 5: Having a purpose
Why exactly are you assessing candidates? What is the purpose of the exercise? The answers to these questions have a bearing on the kinds of assessment that would be most useful. Are you most concerned with fit, retention or performance? If you are concerned with fit, is that fit with the culture of the organisation (in which case you might assess values)? Or fit within a team (in which assessment of emotional intelligence or risk type might be useful)? Or fit with the organisation’s competency framework (in which case a personality questionnaire tailored to selected competencies might be best)? There is no one type of test or questionnaire that can do everything.
Discussion
All this has important implications for test developers, as well as for users. After all, we use complex technology every day without giving it a thought. We do not have to understand how it works in order to make use of it. Computers, TVs, mobile phones, cars and fridges all deliver painlessly. The assessment process can undoubtedly be made more conveniently accessible than it is. Today’s domestic and telecoms devices routinely deliver the fruits of incredible technology that was once only accessible to a tiny minority. Consensual technology, typically, becomes commoditised and, although we may not yet be quite at that stage with personality assessment, increasing numbers clearly do want it and would benefit from it.
Inexpert use is a concern and potentially a threat to the integrity and reputation of personality assessment. At the same time, efforts by professional associations to control training and access are clearly failing to keep up with the rate of change.
There can be no turning back to a more sedate era. Maybe it is time that these issues were addressed, making the process more accessible and resilient. Most importantly, there is a need to provide interfaces that navigate smoothly through the complexities and, by better design, embrace the possibilities, while taking some of these pitfalls out of the equation.
The original article, published by Personnel Today, can be viewed here.