What is Test Retest?

Sometimes referred to as ‘temporal stability’, ‘test retest’ is a form of reliability that assesses how consistent an individual’s scores remain over time. This is achieved by correlating two scale scores obtained from the same individual at two different points in time.

Creating assessments that obtain strong correlation coefficients is a priority for test publishers who are keen to assess traits that they deem to be stable. Weak correlations could indicate that (1) the trait is inherently unstable, or (2) that the measures used to assess it are flawed.

The British Psychological Society’s Psychological Testing Centre utilises the European Federation of Psychologists’ Associations’ (EFPA) test review framework to help practitioners understand the merits of the various psychometrics on offer. In the case of test retest reliability, coefficient strength is rated as follows:

Risk Type Compass Test Retest Analysis

As part of PCL’s ongoing research and validation work, Dr Simon Toms conducted a test retest analysis on PCL’s Risk Type Compass (RTC) tool. RTC is a personality assessment tool developed to evaluate the differences in the way individuals perceive, react to and manage risk, and how this affects their decision making. Individuals are placed into one of eight distinctive Risk Types. The RTC can be used for a range of applications such as employee selection and development, team building, risk management strategies and developing risk culture.

Participants completed the RTC assessment twice with each completion at least a week apart. The analysis noted that, whilst the majority of completions occurred within three months of each other, several dozen completions exceeded this time frame (to a maximum of 1,011 days). This enabled the analysis to also assess long term temporal stability of the two scales used by the RTC. The results of the analysis were as follows:

The Findings

The headline finding was the exceedingly high correlation coefficient observed for both scales, providing considerable evidence for the stability of the Risk Type Compass. As expected, correlations between the first and second assessment drop slightly when the time between completions increases, but they still remain comfortably above the EFPA’s maximum rating of 4, classified as ‘Excellent’.

Ultimately, the analysis provides evidence for the reliability and stability of PCL’s Risk Type Compass tool.