Measuring Validity Requires Measuring Job Performance

Measuring Validity Requires Measuring Job Performance

It seems that one question almost inevitably begets another.  Last month’s feature article talked about what validity means, why it is important and how it is measured.  The article contained an offer to assist Prevue users in proving the internal validity and/or the return on investment (“ROI”) on their use of Prevue Assessments through the Prevue internal validity/ROI program.  Some of those who sought more information about the program also wanted to know how they can establish measures of job performance that are required to be provided under the program.

The primary objective of the Prevue Internal Validity/ROI program is to confirm a direct relationship between job performance criteria for a specified job and test scores on one or more scales of the Prevue Assessments.  To fulfill that objective we have to identify relevant on the job performance criteria.

The most common job performance criteria ratings used in validity studies is a simple rating of overall job performance based on a 1 to 5 rating scale.  Although a very simple measure of job performance, this type of rating process often lacks the objectivity required to generate the solid metrics needed to prove validity or ROI, particularly where the sample size is small (less than two hundred participants. The Prevue Internal Validity/ROI program recommends the following process be implemented to identify relevant job performance criteria:

  1. Outline the general expectations and/or performance objectives of a position including organizational goals.  These may be contained in job descriptions or competencies that have been developed for the specified position.
  2. Develop job performance criteria by identifying and ranking the most important specific measures of performance or productivity for the position based on the following three types of performance data:
  • Production data – objective data that relates to productivity in job tasks such as number of units produced, sold or delivered, number of errors/mistakes, new account generation or average telephone call time,
  • Personnel data – does not directly measure productivity but rather reflects workplace behavior that includes organizational effectiveness as well as counter-productive behaviors that produce lower rates of productivity such as absenteeism, customer complaints and disciplinary warnings.
  • Subject ratings – typically supervisor review ratings and 360 degree feedback ratings (if available) that are more subjective.

Our experts will assist those Prevue users in developing job performance criteria who have not already developed job performance criteria through job analysis or other HR management activities that relate to the subject position.

For more information on the Prevue Internal Validity/ROI program, contact your authorized Prevue Distributor or send a request to [email protected].

Written by Ken Danderfer | August 2010