Thinking Past the Assessment – Unfinished Goals and Unrealized Expectations

by Stacey Lett, Director of Operations – Eastern U.S. – Proactive Technologies, Inc.

Literally speaking, an “assessment” is the process of measuring the value, quality and/or quantity of something. There are many types of assessments,  and methods for assessing. In theory, it is the process of evaluating one thing against a set of criteria to determine the match/mismatch.

There are assessments for risk, for taxes, vulnerability. There are psychological, health, and political assessments. There is a group of educational assessments that measure a variety of outcomes such as educational attainment – assessments of course content mastery, assessment of grade level attainment, assessments of Scholastic Aptitude Tests (“SAT”) that compare a student to their peers nationally and a variety of college readiness exams.

Determining that you, indeed, hired the right person for the job will not automatically ensure the person is successful in learning and mastering the job. The most important step in the employment process is seeing to it that the individual’s core knowledge, skills and abilities are applied in learning and mastering the tasks which they were hired to perform. That is where the money is made. 

Educational assessments have been adapted for use in workforce development and employment, used to assess a prospective employee’s suitability for a job opening, with limited success. They often measure more of what, if anything, a student learned and retained before graduating than how they match the employer’s actual job opening. Psychological assessments have been adapted to measure a prospective employee’s sociability to the workplace, morphing into a new category called “psychometric assessments.”

We have seen a growth in the employment assessment industry over the past 2 decades, particularly after 9-11. There are assessments for cognitive tests, physical abilities, “trustworthiness,” credit history, personality, criminal background and more. When used improperly, the methods have been challenged in court for their appropriateness and intent.

An assessment is a “test,” and has been held as such by court rulings over the years. The instrument determines a positive or negative outcome for the employee or prospective employee. The court has ruled, in many cases referring to the Equal Employment Opportunity Commission’s Uniform Guidelines for Employee Selection Procedures, that anything used to evaluate a prospective employee’s access to employment, or an existing employee’s retention, promotion and movement within a job, must meet certain standards to be legally valid.

According to Wikipedia, “There are a number of different forms of validity.  Criterion-related validity can be assessed by correlating a measure with a criterion measure theoretically expected to be related. When the criterion measure is collected at the same time as the measure being validated the goal is to establish concurrent validity; when the criterion is collected later the goal is to establish predictive validity. A measure has construct validity if it is related to measures of other constructs as required by theory. Content validity is a demonstration that the items of a test do an adequate job of covering the domain being measured. In a personnel selection example, test content is based on a defined statement or set of statements of knowledge, skill, ability, or other characteristics obtained from a job analysis.” What is “an adequate job of covering the domain being measured” depends on how the word “domain” is defined; industry-general or employer-specific.

In many court rulings involving hiring and employment, the court has said that no matter if an assessment comes from a reputable national organization that has done its job of testing and documenting criterion, concurrent, predictive or construct validity (which tend to test an assessment’s intent and accuracy for a general target population), if the assessment has not been proven content valid for the employer’s job classification – sometimes referred to as “job-relatedness” – it is not appropriate and its use may be unlawful. In other words, an employer cannot base hiring or employment decisions for their firm on assessments for traits, skills, knowledge and abilities that cannot be directly correlated to a “bona fide occupational qualification” of the job for which employment is concerned.

All of the methods of testing validity, except content validity, are difficult to administer in the workplace because the target groups are small – 5 to 10 employees in a job classification. Any statistical analysis, and inferences drawn from it, would be suspect. However content validity works in all cases when a proper job/task analysis is performed because subject matter experts who currently hold the target job classification are used to define each required task step-by-step, and to discover the prerequisite core knowledge, skills and abilities that the prospective employee needs to learn, master and be promoted in the job classification. Other peers in the job classification review and confirm that the documented requirements are bond fide. Courts have ruled that if the company has a credible methodology for analyzing jobs for prerequisite requirements, has properly documented these requirements, content validity has been established. Other validity studies can be attempted in addition to content validity to try and measure any adverse impact on segments of the population, but content validity is sufficient to demonstrate job-relatedness.

Inappropriate selection and use of an assessment can be costly to an employer. But let’s assume the employer has selected the right assessment. Once the individual has been assessed for suitability for hiring for the job, or retention and movement within the job, what is next? Hopefully the assessment was accurately designed and selected to identify prior attainment of prerequisite core skill (e.g. reading, math, communication) and core ability (e.g. vision, hearing, social, strength) levels, and a decision could be comfortably made whether to develop any missing skills or remediate inadequate skill levels. But after that, what is next?

Determining that you, indeed, hired the right person for the job will not automatically ensure the person is successful in learning and mastering the job. The most important step in the employment process is seeing to it that the individual’s core knowledge, skills and abilities are applied in learning and mastering the tasks which they were hired to perform. That is where the money is made. If no structured on-the-job training is in place to train the individual and measure performance, all the money and time that went into assessing the individual is wasted and any raw talent that does exist remains un/underdeveloped.

Many companies have been persuaded to use “canned,” nationally validated assessments because of their endorsement by national industry groups or academic institutions – sometimes unaware of the validity requirements as they apply to the employer. An assessment that is validated for a job title in an industry is not automatically valid for a specific employer’s job classification, or useful for that matter. Many states have come up with their own version of assessments and are aggressively promoting them to employers.

We have come across instances where the assessment was over-marketed to the employer as the only solution necessary for workforce selection and development, leading to ineffectiveness and disappointment. Will those groups promoting the assessments be there for the company if the desired results were not achieved or the assessment is challenged?

Once Proactive Technologies performs a job/task analysis to set up a structured on-the-job training program, the data is also available to select or “assess” any assessments currently in use, or to be used, by the employer for content validity and, therefore, legal defensibility. Assessment criteria of an assessment in use may be shown to not be content invalid and not job-related, and use of that criteria should be modified or stopped. The job/task analysis data may also identify additional areas that could be, and should be, assessed as a more predictive evaluation of potential success in learning, and performance in, the job. Proactive Technologies has partners that provide and administer assessments and routinely use the job/task analysis data provided by Proactive Technologies to offer the mutual client the most appropriate selection and evaluation instruments, aligned with the expected training and performance outcome.

It is this combined approach of solid assessment of candidates and structured on-the-job training that lets the employer realize the intent of employment; hiring the most qualified candidates to learn and master the exact tasks of the job classification. Even if the most accurate assessment tool is used, assessing an individual without planning for the next step is literally like buying the latest technology machine based on perfect world specifications, only to find that in your world its output is not any where near expectations and the tools are lacking to increase it.

Redirecting your focus from selection alone to selection and development will maximize your results and maximize the return on worker investment.