3 steps for choosing the right assessment

Posted by | No Tags | Articles | No Comments on 3 steps for choosing the right assessment

Summary:
This article aims to inform HR (Human Resources), OD (Organizational Development), and other professionals on three simple steps to for choosing a test or assessment for use in their organization. Specifically, this article discusses (a) the goal of the assessment, (b) where to find information about specific published assessments, and (c) what information should be used to gauge an assessment’s quality.

 
Introduction

Organizations are increasingly recognizing the value that personnel tests and assessments can bring towards improving their effectiveness. Typing the words “personality assessment” in Google alone now yields over four million hits. While not all of these represent distinct assessments, it does provide a good indication of its growing popularity.

 
Needless to say, when carefully selected and used appropriately, assessments can add tremendous value to organizations; be that for personnel/executive selection, training and development, improving group performance, managing conflict, or as a general tool of enhancing self-awareness. Given the overwhelming number of assessments in existence, however, it can be daunting for the average consumer to choose the right assessment to fit their need. This is one reason why word-of-mouth endorsements are one of the most common methods by which decisions are made.

 
While word-of-mouth testimonies are useful, this approach can introduce more subjectivity (i.e., over-reliance on others’ opinions) than is ideal. It is, in fact, more prudent to do our own due-diligence to more objectively determine the relative quality of assessments under consideration.

 
I outline here three simple steps to follow before deciding on an assessment. If one wishes to be more thorough, there are additional considerations – e.g., test creator’s background, theoretical rigor, validation sample characteristics, fairness/adverse impact, etc. – the last of which becomes increasingly important as the stakes increase. However, for simplicity’s sake, these topics will be addressed in greater detail in later articles. For the present article, my goal is to provide the bare minimum steps one should consider.

 
1. Purpose of assessment: What am I trying to measure?

This first question might seem obvious but it’s more complex than one might think. Generally speaking, the more closely an assessment is to the purpose for which it is being used, the more valuable will be the information obtained. Because all assessments only provide a partial picture of an individual’s complex and dynamic set of personal qualities, it is best to start by asking yourself, what am I trying to achieve?

 
The second reason why it is important to start with the goal (rather than starting with an assessment) is because if we opt for the latter, we tend to squeeze individuals into categories defined by the assessment. Think of this as going to a shoe-store and randomly selecting a pair of shoes and doing whatever needs to be done to fit into the shoes. Knowing that you’re looking for a size 11, men’s running shoes, in white, around $75 is a better way and when it comes to assessments, this approach will maximize the relevance (or validity) of the obtained scores and interpretation.

 
2. The search: Where do I go to find the right assessment?

For most reputable tests (i.e., those that have at least a semblance of scientific rigor), there is a convenient online website that provides quick summary as well as critical psychometric properties for most published assessments. Buros Institute (http://buros.unl.edu/buros/jsp/search.jsp) of University of Nebraska-Lincoln, publishes review on over 3500 assessments – both in print and online. There is a small fee to view reviews on individual assessments but most public and university libraries carry a print version that can easily be used as a source of reference.

 
Individual reviews contain a brief description as well as key specifications such as population for which the assessment is designed, publisher, cost, and purpose. More importantly, they contain information about test length (average time to complete), format (multiple-choice, fill-in etc.), norm information (the population on which information was gathered to develop the assessment), and lastly, information related to its reliability (a measure of consistency) and validity (a measure of truthfulness or how well the assessment is actually measuring what it is intending to measure). Given the importance of reliability and validity, I devote the third step to these two issues.

 
3. Evaluation: What do I look for?

Reliability

If you hop on a bathroom scale every morning and it gives you a different reading, there’s a good chance that your scale isn’t a reliable gauge of your actual weight. The same applies to assessments. If an assessment yields a dramatically different score each time it is taken, it is considered unstable or unreliable. Reliabilities are gauged using various methods but the most common index is referred to as Cronbach’s alpha and the value that this formula yields is known as the alpha coefficient. A general rule of thumb is that an alpha coefficient of .70 or higher is considered to be sufficiently reliable.

 

Validity

The final consideration is the test’s validity – or the question related to whether the test is actually measuring what it is intended to measure. This concept, like reliability, should not be viewed in terms of yes (it is valid) or no (it isn’t valid). Validity is gauged on a continuum from zero or no validity evidence to acceptable or high degree of evidence. Like reliability, there are multiple sources of validity evidence. In general, one needs to simply remember that validity coefficients of .30 or higher represents sufficient validity evidence.


No Comments

Leave a comment