Designing Leadership Competencies – Five Most Common Mistakes
Designing and using a set of reliable and valid leadership competencies is critical for organizations seeking to go from point A to point B. Leadership competencies are ideal for specifying the types of skills, knowledges, and behavioral traits you want your leaders to possess, and therefore, exhibit. Leadership competencies can used as part of a broader talent management system and succession planning, for leadership identification and development purposes, and/or as part of a performance management system – e.g., via a 360 degree feedback.
Despite the prohibitive cost associated with its design and use – anywhere from say, $75,000 to $400,000, depending on the size and scope of the project – many organizations fall short of the reliability and validity requirements set forth by such governing bodies as APA (American Psychological Association) and SIOP (Society for Industrial-Organizational Psychologists). As a psychometrician and a consultant, I have found the following five steps to be common mistakes in leadership competency design.
1. Starting from the top – Many believe (HR/OD professionals and consultants alike) that there are specific leadership competencies that are fixed – meaning that if you find leaders possessing certain qualities (critical thinking skills, strategic orientation, market awareness etc.), you can bring him/her onboard and he/she will bring out the magic. This is simply not true. Given the large number (in the hundreds) and variations in competencies purported to be important, it is highly unlikely that one set of competencies apply to all organizations. As with Olympic athletes, while one generally needs to be “fit,” this does not guarantee a gold medal in every event. In other words, like athletes, different organization need different sets of competencies in order to optimize their competitive edge. This means starting from the top. A solid set of competency, therefore, is modeled after what an organization needs; not something offered “off-the-shelf.” This step ensures that you identify precisely those skills you feel is “necessary” for your organization to be successful.
2. Validating with existing leaders – Once you have a set of competencies deemed important, many organizations fail to validate the competencies against incumbents. It is critical to check the competencies with existing leaders to obtain their thoughts on how important each competency (and related questions) is. This step is important for two reasons: First, it legitimizes the competencies in the eyes of those who are being assessed – i.e., the leaders themselves; thus, getting them to buy-in on the idea. Second, it satisfies the “face validity” requirement set forth in the major governing bodies. In other words, if I am design a test to select competent mechanics, I need to run the test by real mechanics in order to be sure that the test is measuring what it is intending to measure.
3. Validating against actual performance – Even if organizations are savvy enough to complete the first two steps, many skip over what is known as a “concurrent validation” process – i.e., assessing existing leaders on the new competencies and comparing those scores against their performance evaluations. The most objective was to do this is through a 360 degree assessment – i.e., obtaining judgments from subordinates, peers, and supervisors. In this way, an objective scores on each of the competency is obtained without the inflated scores often seen with self-rated assessments. After some reliability checks and data cleaning, the aggregated scores are compared against the same leaders’ past performance scores. If the set of competencies deemed important for an organization are, in fact, what the organization values, then the correlation between those same competencies and performance ratings should be positive. This is called “concurrent validation.”
4. Validation against future performance – In addition to validating against current performance levels, the same competencies should be compared against performance at some point in the future – e.g., 9 to 12 months. This technique, known as “predictive validation,” further ensures that the set of competencies are, in fact, deemed important and that leaders are being assessed – at least partially – on those competencies. Unfortunately, this step is rarely done in organizations. **Note: Although unlikely, it is possible that performance management system is entirely different, and event, opposed to the set of competencies being assessed. In such a case, the performance appraisal system needs to be refined to be aligned with the competencies.
5. Setting competency cut-off scores or categories – Finally, once data have been obtained on the competencies from each leader, there is a need classify individuals according to their scores on each of the competencies. While it may be tempting to choose an arbitrary category, this may lead to classifying most, if not all, leaders as needing improvement. Thus, it is better to allow the data to choose the size of the difference that is meaningful to the population being tested. This approach, known as the “data-driven” method, is done by using the “standard deviation” (or average difference) of the competency scores. This approach ensures that the different score categories are based on the population at hand (each organization will differ on what this deviation score is) and is a known “legally defensible” strategy.
In sum, there are a number of common mistakes (mostly by omission) that occur in the development of leadership competencies. While the above steps may seem overly rigorous, there are ways to incorporate them without making the process overly daunting. The result is a set of competencies that (a) will be viewed by incumbents as legitimate, (b) demonstrates proof that it is measuring what it is supposed to be measuring, and (c) ultimately allow you to place complete confidence in the intervention stemming from its use.