Guest post by Jack E. Clark, PhD, Clark & Chase Research, Inc.With executive compensation under scrutiny, salary surveys are increasingly important. If done correctly, surveys provide reasonable estimates of what you could expect if you had access to this data for every credit union. Surveys are far superior to decisions based on guesswork or anecdotal information picked up from some of your colleagues. However, not all surveys are the same and the user needs to know what to look for.
How representative is the sample?
A good survey should obtain a sample representative of the larger population. For example, were all credit unions in the population of interest approached or did they at least have an equal chance of being approached? Was there anything in the selection that may have excluded important groups or favored others? If so, how would that affect results?
Some factors cannot be controlled but can be taken into account when analyzing the results. For example, the smaller the credit union, the more likely it is to forego this type of survey. As a result, the total sample may be over-represented by larger credit unions. This can have a misleading impact on overall findings especially anything influenced by asset size. However, an adjustment can be made during analysis (weighting) that brings asset size back into proper proportion, allowing for a more realistic estimate.
Is the sample size adequate?
A larger sample lowers a survey’s margin of error and increases our level of confidence in the results. However, it is not enough to only pay attention to the overall survey’s sample size. Since decision-makers are mostly interested in the findings for credit unions similar to theirs, the results are broken down by various groups. And the sample size for each group is what really matters.
The problem occurs when findings are reported based on very small sample sizes. For example, findings based on a sample size of five have such a high margin of error that the results are unreliable and meaningless, even if they look official.
What minimum sample size is recommended? There is no single standard but most recommendations range between 30 and 100. A single standard is difficult because it is a matter of degree – we cannot say a sample size of 30 is clearly adequate but 29 is clearly inadequate. Decision-makers would be well-served to simply pay attention to sample size and adjust their level of caution based on that.
Percentile Ranges
Compensation is typically presented in percentiles. If compensation is $90,000 at the 25th percentile – it means 25% of those surveyed are compensated at or below $90,000. Percentiles can help you see the range of compensation paid by credit unions.
It is recommended the decision-maker initially focus on the compensation between the 25th and 75th percentiles. Some compensation experts feel this inter-quartile range (IQR) provides a more meaningful measure.[1] We can see the reason for this in the following table from a recent survey.
When we focus on the amounts at the 10th and 90th percentiles, there is overlap as you move from one asset group to the next. The compensation at the 90th percentile is often higher than the 10th percentile in the next larger asset group – seemingly contradicting the tendency for compensation to increase with asset size.
The 10th and 90th percentiles represent values at extremes of these ranges – sometimes referred to as outliers. When we focus on values between the 25th and 75th percentiles, there tends to be much less overlap.
Besides asset size, there are other reasons that may influence the compensation figures we see at the extremes of these ranges. Every credit union’s situation is unique. And even though you may initially focus attention on the IQR, your credit union’s situation may justify a figure above or even below that range. In short, the IQR is meant as a guideline but should not be seen as a constraint.
Comparing Results From Two or More Surveys
You can enhance your compensation strategy by reviewing the results of more than one salary survey, if they are conducted in a valid and reliable manner. However, you will want to consider if there are differences between them and how those differences may impact results.
For example, we conduct a survey of all Federal credit unions for NAFCU and Burns-Fazzi Brock. If another survey includes both state and Federal credit unions, you would want to consider if and how that difference affects comparability.
Perhaps most difficulties in comparing survey results are due to differences in how each survey groups the data. For example, your credit union has $200 million in total assets. One survey includes your credit union in an asset group of $100 million to $300 million and the other survey includes it in a group of $200 million to $400 million. Because asset size and compensation are strongly related, the compensation figures for these two asset ranges will be different, making the comparison more challenging.
The Methodology section for any survey report is worthwhile reading. It can provide you information to help you decide on the survey’s worth as well as its limitations. And if the information you are looking for is not provided, you may want to consider asking for it.
Opportunity Alert! Sign up to participate in the 2012 NAFCU & Burns-Fazzi, Brock Executive Compensation & Benefits Survey for Federal Credit Unions and get the results report FREE!
[1] Haley, R.M., Measures of Central Tendency, Location and Dispersion in Salary Survey Research, Compensation and Benefits Review, September/October, 2004, pp. 39-52.
How representative is the sample?
A good survey should obtain a sample representative of the larger population. For example, were all credit unions in the population of interest approached or did they at least have an equal chance of being approached? Was there anything in the selection that may have excluded important groups or favored others? If so, how would that affect results?
Some factors cannot be controlled but can be taken into account when analyzing the results. For example, the smaller the credit union, the more likely it is to forego this type of survey. As a result, the total sample may be over-represented by larger credit unions. This can have a misleading impact on overall findings especially anything influenced by asset size. However, an adjustment can be made during analysis (weighting) that brings asset size back into proper proportion, allowing for a more realistic estimate.
Is the sample size adequate?
A larger sample lowers a survey’s margin of error and increases our level of confidence in the results. However, it is not enough to only pay attention to the overall survey’s sample size. Since decision-makers are mostly interested in the findings for credit unions similar to theirs, the results are broken down by various groups. And the sample size for each group is what really matters.
The problem occurs when findings are reported based on very small sample sizes. For example, findings based on a sample size of five have such a high margin of error that the results are unreliable and meaningless, even if they look official.
What minimum sample size is recommended? There is no single standard but most recommendations range between 30 and 100. A single standard is difficult because it is a matter of degree – we cannot say a sample size of 30 is clearly adequate but 29 is clearly inadequate. Decision-makers would be well-served to simply pay attention to sample size and adjust their level of caution based on that.
Percentile Ranges
Compensation is typically presented in percentiles. If compensation is $90,000 at the 25th percentile – it means 25% of those surveyed are compensated at or below $90,000. Percentiles can help you see the range of compensation paid by credit unions.
It is recommended the decision-maker initially focus on the compensation between the 25th and 75th percentiles. Some compensation experts feel this inter-quartile range (IQR) provides a more meaningful measure.[1] We can see the reason for this in the following table from a recent survey.
When we focus on the amounts at the 10th and 90th percentiles, there is overlap as you move from one asset group to the next. The compensation at the 90th percentile is often higher than the 10th percentile in the next larger asset group – seemingly contradicting the tendency for compensation to increase with asset size.
The 10th and 90th percentiles represent values at extremes of these ranges – sometimes referred to as outliers. When we focus on values between the 25th and 75th percentiles, there tends to be much less overlap.
Besides asset size, there are other reasons that may influence the compensation figures we see at the extremes of these ranges. Every credit union’s situation is unique. And even though you may initially focus attention on the IQR, your credit union’s situation may justify a figure above or even below that range. In short, the IQR is meant as a guideline but should not be seen as a constraint.
Comparing Results From Two or More Surveys
You can enhance your compensation strategy by reviewing the results of more than one salary survey, if they are conducted in a valid and reliable manner. However, you will want to consider if there are differences between them and how those differences may impact results.
For example, we conduct a survey of all Federal credit unions for NAFCU and Burns-Fazzi Brock. If another survey includes both state and Federal credit unions, you would want to consider if and how that difference affects comparability.
Perhaps most difficulties in comparing survey results are due to differences in how each survey groups the data. For example, your credit union has $200 million in total assets. One survey includes your credit union in an asset group of $100 million to $300 million and the other survey includes it in a group of $200 million to $400 million. Because asset size and compensation are strongly related, the compensation figures for these two asset ranges will be different, making the comparison more challenging.
The Methodology section for any survey report is worthwhile reading. It can provide you information to help you decide on the survey’s worth as well as its limitations. And if the information you are looking for is not provided, you may want to consider asking for it.
Opportunity Alert! Sign up to participate in the 2012 NAFCU & Burns-Fazzi, Brock Executive Compensation & Benefits Survey for Federal Credit Unions and get the results report FREE!
[1] Haley, R.M., Measures of Central Tendency, Location and Dispersion in Salary Survey Research, Compensation and Benefits Review, September/October, 2004, pp. 39-52.
Comments
Post a Comment
Please no profanity or political comments.