Developing Predictive Selection Criteria

At ECI, we have been devising selection criteria for many years to identify people who will be top performing sales people.  The trick to identifying predictive, reliable criteria is to make sure you have a good linear relationship between the results that top performers deliver and the results that bottom performers deliver.   Top performers deliver better results and bottom performers deliver poorer results, so we pay the good guys lots more than the guys who are less successful.  Sounds logical?  Well, for many companies, it isn’t so simple any more.

In an effort to devise a compensation or rating plan that keeps everyone happy and generates a proper reward for good work, companies often create quite complex formulae to use in tracking the sales rep’s performance to determine final pay outs in compensation.  The most complicated of these will allow for several different factors to figure into the final payout, such as the product mix or territory potential multipliers, along with some complicated additives to offset how the overall corporation performed.  In the better of these, at least there is an equalizing factor that can be reviewed and used that really is linear in demonstrating the comparison of top and bottom performers.

So why not just use the top performing group?  This is a question people ask us all the time.  The reason you can’t just look at the top is that you won’t be able to tell which of the common factors are true of the total population and which are predictive of success for only the top group. Without a better comparative, you are in effect guessing.

One would think that if you work for Company A and are the best sales rep , you will probably take home the biggest check.  But these days, if you work for Company A and are the best performer and live in Podunkia, South Dakota, you might earn 80% of the highest pay rate, because it costs a lot less to live there than it does in San Diego, California.   All of this is part of the complexity that plays into doing a statistical analysis of data in order to identify predictive selection criteria.

So simple is always better.  Sales rank works, providing it is not complicated by combining the results for multiple products, which it often does.  In these cases, we have to take a look at results for each product, then divide the groups by quartiles, then by product to identify commonalities of the highest group v. the lower groups.   What we are really concerned about is not the difference between performer #1 and performer #6, but rather identifying the factors that differentiate performers #1-150 from performers #1207-2509.

By getting the criteria straight first, the resulting models have a much better opportunity to be predictive.   At ECI it is about putting science behind behavioral factors.  Sometimes that endeavor is a bit more complicated than it should be these days.

Get Ready for Changes in Interviewing!

The most read topic on The ECI Blog is about behavioral event interviewing and how to properly answer these questions.  Many people are becoming very familiar with the behavioral event interviewing process and are getting coaching prior to their interviews from their placement counselors.  This, of course, has the effect of masking an individual candidate’s true potentials.

For many years in Europe, employers have been inviting potential candidates to simulation workshops as a part of the selection process.  This is a very time-consuming and costly step, but considering that the employment dismissal regulations are so much more stringent in many countries than they are in the US,  the process provides real understanding of a candidate’s real capabilities.

ECI has experience in simulations, given that we offer workshops and assessment centers for current employees.  Taking that knowledge into account, and knowing that simulations are good assessment tools, we have recently incorporated scenarios within our updates and improvements to our Structured Interview Process.

What is a scenario?  This is a simulated exercise that candidates complete during the interview process to see how they will respond under pressure, how able they are to think on their feet, and to evaluate what type of experiences they have had.  Good scenarios enable the interviewer to assess some of the more subtle talents, such as judgment, decision-making, knowledge of the business environment and how to devise a good strategy for solving a typical problem.

By seeing how candidates address the scenarios, how much urgency they put behind taking action or not taking, action as the case may be, and how much depth of the business environment they actually apply, the interviewer gains a real understanding of how the individual might react after the hire.  The very best scenarios are those which might have occurred in the particular employer’s environment.

Think about several situations at your company that have been mishandled or that have the potential to be mishandled by employees who lack the skills and abilities you require in a particular position.  The best scenarios are realistic and typical of the work that is part of the job you are filling.  Here are a couple of scenarios for your reference:

1. Our organization has a very flat managerial structure.  You will find yourself working with people from different disciplines who often have differing objectives and work styles.  Tell me how you would handle working with the President of the company when she knows nothing of your work, yet is trying to demand that you follow her lead on the approach to a particular project.  What strategy would you recommend?  How would you challenge her ideas?
2. Our clients drive our projects.  We cannot control clients’ calendars and so we must adjust our own priorities in order to keep several projects moving along successfully.  Tell me how you would respond when two clients call on the same day and ask for completion dates that overlap on major projects.  Assume that both of these clients are major accounts and each has a big-ticket project for you to complete.  What problems are you likely to encounter?  How will you address these problems?

Once you have assembled six to ten examples, ask a broad range of your current employees in the job to tell you how they would address each scenario.  Evaluate all the responses and determine which responses are best, which are poorest, and which are only average.  If you have one of your scenarios that everyone does well on or everyone does poorly on, eliminate it.   This scenario is either too hard or too easy and is one that really will not return much in the interview process, since it is unable to separate top performers from less effective performers.

By taking time to prepare your scenarios in this way, you will have a better idea of how to assess your candidates’ responses and will be able to recognize candidates who have better potential than others from your pool.   As you use the scenarios, be careful not to lead candidates by offering any feedback or response confirmation.  Some candidates are very good assessors of people and you could be influencing the responses by providing responses.

If you would like more information about using scenarios in your interview process, give ECI a call and we can assist you in upgrading your selection process.

How to Hire Without Getting Sued

I used to do a lot of seminars and speaking engagements for various groups on topics relating to human capital.  One of the more requested topics was the title of this post.  There are a few good practices to be sure to consider in order to avoid suits from your candidates as you hire new people.  And hiring is beginning to increase!  This is the good news.

The bad news is that some new regulations have impacted the hiring environment and you need to be aware of these as you set about finding that long-awaited new hire.

The OFCCP has released recent guidelines around internet candidates.  If you use web job search services, such as Monster, Career Builder, Snag-A-Job or other such applications to advertise your open positions, be sure that you are clear about stating the steps in the process for applying for the job and the background and experiences you will require in order to be considered as a qualified applicant .   Otherwise, anyone who leaves a post or submits a partial request must be considered, whether they have the qualifications or not.

Also, ask all QUALIFIED applicants to respond to various requests throughout the process, such as attaching a resume, completing an application and returning it within the deadlines you communicate, providing complete references (phone numbers, names and job titles) of supervisors within any prior work experiences, or answering job related questions, in order to show their continued interest in the position.  If the candidate doesn’t respond, then drop them from the selection process.

One of the issues with using on-line services is that they are so accessible and you are likely to get many, many responses to any job postings you open up.  Without the extra steps, anyone who even leaves a partial interest has to be considered in the process, whether they have the qualifications or not.

Make sure that you establish the criteria for selection and stick with it.  If you post the job qualifications, then ultimately hire someone who lacks these qualifications, anyone who does have the qualifications and who applied has a cause of action to sue you for discrimination.   Not good.  If you find that the person you require is not within the pool of candidates who apply, better to close out the listing, refine the qualifications and repost the job opening than to just update the current listing.

Here is an example for you to ponder over.  Recently, a school district in New Jersey posted a job opening for a Kindergarten teacher.  In the ad, they stated that all candidates must be NJ certified, have security clearance, have a degree in Early Childhood Education and that they have 1 year in teaching children in grades Pre-K through grade 3.  Over 600 applicants applied for the job.

The School Board ultimately hired an internal candidate who had served as an aide for the Kindergarten class for 2 years.  She had a degree in Social Studies, not Early Childhood Ed, and had supported the teacher (not led the instruction of the class).  So in essence, the only requirement she met was that she had security clearance.  Hmmmm.  I would bet the farm that there were MANY individuals who did apply for the position who met the criteria better than the individual who was ultimately hired.  This would be a good claim to pursue, were you one of the candidates who was not interviewed or considered and who had the proper qualifications.

Treat all applicants the same and consider them on the merits of their qualifications, background and experiences.  If you don’t believe a person will fit your environment, because he/she has green hair, lots of visable tattoos and does not dress appropriately for a business interface with customers,  make sure that the person you do hire doesn’t also have a bunch of tattoos that you just didn’t see during the interview process, or that your secretary isn’t sitting at the front desk with his hair dyed Daffodil this week, since it is springtime.  Do you follow the line of reasoning?

When you start splitting hairs over how people dress and what they do in their personal life, you are bound to have some problems in the selection process over inconsistent selection criteria.  I am not even sure that we can require that people not to have Daffodil hair or visable tattoos these days.   But if you choose to set such requirements at your place of business,  ensure that you are expecting not just applicants to live within the guidelines and that you act to correct those individuals who work for you and fail to meet your guidelines.

Wait until the successful candidate has accepted the position and actually started work before you notify other candidates that the position has been filled.  I hate to tell you how many times I have been told that someone thought they had the job filled, then found out that the finalist has chosen to take another more suitable (in their minds!) position at the last moment.  This step of waiting will enable you to move to Plan B on your list, should Plan A not work out as planned.

There are many, many, many other things to think about as you hire, such as taking care when  handling confidential data, being sure to gather references and check backgrounds, paying the appropriate wage for the work of the position (rather than for what the candidate attempts to negotiate) and a raft of other considerations.  Happy Hiring!

Favorite Interview Questions for Sales People

Some time ago, we began to develop behavioral event interview guides for our clients to help them to assess the potentials and capabilities of candidates during the selection process.   This approach can be used to assess candidates for any type of job, but when the questions are designed, you need to be sure they are relevant to the content of the particular role.

You can learn a lot about someone from using this approach to interviewing.  The notion behind behavioral event interviewing is to ask about a specific situation in which the candidate demonstrated a particular capability in order to determine the individual’s effectiveness in that situation.   Situation, Action and Result, often referred to by one particular vendor as the STAR model…the T is for Task.

The trick to utilizing this approach is to pick questions that are key to the job and to understand that people who are good at a particular skill will answer the question differently than others might.  Let’s look at sales people, for example.  One of the primary skills required is that the individual has to be able to ask for the sale.  To assess this capability, here is a question to use:

• Tell me about a time when you failed to close a sale.  What did you do in this situation?  What was the outcome?  What would you do differently?

Effective sales people will have a unique tale to tell you about a nightmare buyer who was thought to be the decision maker, but was not, or was someone who was simply shopping the competition.  They will include a myriad of details about the interaction and tell you how they reacted when they faced a similar situation in the future.

Less effective sales people will tell you that they were unable to connect with the particular buyer and that the buyer shut them out before they had the chance to ask for the business.  They will also tell you that next time, they worked hard to identify the actual decision maker.

See the difference?  One is more realistic (the effective sales person) and the other is more process driven, following the sales protocol.  High performers in sales need to make an impact quickly and when they are faced with a challenge, they do creative things to get around the issues.

Here’s another question:

• Tell me about a time when you didn’t get the sale, when you were sure you would.  Why did this occur?  What would you do differently?

In this case, the high performer will give you an example and tell you why they believe they were beaten out.  It is rarely about price!  He/she will also give you a great analysis of why they failed and will look at the situation as an unique incident from which they learned something important to them.  The lower performers will tell you  “you can’t win them all” and that if you throw enough proposals at buyers, eventually something will stick.  Not a very good way of focusing your use of time, if you are in sales.

You are probably asking, so how do I know what a good answer is and what a bad answer is so that when I interview I can tell the difference?  There are so many variables in interviewing that the answer really is through experience.  If you are new to interviewing, ask someone who is experienced in sales, for example, to sit with you while you interview the candidate.  After the interview has been completed, ask your partner to discuss the candidate’s responses and to tell you what they think of him or her.  Compare the results of your impressions with your partner’s results. Did you agree or disagree? Why?  Can you see what the experienced sales person felt differently about the responses when you disagreed?

After you have interviewed several candidates and have some experience, you will learn how to assess the quality of the answers.  Some questions have been proven to be more valid in assessing sales skills.  That’s why you might want to consider calling ECI to assist your company in developing and focusing questions on the skills and abilities that count in your marketplace!  You can select better people and enhance results for your organization.

Who’s Hiring?

The short answer, lots of companies.  The more complete answer, however, is companies where there is expected turn over and companies who provide services to people.  The companies who are hiring are often the larger organizations who always have openings because of people moving up, people moving on and people retiring.  To apply and get noticed by these organizations, you need to look at what jobs are offered and how well you match the posted job requirements. These are often the US major employers.

The second group, the service providers, are companies or organizations that provide services to the population, the elderly, home health care, governmental agencies, and those organizations where people go to get help for a particular need.  There are a good many opportunities in these sectors for jobs.  Nursing and health providers, lab positions, claims managers and insurance positions, support staff in hospitals and institutions, clinics and medical facilities have openings as well.

If you want to get noticed by these potential employers, how will you do this?  Here are some ideas to help you along.

1.  Make sure to read the job qualifications and only apply if you can meet these qualifications.  Many on-line search engines use coded queries to pick out those applicants who list the specific experiences presented in the job posting.  If you have the experience, then show it in your application in the terminology used to describe the position in the job posting.  Don’t trust that someone will read your written responses and be able to figure out that you really do have the experience, particularly if you didn’t list it the way it was stated in the job posting.   You are probably dealing with software doing the job of the initial screening and it will do this through a word-matching and number of years matching process.

2.  Follow the directions.  If the posting says “no calls, please”, then don’t call, unless you know someone who works at the company who can put in a good word for you.  You want the potential employer to know that you understand how to follow directions.

3.  Meet the deadlines.  If the posting says “submit your complete information by Friday, April 5, then have your information completed and submitted by close of business on Friday, April 5th.

4.  If you need special certifications for particular jobs, such as licenses or training, be sure that you have completed this training, testing, or classroom activities, and that you have the proof of your successful passing of any tests required.  If you are interested in particular jobs, such as nursing or lab technician positions, then go to school and get your certifications while you are looking for a new job.  This way, you may be able to get your school to help you find a job in this area, as many companies recruit from local training institutions.

5.  If you graduated from a college, go back to your alma mater and ask them for assistance with job hunting.  Many colleges have excellent staff who can help you find a new position and often have requests for people of a specific background, just waiting for someone to express interest.

The long and the short of it is that there are still  jobs out there.  The caveat is that the job you think you want may not be the one you will find.  A lot of the old standard jobs aren’t available any longer.  But there are some new jobs for you to consider.   Be open minded and look at the posted experience requirements.  If you can comply, go ahead and apply.  Be persistent every day.  You just might find a job you will really enjoy and one you never expected you might get hired for!

Is the Downturn Opportunity for Businesses?

Let’s add another positive point for the opportunities that arise when the economy is in a downturn.  For those of you company owners/managers who have job openings, and this will happen as a result of normal turnover, people moving around to position themselves more favorably in a new role, and for a variety of other reasons, the downturn in the economy can be a real opportunity to enhance the quality of your staff.

I recently read that the “real” number for unemployment is somewhere around 12.5% ,if you include all the people who have given up on finding a job, those who don’t enroll to collect unemployment, and those whose unemployment has run out, or those who work part-time and who cannot apply for unemployment.  As an employer, there really are some very good people out there looking for work.  This is a time when a good selection process can really help you find someone who will in fact do well for you for the long-term.  You have the time to look into the background of the candidate, you have the time to conduct several interviews, because you aren’t worried that someone else may snatch the candidate up, and you have time to have several people interview the candidate.  By taking your time, you can spread your net far enough to attract people who might not have looked at your open position a year ago because your company was too big, too small, too far away, or didn’t really fit their bill of expectations at that moment.  These days, people will often consider most any job if it means it is a viable position where the pay is steady and the expectation is that the job will last for awhile.

Follow the best practices of hiring when advertising your open position.  Use a job search engine, such as CareerBuilder, Monster or for hourly positions, Snag-A-Job.  Most services cost around $250 for 90 days, but given that a newspaper ad will cost you about the same, this is a better way to get a lot of interest in the job.

Second, be sure to describe the job and the background you will be requiring of each qualified candidate.   Do they need prior experience? A degree, knowledge or certifications?  Be sure to describe what the environment is like as well.  Add your logo and your website so that people can visit and find out more about your company.

Third, review every applicant you receive and call each person whose information meets your requirements.  Schedule your interviews and talk with each candidate.  It is a good idea to have more than one person interview your candidates.  Then, compare impressions to select the best person.

Today, you will find that there are many people who are over-qualified for the job applying for the position.  Be up front with your applicants and let them know what the going rate is for the work.  If the candidate believes they should be paid more, it is your decision to negotiate higher, but I advise against that.  In my mind, you should be able to find someone who will be willing to do the job at the fair rate.  While this perspective might sound a bit harsh to some folks, there are a lot of very able people out there who will be willing to take the job and be paid fairly to do the work required.  For employers, you just need to find them and offer them the chance!

Advice for Job Seekers

Tough times out there for job hunters and I am more than sympathetic.  Ted told me yesterday that the economy is really driven on consumer purchasing.  When people are not afraid, they spend, whether they have the money or not.   This is because they believe that tomorrow will be a better day.  Today, most people are wondering about that one and many are worried that things will be so different that they won’t be able to survive.

As a caveat, I personally believe that the US government should allow people to collect a paycheck in a job and augment that with partial unemployment payments to yield a higher, livable wage, to encourage people to get working again.  Most of the entrepreneurs in the job market have taken a couple of part-time or full time minimum wage jobs to yield the same results.  To my mind, I would rather see 2 people working, rather than one person trying to make ends meet by working a 16-hour day.

Further, it is not a good practice to have someone say “I make more on unemployment than I would if I worked,” and thereby cut out the incentive to find a job.  All this does is encourage long-term collection of unemployment benefits and fails to get money into people’s hands so that they can start spending again.  Remember Ted’s perspective on the consumer spending economy?

Let’s talk about how to position yourself to find a meaningful job, hopefully better than 2 part-time or full-time filler jobs.  Even if you are applying for an hourly position, you should put your resume together.  This is a handy document to reference if you are required to fill out an application, since all the information is right there on a sheet of paper.  Also, it makes a great impression on your potential employer when you bring along your resume.

So, to put your best foot forward, take some time to visit MS Office on the web and download a resume template that suits your style.  Accurately fill in the blanks with your education, background, prior work experience and job objectives.  DO NOT go over 2 pages in explaining what you have done on your resume.

Also, I recommend that you include only the last 10 years of experience and work information by employer.  If your experience is older than 10 years, you probably are not prepared to qualify for your old job today.  If you want to include this information, under your last employer bullet on the resume, put a general statement, such as this one for an engineer  “prior employment – experience in leading an engineering team in process improvement, experience managing rolling process team in a manufacturing environment, experience in retail sales.”

1. Proof read and spell check your resume.  Use clean paper and do not have any white-outs or erasures.  Do not prepare your resume as you eat your lunch and spill food on it.  All of these things get your resume pushed to the bottom of the circular file.
2. Your resume is your first impression to your future employer.  In the job objective section, tell why you think you would be a good prospect for the employer.  What are the contributions you will bring to the employer? A couple good sentences are all you need.  It’s not about you…it’s about them…
3. In searching for companies to submit your nice new resume to, identify key words that appear in their description of the job, particularly as it relates to qualifications.  Be sure to incorporate those key words within your resume, since most companies use key word searches to pick out resumes from the thousands they receive.
4. Search for jobs that you qualify for.  This doesn’t mean you have to get a job like the one you used to have at all.  It just means that you need to be able to qualify for the position, based on your past experiences and educational background.  If the job posting says 4-year degree required and you don’t have a 4-year degree, submitting your resume will most likely get you screened out immediately.  Apply for the jobs in which the background and requirements match your qualifications.  With all the new legislation surrounding the internet tracking of candidates, companies today shouldn’t be considering candidates who do not meet their posted job requirements.
5. Be open minded.  The world is changing and things aren’t like they used to be. Unless you are engaged in a job where there are plenty of openings, think outside the box by going to various job search sites and putting in critical skills you have and searching that way, rather than by job title.  Then read all about these available jobs and apply for those that interest you.
6. Attend training or skills improvement classes.  By showing that you are actively preparing for a new role, your employer will see that you have the initiative they just might value in a new employee.

Next time, let’s talk about the interview process.  I’ll share some good advice on how to present yourself to make a good, positive and memorable impression on your potential employer.

Establishing Validity of Selection Criteria

We build structured hiring processes for our corporate clients.  Some of the processes we see, prior to the revision, are long-drawn out systems of multiple steps and the collection of data that one would be hard pressed to review, much less show that the step was job related.  The issue of valid selection criteria is emerging as a area of interest to employment lawyers.  Today, you need to be able to show the validity and business necessity of any selection criteria you use.  You must be able to document that the standards you use for selection are in fact essential to the job and/or predictive of success on the job, while not causing disparate impact on protected classes.  You must be able to show through statistical analysis of your workforce data that the selection criteria is in fact related to factors demonstrated by your own workforce at particular established levels.

Criteria Validation:  There is a lot of new information out there to be concerned with in devising your selection criteria.  If you can answer these questions with a “yes” you are probably on the right track:

1. Did you do Job Analysis on the job with several individuals (high, medium and low performers) to determine the essential functions of the position, the context and working conditions of the job, the technical skills or knowledge needed to do the job, the ADA standards required of workers?
2. Do you have a compliant job description which is up-to-date and incorporates essential functions of the job based on the information you gathered in Job Analysis?
3. When you established your job posting for the position, did you ensure that you are using criteria that is evident in the current work force?
4. Do you have multiple steps in the process that produce job-related information on the candidate upon which you base your decision?
5. Do you rely too heavily on a single phase of the process for making your final decision?
6. Do you handle each applicant the same way and does every qualified candidate go through every step?
7. Is your selection process free from steps that cause disparate impact within protected groups?
8. Do you know whether your selection criteria is valid?  And by this, I don’t mean that you pulled a focus group together to look at the new criteria and everyone agreed it was fine.

Establishing Valid Selection Criteria:  In these times, employers need to be particularly careful in managing the selection process because people are so desparate for work and many candidates are willing to assume significant risk in challenging an employment decision which negatively impacts them.

Here is a recent posting on the Department of Labor website regarding the Uniform Guidelines for Selection:

The degree of relationship between selection procedure scores and criterion measures should be examined and computed, using professionally acceptable statistical procedures. Generally, a selection procedure is considered related to the criterion, for the purposes of these guidelines, when the relationship between performance on the procedure and performance on the criterion measure is statistically significant at the 0.05 level of significance, which means that it is sufficiently high as to have a probability of no more than one (1) in twenty (20) to have occurred by chance. Absence of a statistically significant relationship between a selection procedure and job performance should not necessarily discourage other investigations of the validity of that selection procedure.

While you might wonder how you are going to accomplish this, it might not be as difficult as you think.  What this particular guideline is requiring is that you can show the relationship between the criteria or steps you use in the selection process and actual performance of the job.  For example, if you do an analysis of your workforce and find that only 15% of the workforce has a 4-year degree and you are requiring the degree in your job posting or in the job description, you have a problem.  However, if you say that a degree is desirable and will accept relevant work experience instead (providing this is true of your analysis of the existing workforce), your criteria should pass.

If you say you require a particular level of math skills, such as would be demonstrated through the administration and scoring of a test of basic math, but the job does not require the use of basic math, here again, you have a problem.

Begin by asking what particular skills are required in the job.  Does the person need to write memos?  You can find out by doing some job analysis with people in the job in your own company.  If you find that this is an essential part of the job, then you might assess writing skills.  But be sure that the content of the test is job specific and that the scores you use for establishing the pass rate are statistically significant at the .05 level with the scores demonstrated by existing  people in the job.   Unless you are up on statistics and test construction, I wouldn’t advise that you attempt this yourself.  There are plenty of good consulting firms you can hire at a reasonble cost to do the work for you.

Personality Tests:  How about behavioral measures, such as personality skills assessments?  Again, avoid any measures that incorporate non-job related content.  If you can show the relationship through statistical analysis between the results of the assessment and its ability to predict success on the job at the required level, then you are on a far better footing than if you simply use the test provider’s documentation of validity.

Many test providers give you an extensive validity report that shows that there is no disparate impact caused by the instrument or that the instrument is valid and reliable in measuring what it purports to measure.  The problem is that the OFCCP or the EEO won’t allow you to rely solely on the documentation for the instrument, unless you do the diligence of validating your selection standards within in your own workplace.  And of course, if there are holes in the test documentation, they will use that in making their case against your invalid selection criteria.

There are a few exceptions to this on-site study guideline, such as you have too few people in a role to do a study, performing a study is virtually impossible since everyone with the job does considerably different work, the role might be new to your organization, AND  the job you have is similar to one in which the validation study work was originally completed.  However, you still will have the obligation to monitor the impact of the assessment regularly, to show you did some Job Analysis after a period of time to understand the work context, and that you have a good and up-to-date job description.   If you are a good sized organization (over 50, although many regulations are applied at the 15 or above level), you need to look into the establishment of valid selection criteria and show how the criteria is statistically related to workers or the work actually being performed.

Tomorrow, I will discuss validation of interview questions, so tune back in.

If you would like additional information on validating selection criteria, visit the Department of Labor at either the www.eeoc.gov/policy/docs/factemployment_procedures.html or www.ofccp.gov/policy/docs and you will find some very useful documents there to enable you to understand the issues more fully.

Look for our upcoming Webinar on Establishing the Validity of Selection Criteria.  The sign-up sheet will be posted on our blog.

Part 2: Devising Selection Standards for Hiring New Staff

Adverse impact occurs where there is a statistically significant difference between selection, promotional, or compensation levels between members of a protected class and an appropriate comparison group.[1] However, according to the 1991 Civil Rights Act, this difference only amounts to discrimination if the tools used to make the business decision are invalid and not consistent with business necessity.  In other words, numerical and significant differences between the selection rates of groups are allowed, as long as the tools responsible for those differences are in and of themselves valid and reliable and have been developed following a rigid, standard, and legally defensible validation strategy.  But the employer is still responsible to show how the test is job-related and meets a business necessity, even in these cases.

Since the release of the April 2008 OFCCP guidelines update and some recent case law, the determination process now includes looking at the actual tools and systems used within the selection and promotion systems.  Validity and reliability tests will be completed by OFCCP’s new statisticians on these tools to ensure that they do not treat protected classes differently than they treat comparison groups.   In the old days, OFCCP audits were not too bad.  Today, they look like the process EEOC uses to evaluate disparate impact.

This new OFCCP analysis for disparate impact includes a thorough review of:

1. the Job Analysis results in which the skills, knowledge and abilities and personal characteristics were evaluated within a particular job are identified
2. how these skills, knowledge and abilities map back to the specific steps of the selection, promotion or termination processes
3. how tests that are administered in fact measure the skills, knowledge or abilities essential for success in the position
4. the job description, to ensure that it is current and accurately reflects the essential functions the position  within the company’s own environment

This much more rigorous assessment of selection, promotional and termination criteria could cause significant difficulties for many employers who are relying on old, non-existent practices or those who allow their hiring managers to utilize their own preferred processes.

The environment is further compounded through the use of popular internet selection systems, where potential candidates are allowed to apply for any job opening or can post their resumes on job sites whether they meet the stated qualifications or not.  Job seekers can perform searches to attach their resume to any number of open and advertised positions, with little consideration for the job title or understanding the selection process of the company offering the position.  Recent guidelines hold the potential employer accountable to track race, age, gender, and ethnicity data about all applicants.  This is why, at ECI, we added the tracking criteria to our eci-assessments site for all test takers.  When employers need the data, it will be available for all candidates who reached the assessment step.

In the absence of clear definition of an “applicant” by the employer, potential problems arise in determining how to track candidate data and determining whether the applicant is in fact even interested in being considered for the position.   When employers have no standardized steps in their selection process, if the selection criteria is vague or if the steps are very loose, then almost every person who provides information may be considered a viable candidate for the job opening, regardless of whether they meet the qualifications for the job or not.

While the OFCCP has recently released guidelines on who internet “applicants” are, these guidelines are quite broad and can be troublesome when the employer has no additional, consistent steps through which all applicants must pass before becoming a candidate for consideration.  Having in place a selection process that includes multiple, valid, job-specific criteria against which applicants are assessed is the first step, but not the only step necessary to comply.[2]

Many companies incorporate on-line, computer or paper-based testing within their processes to evaluate a candidate’s particular knowledge, behavioral skills and abilities.  Any tests that you use should meet the standards for testing set out by the American Psychological Association and should follow the guidelines set forth by the EEOC.  Online tests fall under the same rigorous standards as do all other types of tests used for selection.

Best practices for testing and selection cited by the EEOC[3] include:

1. Employers should administer tests and other selection procedures without regard to race, color, national origin, sex, religion, age (40 or older), or disability.

2. Employers should ensure that employment tests and other selection procedures are properly validated for the positions and purposes for which they are used.  The test or selection procedure must be job-related and its results appropriate for the employer’s purpose.  While a test vendor’s documentation supporting the validity of a test may be helpful, the employer is still responsible for ensuring that its tests are valid under the Uniform Guidelines for Employment Selection Procedures.[4]

3. If a selection procedure screens out a protected group, the employer should determine whether there is an equally effective alternative selection procedure that has less adverse impact and, if so, adopt the alternative procedure.  For example, if the selection procedure is a test, the employer should determine whether another test would predict job performance, but not disproportionally exclude the protected group.

4. To ensure that a test or selection procedure remains predictive of success in a job, employers should keep abreast of changes in job requirements and should update the test specifications or selection procedures accordingly.

5. Employers such ensure that tests and selection procedures are not adopted casually by managers who know little about these processes.  A test or selection procedure can be an effective management tool, but no tool or selection procedure should be implemented without an understanding of its effectiveness and limitations for the organization, its appropriateness for a specific job, and whether it can be appropriately administered and score.

Or is your test used as a tool to provide additional information to support your interview process?  In either case, the

results from any test should not make up more than 20% of the hiring decision in order to comply with fair selection

standards.  The results from tests, such as personality measures, can enhance the reliability and validity of your

processes, providing that these tests have been professionally prepared and rigorously validated.  Ensuring that any

tests you administer are in fact valid and reliable, and that they are shown to be valid and reliable against your own

population and environment, are additional steps you will need to complete.

It is no longer acceptable to take the word of your outside test or system provider that tools or a criterion is valid.  You will need to ask to see the diligence that went into validation and reliability studies by requesting the technical report or validation report for these tools.  These reports should include item analysis, EEO scores comparison for protected classes, show that no disparate impact results through the administration of the tool, and should provide statistical data concerning face validity and content reliability, all based on a statistically significant population of results.  If no report is available, or the study lacks rigor, you should immediately find another, more valid assessment tool.

Also, your test or system provider should have professional experts who can revalidate their tools against your jobs and in your own environment.  They should be able to show that no adverse impact occurs as a result of the use of their tools for your particular jobs through a statistical study process. If your providers cannot do this for you, look for another vendor.

---

[1] EEO & Testing Quarterly Review. HR News & Information (July – Sept 2004) p. 3.

[2] For additional information, visit the US Department of Labor website at www.dol.gov

[3] Fact Sheet on Employment Tests and Selection Procedures, www.eeoc.gov/policy/docs/factemployment_procddures.html.   Pg. 5

[4]The US Department of Labor and Justice and the Office of Personnel Management issued the UGESP.  A copy of the regulations are available online through http://www.eeoc.gov/policy/docs.html