Case Study for Leading US Manufacturing firm

Pickers and Packers Position



Background

The purpose of the case study using Harrison InnerView was to determine the ability of the Harrison InnerView software to predict success for the “Pickers and Packers” position. The company provided 91 profiles of current employees. Nineteen profiles were eliminated due to very low consistency scores (less than –110). The low consistency score indicates that the individual either attempted to fool the questionnaire or tried to answer the questions too quickly. Therefore, the total sample population of valid profiles was 72.

All the employees in the sample were rated according to their job performance by supervisors. There were 4 equally weighted criteria used for evaluation: Productivity, Quality, Safety Regulation Compliance, and Attendance.

Performance scoring

Each employee in the sample was rated on each criterion according to the following scale:

Productivity

90-100 25% or more above standard
80-89 20% above standard
70-79 15% above standard
60-69 10% above standard
50-59 Standard
40-49 10% below standard
30-39 15% below standard
20-29 20% below standard
10-20 25% below standard

Quality

90-100 25% or more above standard
80-89 20% above standard
70-79 15% above standard
60-69 10% above standard
50-59 Standard
40-49 10% below standard
30-39 15% below standard
20-29 20% below standard
10-20 25% below standard

Safety

90-100 No Accidents/Participate in safety committee
80-89 No Accidents/No safety violations
70-79 No Accidents/One minor safety violation
60-69 No Accidents/Two minor safety violations
50-59 One Accident/One significant safety violation
40-49 One Accident/Two significant safety violations
30-39 Two Accidents
20-29 Two Accidents plus safety violations
10-20 More than two Accidents

Attendance

90-100 Perfect
80-89 One absence
70-79 Two absences
60-69 Three absences
50-59 Four absences
40-49 Six absences
30-39 Eight absences
20-29 Nine absences
10-20 Ten absences

The total performance score was derived by averaging these four scores.

Since this scale (0-100 is different than the scale related to the suitability predictions in the Harrison InnerView software, it was necessary to adjust these performance scores to match the Harrison InnerView scale. The Harrison InnerView scale is as follows:

90-100 Excellent to Best
80-89 Very Good
70-79 Average to Good
60-69 Below average
50-59 Poor

To convert the original performance rating into the final performance rating, the following formula was used:

OPR(Original Performance Rating) /2 + 50 = Final Performance Rating

This Study

The aim of the research was to determine the exact traits that make Pickers and Packers successful at the company and to determine the relative accuracy of the InnerView system to predict success based upon the “job template” that incorporates those traits. The template formulates those traits into “traits to have” as well as “traits to avoid”. The template is then used to measure future applicants and serve as a developmental guide for current employees. The template is shown by a graph that indicates how a person scores against each of the required traits and then offers a final “bottom line” score between zero and one-hundred that represents the individual’s level of “total suitability” for a particular position. A score of one hundred represents a person who is completely suitable for that position. Assuming the person is eligible for a position (has the education, experience and technical skills), a suitability score of 75 or greater represents a person who has a good probability of performing effectively in that position. A score of 74 or less represents a person who is considered to be unlikely to perform well in that position.

Accuracy level

The results showed a very high predictive accuracy and a strong correlation between the Harrison InnerView suitability score and the actual job performance. This indicates that the template includes a comprehensive set of traits related to suitability for this position.

The results were analyzed in two ways: the predictive accuracy and the variation between the suitability scores and the performance ratings:

1) Predictive accuracy - A prediction is considered accurate if one or more of the following conditions are met:

a) The suitability score is 75 or greater and the performance score is 75 or greater

b) The suitability score is less than 75 and the performance score is less than 75.

c) The suitability score is within 6 points of the performance score.

The logic behind this definition - If the suitability score were 75 or above and the person were eligible for the position, it would indicate a prediction that the person would probably succeed in the position. If the suitability score were less than 75 and the person was moderately eligible (not highly eligible) for the position then this would indicate a prediction of below average performance. Also, if the suitability score were within 6 points of the performance score it would indicate a very close prediction and thus should also be considered accurate. The suitability score for each person appears on the template in the far right-hand column of the table.

From the research sample, 66 of the 72 employees (92%) showed a correlation between the InnerView suitability score and performance to be accurate, according to the definition above.

2) Variation between the suitability ratings and the performance ratings

The degree of variation between the suitability score and the performance ratings also provides an effective means of understanding the accuracy level of the methodology. The variation was examined in two ways: the average variation between the predictive suitability rating and the performance rating and the percentage of predictions that fall within different ranges of closeness.

The average variation between the suitability score and the performance rating was 7.4 points. This indicates a close relationship between the suitability score and the performance ratings.

Next we examined the percentage of employees whose two scores (performance rating and predicted suitability score) were within 5 points, 8 points, 10 points and 15 points. The table below shows these percentages for each group.

Variation between performance rating and predictive suitability score

Within 5 points Within 8 points Within 10 points Within 15 points
37% 51% 61% 89%

If the two scores were within five points, the prediction would be considered to be extremely accurate. If within 8 points, the prediction would be considered to be quite accurate. If within 10 points, the prediction should still be considered to be reasonably accurate. If the two scores are greater than 10 points apart, but within 15 points, it could not be considered to be accurate, but at the same time could not be considered to be very inaccurate. If the two scores are greater than 15 points apart, they should be considered to be inaccurate.

Traits required for success

The results show that the template developed is able to predict job success with a high level of accuracy. The sample size was sufficient to assume reliable results for future applicants.

The research indicates that the success factors for this position include:

  • A tolerance of less sleep than most people
  • The willingness to take initiative (this relates to performance, quality and attendance)
  • Being energetic
  • Enjoys working with one’s hands
  • The tolerance of standing (also related to physical/manual work)
  • The tolerance of pressure of deadlines
  • The willingness to do physical work
  • The tolerance of structure (including safety regulations)
  • The willingness to do work that requires a fast pace
  • The willingness to be helpful
  • The tendency to be optimistic

    The research indicates that there are also factors that will hinder success in this position. These include:

  • A severe lack of enthusiasm (or apathy)
  • The tendency to be harsh or punitive
  • The desire to avoid responsibility
  • Having strong desires to earn high pay without having the necessary self-motivation
  • The tendency to be defensive or have difficulty receiving corrective feedback
  • A severe lack of diplomacy
  • A severe lack of precision (poor attention to quality)
  • A lack of interest in self-improvement (little desire to get better at what one does)
  • The tendency to be extremely reserved or withdraw from social interaction

    There is one additional trait related to success:

  • The willingness to communicate (good communication)

    These traits as formulated in the Harrison InnerView software will enable a future prediction level of success for the Pickers and Packers position of at least 80%.