Do-It-Yourself Safety Perception Survey: Nine Step Process

Perception Survey Image

The purpose of this article is to clarify some of the misguided do-it-yourself safety perception survey information published on the internet.  When it comes to survey consultants, it is a little bit like the Wild West, anyone can hang out a shingle.  Unfortunately, there are self proclaimed survey consultants that really are not equipped to conduct your survey.  They do not have a complete knowledge of the survey process and most do not have an appropriate database to manage the survey data.  Our company is Compass Health & Safety Ltd.  We do not compete with these consultants for your survey business.  Our goal is to provide companies with the ability to conduct their own survey.  We advocate a do-it-yourself approach because we want every company to be able to afford to benefit from this assessment method.

In this article, we will provide you with some solid information on the survey process.  We also want to caution you that this is only a summary of the process.  Compass promotes a Nine Step do-it yourself Safety Perception Survey Process.  We have written a book on the subject entitled ‘Yes You Can….Conduct your own safety perception survey ‘ We have written numerous articles on the subject, created a web-based survey application to help manage the survey data and have developed a two day course on how to successfully conduct safety perception surveys.  We have taught our course numerous times to students in several universities and colleges here in Alberta.  We are qualified to write this do-it-yourself survey process summary.

Safety perception surveys first started to gain popularity in the 1970’s.  Companies were looking for additional measurement options to augment the information they were obtaining from health and safety audits.  Safety system audits are considered a primary measurement indicator and their focus is on measuring compliance to basic safety elements such as inspection, investigation, etc. as well as compliance to legislation.  In short, their focus is on “what is in place”.  To assess compliance, auditors review employer documents, conduct site observations and interview employees using questions prescribed in an audit protocol or script.  What audits do a poor job of assessing is how well some of the elements are working.  For example, where an audit may identify if pre-job assessments are conducted, the safety perception survey may identify if employees are getting any value out of completing them.  It is this additional information that is very complimentary to the audit process and it is essential to establishing an effective health and safety management system.

The call for a do-it-yourself survey approach is the result of the high costs associated with consultants offering safety perception survey services.  Few companies can afford to hire a consultant to conduct a survey.  The purpose of this article is not to reveal or rationalize the cost of conducting safety perception surveys but to provide some guidance on how safety perception surveys can affordably be conducted without the expensive consultant.  It is our belief that every company should be able to afford to conduct a safety perception survey.  The information surveys provide is necessary to the progression to a best in class health and safety management system.  So here is our Nine Step Process for conducting a safety perception survey.  More detailed information on this process can be found in our book.

Step 1 – DEVELOP

A poorly developed survey will yield poor information.  Sufficient time should be budgeted to properly develop a survey that is certain to work.  Here are some tips:

  • First, select or develop survey questions or statements that are suitable to the organization. What information do you hope to obtain from employees?  Remember you are not tethered to, or confined to, the audit process so perhaps start by asking what the audit process does not.
  • Be aware that if you ask about more than one idea or concept in any of your survey statements, the response provided may not identify if the respondent is assessing both of the concepts or just one. For example, “Safety meetings are effective and well attended.”  This statement is asking two different things.  This suggestion does not say that you cannot include two concepts in one question but we suggest you avoid it if possible.
  • Do not use subjective adjectives such as “good,” “fair”, and “bad.” in your survey statements. What is “good” or “bad” to one respondent may not be for another.  Instead of “There is good communication at safety meetings” try “Safety meetings are effective in communicating safety issues.”
  • Avoid using terms such as “always” and “never.” Using these terms sometimes forces the respondent to respond more negatively than they normally would. For example, “Employees always bring up safety issues at safety meetings”.  If one out of 50 times the meetings do not, the respondent may score the statement less than they might have had it simply been worded as “Employees participate by bringing up safety issues at safety meetings.”
  • Do not frame a statement in the negative if all the other expected/desired responses are in the positive. This will confuse your database and probably you.
  • Ensure that all respondents, no matter what their position in the organization, have the knowledge or information they need to understand and respond to the statements presented. For example, do not ask an administration employee about a field level risk assessments.  If you do, your average scores to these questions will be skewed to the negative.
  • Do not use statements that “lead” the respondent. For example, “Given that our incident statistics have increased substantially this year, are you satisfied our health and safety program has improved?”

If your survey statements are poorly worded you will not get good value out of the responses.

Select an appropriate scale for the survey.  Most surveys employ the Likert 1–5 scale, where 1=0% positive and 5=100% positive.  There are many other possible scales options available.  For example, most system audits use only two options, Yes or No.  When responding to this scale the employee has to choose if the answer is more Yes or No.  They have to make their score selection based on if their answer is more or less than 50% positive or negative.  This is a very imprecise scale and leads to errors in determining opportunities for improvement.  Many consumer surveys will have up to ten response options.  This is not a good choice for longer surveys as they will exhaust the respondent.

Next, develop the reporting parameters.  The reporting parameters, such as location and employee position, are invaluable in telling the database how to sort the data.  Typically, the reporting parameters are placed at the front of the survey.  An example of four reporting parameters (ie. Department, Job, location, Position) is shown below.

 

Choose the reporting parameters carefully, because they will determine the different ways that the data can be reported out once the survey is finalize and this influences what can or cannot be analyzed.  Care must be taken to ensure no individual employee response is revealed because too many parameters were used.  For example, once the only Manager of Operations in Timbuktu, he/she has checked off the reporting parameter boxes, that manager reveals their identify.  All of people reviewing the survey data will know how that Manager responded to every question.  When anonymity is promised it must be respected.

Step 2 SAMPLE

How many respondents should participate in the survey?  Is it best to survey all employees, or only some of them?  There are clear advantages and disadvantages to each of these options.  Including all employees in the survey gives everyone the opportunity to participate in the process.  Employees feel more involved in the company safety program, because they’ve been invited to express their issues or concerns.  As with any sample, the reliability of the score increases with the increase in sample size.  On the other hand, time is money and it takes time to complete surveys.  If the company chooses to have only a sample of employees respond, there are sampling charts available on the internet that will help select an appropriate sample size.  For example, if you select a chart providing 95% confidence in the sample size, a company with a1000 employees should have a survey sample of 278 employees.  What is most important is that sufficient respondents complete the survey to ensure their anonymity.  In order to make this determination one has to view the reporting parameters and ensure a sufficient sample size has been selected.  If you cannot be assured of full participation you should consider allowing for extra participants.

Step 3 TEST

It’s a good idea to pre-test the survey by administering it to a small group of employees before you administer throughout the company. This pre-testing process will help to:

  • Ensure that the reporting parameters are clear, correct and understood by respondents.
  • Ensure that respondents will correctly interpret the survey statements.

No matter how tempting it may seem, do not skip the pre-test step.  Once the survey has been administered and completed it is too late to make corrections to the survey.

Step 4 COMMUNICATE

Employees generally do not appreciate surprise surveys.  Some communication should be sent out ahead of time to apprise the employees of the impending survey.  Cover the following points:

  • The purpose of the survey and what the company hopes to achieve by administering it.
  • Explain by what authority the survey is being administered (for example, if the president has approved it, say so).
  • Request respondents’ full participation.
  • Explain the confidentiality protection measures that are being taken to ensure anonymity.
  • What will be done with the results.
  • How the findings will be communicated to the respondents.

Step 5 ADMINISTER

There are many ways to administer a survey.  The survey can be administered by hardcopy such as at a safety meeting.  To ensure anonymity, a trusted employee such as a Safety Committee Representative should oversee the completion of the surveys and see to it that once completed, they are confidentially placed into an envelope.  This administration approach requires someone later to input all of the information from each survey into the database.  There are many other administration methods but the one we like best is electronic.  Since our database is web-based, it is possible to allow employees access to the survey from anywhere where there is internet access.  They complete the survey online and therefore they do all of the data inputting.

If your industry contains a number of people of different cultures, races and religions, it may be useful to use a database that can translate your survey into their native language.  This will ensure they have full understanding of each of each survey statement.  The downside of this is that someone is then needed to interpret the comments that may be written in another language.  Alternatively, one could employ a translator.  We have made use of a plug in program from Google that gives us the ability to translate the survey into over 100 different languages.

Step 6 ANALYZE

Once all of the surveys responses are in, it is time to analyze the data.  Your analysis will help shape the future of the program going forward so it is important you take the time to assess what the data is telling you.  This is when you will be happy that you selected a database that can provide you with the reports needed to conduct a proper analysis.  Data analysis is not an easy job.  It is not possible to cover all aspects of the analysis process here in this summary article but here are some tips to get you started.

  • First, your database should provide you with reports that reveal scores and percent positive scores by each of the survey reporting parameters. Review this information, statement by statement and look for averages that warrant a recommendation.  For example, you might select 75% positive as the lowest acceptable score.  Any scores below this figure require a recommendation.
  • Next, look for perception gaps between non-supervisory and supervisory or management. Large gaps suggest the groups are not aligned.
  • Review your comments report. This report will justify the score awarded and give you the reason employees scored the way that they did.  By correlating the scores with the comments, one can immediately see the benefits of including comments in a survey.  Without comments, the data shows only that some groups of respondents scored high and some scored low, but gives no rationale for the score.  Scores alone will not provide the direction needed to take specific action. It can be very time consuming and expensive collecting perceptions to scores by way of focus groups after the survey is completed.

 

Respondent comments sometimes reveal more information than what you asked for.  Under the cloak of anonymity, there may be one or perhaps more employees that provide comments that are less than flattering.  Revealing these kind of comments may serve to distract and even anger key employees crucial to making decisions on improving the system.  We generally do not provide all of the comments in the final report.  Our database offers an optional pie chart that depicts the percent positive, negative and neutral comments.  We use these charts in our reports and then provide management with a brief summary of the comments both positive and negative.  Some companies would like to include standard deviation, significant difference and other analysis aids.  Your database will determine what you can or cannot provide.

 Step 7 VALIDATE

As previously indicated, we ensure there is a comment option available for our survey respondents.  The reason for this is that their comments help to validate the scores.  The main reason safety perception surveys have earned the reputation for being expensive is that some survey consultants do not collect comments to each survey statement.  In the absence of comments, scores need to be validated by interview, focus groups or some other method.  This process can translate into a great deal of billable time.

Even with the survey comments option, there may be times when the comments provided from the survey do not give sufficient information for you to take action.  In these cases, focus groups can be very helpful for getting the extra information needed.  Another method that we have used is to set up a kiosk with a computer and then ask employees to stop at the station to respond to certain survey statements to help validate them.  Sometimes it is advantageous to randomly select employees to participate in this validation process.

Step 8 FEEDBACK

Before the survey was administered, hopefully you promised employees that survey findings would be communicated back to them.  Do not overwhelm them with survey data that can be misinterpreted.  Instead, it is best to provide an easy-to-understand summary of the survey findings.  If employees ask to review the full report, the surveyor may want to guide them through the data.  Another effective means of communicating survey results is to release a short newsletter outlining key strengths, opportunities for improvement, and action steps that will be taken.

Keep all employees up-to-date on the actions the company is taking, or planning to take, in response to the survey results.  Action on report recommendations must be taken. Experience has shown that conducting a safety perception survey creates a strong expectation that the company will take action on the findings.  Employees take their investment in the safety perception survey process seriously and expect to see results.  If you disappoint them, it may be difficult to get their cooperation on future surveys.

Step 9 RE-EVALUATE

The safety perception survey yields more information about a company’s health and safety management system than any other method of safety measurement currently in use.  Some say it yields too much about the culture of the organization and this can make some managers uncomfortable.  Some survey findings such as those culturally related, take more than a year to improve.  For example, if management has gained a reputation for not following up on health and safety issues, they have lost credibility.  Credibility can be redeemed but redemption takes time.  Some of the remedies for survey issues are deeply rooted and take time to improve.  For this reason, safety perception surveys are typically not conducted every year.  Our recommendation is to conduct a survey every second year.

Conclusion

Now, more than ever before, employee opinions and perceptions play a key role in the success of an organization.  As companies start to realize that the solutions to most of their problems are already well known by their stakeholders, more and more companies are seeing the value in soliciting opinions and perceptions from employees, customers, and constituents.  It is no exaggeration to say that management cannot lead effectively without the information safety perceptions can provide.  Too many companies with their 95% audit scores have been lulled into thinking their safety systems are best in class.  These companies do not know they are vulnerable to a serious incident because they have not listened to their employees.  When properly conducted a safety perception survey will reveal potential surprise events.  It will provide insight not only to your health and safety management system but others such as operations, quality assurance and service.  The safety issues revealed by safety perception surveys are often the same issues existing in other aspects of the business.  From a value added perspective, the survey approach yields a good return on the investment.

Anyone can conduct a safety perception survey but if it is your first survey initiative, you will need some guidance.  This article only scratches the surface of the survey process.  More detailed information can be found in our book ‘Yes You Can… Conduct your own safety perception survey.  Good luck.