We’ve all been there. It’s Friday afternoon and you get an email from your company’s HR department asking you to complete another survey. Sometimes you respond, sometimes you don’t. When you do feel inspired to click on the link and provide your thoughts, you usually move pretty quickly through the closed-ended items. Read, click. Read, click. Read, click. Then, you come to the fork in the road:
Please use the space below to provide additional comments.
This is where it gets interesting - well, at least this is where it gets interesting for me - a survey researcher and I/O practitioner who is particularly interested in survey commenting behavior. Will you comment or skip to the next item?
Organizational surveys like this are very common (Kraut, 2006), but only about 34-40% of survey respondents provide comments when prompted (Borg, 2005; Poncheri, Lindberg, Thompson, & Surface, 2008; Siem, 2005), so if I was a betting woman, I would place my bet that you will skip to the next item leaving me to wonder why you didn’t respond and what insights I may be missing.
My research has focused on trying to find answers to some of these questions. In 2008, my colleagues and I (Poncheri et al., 2008) published an article in Organizational Research Methods exploring negativity bias in open-ended responses. Our study showed that commenters were relatively dissatisfied and provided comments that were negative in tone. Not only that, but the comment length increased as the tone became more negative.
So, back to the scenario. Maybe it was good that you didn’t respond – after all, commenters usually provide negative comments and are dissatisfied. Perhaps your non-response is an indication of satisfaction or no issues to report (i.e., “everything’s good, but thanks for asking.”). Or maybe you had plans to go out with friends on this particular Friday night and spending time responding to the open-ended comments would have made you late. There are many potential explanations for why you didn’t respond.
So, why does all of this matter? Because the issue of comment non-response has important implications for organizations and for employees. Comment are often vivid and can distract for quantitative findings that may be more representative of the respondents’ attitudes and opinions. One inflammatory comment can have more influence than quantitative ratings from hundreds of respondents. If only the negative comments are being expressed and those get the most attention, other important information may be missed. So, a few practical suggestions for HR departments when providing survey results to managers:
1. Provide warnings about the tone of open-ended comments in reports to prevent managers (Borg, 2005; Macey, 1996),
2. Add comparisons of quantitative ratings with the tone of comments to reports for managers (Poncheri et al., 2008),
3. Include results by subgroup to highlight any differences so that comments can be appropriately interpreted and interventions can be targeted (Poncheri et al., 2008), and
4. Make sure employees know you care about their comments to encourage more response – read them and help facilitate action! (Kraut, 2006). For this final suggestion, you can modify the wording of open-ended item prompts to communicate importance or write more targeted items prompts to get the information you are seeking.
There are lots of questions that still need to be answered and I’m working on some of those in my program of research. More to follow on that, but before then, I’d like to hear your thoughts on reasons for non-response or strategies for encouraging response that you have found to be effective. Please provide your comments in the space below. I look forward to reading your comments!
Borg, I. (2005, April). Who writes what kinds of comments? Some new findings. In A. I. Kraut (Chair), Grappling with write-in comments in a web-enabled survey world. Symposium conducted at the 20th annual conference of the Society for Industrial and Organizational Psychology, Los Angeles.
Kraut, A. I. (2006). Moving the needle: Getting action after a survey. In A. I. Kraut (Ed.), Getting action from organizational surveys: New concepts technologies and applications (pp. 1-30). San Francisco: Jossey-Bass.
Macey, W. H. (1996). Dealing with the data: Collecting, processing, and analysis. In A. I., Kraut (Ed.), Organizational surveys (pp. 204-232). San Francisco: Jossey-Bass.
Poncheri, R. M., Lindberg, J.T., Thompson, L. F., & Surface, E. A. (2008). A comment on employee surveys: Negativity bias in open-ended responses. Organizational Research Methods, 11, 614-630.
Siem, F. (2005, April). History of survey comments at the Boeing Company. In K. J. Fenlason (Chair), Comments: Where have we been? Where are we going? Symposium conducted at the 20th annual conference of the Society for Industrial and Organizational Psychology, Los Angeles.