Open-ended pulse survey responses are often underexploited by organizations, many of which rely on basic topic and sentiment analysis to derive face-value insights. Employing a psychological approach to examining the language contained in open-ended responses can provide a wealth of information concerning workforce well-being.
For example, an analysis of language data, either aggregated across the organization or by department, can reveal the degree to which the workforce's open-ended responses reflect elevated levels of stress, anxiety, or other indicators of burnout.
Tracking workforce burnout using open-ended pulse survey data
The steps below demonstrate how one can prepare, visualize, and interpret aggregated open-ended pulse survey response data from hypothetical Organization X.
Organization X administers pulse surveys quarterly (at the beginning of every March, June, September, and December). The survey includes one open-ended question asking employees about their thoughts and feelings concerning workplace experiences. Based on the aggregated language contained in open-ended responses from four quarters, Organization X is interested in examining its workforce's overall stress, anxiety, excitement, and cognitive load over the course of a year (i.e., longitudinally).
The following Receptiviti measures can be used:
Stress, using stress_prone, a facet of neuroticism, in Receptiviti’s Big Five Personality Framework
Emotions, such as fear (i.e., anxiety) and excitement, using Receptiviti’s Emotions Framework
Cognitive load, using cognitive processing in Receptiviti’s Cognition Framework
Cognitive load is the amount of cognitive resources (or mental energy) that a person uses. It can be a precursor to burnout and can inform well-being. For example, certain factors, such as stress or task complexity, often increase cognitive load and, consequently, lead to decreases in performance.
1. Data preparation
If Organization X currently has pulse survey data from four time points in 2022 (i.e., March 1st, June 1st, September 1st, and December 1st), then the data should be organized in a way that identifies each employee with four rows of data in the spreadsheet, corresponding to each of the four time points. For example, as shown below, “Employee_Number 1" represents data from employee 1’s open-ended responses for March, June, September, and December.
It is also necessary, for calling Receptivi’s API, to have a “Case_Number” variable that identifies each unique row of data. Additionally, while the “Date” variable identifies the specific date on which the pulse survey data was collected, it can be helpful to include a “Quarter” variable that designates whether the data is from Quarter 1 (3/1/22), Quarter 2 (6/1/22), Quarter 3 (9/1/22), and Quarter 4 (12/1/22). This can facilitate the process of creating figures to help track employee well-being and burnout over time.
2. Calling Receptiviti’s API
To generate language scores based on your open-ended pulse survey response data, you can start by accessing Receptiviti’s API through our CSV upload tool using your registered account information here. (Instructions for calling the API using Postman, cURL, Python, and R are also available for those who prefer interacting with programming interfaces.) When uploading your open-ended pulse survey response data using the CSV upload tool, make sure the file only includes two columns (i.e., the Case_Number and Raw_Text variables) without headers and is in the CSV file format. After uploading and submitting your CSV file, you should almost instantly receive an output with scores corresponding to each row of data in your table that you can merge with your original dataset.
3. Visualizing and interpreting longitudinal open-ended pulse survey data
As a way to visualize and track company-wide changes in well-being over time, we can graph language-based measures of stress, anxiety, excitement, and cognitive load apparent in employees’ open-ended pulse survey responses for each quarter of 2022 by using the splot package in R (a free statistical software that can be downloaded here). Below is R code for creating plots to track workforce well-being throughout each quarter:
The figures above depict employees’ stress, anxiety, excitement, and cognitive load from Quarter 1 to Quarter 4, where stress, anxiety, and cognitive load increase from Quarter 1 to Quarter 2 but decrease and eventually return to baseline from Quarter 2 to Quarter 4. In contrast, there is a decrease in excitement from Quarter 1 to Quarter 2, where excitement then gradually increases and returns to baseline from Quarter 2 to Quarter 4.
While this approach provides for an aggregated organizational-level view, subsetting your data at departmental or functional levels can provide for more granular insights.
