Introduction & Research Question

The amount of HR data is growing- But even more important, HR data is becoming more accessible as companies are digitizing HR Processes in ther HRIS systems. Nevertheless, the creation of business value via data driven recommendations remains a challenge. There is a significant “execution gap”, as companies consider people analytics as important (Bersin 2017), but the actual implementation lacks behind.

One hypothese is, that the lack of people analytics skills of today’s HR leaders is a key impediment. This gives raise to our reseach questions: “Are specific people analytics skills needed to make make better decision in the HR problem space?

We probe into this research question by testing three hypotheses via an experimental an online game on employee retentention (see video above). 

Figure 1: Impact of People Analytics on decision making

Research Hypotheses

Understanding how humans investigate a problem to arrive at a decision, is a long-standing research question.  As early as in 1972, Newell and Simon state in their book “Human Problem Solving”, that humans use primarily their own experiences to explore a specific problem space. One of key findings was, that good decision are driven by specific domain knowledge rather than general work experience for defined problem spaces (such as HR for example).

The following three research hypotheses were tested:

H1: General work experience has no impact on the problem solving capability in the HR problem space.

H2: People with coding experience (but no HR Experience) perform better than people without coding experience (but with HR Experience).

H3: People with human resource and coding experience perform better than people with only coding or only HR experience.

These hypotheses were tested with collected data in an expiremental employee attrition game, played by 52 online players in the period February-March 2021. The game design was as follows:

• Objective: Players had to maximize profits of a fictional company by managing employee attrition risk via salary increases as well as investment in training
• Information Design: Players could access different visualizations and data tables on the churn probability for the approx. 1.500 employees. Additional data, such as demographics, gender, past salary development, promotion history, and training history was provided as well.
• Model and Data Capturing: Two complementary approaches were used:
  a)  We calculate a profit function by estimating a churn probability (with a ML Model), therefore determining the quality of the decision of the players.
  b) We observed the player’s consumption of information via web tracking to determine how actual salary and training decisions were taken

Results

We were able to attract 52 people (of which 31 were students) with an average work experience of 4.4 years. The average player conducted 255 clicks and spent 13.5 minutes inside the game.

H1: General work experience has no impact on the problem solving capability in the HR problem space: Confirmed.  General work experience  did not significant predict performance and thereby the first hypotheses (H1) can be accepted.

H2: People with coding experience (but no HR Experience) perform better than people without coding experience (but with HR Experience): Declined. We split the data into two groups (with coding and with HR experience) and conducted a two sample Welch test, which resulted in a p-value of 0.59. The test showed that neither people with coding experience nor people with HR experience performed significantly better than the other group. This results give a first indication that analytical capabilities alone do not need to superior performance in people analytics problem space

H3: People with human resource and coding experience perform better than people with only coding or only HR experience: Sample Size. Unfortunately, only four players did have experience in both areas. While there is preliminary evidence (Performance of these player was higher), we were not able to test this due to the sample size.

Conclusion

HR-game are low-cost and attractive option to test hypotheses in the people analytics space. While our findings should be treated with caution (due to sample size and sample bias), we believe that gaming with HR executives is a powerful mean to teach and test your hypotheses.