The Experiment: What Are Errors and Why Do We Keep on Making Them?

 

This game was developed during a group discussion on challenges and errors in science at a joint workshop titled “Un/Certainty” hosted at the University of Konstanz in June 2018. The conference’s participants and this post’s authors all come from the Martin Buber Society of Fellows at the Hebrew University of Jerusalem or the Zukunftskolleg at the University of Konstanz.

None of us are professional game developers, but are scientist from different fields: history, anthropology, literature, and biology. In our group discussion, we discussed paradoxes related to scholars who seek to factor in their own chances of errors. We asked what factors can be responsible for errors, and whether it must necessarily involve an internal miscalculation on the part of the researcher or could it equally stem from external factors. This discussion then shifted to recognizing the importance of errors for boundary orientation and eventual learning in the research setting as in other aspects of human lives, a discussion that drew lessons from developmental psychology and even from our personal relationships. Above all, the discussion emphasized that the very implication that something is an error is fundamentally grounded in a backdrop concept of rules in view of which evaluation is taking place.

This was the starting point for the development of the game presented below. In what follows, we first present the game and then discuss the lessons drawn from it.

The Game*

You need 4 people and a dice to play this game and tackle the important scientific quandry: What constitutes an error and how can it be addressed?

The game simulates a funding process, in which funding is granted to those who "perform" to the expectation of the evaluating committee. To this aim, the players assume the role of 3 scientists at the beginning of their career, and the game’s organizer—who serves as a referee and is the one reading these rules and setting up the game—assumes the role of a professor on their evaluation committee. The scientists are playing to get fun-ding! A fixed number of rounds (e.g. 15 or 20) is played, following the rules mentioned below. Throughout the game, all players must keep silent except for the professor, and they should all pay close attention to the mimics and gestures of the other players.

Rules of the Game (**only to be read by the player playing the professor**)

The rules of the game are only handed to the person playing the role of the professor. These rules are explained in four different cards, one for each of the players. The professor assigns one card to each of the participants and keeps the professor’s card to him-/herself, and distributes ten tokens to each player. The players’ cards describe six different "expectations" of the funding committee to different rounds of funding. The expectation for each round is determined by a dice, whereby each roll informs the players what will be "funded” in each call/round (e.g. according to the cards below, if the dice rolled 4 then scientist 1 should look up, scientist 2 should cross his arms, and scientist 3 should clear her throat). The professor rolls the dice and rewards the player who performed correctly—that is, in a way that corresponded to what appears on the professor’s card—with funding (e.g., a candy or simply a point). However, if a player has “erred”, one token is taken away from him/her. Every player’s card contains two correct responses (i.e. corresponding to the professor’s card) and four incorrect responses, meaning that, on average, each player will be rewarded a third of the time, but will be penalized two thirds of the time. To consistently win, the players have to rely on more than just luck: they must learn from observation the correct responses for every roll, and know when to depart from the responses appearing on their card.

Example cards (players must NOT show them throughout the game):

 

Scientist 1                                                          Scientist 2

1 – touch your earlobe                                        1 – touch your nose

2 – clear your throat                                           2 – touch your earlobe

3 – touch your nose                                            3 – look up

4 – look up                                                          4 – cross your arms

5 – cross your arms                                             5 – puff your cheeks

6 – puff your cheeks                                           6 – clear your throat

Do NOT share your rules!                                  Do NOT share your rules!

 

Scientist 3                                                          Professor

1 – puff your cheeks                                           1 – touch your nose

2 – cross your arms                                             2 – clear your throat

3 – touch your earlobe                                        3 – touch your earlobe

4 – clear your throat                                           4 – cross your arms

5 – look up                                                          5 – look up

6 – touch your nose                                            6 – puff your cheeks

Do NOT share your rules!                                  Do NOT share your rules!

 

After the Game

After a fixed number of game rounds, the winner is declared and the players are invited to reflect on their learning experience. How did they respond when their actions did not produce the expected result (receiving a chocolate token)? At what point did they understand that they should combine the use of the (imperfect) rules that they have before their eyes with empirical observation of the actions that are being rewarded, and how difficult was it for them to adjust? Beyond that, the game should lead to a basic discussion about the essence of errors. If conclusions are not reached organically, the professor might give hints in order to kick off the discussion. The discussion will typically note that winning players have understood that everyone is following a different set of rules and that the professor is not distributing funding arbitrarily or erroneously. Rather, everyone erred sometimes, because they did not have the same set of rules. Consequently, in order to avoid errors, it is required to have a set of defining rules that everyone agrees on. Moreover, we can see that errors can be created when there is a different understanding within the same basic set of rules. While all of us have access to the same rules when we apply to funding—or perform any other action that depends on positive social feedback—we have a different understanding of how to follow them. As the painting by Magritte illustrates, our background and perspectives skew our vision and give us a different interpretation of things.

Preliminary Observations

Among other things, the game uncovers an emotional response mechanism: at times, players responded with a sense of indignancy when their supposedly-correct actions were penalized rather than rewarded and the game felt arbitrary and unfair. This, in turn, adversely affected their openness and willingness to look outwardly and adjust themselves according to their observations. In other cases, one or more players understood the game quite rapidly and adjusted their responses to get continuously rewarded.

Have fun trying it out and please share your observations and suggestions for a better gaming experience in the comments section!

* This game is partially inspired by the game Barnga, “a simulation game on cultural clashes” (Thiagarajan 2006).

 

 

https://buberfellows.huji.ac.il/sites/default/files/buberinstitute/files/rene_magritte-_empire_of_light.jpg?m=1575376626
rene_magritte-_empire_of_light