Improving Citizen Science Games through Open Analytics Data


  • Jesse Himmelstein Centre de Recherches Interdisciplinaires



cyberlab, video games, analytics, metrics, citizen science, synthetic biology, open data, open source


Game developers make heavy use of "metrics" (or "analytics") in order to understand how their games are being played by the public. Through recording players' actions within a game, designers can identify weaknesses in the game design, such as sections of the game that are too difficult, or never explored. Meanwhile, a genre of “scientific games” has emerged, in which players contribute to the advancement of science through their actions in a game, learn scientific concepts, or both. Although most scientific games incorporate metrics, the data is meant only for the developers and is rarely shared with the public. This represents a missed opportunity for the public to learn from the data that they and their fellow players are generating.Inspired by the belief that anyone can contribute to science, we have created an open source game analytics service called "RedMetrics" in which all data gathered is freely and immediately available online. RedMetrics can gather data from any platform (web, PC, console, etc.) and stores it on an open repository. The data is available via a web API as well as a web application. To ease integration, we provide interfaces for the popular game engine Unity as well as for the web browser.In this paper, we demonstrate RedMetrics through a case study of Hero.Coli, a game to learn synthetic biology. We study the progression of players through the game and use RedMetrics to identify bottlenecks in the game design that hinder learning.

Author Biography

Jesse Himmelstein, Centre de Recherches Interdisciplinaires

Game Lab Director


Bennett, J., & Lanning, S. (2007). The Netflix Prize. KDD Cup and Workshop, 3–6.

Booth, D., Haas, H., McCabe, F., Newcomer, E., Champion, M., Ferris, C., & Orchard, D. (2004). Web Services Architecture. Group, 22(February), 19–26.

Christian, B. (2015). The A/B Test: Inside the Technology That’s Changing the Rules of Business. Wired Magazine.

Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., … Players, F. (2010). Predicting protein structures with a multiplayer online game. Nature, 466(7307), 756–60.

Culbertson, G., Andersen, E., White, W., Zhang, D., & Jung, M. (2016). Crystallize: An Immersive, Collaborative Game for Second Language Learning. In CSCW.

Denzin, N. K., Simon, H. A., & Ericsson, K. A. (1985). Protocol Analysis: Verbal Reports as Data. Contemporary Sociology (Vol. 14).

El-Nasr, M. S., Drachen, A., & Canossa, A. (2013). Game Analytics: Maximizing the Value of Player Data. Retrieved from

Fielding, R. T. (2000). Architectural Styles and the Design of Network-based Software Architectures. Building, 54, 162.

Himmelstein, J., Couzic, M., Jennett, C. I., Cox, A. L., Goujet, R., Lindner, A., & Taddei, F. (2014). RedWire: A Novel Way to Create and Re-Mix Games. In Proceedings of the first ACM SIGCHI annual symposium on Computer-human interaction in play - CHI PLAY ’14.

Hunt, J. W., & Mcilroy, M. D. (1976). An Algorithm for Differential File Comparison. Bell Laboratories Computing Science Technical Report (Vol. 41). Retrieved from

Janssen, M., Charalabidis, Y., & Zuiderwijk, A. (2012). Benefits, Adoption Barriers and Myths of Open Data and Open Government. Information Systems Management, 29(4), 258–268.

Kawrykow, A., Roumanis, G., Kam, A., Kwak, D., Leung, C., Wu, C., … Waldispühl, J. (2012). Phylo: a citizen science approach for improving multiple sequence alignment. PloS One, 7(3), e31362.

Kim, J. S., Greene, M. J., Zlateski, A., Lee, K., Richardson, M., Turaga, S. C., … Seung, H. S. (2014). Space-time wiring specificity supports direction selectivity in the retina. Nature, 509(7500), 331–6.

Kohavi, R., Henne, R. M. R., & Sommerfield, D. (2007). Practical guide to controlled experiments on the web. Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ’07, 2007, 959–967.

Lee, J., Kladwang, W., Lee, M., Cantu, D., Azizyan, M., Kim, H., … Das, R. (2014). RNA design rules from a massive open laboratory. Proceedings of the National Academy of Sciences of the United States of America, 111(6), 2122–7.

Miller, S. A. (2012). Piwik Web Analytics Essentials. Retrieved from

Raddick, M. J., Bracey, G., Gay, P. L., Lintott, C. J., Murray, P., Schawinski, K., … Vandenberg, J. (2010). Galaxy Zoo: Exploring the Motivations of Citizen Science Volunteers. Astronomy Education Review, 9(1), 010103. Retrieved from

Schell, J. (2008). The Art of Game Design: A Book of Lenses. Annals of Physics (Vol. 54).

Sharp, L. (2012). Stealth Learning: Unexpected Learning Opportunities Through Games. Journal of Instructional Research, 1.

Steele, C. (2013). Open Access (The MIT Press Essential Knowledge Series). Australian Library Journal, 62(3), 248–249.

Tenopir, C., Allard, S., Douglass, K., Aydinoglu, A. U., Wu, L., Read, E., … Frame, M. (2011). Data sharing by scientists: practices and perceptions. PloS One, 6(6), e21101. Retrieved from




How to Cite

Himmelstein, J. (2016). Improving Citizen Science Games through Open Analytics Data. Human Computation, 3(1), 119-141.