With the advent of AI, one of the most important things we can do to prepare for the best possible future is to become more deeply aware of what it means to be human. How we transform that knowledge into useful data—and who has control of it could have profound implications for the type of society we will be living in.
In this day and age, it is extremely profitable for companies to learn more about human (inter)actions. Without question, Google, Facebook, Amazon, and Apple take the cake on aggregating such data. The trade secret is to create digital neural networks to process and analyze every action—ultimately leading to impressive corporate innovation. Recently, Cambridge Analytica showed the world the potential of massive cognitive profiling but also the potential dark sides. Thus, these companies are also exceptional at manipulating “the fundamental emotional needs that have driven us since our ancestors lived in caves, at a speed and scope others can’t match...Whether you want to compete with them, do business with them, or simply live in the world they dominate, you need to understand [the way these tech companies work]”.
More targeted companies like Knack and Lumosity, are attempting to use games to capitalize on our desire to “brain-train” and get assistance in HR applications such as hiring and efficient team formation. However, the companies rarely scientifically document their findings. In the massive BBC-sponsored Brain Train Britain, no effect was discovered in 60,000 participants and Lumosity has since then been fined for false marketing. Social science and behavioral economics research groups have undoubtedly made great leaps towards understanding and analyzing how we tick. However, due to methodological shortcomings such as limited and biased samples they are increasingly falling victim to the reproducibility crisis.
So now, imagine what the world would look like if we ‘democratize’ the understanding of human behavior? What if we could conduct responsible, non-proprietary human behavioral research on nearly the same scale? If everyone could tap into those neural network nodes, then everyone could develop services just as personalized as Google provides—opening access to innovation for smaller companies to be able to deliver competitive products. There is a dire need for a large-scale public effort to both give access to all the benefits of this knowledge and to spark important public discussions on its dangers. Even the act of presenting these ideas to the public and including them in the scientific process of inquiry is a critical step to democratizing the data.
For these purposes, ScienceAtHome created Skill Lab: Science Detective, a suite of mini-games exploring basic and higher cognitive skills such as 2D and 3D visuospatial reasoning, response inhibition, selective visual attention, visuospatial working memory, reaction times, and written language comprehension of English language. We compare the skill-indicators from the games with classical, validated psychological tasks evaluating the same skills. The game and several applications will be described below but first more about our initial motivations for the development.
Investigating Human Problem Solving, Creativity, and Innovation
While machine learning algorithms are becoming increasingly powerful, human intelligence is still superior in many respects. It is essential to understand the difference between human and machine intelligence, and develop hybrid-intelligence interfaces that optimally exploit the best of both worlds. By offering complex research challenges to the general public, citizen science does exactly this. Numerous citizen science projects have shown that humans can compete with state-of-the-art algorithms in terms of solving complex, natural science problems. However, much less is understood as to why a collective of citizen scientists can solve such complex problems. Many research institutions around the world have groups studying this facet of human creativity. Typically, each of these groups has one or two games that they have designed to test creativity in a particular context. However, just like in Google’s algorithms, one needs lots of diverse data to understand subtle interactions.
At ScienceAtHome, we believe that we can only unlock the subtle patterns of human creativity by studying it in many shapes and forms simultaneously. We have developed a portfolio of high-dimensional, natural science challenges (within physics, chemistry, and mathematics). At the same time, we are setting up a portfolio of more controlled individual and collective games. We believe that such a “social science supercollider” infrastructure will allow the massive and detailed investigations necessary to overcome current methodological hurdles. In e.g. the Quantum Minds and Alice Challenge projects, we have taken first steps into combining natural and social science investigations. However, it is also becoming increasingly clear that even with data from our current 300,000 volunteers, typical big data methodologies struggle to give clear insights into subtle human strategies.
One of the mind-boggling (and perhaps comforting) realizations coming from the social sciences in recent years is the fact that humans are often superior to algorithms because and not despite having limited computational, psychological, and physiological capabilities. As a simple example, human minds have limited storage capacity but this has forced us to develop sophisticated information prioritization algorithms (forgetting :)) that make us immensely efficient when forced to make fast, heuristic choices. Similarly, when balancing exploration and exploitation in complex innovation tasks, both skills and weaknesses play together to influence our choice of strategies. Out of these insights and our big-data frustrations came the concept for the Skill Lab games: by simultaneously investigating complex problem solving and knowing the cognitive profile of each player, we would be able to perform an a priori categorization of players into persona types and then look for common patterns of learning and exploration within each group.
Skill Lab: Science Detective
Skill Lab: Science Detective is designed to be a real gaming experience instead of a gamified experience for citizen science research (lead scientist: Carlos Días, Phd). It is based on the principles of stealth assessment and evidence-centered design. The mini-games are embedded in a story-driven game where the player is a detective. The player’s challenge is to solve the mystery behind strange events happening at a university campus while helping researchers at different labs.
We digitally adapted a series of 16 challenges which players can access at points inside the game. The game establishes a pact between citizen and scientist. We give players access to the newest psychologically-founded personal cognitive map allowing them to self-assess strengths and weaknesses in their cognitive skills while donating their data. In the fall of 2018 we launched a prototype of this large-scale profiling in collaboration with Danish Public Broadcasting. More than 15,000 people have already participated and because of the citizen science methodology and the publicity by a public service channel, participation is very evenly distributed across ages and by gender. This is something truly unique in the field of social science.
Bridging the Gap Between Biology and Social Phenomena
Denmark maintains a Central Personal Registry (CPR) database for government data on each citizen, which provides for up-to-date services. This has established Denmark as a haven for registry-based research, in which any social-science question (such as entrepreneurship or health issues) can potentially be investigated by correlating to the CPR-data. In the future, genome mapping done by companies such as 23andMe will allow research into how our biology might impact the emergent phenomenology of cognitive and social behavior. However, first efforts clearly demonstrate that we are far from establishing a full microscopic understanding.
So far, research based directly on cognitive indicators has either relied on self-reporting, which is small-scale and unreliable, or on the one big database that exists for cognitive profiles: military draft data. Although the latter has given rise to a number of insights it harbors an intrinsic problem: such data is predominantly available for one demographic group: young men. It has recently come to light that much medical testing has been conducted predominantly on males and therefore many documented side effects are much more relevant to men than women. Of course, this is a huge medical problem and a big social issue if we view human interactions based on insights derived in a gender-unbalanced way.
In our Danish 2018 pilot, we took steps to demonstrate the applicability of this new database for societal registry-based research. We called for researchers to pose a few brief survey questions for Skill Lab users. These responses can then be correlated with the game-generated individual cognitive profiles to investigate e.g. entrepreneurial intent, risk preferences, cognitive aging, political ideology, as well as relations between language skill and working memory.
Imagine how this work can help develop a model of how humans solve problems and reason through choices. It can lead us to a better understanding of cognitive biases, logical fallacies, societal interactions and issues, turning the data into a powerful tool for humanity to understand cultural and political dynamics. Understanding these issues can help us improve our society in the future.
Consider what we might be able to do if we could examine cognitive profiles from the whole world? Could you create a social media digital assistant that detects when you are being manipulated? For that you need much deeper insights than what is currently available. Let’s also consider other challenges: Could we help overworked teachers deliver on the promise of personalized education? Could businesses be more systematically successful if they were less dependent on the intuitions of exceptional, natural born leaders?
Just like the thrill of discovering a new continent 500 years ago, today we can get a thrill of uncovering some of the hidden landscapes of our knowledge. One of the most important dark spots is how the individual mind works and how many individuals join collectively to accomplish important achievements.
About the authors: Jacob Sherson is Founder and Executive Director at ScienceatHome and Professor at Aarhus University. Janet Rafner is Director and International Coordinator at ScienceatHome.
a global affairs media network
Understanding of Human Behavior Through Citizen Science
January 18, 2019
With the advent of AI, one of the most important things we can do to prepare for the best possible future is to become more deeply aware of what it means to be human. How we transform that knowledge into useful data—and who has control of it could have profound implications for the type of society we will be living in.
In this day and age, it is extremely profitable for companies to learn more about human (inter)actions. Without question, Google, Facebook, Amazon, and Apple take the cake on aggregating such data. The trade secret is to create digital neural networks to process and analyze every action—ultimately leading to impressive corporate innovation. Recently, Cambridge Analytica showed the world the potential of massive cognitive profiling but also the potential dark sides. Thus, these companies are also exceptional at manipulating “the fundamental emotional needs that have driven us since our ancestors lived in caves, at a speed and scope others can’t match...Whether you want to compete with them, do business with them, or simply live in the world they dominate, you need to understand [the way these tech companies work]”.
More targeted companies like Knack and Lumosity, are attempting to use games to capitalize on our desire to “brain-train” and get assistance in HR applications such as hiring and efficient team formation. However, the companies rarely scientifically document their findings. In the massive BBC-sponsored Brain Train Britain, no effect was discovered in 60,000 participants and Lumosity has since then been fined for false marketing. Social science and behavioral economics research groups have undoubtedly made great leaps towards understanding and analyzing how we tick. However, due to methodological shortcomings such as limited and biased samples they are increasingly falling victim to the reproducibility crisis.
So now, imagine what the world would look like if we ‘democratize’ the understanding of human behavior? What if we could conduct responsible, non-proprietary human behavioral research on nearly the same scale? If everyone could tap into those neural network nodes, then everyone could develop services just as personalized as Google provides—opening access to innovation for smaller companies to be able to deliver competitive products. There is a dire need for a large-scale public effort to both give access to all the benefits of this knowledge and to spark important public discussions on its dangers. Even the act of presenting these ideas to the public and including them in the scientific process of inquiry is a critical step to democratizing the data.
For these purposes, ScienceAtHome created Skill Lab: Science Detective, a suite of mini-games exploring basic and higher cognitive skills such as 2D and 3D visuospatial reasoning, response inhibition, selective visual attention, visuospatial working memory, reaction times, and written language comprehension of English language. We compare the skill-indicators from the games with classical, validated psychological tasks evaluating the same skills. The game and several applications will be described below but first more about our initial motivations for the development.
Investigating Human Problem Solving, Creativity, and Innovation
While machine learning algorithms are becoming increasingly powerful, human intelligence is still superior in many respects. It is essential to understand the difference between human and machine intelligence, and develop hybrid-intelligence interfaces that optimally exploit the best of both worlds. By offering complex research challenges to the general public, citizen science does exactly this. Numerous citizen science projects have shown that humans can compete with state-of-the-art algorithms in terms of solving complex, natural science problems. However, much less is understood as to why a collective of citizen scientists can solve such complex problems. Many research institutions around the world have groups studying this facet of human creativity. Typically, each of these groups has one or two games that they have designed to test creativity in a particular context. However, just like in Google’s algorithms, one needs lots of diverse data to understand subtle interactions.
At ScienceAtHome, we believe that we can only unlock the subtle patterns of human creativity by studying it in many shapes and forms simultaneously. We have developed a portfolio of high-dimensional, natural science challenges (within physics, chemistry, and mathematics). At the same time, we are setting up a portfolio of more controlled individual and collective games. We believe that such a “social science supercollider” infrastructure will allow the massive and detailed investigations necessary to overcome current methodological hurdles. In e.g. the Quantum Minds and Alice Challenge projects, we have taken first steps into combining natural and social science investigations. However, it is also becoming increasingly clear that even with data from our current 300,000 volunteers, typical big data methodologies struggle to give clear insights into subtle human strategies.
One of the mind-boggling (and perhaps comforting) realizations coming from the social sciences in recent years is the fact that humans are often superior to algorithms because and not despite having limited computational, psychological, and physiological capabilities. As a simple example, human minds have limited storage capacity but this has forced us to develop sophisticated information prioritization algorithms (forgetting :)) that make us immensely efficient when forced to make fast, heuristic choices. Similarly, when balancing exploration and exploitation in complex innovation tasks, both skills and weaknesses play together to influence our choice of strategies. Out of these insights and our big-data frustrations came the concept for the Skill Lab games: by simultaneously investigating complex problem solving and knowing the cognitive profile of each player, we would be able to perform an a priori categorization of players into persona types and then look for common patterns of learning and exploration within each group.
Skill Lab: Science Detective
Skill Lab: Science Detective is designed to be a real gaming experience instead of a gamified experience for citizen science research (lead scientist: Carlos Días, Phd). It is based on the principles of stealth assessment and evidence-centered design. The mini-games are embedded in a story-driven game where the player is a detective. The player’s challenge is to solve the mystery behind strange events happening at a university campus while helping researchers at different labs.
We digitally adapted a series of 16 challenges which players can access at points inside the game. The game establishes a pact between citizen and scientist. We give players access to the newest psychologically-founded personal cognitive map allowing them to self-assess strengths and weaknesses in their cognitive skills while donating their data. In the fall of 2018 we launched a prototype of this large-scale profiling in collaboration with Danish Public Broadcasting. More than 15,000 people have already participated and because of the citizen science methodology and the publicity by a public service channel, participation is very evenly distributed across ages and by gender. This is something truly unique in the field of social science.
Bridging the Gap Between Biology and Social Phenomena
Denmark maintains a Central Personal Registry (CPR) database for government data on each citizen, which provides for up-to-date services. This has established Denmark as a haven for registry-based research, in which any social-science question (such as entrepreneurship or health issues) can potentially be investigated by correlating to the CPR-data. In the future, genome mapping done by companies such as 23andMe will allow research into how our biology might impact the emergent phenomenology of cognitive and social behavior. However, first efforts clearly demonstrate that we are far from establishing a full microscopic understanding.
So far, research based directly on cognitive indicators has either relied on self-reporting, which is small-scale and unreliable, or on the one big database that exists for cognitive profiles: military draft data. Although the latter has given rise to a number of insights it harbors an intrinsic problem: such data is predominantly available for one demographic group: young men. It has recently come to light that much medical testing has been conducted predominantly on males and therefore many documented side effects are much more relevant to men than women. Of course, this is a huge medical problem and a big social issue if we view human interactions based on insights derived in a gender-unbalanced way.
In our Danish 2018 pilot, we took steps to demonstrate the applicability of this new database for societal registry-based research. We called for researchers to pose a few brief survey questions for Skill Lab users. These responses can then be correlated with the game-generated individual cognitive profiles to investigate e.g. entrepreneurial intent, risk preferences, cognitive aging, political ideology, as well as relations between language skill and working memory.
Imagine how this work can help develop a model of how humans solve problems and reason through choices. It can lead us to a better understanding of cognitive biases, logical fallacies, societal interactions and issues, turning the data into a powerful tool for humanity to understand cultural and political dynamics. Understanding these issues can help us improve our society in the future.
Consider what we might be able to do if we could examine cognitive profiles from the whole world? Could you create a social media digital assistant that detects when you are being manipulated? For that you need much deeper insights than what is currently available. Let’s also consider other challenges: Could we help overworked teachers deliver on the promise of personalized education? Could businesses be more systematically successful if they were less dependent on the intuitions of exceptional, natural born leaders?
Just like the thrill of discovering a new continent 500 years ago, today we can get a thrill of uncovering some of the hidden landscapes of our knowledge. One of the most important dark spots is how the individual mind works and how many individuals join collectively to accomplish important achievements.
About the authors: Jacob Sherson is Founder and Executive Director at ScienceatHome and Professor at Aarhus University. Janet Rafner is Director and International Coordinator at ScienceatHome.
