The data are collected from a human subjects study in which 100 participants solve chess puzzle problems with artificial intelligence (AI) assistance. The participants are assigned to one of the two experimental conditions determined by the direction of the change in AI performance at problem 20: 1) high- to low-performing and 2) low- to high-performing. The dataset contains information about the participants’ move before an AI suggestion, the goodness evaluation score of these moves, AI suggestion, feedback, and the participants’ confidence in AI and self-confidence during three initial practice problems and 30 experimental problems. The dataset contains 100 CSV files, one per participant. There is opportunity for this dataset to be utilized in various domains that research human-AI collaboration scenarios such as human-computer interaction, psychology, computer science, and team management in engineering/business. Not only can the dataset enable further cognitive and behavioral analysis in human-AI collaboration contexts but also provide an experimental platform to develop and test future confidence calibration methods.
AI assistance; Artificial intelligence; Confidence; Decision-making; Human-AI interaction; Trust.
© 2023 The Author(s).
Conflict of interest statement
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Chong L., Zhang G., Goucher-Lambert K., Kotovsky K., Cagan J. Human confidence in artificial intelligence and in themselves: The evolution and impact of confidence on adoption of AI advice. Comput. Hum. Behav. 2022;127 doi: 10.1016/j.chb.2021.107018.