We use a few cookies

Essential cookies keep the library working. With your permission, we'd also like to use analytics to see what's useful and improve the experience. Read our Privacy Policy or Cookie Policy.

Skip to main content
Home/e-Resources/Artificial Intelligence & Machine Learning/People cannot distinguish GPT-4 from a human in a Turing test
Conference ProceedingsArtificial Intelligence & Machine Learning

People cannot distinguish GPT-4 from a human in a Turing test

Checking your access to this record.

Abstract & Overview

AI systems that can fool people into thinking that they are human could have widespread social and economic consequences.In order to measure this ability, we evaluated 3 systems (ELIZA, GPT-3.5 and GPT-4) in a randomized, controlled, and preregistered Turing test.Human participants had a 5 minute conversation with either a human or an AI, and judged whether or not they thought their interlocutor was human.GPT-4 was judged to be a human 54% of the time, significantly more often than ELIZA (22%) but less often than actual humans (67%).In order to test the generalizability of our results, we replicated the study on a second population (undergraduate students) and found that the same prompt with GPT-4o achieved a pass rate of 77%, slightly higher than the human pass rate of 71%.On some interpretations, the results provide the first robust empirical demonstration that any artificial system passes an interactive 2-player Turing test.The results have implications for debates around machine intelligence and, more urgently, suggest that deception by current AI systems may go undetected.Analysis of participants' strategies and reasoning suggests that stylistic and socio-emotional factors play a larger role in passing the Turing test than traditional notions of intelligence.We release the full transcripts of the replication data to enable further investigation of human-AI interaction dynamics and deception.

Cite this record

Cameron R. Jones, Ishika Rathi, Sydney Taylor, Benjamin K. Bergen (2025). People cannot distinguish GPT-4 from a human in a Turing test. ACM Conference on Fairness, Accountability, and Transparency (FAccT). https://doi.org/10.1145/3715275.3732108

Copy in another style

Jones, C. R., Rathi, I., Taylor, S., & Bergen, B. K. (2025). People cannot distinguish GPT-4 from a human in a Turing test. ACM Conference on Fairness, Accountability, and Transparency (FAccT). https://doi.org/10.1145/3715275.3732108
Author names are used as deposited, so check their order before you submit.

Metadata is harvested from open-access indexes and can be incomplete. Check a citation against the published version before relying on it — see our open access & attribution policy.