Winograd schema challenge

The Winograd schema challenge (WSC) is a test of machine intelligence proposed in 2012 by Hector Levesque, a computer scientist at the University of Toronto. Designed to be an improvement on the Turing test, it is a multiple-choice test that employs questions of a very specific structure: they are instances of what are called Winograd schemas, named after Terry Winograd, professor of computer science at Stanford University.[1]

On the surface, Winograd schema questions simply require the resolution of anaphora: the machine must identify the antecedent of an ambiguous pronoun in a statement. This makes it a task of natural language processing, but Levesque argues that for Winograd schemas, the task requires the use of knowledge and commonsense reasoning.[2]

The challenge is considered defeated in 2019 since a number of transformer-based language models achieved accuracies of over 90%.[3]

  1. ^ Ackerman, Evan (29 July 2014). "Can Winograd Schemas Replace Turing Test for Defining Human-level AI". IEEE Spectrum. Retrieved 29 October 2014.
  2. ^ Levesque, H. J. (2014). "On our best behaviour". Artificial Intelligence. 212: 27–35. doi:10.1016/j.artint.2014.03.007.
  3. ^ Kocijan, Vid; Davis, Ernest; Lukasiewicz, Thomas; Marcus, Gary; Morgenstern, Leora (11 July 2023). "The defeat of the Winograd Schema Challenge". Artificial Intelligence. 325: 103971. arXiv:2201.02387. doi:10.1016/j.artint.2023.103971. ISSN 0004-3702. S2CID 245827747.