Non-cooperative strategies in the Loebner contest

The main aim of this project is to investigate non-cooperative strategies used in the practical realisation of the Turing test, namely the Loebner Contest.

[The project was partially supported by the Institute of Pychology AMU Grant for young scientists “Badanie niekooperatywnych strategii graczy w konkursie Loebnera” (Non-cooperative strategies of players in the Loebner contest).]

Conferences:

  • Paweł Łupkowski, Non-cooperative strategies in the Loebner contest, Investigating semantics: Empirical and philosophical approaches, Bochum, October 10–12, 2013.
  • Paweł Łupkowski, Aleksandra Rybacka and Jakub Marszałkowski, Measuring Non-Cooperation of Dialogue Participants. Loebner Contest Case Study, 12th ArgDiaP: From Real Data to Argument Mining, Warsaw, May 23–24, 2014.
  • Paweł Łupkowski, Aleksandra Rybacka, Niekooperatywne strategie graczy w konkursie Loebnera — ludzie a maszyny, Kulturotwórcza funkcja gier — Game-based learning / Game-biased learning, Poznań, 15–16.11.2014.

Papers:

  • Łupkowski, P. [2013]. Measuring the non-cooperation of players—a Loebner contest case study. Homo Ludens, 5(1):13–22. [PDF]

Abstract: The B. Pluss’ measure of degrees of non-cooperation in a dialogue is used for the Loebner Contest. Types of non-cooperative features of the contest’s dialogues are described and the reliability of annotation with these types is analysed. Degrees of non-cooperation of both players in four rounds of the Loebner Contest 2010 are counted and discussed.

  • Rybacka A. [2015]. The Study of Verbal Non-Cooperative Behaviours of Players in the Loebner Contest, M.A. Thesis (supervisor: Paweł Łupkowski), Institute of Psychology, Adam Mickiewicz University in Poznań, Poland.

Abstract: In this thesis the idea of the Turing test is described together with its practical realisations with the emphasis put on the Loebner contest. The Brian Plüss’ measure of the degrees of non-cooperation in a dialogue is applied to the dialogues of the Loebner contest. The proposal of types of non-cooperative features in the contest’s dialogues is discussed and the reliability of annotation with the use of these types of features is analysed. The degrees of non-cooperation of judges and programs for the Loebner contest (editions 2009–2012) are presented and discussed.

  • Łupkowski P. and Rybacka A. [2016]. Non-cooperative Strategies of Players in the Loebner Contest, Organon F, 23 (3) 2016:324–366. [PDF]

Abstract: In this paper the idea of the Loebner contest as a practical implementation of the Turing test is presented. The Brian Plüss’ measure of the degrees of non-cooperation in a dialogue is applied to the dialogues of the Loebner contest. The proposal of a typology of non-cooperative features in the contest’s dialogues is discussed and the reliability of annotation with the use of this typology of features is analyzed. The degrees of non-cooperation of judges and programs for the Loebner contest (editions 2009–2012) are presented and discussed. On the basis of the results the role of a judge and the strategies used by programs are discussed for the contest and the Turing test.