Natural language referring expressions in a person/machine dialogue.


G. Dudek, R. DeMori

The scenario consists of a human user who commands a spatial exploration system to perform certain tasks. The format of the command comprises two parts: an action and a description of the object which is to be acted upon, e.g.: "Get the small manila envelope". It is then the system's task to determine the object to which the user is referring. To accomplish this, the system uses fuzzy logic to map the information provided by the user onto the data collected by the system's sensors about the world objects. The system selects as the referent the single, remaining candidate surpassing a given threshold of acceptability. If, however, the supplied information is insufficient for the system to uniquely determine the referent, it must resort to questioning the user about a particular attribute. A key aspect of the problem is the tradeoff between asking the user further questions (which may be expensive or time consuming in some contexts) and carrying out sensing or exploration autonomously. Entropy is used to select the attribute which will maximally discriminate between the candidates. Questioning continues until one candidate stands out as the most likely referent. Click here for a compressed postscript copy of a recent paper on this work.