Skip to content. Skip to navigation
CIM Menus

Inferring External Stimulus Properties from Internal Sensory States Alone Through Action and Invariance

Dr. Yoonstuck Choe < >
Assistant Professor, Dept. of Computer Science Texas A&M University

August 3, 2005 at  11:00 AM
Zames Seminar Room - MC437

How can we build artificial agents that can autonomously explore and understand their environments? An immediate requirement for such an agent is to understand what its own internal sensory state means, i.e., to learn the semantics of its internal sensory state. In principle, we as designers can provide the agents with the required semantics, but this turns out to be a challenging engineering problem. To overcome this challenge, we need to first look at natural agents and see how they acquire meaning of their own sensory states -- their neural firing patterns. We (as external observers) can learn a lot about what certain neural spikes mean by carefully controlling the input stimulus and observing how the neurons fire. However, neurons embedded in the brain do not have direct access to the outside stimuli, so such a stimulus-to-spike association may not be possible. How then does the brain solve this problem?

We propose that motor action is necessary to overcome this conundrum. Further, we provide a simple yet powerful criterion: sensory-invariance, for learning the meaning of sensory states. The basic idea is that a particular action sequence, which maintains invariance of a certain internal sensory state, reflects important properties of the environmental stimulus that triggered the sensory state. Our experiments, with a simplified sensorimotor agent, shows that sensory-invariance can indeed serve as a powerful objective for learning internal semantics. The results from the experiments will be discussed in relation to other relevant work, and several unresolved issues (such as the privileged status of action-based knowledge, need for structure in the input, role of attention, etc.) will be considered.