Understanding and Reverse-Engineering Volitional Agent Systems


The ultimate goal of artificial intelligence has been to engineer human intelligence—the cognitive ability of humans—in machines. Traditional computational approaches have tried to design animal and human intelligence from scratch, the implicit idea being that an army of clever programmers could encode the algorithms of human behavior, so that the resulting system would behave exactly like a human and, therefore, embody human intelligence.

The trouble with this traditional approach is that it looks at the surface behavior of animals and humans and tries to make a leap to algorithms that model that surface behavior. No attention is paid by the researchers to the very complex neural mechanisms that give rise to animal behavior, or to the actually quite extensive knowledge the neurosciences have gathered about the architecture of mammalian brains. In the absence of attention to neuroscience, the traditional AI researcher has an enormous space of methods and algorithms to search. To date, after over 50 years of research in the field, the fruits of this traditional approach have fallen far short of the expectations. We do not yet have an AI system that even has the intelligence of a human toddler, or even, I would argue, that of a humble rat.

A better, faster approach to solving the problem of human intelligence would be to pay closer attention to the neural mechanisms animals use in cognition and behavior, and to engineer systems that have the same design principles: systems that process information, learn, and behave in a similar way to animal neural systems. Instead of trying to "hard-code" every aspect of human knowledge into such systems, we would encode into them the means that allow them the capacity of active learning: the ability to interact with their environments as an autonomous agent and learn from the results of those interactions, for example, from the reward and punishment signals the environment sends the agent.

In addition to its inattention to animal models of behavior, I believe traditional AI's focus on perception and the input stages of cognition has shifted focus away from the importance of the output / behavior stage mechanisms of agents. Neuroscience, too, has long had the same input stage focus: the visual and auditory systems are still far better understood than the motor and behavioral systems. What is needed is more attention to the behavior components of the system, especially to mechanisms that determine how behaviors, and also goals, are adaptively selected from among alternatives.

An approach I believe would be fruitful and illuminating would be to attempt to reverse-engineer specifically human volition, where volition is defined as: the capacity for adaptive decision-making. (See my paper published in Adaptive Behavior, Assessing Machine Volition, for more a more detailed formulation of volition.) The higher semantic processing of human natural language, formal logic, etc., may be ultimately important in engineering systems that perform at human levels of cognitive ability. However, there is a whole non-verbal active-learning agent architecture that sub-primate animals possess that is quite clever and probably the foundation on which the higher, symbolic features of human cognition are built. Reverse-engineering this non-verbal volitional architecture seems to me to be the first step in achieving the goals AI has long sought. Once these mechanisms are operational, the symbolic information processing we do will be more likely to emerge from our artificial agents because the same learning architecture is used for symbolic and non-verbal aspects of human cognition.

The focus of my research interest, then, is in the reverse-engineering of animal volition and its application to state-of-the-art AI technology. There are two components of my research interests:

Computational Cognitive Neuroscience: Investigating the Neural Mechanisms of Animal Volition

A detailed understanding of the neural mechanisms underlying human behavior and cognition may be one of the most important objectives of twenty-first century science. Such an understanding would not only satisfy our philosophical curiosity, but would allow us to better clinically treat mental disorders, and also engineer artificial systems that are capable of levels of intelligent behavior currently only possessed by natural systems.

The neuroscience component of my research interests is focused around an attempt to understand the functional and neural basis of human volition. The fundamental question is how the brain selects which behaviors are active among the vast repertoire of possible behaviors, in response to both current environmental stimuli and conditions and internal bodily state and representations of current goals. Understanding how areas of mammalian brain such as the prefrontal cortex, basal ganglia, ventral tegmental area, anterior cingulate cortex, hippocampus, and cerebellum function together to allow learning, planning, and execution of well-adapted behavior is the immediate target of my research. An integrative large-scale computational modeling approach is the primary technique I'm interested in applying to the problem, with an emphasis on developing simulations or autonomous systems that illuminate the functional principles operative in mammalian brain. The operation of neuromodulatory neurotransmitters such as dopamine, acetylcholine, norepinephrine, and serotonin are likely to critically regulate the function and interrelationship of the above-described areas of brain, so I regard neuromodulatory mechanisms as a focus area of investigation. I'm also interested in how the functionality of the volitional system and individual variation in that functionality factor into differences in personality and temperament, and how pathologies in their function may generate symptoms of cognitive, affective, and behavioral dysfunction.

Neuromorphic AI Development: Reverse-Engineering Animal Volition

A thorough understanding of the neural mechanisms of the volitional architecture in mammalian brains is likely to revolutionize psychiatry by giving it a firm physiological theoretical foundation. It may also provide a schema for advancing artificial intelligence to the levels long envisioned in popular fiction. One of the first challenges that might be tackled once a better understanding is gained of the neural volitional architecture is the development of robots that are capable of general-purpose task learning based on methods analogous to animal conditioning. An example application might be a domestic robot capable of learning a wide repertoire of routine household chores through conditioned emulation of human behaviors.

The AI development component of my research interests is focused around embodying the principles of the neural architecture of animal volition in artificial systems. One of the first steps I would like to take is to develop a neural network architecture based on the neural mechanisms of reinforcement learning. Such an architecture would attempt to translate current theory about the functioning of the basal ganglia, frontal cortex, midbrain dopaminergic areas, hippocampus, the cerebellum, and the anterior cingulate cortex into a general learning engine that can be used in a wide array of robotic and/or virtual agent domains. Such a system would be capable of active reinforcement learning of behaviors, allowing the same system to build up a repertoire of behavior in multiple domains, with the primary limit being the sensory and motor interfaces provided to the system.