A Neuromorphic Robot that Interacts with People Through Tactile Sensing and Bi-directional Learning

Brief Description

The device is an interactive neuromorphic robot, using to mimic neuro-biological architectures and learning.Properties include:a spiking neural network to control robot behavior, inexpensive parts which are easily available, and two-way learning and behavior shaping.The technology is autonomous, highly mobile, and includes on-board measurement equipment.

Full Description

This invention is a highly interactive neuromorphic robot or neurorobot, called CARL-SJR (Cognitive Anteater Robotics Laboratory- Spiking Judgment Robot), which is capable of tactile sensing and interaction with people. CARL-SJR can exhibit a wide range of behaviors and has the learning potential to tailor its behavior to a particular person.

A spiking neural network (SNN), based on neuronal activity in the brain, controls the robot’s behavior.This SNN can learn to recognize different hand movements across the robot’s surface. In particular, the spatial temporal nature of the interaction is closely coupled with the response of the SNN and its ability to learn spatiotemporal patterns.The robot’s surface is covered by an array of trackballs, which serve as tactile sensors. Inputs to the trackballs, such as sweeps of a hand across the shell, are coded into neural spikes that retain spatial and temporal information and colors and patterns.The robot can respond to these touches with sound, movement, and displaying a color pattern across its shell. These responses can communicate behaviors, desires, or useful information to a user or a patient.In addition, an onboard camera can be used to recognize the user.

CARL-SJR can take advantage of two way learning or behavior shaping, where CARL-SJR learns from the user and the user learns from CARL-SJR.Different hand sweeps can be coupled to reward and punishment. For example, the robot might signal a soothing color and emit a pleasant sound when it is rubbed from front to back, but will signal a harsh color and noxious sound when petted back to front.In another interaction scenario, CARL-SJR could learn to recognize and tailor its response to a particular user.For example, a child might pet CARL-SJR in a non-pleasing way, CARL-SJR can move away from this child when it gets touched or recognizes the child.

In addition, the robot is autonomous and does not need constant supervision by a caretaker.Moreover, many measurements that a clinician might be interested in, such as rate of interaction, rate of learning, who interacted with the robot, the type of interaction, etc. can be saved on-board the robot. This frees the caretaker from monitoring progress in a clinic or specially outfitted room.

The robot’s unique form was designed to encourage users to rub or pet its surface. Moreover, the robot has the capability to signal or communicate by flashing different colors in response to touch or other stimuli. Rather than using expensive custom-made sensors, the researchers incorporated inexpensive trackballs, which are typically found in cellphones and other devices, to signal the direction and velocity of tactile stimuli. The choice to use cheaply produced trackballs allows for CARL-SJR to have sensor input over a large portion of its surface. In addition to the trackball sensing, CARL-SJR carries a suite of sensors and communication devices including: a pan-tilt-zoom network camera, microphone, speaker, WiFi, Bluetooth, wheel encoders, and bump sensors. The robot also employs a holonomic drive system using three, triangularly oriented omni-wheels allowing for mobility within the confined spaces of a home clinic or office building.

Suggested uses

• Treatment for dementia and Alzheimers, where patients have been shown to respond well to a platform that can be handled and can respond to contact.
• Treatment for children with developmental disorders, such as Autism Spectrum Disorder and Attention Deficit Hyperactivity Disorder, where patients have been shown to respond well to robotics.
• Interactive entertainment
• Programming modules, allowing users to change color and sensor schemes as well as develop custom games and interactive features, including visual and voice recognition
• Interchangeable modules, allowing users to incorporate different sensors and actuators for multiple reactions to human interfaces, including sound, light, and vibration

Advantages

• Autonomous system, CARL-SJR does not need constant supervision by a caretaker.
• On board computer, hard-drive, and sensing.. As opposed to relying on external equipment, measurements that a clinician might be interested in can be saved on-board the robot.
• High mobility. The form and the drive system allow CARL-SJR to move in any direction and through tight spaces.
• Two-way learning. Learning is inspired by neurobiology and cognitive science. Interaction that is adaptive, shaped over time, and tailored to a child or patient’s specific needs.

Patent Status

Country	Type	Number	Dated	Case
United States Of America	Issued Patent	10,369,459	08/06/2019	2014-263
United States Of America	Issued Patent	9,975,038	05/22/2018	2014-263

Publications

UC Irvine Cognitive Robots

Sensory Decoding in a Tactile, Interactive Neurorobot

Lead Inventor

Jeff Krichmar
Department of Cognitive Sciences
School of Social Sciences
University of California, Irvine

http://www.socsci.uci.edu/~jkrichma