Why teaching robots to blink is hard but important

It's my first time drumming along with a robot.

I'm sitting across a table from an adorable humanoid robot called the iCub. We each have our own stick and box, and are meant to strike the box with the stick in sync with a light pattern. But of course I'm also watching the robot - and I'm aware that it's watching me.

This drumming experiment has been designed to test how the presence of a robot doing the same task affects a human's behaviour.

It's one of many experiments of human-robot interaction being carried out by the research group CONTACT (Cognitive Architecture for Collaborative Technologies) at the Italian Institute of Technology in Genoa.

Though I'm aware of the lights and the other people in the room, my gaze keeps being drawn to the iCub's heavy white eyelids.

For one thing, there's a noticeable noise when the robot blinks every few seconds. And, because this childlike robot has expressive, oversized eyes, its gaze is compelling.

This matters because with blinking, as with other aspects of eye gaze, there's more than strictly meets the eye.

"While it is often assumed that blinking is just a reflexive physiological function associated with protective functions and ocular lubrication, it also serves an important role in reciprocal interaction," comments Helena Kiilavuori, a psychology researcher at the University of Tampere in Finland.

Human blinking conveys attention and emotion. As a means of non-verbal communication, it expresses a number of things we aren't consciously aware of, such as who should take a turn in a conversation.

Thus, it's just one of many social signals humans exchange constantly without being aware of them, yet while deriving a great deal of information and comfort from them.

So social roboticists have been studying both the physical and psychological properties of human blinking to understand why it might be useful to match it in robots.

"Due to the many important functions of blinking in human behaviour, it could be assumed that having robots blinking their eyes could significantly enhance their perceived human-likeness. This could, in turn, facilitate human-robot interaction," says Ms Kiilavuori.

Put another way, "When the robot's blinking well, people feel for your character," says David Hanson, who leads Hanson Robotics.

Indeed, CONTACT research with 13-year-olds and adults in Italy shows that both groups simply like blinking robots more. An unblinking robot can give a person the unpleasant impression of being stared at, points out Alessandra Sciutti, who leads the CONTACT unit.

Humans also perceive robots that blink more naturally as more intelligent. And intelligence matters in situations where humans depend on robots to provide information, such as in train stations.

Despite the benefits of natural-looking blinking, integrating it into robots (apart from avatars and robots with screen displays for faces) is technically challenging.

"Blinks are one of the subtlest of human movements, so designing mechanisms that can mimic these movements require advanced technology, such as high-precision motors," Ms Kiilavuori explains. The roboticists at Engineered Arts, for example, use expensive, aerospace-grade motors as well as designing their own control electronics.

Specifically, says Mr Hanson of Hanson Robotics, "The speed of the motors moving skin material when blinking is really challenging and making the shape of the eye look natural while blinking is happening is also challenging. As well as reducing friction between artificial eyelids and the eye surface itself."

Another issue here is a trade-off between the speed and sound of motorised blinking. Francesco Rea, a senior technician at the CONTACT unit, says that in the iCub robot a quieter motor could dampen the sound of a blink. But the slower movement then makes the robot look sleepy or catlike.

Very slow blinks also risk a loss of visual information, as the iCub's camera is located behind the eyes. "In vision, missing two frames is not that big an issue," says Mr Rea. "Missing ten frames starts to be an issue."

Ms Kiilavuori adds, "Another challenge is the correct timing and duration of blinks." The different functions that blinking performs - such as how a person changes blinking speed while telling a lie - involve different eyelid dynamics as well as different emotional states.

"Any deviation from the natural and appropriate blinking times and duration, in a given context, can make the robot appear odd and disturbing," she says.

The CONTACT team use a software program that partially randomises the intervals between single and double blinking. After all, blinking fixedly wouldn't look very natural either.

Over at Disney Research, roboticists have joined forces with character animators to develop a research prototype for realistic robotic gaze. The aim is to design an expressive system of eye gaze that is easy for animators to control in order to convey subtle emotion.

With elements like motion curves of eyelids, "we can kind of isolate these individual behaviours, which makes it much easier to really concentrate on getting small aspects and small details correct," says James Kennedy, a research scientist at Disney Research.

They've patented their system of robotic sensing and control of eye gaze. This includes software for processing images taken by a camera in the robot's chest, and generating control signals for movements like opening and closing the eyelids.

Mr Kennedy says that the research remains more experimental, and isn't yet being applied in Disney's theme parks. "The goal here was to really select a single social cue that we were interested in and push it as far as we could in making lifelike believable motion and behaviour that we felt would provide a platform for engagement with people."

Another general challenge would be getting humanoid robots to begin syncing their blinking patterns to those of humans, as humans do in conversation.

These types of challenges remain compelling to some roboticists. And, contrary to exaggerated pop culture depictions of androids that are indistinguishable from humans, blinking is one tiny example of the many complexities that still keep robotic interactions from seeming completely natural.

When trying to replicate a mechanism as tiny and sometimes underappreciated as blinking, "actually you reveal how complex this mechanism is, and then how much subtle movement there is," notes Mr Kennedy. "And that's where we have this really great opportunity for exploration and invention."