Teaching The NAO Robot Japanese Calligraphy

110720 NAO Robot Calligraphy 023

We spent a wonderful, and learning packed, afternoon with the Robot Japan team in Tokyo figuring out how to teach the Aldebaran NAO humanoid robot to draw Japanese kanji characters in the traditional calligraphy style called "Shodo". It may look simple, but take it from me, drawing the characters correctly, in the right stroke order, and with the proper energy and spirit can be a real challenge, even for a human.

In the beginning the task looked fairly straight forward. At least that's what we thought. Setup a traditional calligraphy desk with an assortment of brushes, put the NAO robot into learning mode using its rich software environment with some modification by Taylor Veltrop, then show NAO the proper moves to form the desired character while he learns (records) it, then play it back. It turned out to be much more difficult, and much more fun, than we imagined.

110720 NAO Robot Calligraphy 021

Setting up the calligraphy environment.

110720 NAO Robot Calligraphy 028

Holding the calligraphy brushes turned out to be a difficult but totally solvable challenge. The NAO robot hand only has three fingers. It's enough, but we did have to build up the brush handle diameter using tape to get a secure, stable fit.

110720 NAO Robot Calligraphy 029

Taylor Veltrop - The resident NAO robot guru as well as being a RoboGames champion several times over. Taylor was responsible for the vast majority of the work, while the rest of us were happy backseat drivers chiming in with lots of what we thought would be helpful suggestions and observations.

110720 NAO Robot Calligraphy 025

Yoshihiro Shibata - Technical advisor to Robot Japan and a famed humanoid robot designer, Shibata was the lead engineer for several Kondo KHR series robots as well as the Kyosho MANOI AT-01 robot. In addition to being a ROBO-ONE champion level competitor, recently he expanded his scope internationally and in April this year became a RoboGames Gold Medal champion also.

Today he provided valuable insight and advice as the project evolved and enabled us to make a lot more progress than we had  expected. Along the way he also managed to learn a lot about the NAO robot and its supporting software system and applications. He's really a 'quick study' and picks up new concepts extremely quickly.

110720 NAO Robot Calligraphy 031

Grasping NAO's hand and moving it over the paper involved a large number of variables including the multi-bar linkage consisting of the robots fingers, hands, wrist, elbow, and its relationship to the torso and legs.

110720 NAO Robot Calligraphy 036

It took quite a few attempts before the robot was producing anything close to the desired results. Factors like the angle of its back; the positioning of the brush within the robots hand; the elbow and other joint angles; and the stability of the work surface; all had to be considered and a strategy for dealing with the effects of each factor was a high priority.

110720 NAO Robot Calligraphy 035

110720 NAO Robot Calligraphy 037

About 2/3rds through the afternoon it occurred to us that the vertical positioning of the brush was sometimes defeating our efforts. Marking the outline of each finger provided a quick and easy fix.

One of the more difficult challenges was to compensate for the different loading of NAO's shoulder/arm/elbow/. For example, when the robot was learning a set of strokes so that it could reproduce the kanji, its hand and lower arm were held by the trainer. Later, when the movement sequence was replayed, its arm didn't have the human hands supporting it. So the calligraphy brush would end up being jammed into the table top at times.

110720 NAO Robot Calligraphy 038


110720 NAO Robot Calligraphy 042

They almost resemble a proud mother and father - and that's not far from the truth. Everyone was totally overjoyed when NAO successfully completed the initial tests.

110720 NAO Robot Calligraphy 040

If you noticed that this particular NAO looks like it's sporting a custom color job, you would be right on target. Taylor told us that somehow he had managed to inflict major scratches on the robot when testing it outside in the real world. So, he committed to making it all right. In the end he singlehandedly did the new blue paint job. Even though we knew that he had repainted his NAO, it was hard to tell that the robot wasn't a stock unit - Taylor's paint job was that good. I'm definitely jealous.

110720 NAO Robot Calligraphy 043

We toyed with the idea of using a continuous roll of paper with the unused portion of the roll hidden under the writing surface. We'll probably switch to hacking a cheap, small table where the unused portion of the paper roll can be concealed from the audience's view.

110720 NAO Robot Calligraphy 041

NAO's calligraphy is often so unusual, unexpected, and surprising that one of these days his works may appear in one of the major art museums and galleries.

110720 NAO Robot Calligraphy 044

Taylor and Shibata review the day's results and develop their initial plans for the upcoming Robot Japan event.

110720 NAO Robot Calligraphy 045

After a long hard day, NAO climbed into his suitcase for a well deserved rest during the trip back home.

I have considerable video footage showing how NAO fared. Unfortunately it will take a while to edit and upload to our ISP. Look for it within the next week or week and a half.



Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>