Once the basic architecture was up and running, and could consistently control my Robosapien via the USB-UIRT, it was time to do some design brainstorming. What was it that I liked about the RS remote control? What didn’t I like? What features would I really want to see in a remote control?
I had some clues. For example, one of the European university websites mentioned that RS has 9 degrees of freedom. How could I show that on the control, and make it easy to control each degree of freedom? In thinking about that problem, I recalled how the Japanese Robot Battle competitors program their robots. They typically have their left hand on the robot while their right hand is typing commands into the computer keyboard. They get immediate tactile feedback at a gut level.That lead me to the idea that the controller image should map directly to the Robosapien itself – hopefully on a 1:1 basis.
After some consideration, I came up with the following design guidelines:
- Intuitive – in other words, the remote control function should provide a WYSIWYG type of experience.
- Logical – it should make sense at a gut level and not require a lot of analysis to use.
- Shallow – as much as possible one button press should evoke the behavior you want. Control and multiple key strokes should be minimized.
- Facilitate – it should help the user and anticipate their desires, not get in the way of them.
- Toy – Robosapien, after all is said and done, is a toy. The remote control should be toy like in terms of design, colors, and experience.
- Fun – If it isn’t fun, then no one (including me) is going to use it for very long.
With that in mind, I came up with this design -
(click on the graphic for a full size image)
Once I had the controls laid out the way I wanted them, it was easy to combine the design with the operational software. Using Visual Basic 6, I imported the graphic layout into a form, then created an image array with enough elements for all the buttons. Each element in the image array overlays a button image in the graphic. This makes it easy to do things like tool tip text hints for mouse-overs.
The end result: an extremely easy to use, intuitive, fun to play with, remote control for my Robosapien.
When I have a chance over the next week or so, I’ll put together a post covering the details of using an image array to simulate graphic buttons in Visual Basic.
Here’s what the ‘finished’ remote control looks like-
As of this afternoon (Saturday, February 26th), everything is up and running. Integrating the USBUIRT API and DLL code into the VB6 application I built to control my Robosapien went much smoother than I had expected. The package that Jon provides for developers doesn’t include much in the way of documentation – but I have to say that it doesn’t really need it. The sample code is logically structured and includes extensive comments. It was very straightforward to figure out what was going on, and to strip out the parts that were not necessary for my application.
One additional feature, hacked together today, are the indicator ‘leds’ that show the USBUIRT status and indicate when it’s transmitting. I retained the actual USBUIRT status parameters, but moved them to the About page. After everything seemed to be running, I spent about an hour
playing around testing the application. It was really a blast – almost like rediscovering the Robosapien all over again for the first time.
I certainly enjoyed playing with Robosapien when I first brought him home. Still, I was a little frustrated by the standard remote control, and its learning curve. Back then, I was trying to get him to move forward, or turn, or raise one of his arms, but to do that I had to figure out where the buttons were, and I often had to deal with the multi-level command sequence – press the select button, one or two times, then the function you want Robosapien to perform. Unless you happen to be a quick study, it’s easy to get confused and frustrated. Mark Tilden and the other designers did an excellent job – no question – but trying to jam 62 functions into a low cost remote control in a simple, easy to understand and use way is a tremendous challenge.
What I found today with the PC based remote control was that it is much, much easier to run Robosapien through his paces. One button, one push, and immediately he goes off and does exactly what you tell him to. I found myself spending a lot more time observing what he was doing rather than spending it trying to figure out the remote control buttons. And, in a few minutes, I was playing ‘what if’ games with him. Seeing how he moved, and predicting his movements for a sequence of commands. Really cool.
Next step: Use this as a front end for the choreography program.
Sometimes you just get lucky. There doesn't seem to be any other reasonable explanation for it. Just luck - plain and simple.
I had pretty much given up on doing a full remote control application - primarily because I had visions of trying to write and debug the code necessary to handle the 62+ Robosapien functions. I just knew that at some point - probably around the 20th button or so, I would either get totally frustrated, or extremely bored. The result would be yet another interesting software project that wound up on the shelf gathering dust.
Still, I needed a simple test program to send IR codes from my PC USBUIRT interface to my Robosapien. I had the basic code transmission worked out using Girder, so I knew that part was working. My short term goal was to get a choreography program up and running using Visual Basic 6.0. I chose VB for several reasons-
- I've used VB for years, and know it fairly well [I'm basically lazy]
- I already have the VB development environment setup on my primary systems. [I'm basically cheap]
- The design and syntax should be close enough so that other people can port it to the language of their own choice.
I seriously considered other language development environments, but the thought of having spend the $$$ for another development platform and to learn yet another programming language quickly discouraged me from going down that path.
I started off by just putting a couple of command buttons on a form, and labeling them with Robosapien IR code command names. I wrote some code to read a modified version of the Girder Robosapien commands and codes, and store them in a series of arrays. Then I thought it would be cute to have a picture of the robot on the form..., then I added a few more buttons..., and a few more...
By the time I got to the sixth or seventh command button I could tell that this approach wasn't going to be very practical. So, I started to think about alternative ways to skin this cat. It turns out that every time you cut and paste a command object, VB tells you that you already have one with the same name, and asks if you want to create an array. Suddenly the mental light bulb went on in my head, and within a couple of minutes I had populated the entire form with enough buttons to handle all the existing codes plus a number of empty buttons set aside to use with my special choreography commands.
The next step was to sort the command buttons into groups of similar functions, and place them in some sort of logical, easy to use, order on the form. I'm sure that after I use it for a while I will want to change some of the layout, but for now, this is what it looks like-
Taking a closer look at the form layout, you can see that each command button has a sequentially numbered caption. All of the buttons on this form, with the sole exception of the one labeled 'Refresh', are elements in a single command control array. I also added some status boxes at the bottom of the form to display the current command, which group it belongs to, the function it performs, and the actual USBUIRT IR code to be transmitted. These will probably be replaced later with just some simple indicators.
When the program first runs, it opens a text file containing the list of groups, commands, functions, and codes then stores them in string arrays. The command array is then used to update the captions and ToolTipText on each of the buttons. Later, if I need to change any of the captions or commands it can be done by editing the text file without having to rebuild or recode the application.
Private Sub Titles_Click()
' Sets the caption for each command button to
' match the command in the indexed array.
For i = 0 To 64
Cmd(i).Caption = cmdarray(i)
Cmd(i).ToolTipText = functionarray(i)
The coding that handles the button clicks is extremely simple and straight forward. The button click generates a corresponding Index which is used as a pointer to the correct elements in the arrays.
Private Sub Cmd_Click(Index As Integer)
' Updates the display and selects the appropriate IR code
' whenever a command button is clicked
' Used for all of the individual command buttons as
' a command array. The array index determines which button
' was clicked
Group.Caption = grouparray(Index)
Command.Caption = cmdarray(Index)
Function1.Caption = functionarray(Index)
Code.Caption = codearray(Index)
' add call to USBUIRT transmit routine
The result is a clean, hopefully easy to understand and use, layout that can be modified and expanded with a minimum of effort as time goes on.
Just to make it look a little more professional, I added an About page, and a browser page that automatically links to my robotics website.
- Incorporate the already tested USBUIRT code
- Add command timing (time that the Robosapien actually takes to execute each command)
- Draft a specification for additional functionality including -
- Command strings
- Elapsed time
- Graphical time line
- Programmable delays
- Variable command repeats
Now that my laptop can send commands to Robosapien, it's time to cobble together some sort of a control program. The first iteration will probably be a "player piano roll" type that will eventually get expanded to include a rich function set - hopefully including sensors and some type of video feedback.
The initial pass could be something as simple as an interpreted text file. Using the existing command names, the text file would look something like this-
<larmallup> ' left arm all up <larmout> ' left arm out <leanr> ' lean right
Of course, the interpreter will need to handle the parsing, ignore the comments, and keep track of the elapsed time. My concern is that since there is no direct feedback from Robosapien to indicate the completion of each command, it may be very easy to get out of sync and have the commands sent faster than Robosapien can receive, decode, and act upon them.
Once I have that up and debugged, then the next step will be to add some additional functionality. It would be useful to have a programmed pause or wait state. For example, if I wanted Robosapien to do an ocean 'wave' move, it would look similar to this-
<larmallup> ' left arm all up <larmout> ' left arm out <leanr> ' lean right <rarmalldown> ' right arm all down <rarmin> ' right arm in <wait time=5> ' pause <rarmallup> ' right arm all up <rarmout> ' right arm out <leanl> ' lean left <leanl> ' lean left <larmalldown> ' left arm all down <larmin> ' left arm in <end> ' end of sequence
The <wait time=5> command simply suspends the transmission of new commands to Robosapien for a given period. It will take a little experimentation and testing to determine the best intervals to use. Perhaps tenths or hundreds of a second.
Most music and dance includes lots of repetition - a musical phrase or set of dance moves may be repeated over and over again during a single performance. If the repetition is simple - say just repeating a single command, then we can add a 'repeat=n' command for the interpretor to process. If the repetition is more complex involving a sequence of commands, then it could be implemented using the gosub/return model.
<gosub dosedo> ' <whistle> ' <gosub dosedo> ' <whistle> ' <gosub dosedo> ' <fart repeat=3> ' <end> ' ' <sub dosedo> ' <walkf> ' <wait time=5> ' <walkb> ' <wait time=5> ' <return> '
I'm going to give it some more thought, but will probably start putting together some initial code over the next week or so.
Progress always seems to come in fits and starts. I make some headway, then hit a bump in the road that either slows me down, or totally distracts me. Sometimes it's so severe that I get totally frustrated and have to set a project aside for a while. At other times it seems like everything wants to come together almost effortlessly. Thankfully, I seem to be in one of the latter cycles right now and everything is going pretty smoothly - knock on wood.
I was able to get the USBUIRT working with Girder, and to confirm that I could send commands to my Robosapien. The next step was to explore the USBUIRT API and DLL to see how it could be modified. The API is a 'developer' tool, and isn't provided with the USBUIRT, but if you have purchased a USBUIRT and are trying to develop an application based on it, you can request a copy from Jon via the support email address on his website. Jon was also kind enough to include a sample test program written in Visual Basic - which was great since I have a lot of experience with VB and will probably use it as the platform for a lot of my projects.
The test program was fairly easy to understand - especially since all I really wanted to do at this point was to transmit an IR code to the Robosapien. Getting the IR code format set correctly took a little while, and I was hindered by a bug in my VB installation that caused it to crash unexpectedly. The IR code format concern was sorted out by looking at other people's code on line to see how they had solved the problem. The VB problem was solved by a patch down-loaded from the Microsoft support website. Then, about 10:00 pm on Sunday night, I was finally able to send commands from a VB program running on my laptop to Robosapien!
The next step is to develop a spec for a fully functional control program, and then code it in VB. It looks like it should be very straight forward to develop a remote control program that just emulates the Robosapien remote. However, I think that particular approach might get boring very quickly. Besides, I always have the physical remote in case I need to use it.
At this point I'm leaning towards a choreography type controller. Something that would allow me to build up movement sequences and allow for macro loops, delays, repeats, and the like. It also needs to consider the elapsed time to execute each command into consideration. For example, when Pixar or Disney put together an animated movie they usually do the vocals and music first, then fit the animation timing into the audio tracks. I suspect that the same approach would work well in this case.
Assuming that I take that route, I will need to define a syntax - a command set, and a good place to start is with the commands that have already been setup by other Robosapien experimenters using the Girder application. The Girder GML file is basically a flat text file that includes a lot of parameters that won't be needed in this application. Pulling out just the group names, actions, command names, and the corresponding USBUIRT IR codes should provide a strong base for the choreography application (see the IR control code table below. )
’Progress With A Robosapien Control Application’ continues
Way, way back – last fall to be more specific – when I bought my Robosapien, I thought it would be great if I could control it directly from my PC. I had dreams of making it dance, or chase the dogs, or do any number of similar things simply by harnessing all the computing power in my laptop. The Robosapien remote control was fine, but I really wanted to go beyond its limitations. Initially that would require being able to store long macro arrays, and later the ability to add sensors and to branch behaviors.
Simple – right? All you have to do is hook up an IR transmitter to the laptop and get it to send the command strings to Robosapien. Sounds simple enough . . .
So, here we are, months later, and after a lot of work, a lot of head scratching, and some invaluable assistance from others via the internet, and finally my Robosapien is moving around under the control of my laptop computer!
The first problem was figuring out whether it could actually be done or not. I studied a large number of websites that reference Robosapien, and it became clear that what I wanted to do was definitely possible, but it wasn’t going to be a cakewalk. The RS IR codes were known (see http://www.aibohack.com/robosap/ir_codes.htm) including some interesting undocumented codes, and Palm PDAs had been used to control RS. I finally ran across the Robots Rule website and the section on getting Robosapien to to dance. There’s even a hilarious video featuring two Robosapiens and a robot dog. While it wasn’t exactly what I wanted to do, it was close enough to get me started on the right track. After a few email exchanges with the Robots Rule webmaster, I decided to go with a similar approach using the USBUIRT interface to communicate with my Robosapien. Jon Rhees, the USBUIRT developer, was extremely helpful. He went out of his way to answer my questions, and shipped my interface within 24 hours of my order placement. It took a couple of weeks – Jon is in the US, and I live in Japan.
With the hardware part of the solution in my possesion, the next step was to get the initial software up and running. For that I settled on Girder and downloaded the free evaluation copy. Then I hit the proverbial wall for a while. Girder seems to be a great application, jam packed with fancy bells and whistles. It was designed for general purpose use, and I knew that it would have a lot of features that I wouldn’t use. Still, it came highly recommended, and had been used successfully with the Robosapien. So I decided I would give it a try. Unfortunately all the complexity got the better of me. I tried unsuccessfully to get it running with my robot for about a week or more. Then I swallowed my pride, took my keyboard in hand, and wrote to some of the folks that had a track record of making things like this work. They quickly pointed me in the right direction, and within a couple of days my laptop was able to consistently send the right commands to my RS. One of them was even kind enough to share a file with the RS IR code strings with me.
Girder in combination with the USBUIRT can ‘learn’ the IR strings from your existing remote control – not just for Robosapien but for any device that uses the same IR coding standards – like your television, VCR, stereo, etc.
As you might expect from looking at the remote control, Robosapien has a large number of functions/movements, and each of them requires a separate and unique IR code to be sent. Here’s what the high level command grouping looked like-
The next level down includes the individual commands to do things like “raise your right hand”.
When you get through with them all, including the burps, roars, demos, and other stuff, there are well over 60 different commands.
You can also create multicommands – a series of commands that are executed sequentially. For example, I wanted RS to walk forward for a while, then stop, raise his right arm and turn it, then raise his left arm, and finally whistle. The multicommand to do that looked like this-
It turns out that the Girder GML files are text files so it’s relatively easy to extract the command names and IR code strings that need to be sent. My next step is to use the USBUIRT API and DLL to build a Visual Basic control module for Robosapien. That will probably take a couple of evenings, or perhaps a long weekend. After that – well, we’re going to Rock ‘n Roll!