From IPRE Wiki

Contents 1 Abstract 2 May 26th to May 29th 3 June 1st to June 5th 4 June 8th to June 12th 5 June 15th to June 19th 6 June 22nd to June 26th 7 June 29th to July 3rd 8 July 6th to July 10th 9 July 13th to July 17th 10 July 20th to July 24th 11 July 27th to July 31st

Abstract

Human-robot interaction (HRI) is the study of interactions between robots and people. The underlying goal of HRI is to develop principles and algorithms to allow more natural and effective interaction and communication between humans and robots. HRI is applied to, or will be applied to many fields including law enforcement, entertainment, scientific exploration, hospital care, search and rescue, and military battle. In many cases, a robot becomes more than a tool, instead serving as almost another member of the team. Thus, these robots must not only coordinate their behavior with the requirements and expectations of human team members, but must also be able to integrate their tasks with those of their human counterparts. Using HRI and computer vision, the science and technology of machines that see, I will begin to develop a facial recognition algorithm that can be used to create “eye contact” between a robot and a human being and a humanoid robot platform on which I will conduct future HRI experiments.

May 26th to May 29th

Tuesday May 26th

• Introduction to the Research program
• Talked with Doug about what we will be working on this summer.

Wednesday May 27th

• Installed Pyjama on Fedora machine in the lab with Doug and Meena. Found some problems.
• Windows vista machine died (after all those horrific updates. . .).
• Discussion of Bringing Up Robot, a paper about developmental robotics.
• Search for a "make" for Mac OS X (without having to use XCode) Have to test a few things on my personal computer, but here are the links I found so far:

GNU "make" description page[1]

GNU sources[2]

Perl "make" for Web Design[3] (not at all useful for my purposes, but talks about a built-in make . . . will look into this.)

• Read and completed the first four and a half chapters of Learning Computing with Robots so that I can learn Myro.

Thursday May 28th

• Halfway through chapter 7 in Learning Computing with Robots. Completed many of the exercises.
• Trying to familiarize myself with Myro, installed Myro on my personal computer machine
• Research about makefiles and XCode.
• Spent a long time trying to find the "make" for OS X 10.4, it seems Apple wants to do everything in it's power to prevent you from having useful software once your operating system is outdated. No XCode for me.

Friday May 29th

• Spent the whole day working on learning Myro. Went back to chapter 6 to do more of the Braitenberg exercises. I got a lot of strange error messages throughout my day (Meena witnessed that I am not crazy near the end of the afternoon). Had to switch robots because mine started acting up. Got almost all the way through Chapter 9. Finished my day learning about image processing.

June 1st to June 5th

Monday June 1st

• Abstract workshop
• Continued with imaging processing. Finished chapter 11 in Robots.
• Talked with Doug, Ashley, and Meena about the summer in general and our ideas for future research.

Tuesday June 2nd

• Re-read Bringing up Robot for our talk tomorrow
• Review of image processing in myro
• Introduction to Webots, the simulation we will be using this summer for Robonova

Wednesday June 3rd

• Talked about upper level stuff, only got through neural networks(and some of the related parts of Bringing up Robot).
• Did a little reading about neural networks, Turing Machine, etc.
• Worked with Meena on some simple neural network programs, AND, OR, etc.

Thursday June 4th

• Read about and installed Pyro on my computer. Read about neural networks. I realized that I was thinking about neural networks all wrong and that's why the code from yesterday wasn't entirely correct.
• Programmed a simple neural network that locates an object in a picture by its color.newTest.py
• Learned a little about Webots from Ashley.
• Read about Boosting Algorithms.

Friday June 5th

• Worked on code from yesterday a little more, the updated version is the one linked to above.
• Began to read the Robot Vision section of Pyro.
• Talked to Meena about her neural network code.

June 8th to June 12th

Monday June 8th

• Read more of the vision stuff for Pyro. Realized I need to go back and learn some of the Pyro basics before doing the exercises.
• Began reading An Emergent Framework paper.
• Learned more about Webots from Ashley.

Tuesday June 9th

• Began going through the beginning of the Pyro exercises online. Found a lot of errors.
• We need to install PLAYER, tutorial.cfg is gone (needed for the PyroModuleDirectControl section)
• Microsoft table interface research

Wednesday June 10th

• Continued with Pyro vision. Complete exercises, learned basic commands.
• Read more of An Emergent Framework paper.
• Scientific Ethical Code talk with Paul Grobstein of the Bio department.
• Talked with Meena, Doug, and Ashley about the Microsoft table and some things we may want to do with it.

Thursday June 11th

• Worked with Pyro vision again. Tried to get the cameras for Linux working but Doug and I couldn't figure out what was wrong with them.
• Edited the extended Abstract with Meena and Ashley.
• Read about computer vision, trying to figure out what tests I might like to run
• Talked with the guys from the shop about building our robot, tentatively dubbed "Huba." I'm hoping for a movable neck and two cameras so that I can perhaps do depth perception tests.

Motors and robot parts[4]

Information on Mini Hubo[5]

Friday June 12th

• Looked at more Pyro vision.
• Did a lot of reading on Computer Vision[6] and related topics.

June 15th to June 19th

Monday June 15th

• Went to Swathmore to meet the students working on developmental robotics.
• Learned about their research and talked about the Rovio (a potential replacement for the scribbler)
• Growing Neural Gas, IAC, Self Organizing Maps, RAVQ

Rovio

Pros: Video, no batteries, microphone, really high quality camera, long range, omni-directional wheels, head light

Cons: (if the interface is up and running some of these may not be problems any more) Has to move one small command at a time, heavier and bigger, no smooth turning, charge only lasts 3 hours, IR sensors are not as useful

Questions: Where would we attach a pen for drawing?

Scribbler

Pros: small and light weight, drawing, sounds, streaming movement (smooth turning), long range, more IR sensors, batteries last longer

Cons: BATTERIES

Tuesday June 16th

• Time to try and get a WebCam working under Linux so that I can start some Pyro experiments. Doug picked up a few different WebCam options. The first is the Labtec Plus [7]. We didn't have any other video/picture programs on Linux already so we tried Pyro right away. The LabTec Plus "sort of" worked with the default settings on Pyro, it showed input from the camera very tiny in black and white. But after changing the proportions on the images we quickly realized the camera wasn't really working anywhere near the way we needed it to be. Hence, I began searching for Linux webCam frame grabbing and GUI based programs. Initially I used this site to learn about WebCams on Linux [8]. The programs listed are motion [9], gnomemeeting aka ekiga [10], xawtv [11], CamE [12]. After installing xawtv, CamE, and ekiga. The only one that worked was ekiga and it only worked for the Logitech camera.
• After all that, Doug and I looked at the C code for the pyro V4LCamera hook up.
• I learned a lot about the terminal and different commands like yum, grep, etc.
• We also talked to the guys from the shop about "Mini Huba." It looks like we will be able to have hands (hopefully with seperate digits!!), a pan and tilt neck, modular head set up, and perhaps even the ability to add new sensors once Huba is mostly finished.

Wednesday June 17th

• Received an email from Doug about the camera problems we were having yesterday. It turns out the problem with Pyro is that it needs to be updated to V4L2. Other than that the operating system was just a little outdated for the cameras.
• In the morning we went to a really interesting and well done talk on science journalism, specifically science and the radio by Ari Daniel Shapiro. Very cool.
• Talked with Doug about our summer research. I decided that I will be working on getting a robot to learn to make "eye" contact with a human. I'm going to be looking at Open CV [13] for the next few days.
• Installed Fedora 11 with Doug and Mr. Ice on my lab machine.
• The LabTec Plus worked right away in the Cheese application on Fedora 11! yay!

Thursday June 18th

• Spent the morning reading about OpenCV and following links about Face Recognition. Very interesting stuff. Here is some of what I looked at: Facial Rec [14], Face REc homepage [15], Some OpenCV reference pages main[16] another[17] myfav[18]
• Here is a picture of the camera I might get to use for the mini huba on top of another robot: [19]
• Also did some C reading in my lovely textbook.

Friday June 19th

• Did more reading about OpenCV and Facial Recognition
• Spent most of the day trying to figure out what I should say in my Abstract
• Worked on the C book a little more
• Set up my Pan/Tilt/Zoom camera for my work this summer

June 22nd to June 26th

June 22nd

• Had to move the camera station to a new computer because the video wasn't working under Fedora 11
• Visiting with the Swarthmore students (and one professor from Sarah Lawrence) again only this time they came to Bryn Mawr. Talked about GNG, IAC, etc. again. Their paper is due on the 17th and so there was a lot of debate about why they chose certain methods. We showed them the WEBot simulation.
• Pan Tilt Zoom was only set up in Pyro for the Canon VCC4 so we had to change my work station yet again. Now everything works fine and it's back to OpenCV for me!
• Tested the OpenCV C code and that worked and we think the python code should work as well, there is just some xml stuff that seems a little strange. I'll have to try and figure out what it's needed for. Part of the problem might be that it was written for the fluke with the intention of having the robot say "Hello Human" when it finds a face. Obviously, my PTZ camera will not be speaking.
• The wiki pages that I'm using {1}[20] {2}[21]
• Trying to figure out what the Haar xml is. Here is a helpful website[22]

June 23rd

• Filled out questions on my research for NSF and Bryn Mawr Now. Sent them to Doug.
• Talked with one of the guys from the shop about the battery life of Huba. We're hoping for about an hour, maybe a little less. Looks like we're going to try to use super capacitors instead of a lot of batteries. This will decrease the weight, so hopefully it works out.
• Also did some more reading.

June 24th

• Out sick with Strep Throat. Fun.
• Tried to get some reading done in the room.

June 25th

• Feeling a little better so I'm back to work.
• It took me awhile to find the folder path I was looking for the Haar xml. Finally found it!!!! and the code works but since I'm not using myro or a robot (yet) I'll have to mess with the code in order for me to test it with my camera. Tomorrow I'll have to look more closely at the code and work on learning OpenCV.
• Reading a paper about Facial Recognition systems: "Detecting Faces in Images: A Survey" [23] It's long so it should take me a while to read the whole thing.

June 26th

• Read more of the paper from yesterday.
• Worked on learning OpenCV and messed around with the camera setup a little bit.
• Officially half-way done the summer!

June 29th to July 3rd

June 29th

• My grandparents gave me the July edition of Smithsonian magazine because there is an interesting article about robotics that is really relevant to the type of research we are doing this summer. The article is called Birth of a Robot
• Read the C book
• Picked out the myro parts of the faceRec system code

June 30th to July 2nd

• Did mostly background reading and some of the C book

July 3rd

• Off for the holiday

July 6th to July 10th

July 6th and July 7th

• Background reading, trying to familiarize myself with pyro again. (Since I've been doing so much reading in the last week I haven't really been using pyro and needed to remind myself how everything works)
• Reading through the C book.
• I've been looking over the Face Rec code that I have and I know I need to take out the takePicture() from myro. I'm not sure what to replace it with. The resources I've been looking at for Opencv haven't been very helpful so far. For what I'm trying to work on most assume you have a working knowledge of C, C++, or C#. Unfortunately, I do not and so finding a solution to my problem is proving difficult.
• I'm also not very adept at using pyro yet. It seems I have a lot of catching up to do.

July 8th

• Did another chapter in the C book and continued my attempt to wade through the endlessly confusing Opencv documentation.

July 9th

• Talked with one of the guys from the shop about Huba. He brought us up a drawing of which motors would be used and where they would be placed in the body of the robot (We're leaving the head for last since we still need to decide about the cameras, sensors etc). Then the three of us talked about different sensors we might like to have on the robot such as a gyro, accelerometer, IR sensors. . . Gyro seems to be a popular idea but as it gets closer to when the building starts we'll make those decisions.
• Found another Face Rec python implementation[24] but this one uses something called CVtypes which is a python wrapper. Looks like I have a lot more reading to do.
• Meena helped me locate my python wrappers ("my" as in the ones I already have downloaded). They are highgui and cv, both found within the the opencv folder in python.

July 13th to July 17th

July 13th

• Out of work for doctor.

July 14th

• Learned about Simple Res. Nets with Meena and Doug and then talked to Doug about what I've been up to for the past few weeks.
• Working with Doug on the facial recognition system and it works now!!!!! WOOOOO HOOOOO. Here's the code.facesample.py
• Then we talked about the conference that we are going to at the end of this week. Covered the basics, how it will go, what we will each talk about, when we will be talking, etc.
• Went down to the shop and talked over all the specs for Huba. Discussed what images we will need for the conference and the major differences between Huba and Virginia Tech's Mini Hubo.
• Worked with the Face Rec system in pyrobot and got it working as a brain. Now trying to get the ptz to work with my camera. I need to figure out the exact measurements.

July 15th

• Prepared for presentation at the UKC conference this weekend. Made my slides and researched a few topics.
• Went down to the shop to talk over the Mini Huba specs with the guys.

July 16th - July 18th

• Robotics conference in North Carolina.

July 20th to July 24th

July 20th and July 21st

• Sent Bruce Maxwell from Colby an email about the type of camera we should use for the Mini Huba. He suggested two options: Quickcam 9000 and the Genius Multifunction camera that has a microphone and speakers.
• Went to the shop. Talked about the parts we have to order (on-board PC: Pc104, Robot controller: CM-2, batteries or other power sources). We already have the CM-2 so now I'm learning about how to use it. It's very different from the CM-5, which is what Tevonia used for the TevBot.
• Ordered the PC104, now waiting on it to arrive.
• Trying to get the Face Rec system window working with Pyro. Having issues. I discovered that the error message I keep getting is actually a bug with the interaction between V4L and OpenCv. It turns out the video system will quit automatically at a random time. "When accessing the v4l video interface using the OpenCV cvQueryFrame() method, after an unpredictable amount of time, the bttv driver gives the following error message:

VIDIOC_DQBUF error 5, Input/output error" (from [25])

• During this time I was also downloading the software for the CM-2 so that I can hopefully access it through my current PC to learn more about it before we have the robot platform to use. I also had to find the Bioloid software because the CD has mysteriously disappeared. I downloaded the package from the Robotis site but it seems to be missing some of the parts I need. I might have to explore this further. Here's the site for the software download: [26]
• I also learned about lock objects in Python [27]
• Here's the Robotis site[28] and about the CM-2[29]

July 22nd

• Began to work with CM-2 PLUS. It took a little while to find the correct manual. It seemed that the only one out was for the CM-2 and we needed the one for the CM-2 PLUS. Finally found it by the end of the day.
• Also worked on the Facial Recognition system. Decided to try to get it working under Windows, the opencv downloads were not working. Decided to abandon that project for the time being in favor of preparing for Friday's events.
• Transferred as many of the relevant files as I could from Fedora so that I could get the robotis stuff working under windows, which is where it needs to be anyway.

July 23rd

• Downloaded the software and got the Robotis terminal working. Started working through the manual so as to learn about the controller. I got up to the examples about motor control. It seems to be going well.
• The PC104 has arrived so we may begin to experiment with that soon.
• Went to the shop and talked about how things are going with the Huba. I have the neck motors to work with now.

July 24th

• Career Panel
• Prospective students Open House day. Gave tours of the lab.

July 27th to July 31st

July 27th

• Over the weekend Doug fixed some of the problems with the Face Recognition system so I went through what he had fixed (the corrections made implementing the system different).
• Discovered that only one motor is currently working with the CM-2 Plus. We aren't sure why and Doug and I spent a few hours trying to figure out how to get the rest (or even just another) servo to work. By the end of the day it still wasn't going that great so we ended up taking the CM-5 and a few extra AX servos to use for the python, basic code I now need to write.

July 28th

• Began to work on my poster for the summer research program.
• Continued to read the CM-2 Plus manual to see if there would be anything else to help me with the code I need to write.
• Talked over my task for this week with Doug, he clarified exactly what it is I need to write.

July 29th

• Went down to the shop and saw how some of the pieces are made.
• Worked on my poster
• Worked on the servo code.

July 30th

• Did a little bit of my poster in the morning.
• Spent the rest of the day working with Doug on the Pan/tilt servo code. (input to the servos must be sent as a packet!)
• Code from our current pan/tilt servo project: bioloidMessage.py
• Code from our current Bioloid Controller: Media:cmController.bpg
• Went down to the shop to see the nearly finished neck/shoulder section of the robot.

July 31st

• Last day of summer research. sad face.
• Spent nearly all of the day working on my poster! Finished it and turned it in! Yay!