Applied Object Recognition

John Keogh | May 15, 2013

An earlier blog post discussed a simple approach to object recognition and included a complete project in C#. I ported the code to Objective-C and made a video of the use of the code for object recognition for a small robot that I am building. This short post mostly adds a few points to the video. The earlier post referred to previously is where most of the code is. The robot uses an iPod running EyesBot Driver, which will be submitted to the Apple App Store soon, hopefully next week.

There were two primary problems to solve: how to teach the robot what an object is called, and how to later recognize the object.


The sequence of events that occurs when the robot learns an object is: This turned out to be quite simple to implement, and permits easy communication between the robot and user (see the video to see the learning in action). The code for this choreography is:

-(void)learnNewObject:(NSString *)objectName{ [self performSelector:@selector(turnOffLights) withObject:nil afterDelay:0.1]; [self performSelector:@selector(saveBackgroundPhoto) withObject:nil afterDelay:2.0]; //flash light once to let user know to place object //to learn in field of view [self performSelector:@selector(turnOnLights) withObject:nil afterDelay:2.5]; [self performSelector:@selector(turnOffLights) withObject:nil afterDelay:3.5]; [self performSelector:@selector(saveObjectPhoto) withObject:objectName afterDelay:5.0]; //finally, learn the object [self performSelector:@selector(analyzeLearningPhotos:) withObject:objectName afterDelay:6.0]; }


The recognition is simple determining whether the palette present in the object corresponds to any of the learned objects. Please see the project in the other blog entry for a complete project that implements this approach, but the code to test the extracted color palette against the learned color palettes is:

NSString *matchedObject = @""; NSInteger bestMatches = 0; NSInteger currentMatches = 0; for(LearnedObject *object in learnedObjects){ currentMatches=0; NSString *objectName =; for(NSString *color in object.pallette.keyEnumerator){ if([imageColors valueForKey:color]!=nil){ currentMatches++; if(currentMatches>bestMatches){ matchedObject=objectName; bestMatches=currentMatches; } } } }


The main problem is that this only looks for a palette, whereas objects also have a visual texture and morphology, and the colors for an object should be present in an image in a way that is consistent with the expected visual texture and object morphology. This simple approach is useful for a quick screen to narrow down the possible objects for further screening, but also have the problem of light level changes affecting the apparent color palette.


I'm looking forward to releasing EyesBot Driver in the next few weeks, but it will likely not contain this functionality until the functionality is much more mature. It is quite useful for navigation (since rooms tend to have distinctive color palettes) and also as presumptive object recognition. Lots of work left to do, though.

Eyesbot Company

Computer vision

Artificial intelligence

Effecting the physical world