(Credit: Carnegie Mellon University)
A computer program called the Never Ending Image Learner (NEIL) is now running 24 hours a day at
Carnegie Mellon University,
searching the Web for images, doing its best to understand them. And as
it builds a growing visual database, it is gathering common sense on a
massive scale.
NEIL leverages recent advances in computer vision that enable
computer programs to identify and label objects in images, to
characterize scenes and to recognize attributes, such as colors,
lighting and materials, all with a minimum of human supervision. In
turn, the data it generates will further enhance the ability of
computers to understand the visual world.
But NEIL also makes associations between these things to obtain
common sense information: cars often are found on roads, buildings tend
to be vertical, and ducks look sort of like geese.
“Images are the best way to learn visual properties,” said Abhinav
Gupta, assistant research professor in Carnegie Mellon’s Robotics
Institute. “Images also include a lot of common sense information about
the world. People learn this by themselves and, with NEIL, we hope that
computers will do so as well.”
Since late July, the NEIL program has analyzed three million images,
identifying 1,500 types of objects in half a million images and 1,200
types of scenes in hundreds of thousands of images. It has connected the
dots to learn 2,500 associations from thousands of instances.
Drum (percussion) objects identified by NEIL (credit: CMU)
You can view NEIL’s findings at the project website (or help train it):
http://www.neil-kb.com.
World’s largest structured visual knowledge base
One motivation for the NEIL project is to create the world’s largest
visual structured knowledge base, where objects, scenes, actions,
attributes and contextual relationships are labeled and catalogued.
“What we have learned in the last 5-10 years of computer vision
research is that the more data you have, the better computer vision
becomes,” Gupta said.
Some projects, such as ImageNet and Visipedia, have tried to compile
this structured data with human assistance. But the scale of the
Internet is so vast — Facebook alone holds more than 200 billion images —
that the only hope to analyze it all is to teach computers to do it
largely by themselves.
Shrivastava said NEIL can sometimes make erroneous assumptions that
compound mistakes, so people need to be part of the process. A Google
Image search, for instance, might convince NEIL that “pink” is just the
name of a singer, rather than a color.
“People don’t always know how or what to teach computers,” he
observed. “But humans are good at telling computers when they are
wrong.”
People also tell NEIL what categories of objects, scenes, etc., to
search and analyze. But sometimes, what NEIL finds can surprise even the
researchers. Gupta and his team had no idea that a search for F-18
would identify not only images of a fighter jet, but also of F18-class
catamarans.
As its search proceeds, NEIL develops subcategories of objects — cars
come in a variety of brands and models. And it begins to notice
associations — that zebras tend to be found in savannahs, for instance,
and that stock trading floors are typically crowded.
NEIL is computationally intensive, the research team noted. The
program runs on two clusters of computers that include 200 processing
cores.
Practical uses
NEIL’s knowledge can be used wherever machine perception is required
(e.g., image retrieval, robotics applications, object and scene
recognition, describing images, visual properties of objects and even
visual surveillance,” Gupta explained to
KurzweilAI.
“NEIL has analyzed more than 5 million images and built a database of
0.5 million images and 3000 relationships in 4 months. The NEIL visual
knowledge base also includes visual models of concepts (e.g., car,
crowded, trading floor) and relationships between concepts (e.g., cars
have wheels, trading floors are crowded). These models and relationships
will be made available for academic research use. We also invite
academic users to submit concepts that they would like NEIL to learn
and later use these models for their own research.
“Once the technology is mature (hopefully in the near future), we
expect NEIL’s knowledge base to have multiple commercial applications.”
The research is supported by the Office of Naval Research and Google Inc.
Abstract of International Conference on Computer vision (ICCV) paper
NEIL (Never Ending Image Learner) is a computer program that runs 24
hours per day and 7 days per week to automatically extract visual
knowledge from Internet data. NEIL uses a semi-supervised learning
algorithm that jointly discovers common sense relationships (e.g.,
“Corolla is a kind of/looks similar to Car”,“Wheel is a part of Car”)
and labels instances of the given visual categories. It is an attempt to
develop the world’s largest visual structured knowledge base with
minimum human labeling effort. As of 10th October 2013, NEIL has been
continuously running for 2.5 months on 200 core cluster (more than 350K
CPU hours) and has an ontology of 1152 object categories, 1034 scene
categories and 87 attributes. During this period, NEIL has discovered
more than 1700 relationships and has labeled more than 400K visual
instances.
(¯`*• Global Source and/or more resources at http://goo.gl/zvSV7 │ www.Future-Observatory.blogspot.com and on LinkeIn Group's "Becoming Aware of the Futures" at http://goo.gl/8qKBbK │ @SciCzar │ Point of Contact: www.linkedin.com/in/AndresAgostini
Washington
