khoji parkAI · Canada · 2012
The Image Contest With a Wide Gap
Seeing pixels was easier than recognising what they showed.
Read in slides1. A recognition task
Researchers tested systems on assigning labels to images across many categories. A picture that looked obvious to a person could still challenge a computer. The challenge involved assigning labels to a large collection of images. A common benchmark made different approaches directly comparable.
Seeing pixels was easier than recognising what they showed.
2. Give every system the same examination
Image classification becomes easier to compare when competing methods face a common collection of examples and evaluation rules. The benchmark made a result measurable outside the developers' own demonstrations. It allowed researchers to see a substantial improvement on a defined task rather than rely on a few selected pictures that happened to look impressive.
The test set was the part the demonstration could not choose.
3. The research team
Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton developed a deep convolutional network. They trained it using a large labelled image collection. Training required examples as well as a suitable model. Computing hardware helped make the demanding learning process practical.
The labels supplied the lessons, not a hand-written rule for every object.
4. The hardware
The work used an efficient graphics-processor implementation to make training practical. Computation and network design were both important to the result. Performance was judged against the competition's test conditions. An improvement there provided evidence more concrete than a striking demonstration.
Gaming hardware had acquired a serious research assignment.
5. The improvement
Their 2012 paper reported substantially better classification performance than earlier approaches. The result became a landmark for deep-learning-based computer vision. The result drew attention to a particular technical approach. Other teams could build on a measured achievement instead of a vague promise.
An image contest had made a broader field pay attention.
6. The result other teams could build on
An improvement in a shared competition gave the wider field something specific to examine. Architecture, training and computing choices could be studied as parts of a working approach. The effect extended beyond one team because the result helped change what other researchers considered worth attempting on related visual problems.
The winning entry had become an invitation to try again.
7. The actual claim
The system improved classification on a defined dataset. It still made errors, and its success was not proof that machine vision had become universally reliable. Progress on image classification did not solve all visual understanding. The breakthrough mattered precisely because it showed a strong result on a defined task.
A breakthrough can be impressive without being the finish line.