Why Scientists are Training AI to Take Standardized Tests

by Robert Montenegro

We're generally pretty good at creating computer software that can accomplish simple tasks on a large scale. Take, for example, the search option on your computer. Just input a search term such as "blueberry muffins" and every document on your hard drive featuring those words pops up in an instant, most vitally that important recipe you --  for whatever reason -- stored in your SNES emulator folder (what were you thinking?). A task that would have taken hours for a human -- filtering through every folder for the recipe -- is accomplished in seconds. As software creators, we've got that down pat.

The future of AI and machine learning is going to be a lot more impressive than your run-of-the-mill muffin finder. The only reason we're not there yet is because teaching software how to reason is a lot more difficult than assigning it mindless busy work. As Tom Simonite of MIT Technology Review writes, computer scientists are trying some interesting new strategies for instilling in our computers the main cognitive ability we possess that they don't: common sense.

Microsoft Director of Search Stefan Weitz explains that the future of machine learning consists of teaching artificial intelligence to identify patterns.

Simonite profiles a team of researchers at the Allen Institute for Artificial Intelligence in Seattle. They have created a program called Aristo with the goal of teaching it common sense, though in a somewhat curious way. The team is currently training Aristo to take the New York state fourth grade standard science exams. Oren Etzioni, the Allen Institute's CEO, argues that standardized tests offer a strong benchmark for tracking the progress of machine learning.

To understand what he means, let's revert quickly to standardized tests. They get a bad rap around here and deservedly so, as they're not a great way to guide our school children toward creative thinking or a lifelong love of learning. Luckily for computer scientists, AI isn't like your typical fourth grader. 

Think about the set of skills one needs to do well on a standardized test. Attacking a multiple choice question is as much an exercise in vetting the choices as it is knowing the answer. The whole point of studying questions similar to those you'll find on a test is to prepare your mind to transpose information from one example to the next. There's a reason some kids are better at taking tests than others and it's not all brains. It's a matter of finding the most efficient way to interpret questions and deliver the best possible answers. In many ways it's a matter of common sense, which is why Etzioni is so keen on making sure Aristo passes the test.

Why is common sense the current golden fleece for computer scientists? Mostly because we've seen incredible progress made with systems that get by without it. The Siri program on your phone might be able to interpret your voice and deliver action but it's not applying rational thought to help you out. It's incapable of figuring things out for itself or of interpreting requests in ways it wasn't initially programmed to do so. Now thing of computer systems more advanced and important than Siri. How useful would effective and replicable machine learning be if scientists can make major progress in teaching AI to teach itself?

"Finding a way to put even a dash of common sense into software is a major challenge in AI," writes Simonite, "and one that could lead to computers helping us out in many new ways."

Read more at MIT Technology Review

Wanna meet Aristo and see it in action. Check it out at the Allen Institute website.

Image credit: Vergeles_Andrey / Getty iStock