Google's DeepMind developed an IQ test for AI models

Can machines learn to reason abstractly? That's the subject of a new paper from Google subsidiary DeepMind titled "Measuring abstract reasoning in neural networks," which was presented at the International Conference on Machine Learning in Stockholm, Sweden this week. The researchers define abstract reasoning as the ability to detect patterns and solve problems on a conceptual level. In humans, they note, verbal, spatial, and mathematical reasoning can be measured empirically with tests that task subjects with teasing out the relationships between shape positions and line colors. "Unfortunately, even in the case of humans, such tests can be invalidated if subjects prepare too much, since test-specific heuristics can be learned that shortcut the need for generally applicable reasoning," the researchers explained. "This potential pitfall is even more acute in the case of neural networks, given their striking capacity for memorization."