The takeaway: Unless it's white-on-black hollow icons, they were no less likely to be recognized than solid icons.
Now that iOS 8 is doubling down on the hollow icon formula, is there any truth to the assertion that hollow icons are more difficult for users to understand than solid icons?
To find out, user experience designer Curt Arledge created a web app that measured users' speed and accuracy in selecting icons with different visual styles. Each user was asked to match an icon with its name, from a grid of 20 icons that used both hollow and solid icons on white and black backgrounds.
Arledge discovered that across more than 1,000 tests, hollow icons were no less likely to be recognized than solid icons. There was one exception: Hollow icons that used white lines on black backgrounds were recognized about a quarter second slower than any other type of icons. Most users' brains, it seems, need a moment to recognize hollow icons when they are presented in a negative color space.
That exception aside, on average, Arledge found no real difference between users' recognition of icons based simply on their being hollow or solid-in. He did find that for specific individual icons, the hollow variant was sometimes recognized faster than the solid-in one, or vice versa. So some icon designs work better one way, and others another. Designers must test both.
Unless you're creating white-on-black hollow icons, the major factor contributing to an icon not being recognized is no different post-iOS 7 than it was before. Whether hollow or solid, the icons people have trouble recognizing are the ones that don't take the user--and test data--properly into account. If people don't understand your hollow icon, don't blame iOS 8. Just go back to the drawing board.
You can read more of Arledge's findings here.