Ride the Lightning

NIST Study: Face Masks Defeat Most Facial Recognition Algorithms

August 5, 2020

VentureBeat posted on July 29 that a National Institutes of Science and Technology (NIST) report evaluated studies where researchers measured the performance of facial recognition algorithms on faces partially covered by protective masks. They report that the 89 commercial facial recognition algorithms from Panasonic, Canon, Tencent, and others they tested had error rates between 5% and 50% in matching digitally applied masks with photos of the same person without a mask.

"With the arrival of the pandemic, we need to understand how face recognition technology deals with masked faces," Mei Ngan, a NIST computer scientist and a coauthor of the report, said in a statement. "We have begun by focusing on how an algorithm developed before the pandemic might be affected by subjects wearing face masks. Later this summer, we plan to test the accuracy of algorithms that were intentionally developed with masked faces in mind."

The study explored how well each of the algorithms was able to perform "one-to-one" matching, where a photo is compared with a different photo of the same person. (NIST notes this sort of technique is often used in smartphone unlocking and passport identity verification systems.) The team applied the algorithms to a set of about 6 million photos used in previous Face Recognition Vendor Test (FRVT) studies, but they didn't test "one-to-many" matching, which is used to determine whether a person in a photo matches any in a database of known images.

Since real-world masks differ, the researchers used nine mask variants to test, which included differences in shape, color, and nose coverage. The digital masks were black or a light blue approximately the same color as a blue surgical mask, while the shapes ranged from round masks covering the nose and mouth to a type as wide as the wearer's face. The wider masks had high, medium, and low variants that covered the nose to varying degrees.

Algorithm accuracy with masked faces declined "substantially" across the board. Using unmasked images, the most accurate algorithms failed to authenticate a person about 0.3% of the time, and masked images raised even these top algorithms' failure rate to about 5%, while many "otherwise competent" algorithms failed between 20% and 50% of the time.

NIST didn't take into account systems designed specifically to identify mask wearers, like those from Chinese company Hanwang and researchers affiliated with Wuhan University.

This summer, NIST plans to test algorithms created with face masks in mind and conduct tests with one-to-many searches and other variations.

This is consistent with a number of articles I've read recently. There is widespread recognition that face masks can play havoc with facial recognition, but the algorithms are getting better all the time. So don't assume you can't be identified because you're wearing a mask. It will get harder and harder to fool the emerging algorithms!

Sharon D. Nelson, Esq., President, Sensei Enterprises, Inc.
3975 University Drive, Suite 225|Fairfax, VA 22030
Email: Phone: 703-359-0700
Digital Forensics/Cybersecurity/Information Technology
https://senseient.com
https://twitter.com/sharonnelsonesq
https://www.linkedin.com/in/sharondnelson
https://amazon.com/author/sharonnelson