IREX 10 Evaluation
Neurotechnology's iris recognition algorithm has been judged by NIST as the second most accurate among the IREX 10 participants. The submitted algorithm featured much faster template creation and search time, and much smaller template size than the only more accurate contender.
The IREX 10 evaluation is an ongoing evaluation of automated iris recognition algorithms, which is run by NIST. Different vendors submit their biometric algorithms for the evaluation. The tests are run over operational data for identification (one-to-many) tasks. See IREX 10 official page for more information.
The main goals of the IREX 10 evaluation are: assess the current state of the art, facilitate research and development, assess the impact of demographics, twins and automated quality assessment.
The evaluation has started in October 2019 and continues till now. The participants can submit their algorithms every 3 months, but the results are shown only for the latest submisssion.
Our latests submission was done in October 2020 with our current state-of-the-art algorithm. The submitted algorithm corresponds to the iris recognition algorithm, which is available for our customers in the VeriEye SDK and MegaMatcher SDK.
There were 5 companies participating in the evaluation as of January 2021. The provided accuracy metrics are: FNIR at FPIR 0.01 and Ranks 1, 10, 100.
According to the FNIR at FPIR 0.01 accuracy metric, our algorithm is second most accurate algorithm in the evaluation. Comparing to the only more accurate contender, our submission features:
- 43% faster search (see Figure 1);
- 7 times faster template creation (see Figure 2);
- 12 times smaller template size (see Figure 3).
Figure 1.
Figure 2.
Figure 3.
According to the accuracy metric Rank 1, 10, 100, our algorithm is the most accurate in the evaluation. Comparing to the other contenders, our algorithm is:
- second fastest by search time, but our algorithm showed 27% better accuracy in FNIR at FPIR 0.01 metric (see Figure 1) and 50% better accuracy by Rank 1 miss rate (see Figure 4);
- second fastest by template extraction time, but the fastest contender showed the worst accuracy (see Figure 5);
- third smallest template size, but the contenders with smaller templates sizes showed the worst accuracy (see Figure 6).
Figure 4.
Figure 5.
Figure 6.
The IREX 10 also included testing of automated quality assesment. Neurotechnology provided not just top level accuracy, but also quality metrics, which can predict on how well an iris impression will be matched. This enables users to clear their database from bad images to optimize system performance.
Figure 7 below demonstrates that FNIR (i.e. the 'miss rate') can be reduced by almost 20% by discarding just 1% of the poorest quality searches. Presumably, this 1% involved samples where the subject was blinking, moving, looking off-axis at the moment of capture, etc. The IREX III failure analysis found that matching failures for the most accurate matchers over a different dataset were almost entirely due to poor presentation of the iris.
The stacked barplot (Figure 8) shows how sample quality impacts the probability that a search will miss (i.e. fail to return the correct mate). Samples assigned low quality values should be more likely to miss. For Neurotechnology's matcher, when the assigned value is 0 the probability of a miss is greater than 50%. FPIR is set to 0.01.
Click to zoom
Click to zoom
See the IREX 10 official page and full report for more information on the IREX program and testing methodology.
