UIDAI's sham studies
The UIDAI published this sham study on the efficacy of their registration and de dup.
http://uidai.gov.in/images/FrontPageUpdates/role_of_biometric_technology_in_aadhaar_jan21_2012.pdf
Besides a lot of mutual back slapping and verbose rubbish, there are only a few paras that we need to look into.
These figures are now being used to justify and show that the system is fine.
http://uidai.gov.in/images/FrontPageUpdates/role_of_biometric_technology_in_aadhaar_jan21_2012.pdf
Besides a lot of mutual back slapping and verbose rubbish, there are only a few paras that we need to look into.
- The POC performed in June 2010. http://uidai.gov.in/images/FrontPageUpdates/uid_enrolment_poc_report.pdf
- The UIDAI also carried out biometric enrolment of school children in the vicinity of Bangalore. About seventy five thousand people in all were enrolled during the first phase of the PoC study including people over 90 years of age,, and sixty thousand of the same people were reenrolled during the second phase after a gap of three weeks, in order to test the biometric matching efficiency using known duplicates.
- False Negative Identification Rate (FNIR): 0.035%. This implies that 99.965% of all duplicates submitted to the biometric de-duplication system are correctly caught by the system as duplicates. Given that currently approximately 0.5% of enrolments are duplicate submissions, only a few thousand duplicate Aadhaars would possibly be issued when the entire country of 120 crores is enrolled.
These figures are now being used to justify and show that the system is fine.
HOWEVER besides the mistake of "rounding" The POC does not test for biometric variability of natural systems.
The problem starts when you do a fresh capture after time intervals of 3 months and longer. Given that the UDAI quotes some fancy names, it might have surely heard about Template ageing and this study
Kim_Analysis_of_Effect_of_Fingerprint_Sample_Quality_in_Template_Ageing.pdf
Page 23 of the UIDAI report simply extrpolates the false metrics of comparing a small size within 3 weeks, rather clever, because failure to auth happens in about 4 weeks, aka error rates will start to render the system unstable AFTER this period and will climb steadily to 50% in 2 years.
Given the above study, they would have to hold their back slapping for a period of atleast two years and do those tests every 3 months with an ever increasing population sample size. At the least the sampling size would have to be an order of magnitude larger and include all those corner cases eliminated in their June 2010 POC. I must remind everyone of the UIDAI's social upliftment lies, wherein the corner cases would constitute 70% of the populace.
The UIDAI with some fancy verbiage then makes a claim that the error rates will be linear, and consequently nothing to worry about
Hers is some more dope for them to mull over.
http://www.cse.nd.edu/courses/cse40151/www/Iris_Fall_2011_11.pdf
In this non study, done on an astronomically small sample size, the error rates on Iris template ageing are horrendous to say the least.
Notice how heavily qualified statement of the patentees and vendors of these tehcnologies, morphs into gibberish.
"The assessment in Flom and Safir,
that re-enrollment might possibly
be needed, morphed into the
marketing myth of “a single
enrollment can last a lifetime” –
even in the total absence of any
study on iris template aging."
One should note that these figures assume an extraordinarily high initial image quality. Not the UIDAI's practice of overriding all quality checks on 4th try. The 4th try case occurs in 15% of the cases in rural areas - the poor. ALL of these 15% will Fail to enroll. But given the UIDAIs silence on reject rates due to image quality we cant make a sane statement on this issue. Indeed the UIDAI claims that these persons are assigned a UIN by manual process. For such cases, not only will auth most likely fail at the first try, it opens up a hole of 15%. This is far higher than all duplicates in all government subsidy schemes!.
The problem starts when you do a fresh capture after time intervals of 3 months and longer. Given that the UDAI quotes some fancy names, it might have surely heard about Template ageing and this study
Kim_Analysis_of_Effect_of_Fingerprint_Sample_Quality_in_Template_Ageing.pdf
Page 23 of the UIDAI report simply extrpolates the false metrics of comparing a small size within 3 weeks, rather clever, because failure to auth happens in about 4 weeks, aka error rates will start to render the system unstable AFTER this period and will climb steadily to 50% in 2 years.
Given the above study, they would have to hold their back slapping for a period of atleast two years and do those tests every 3 months with an ever increasing population sample size. At the least the sampling size would have to be an order of magnitude larger and include all those corner cases eliminated in their June 2010 POC. I must remind everyone of the UIDAI's social upliftment lies, wherein the corner cases would constitute 70% of the populace.
The UIDAI with some fancy verbiage then makes a claim that the error rates will be linear, and consequently nothing to worry about
Not a very meaningful claim without factoring in time scales, and some
more minor details which the UIDAI conveniently obfuscates.
Section 5.1 tells you how to beat the system.
Devious as the UIDAI are, they have to hedge themselves against assaults from real world studies like the above. The UIDAI claims that Multi Modal biometrics (Iris scans) is the reason for their self proclaimed magnificence.
Section 5.1 tells you how to beat the system.
Devious as the UIDAI are, they have to hedge themselves against assaults from real world studies like the above. The UIDAI claims that Multi Modal biometrics (Iris scans) is the reason for their self proclaimed magnificence.
Hers is some more dope for them to mull over.
http://www.cse.nd.edu/courses/cse40151/www/Iris_Fall_2011_11.pdf
In this non study, done on an astronomically small sample size, the error rates on Iris template ageing are horrendous to say the least.
Notice how heavily qualified statement of the patentees and vendors of these tehcnologies, morphs into gibberish.
"The assessment in Flom and Safir,
that re-enrollment might possibly
be needed, morphed into the
marketing myth of “a single
enrollment can last a lifetime” –
even in the total absence of any
study on iris template aging."
One should note that these figures assume an extraordinarily high initial image quality. Not the UIDAI's practice of overriding all quality checks on 4th try. The 4th try case occurs in 15% of the cases in rural areas - the poor. ALL of these 15% will Fail to enroll. But given the UIDAIs silence on reject rates due to image quality we cant make a sane statement on this issue. Indeed the UIDAI claims that these persons are assigned a UIN by manual process. For such cases, not only will auth most likely fail at the first try, it opens up a hole of 15%. This is far higher than all duplicates in all government subsidy schemes!.
Iris based de dup uses gabor transforms to produce match frequencies. I.E.
nothing verifiable was done on establishing biological non variance by
comparison of actual bitmaps of iris images wrt template ageing.
Fingeprint vendors have quickly started to use gabor transforms on fingerprints, especially for de duplication and can thus claim the same benefits attributed to iris (in the abscence of deeper scientific study).
The converse of this argument is that iris is as big a problem as fingerprints!!
The use of Gabor transforms on finger prints for de dup might also be one of the reasons for the UIDAI's claims on error rates. This makes it all the more important to do an actual physical verification on a large scale, as Gabor transforms involve pure probability (as opposed to template matching and it's inherently higher FRR), which will be totally dependent on iris image and iris ring number that is captured on the machine.
Lies, damned lies and the UIDAI.
Fingeprint vendors have quickly started to use gabor transforms on fingerprints, especially for de duplication and can thus claim the same benefits attributed to iris (in the abscence of deeper scientific study).
The converse of this argument is that iris is as big a problem as fingerprints!!
The use of Gabor transforms on finger prints for de dup might also be one of the reasons for the UIDAI's claims on error rates. This makes it all the more important to do an actual physical verification on a large scale, as Gabor transforms involve pure probability (as opposed to template matching and it's inherently higher FRR), which will be totally dependent on iris image and iris ring number that is captured on the machine.
Lies, damned lies and the UIDAI.
