What Is Facial Recognition Based On?

The ability to identify individuals and verify their identity using facial recognition relies on biometric technology. This technology encompasses processes in which the human body is measured in order to obtain knowledge about it. Alongside the fingers and veins, which supply prints and scans[1], the face is the part of the body most commonly used for this. The biometrics of the face – the position of its various features and the distances between them – can be measured when a camera takes a picture of it. This means that we cannot always control when data is collected. It can be recorded without our knowledge or permission.

Why is facial recognition used, and why do some people think it is dangerous?

How Does Facial Recognition Work?

Facial recognition is based on either 2D or 3D scanning. For 2D procedures ordinary photographs are used. There are two methods:

Geometrical features of the face (the position of the nose, eyes, mouth, etc.) are drawn from the picture and analysed. For this purpose, a machine-made artificial network modelled on an individual’s physiognomy is overlaid on the photo. Stored as numbers, this data is converted into a set of vectors. Other pictures from the database can then be compared with the set and aligned with other similar data.[2]
Template matching is also used for facial recognition. This procedure attempts to calculate the similarities between an image of a face and a template modelled on a face. To do this, the system draws on images from a database and compares these with the templates. A vector ascertains the similarities by, for example, counting the ‘empty’ squares between the nose, mouth and eyes.[3]

These methods have been in a process of ongoing development since the early 1990s. Today, 2D facial recognition is also based on complex calculations such as wavelet analysis[4] or the Viola-Jones method[5]. The 2D methods are thus based on mathematical calculations using algorithms.

The disadvantage of 2D methods is that they cannot account for unusual angles, shadows or partial masking in their calculations. At this point 3D methods of facial recognition are a viable alternative. They do not decode the image into distances and values of brightness or colour but rather use information on depth recorded by laser scanners. This generates a netlike structure that recreates the face in virtual form. There are three common methods for using this grid to identify faces:[6]

Curves: the curves of the face are (re)traced, and these provide information about its physiognomy.
Form description: the size and shape of small portions of the grid that are essentially constant – areas on the bridge of the nose, the chin and the forehead, for example – are used for comparison.
Relationships: as in 2D procedures, here the distances between single points in a face are measured. As this takes place in 3D, the measurements differ from those used for 2D photos and include factors like the distance from the tip to the wing of the nose.

These 3D methods are based on artificial intelligence and algorithms. The software independently detects features like the locations of shadow and symmetries, and it can use these to recognise a face. This increases the reliability of facial recognition. However, the complexity generates a considerable volume of data, which makes it more difficult to compare large numbers of images from big databases.

The functionality of facial recognition is continually improving because more people are generating images of themselves and others and making these public – on social media, for example. These images can then be fed into databases. The availability of data capacity needed for processing this information is also constantly on the rise. These two factors mean that more precise models of recognition can be developed.

One reason for the major improvements in the reliability of recognition is deep learning technology.[7] This is a method that enables machines to process information, and artificial intelligence to learn from it. To achieve this, very large amounts of data are analysed and fed into a neuronal network that creates connections between the data. These connections are constantly developing, just like the human brain, and new connections are generated from them. The machine can then not only make predictions and decisions but also question and change them as necessary. As a rule, human intelligence does not interfere with the learning processes in deep learning. Deep learning is used in facial recognition because this method analyses large amounts of data based on parameters and models.

Sometimes different methods are combined. Combinations with other sensors, such as motion detectors or audio recorders, can then be used as aids to facilitate or initiate facial recognition. Motion sensors can, for example, activate the video surveillance needed for facial recognition.

Who Collects Our Data?

Perhaps the best-known and most criticised example of image-based data being collected and stored is the US firm Clearview.[8] Their database is said to contain more than three billion photographs of human faces. To acquire this data, a program searches publicly accessible sites on the internet, such as Facebook, YouTube and Instagram.[9] The pictures found there are automatically downloaded and the faces are scanned for their biometric characteristics. When data correlates, the program supplies more photos and personal data.

Clearview has hitherto been seen as the largest database, but there are other comparable systems. In addition to police data on demonstrators in places like Russia[10] and Hong Kong,[11] there are companies that make use of facial recognition. Apple collects facial data so that people can enable Face ID, which acts like a key that you can use to unlock your smartphone. Google is also presently suspected of deliberately collecting image data and of using unconventional methods:[12] In New York, Google is said to have employed people to offer passers-by vouchers if they provided their facial data – they were then photographed from different angles.

Who Is Interested in Our Data?

There are many different people and organisations interested in our data – companies, governments and public authorities as well as criminal networks and individual actors. By buying in data sets from address brokers, companies acquire information about our purchasing behaviour and can control this by consciously deploying targeted advertising. Facial recognition is not needed for this, nor is it necessary to buy the data – just paying for something by direct debit may be enough for your buying habits to be tracked.[13]

Public authorities also pay for facial data. Six hundred US authorities have allegedly accepted offers from Clearview to assist them in solving sex crimes.[14]

Criminal organisations and individuals can exploit facial recognition to discover and misuse the identities of people. Examples here are stalking, buying illegal goods (drugs, child pornography, etc.) and using false identities.[15]

What Are the Benefits of Facial Recognition?

The police and government security and law enforcement agencies can use facial recognition in the fight against crimes like human trafficking and sex offences as a means to analyse video material and ascertain people’s identities. This technology can also help to uncover identity thieves.[16]

Companies like Migrolino[17] and Valora[18] in Switzerland are experimenting with facial recognition to simplify payment when shopping or to check the age of customers when they wish to buy age-restricted products.

Internet sites like Pornhub use facial recognition software to identify and ‘tag’ performers in videos so that the videos can be indexed with their names as keywords.[19]

Facial recognition is also used to connect people – e.g. in photo albums on Facebook – and so to create new contacts

… and the Drawbacks?

The alleged benefits of facial recognition and the different ways in which it can simplify day-to-day processes also have a serious downside. Facial recognition is a risk to privacy and it affects our basic rights.

In some parts of the world, for example, facial recognition is used by police to identify demonstrators. In 2019 this was done during protests in Hong Kong.[20] The danger with this is that protestors then have to worry that their identities will be registered with the authorities and stored, and that they will face criminal prosecution as a result. This could then lead to a change in behaviour, with people no longer taking part in demonstrations. In the case of the protests in Hong Kong, demonstrators fought back by using laser pointers that disrupted the recognition system.[21] They also used the same technologies against the state’s surveillance system, applying facial recognition to identify police officers and enabling legal action to be taken against the police if necessary. This data could be verified in part with the help of a private database. The authorities in Hong Kong reacted swiftly, implementing strict data protection controls to regulate the private use of facial recognition.[22]

Site currently offline