Where’s Wally ? A Look At Video Content Analysis

Video Content Analysis (VCA) can be broken down into a number of different techniques; object detection, motion detection, recognition (e.g. facial recognition or automatic identification of number plates).

One of the oldest forms of VCA is a process known as Optical Character Recognition (OCR) where images of text, either handwritten or printed, are converted into a digital format. The original OCR systems had to be trained with individual images of each character one font at a time, however the more advanced systems used today are capable of achieving high levels of accuracy recognising a wide variety text fonts in incredibly short spaces of time. A prime example of an advanced OCR system is used on a daily basis on the UK road network.

Every time a car passes an automatic number plate recognition camera, a series of algorithms are used to identify the characters, which are run through database records of vehicles of interest (those known to be stolen, involved in criminal activities or without insurance and MOT) and flagged to the relevant police units. The most prevalent use of this technology is in the average speed cameras found on a number of motorways throughout the country.

The cameras used in ANPR are currently unable to identify more than the 36 characters used on registration plates, hardly close to the ‘Big Brother is watching you’ surveillance from George Orwell’s 1984, but the same cannot be said for new prototype CCTV systems being developed at the moment. Object detection is a relatively new VCA technique that allows users to automatically identify specified objects, and whilst this seems like a conspiracy theorists worst nightmare the real world applications of this kind of software could be extremely beneficial.

For instance, shops all over the UK are involved in a scheme that allows shop owners to upload CCTV images of individuals that have committed a crime such as shoplifting or robbery. Until recently the success of the system has relied upon the eagle eyes of staff members to identify and raise the alarm to the presence of these individuals. Now the founder of the Facewatch scheme, Simon Gordon, has begun testing a form of Facial Recognition to identify the faces of individuals stored in Facewatch’s database.1

There are still a number of issues that hamper the progress of detailed real-time facial recognition systems, for example the use of eyewear can render a person “invisible” by interfering with the system’s ability to recognise facial features, but nothing is impossible and we may be seeing this implemented throughout the UK within the next 50 years.

Facewatch is not the only system to utilise facial recognition as a part of its operation. Facebook has implemented an automatic tagging feature that will recognise faces in uploaded images and suggest who is in the picture. Currently this feature is not available to users within Europe as Privacy Regulations prevent the use of facial recognition software without giving users the ability to ‘opt-in’. It is for this reason that systems, such as Google’s Vision API that can identify the presence of a person and even detect the emotions of people in images but not recognise who they are, have fewer issues regarding breaches of an individual’s privacy.

Video content analysis has vastly improved as the technology and understanding has grown over the years, but what more could it offer further into the future?

Following the growing interest in drone aircraft and the possibilities they can offer in both military and domestic situations, it would be reasonable to assume that video analysis techniques could be integrated into these systems to identify specified objects in potentially hazardous environments. For example, following a disaster i.e. fire, tornado or earthquake, buildings are inspected for damage as they could have become structurally unsound. The use of a drone fitted with a form of object identification software could reduce the potential risk to human life by identifying buckles, tears or holes in a buildings infrastructure, and thus ensuring that any interaction with the building would not result in any adverse situations.

Looking from a marketing angle the technology used in the Google API to identify faces and emotions, could be utilised by the supermarket and advertising industries to provide a new form of targeted marketing. For example as a customer browses for groceries a video analysis program could identify their reaction to a certain product, then alter the current promotional offers advertised on LCD displays around the store, in order to best suit the interests of that customer. Alternatively supermarkets could monitor the popularity of its products based upon the emotional reactions of their customers, if the majority show a negative reaction new stock may be required.

As you can see the range of video content analysis techniques is ever expanding and as technology continues to develop, the scope and sophistication that can be achieved will only continue to improve. As systems such as Google’s Vision API are released these technologies will become more widespread, which will help to lower the cost of what currently can be an expensive process.

Here at Zircon we have been experimenting with and utilising these technologies in both research and full project work, especially in the rail and defence sectors. Some of our older VCA projects have included the development of systems that can detect the presence of unauthorised persons at the side of railway tracks or make checks on the integrity of rail side infrastructures (which can be read about here). One of our most recent projects was used to recognise objects in footage provided by the DSTL.

If you think that you may be interested in utilising any of these techniques or are having issues with a VCA system of your own, we would be more than happy to discuss our experiences and expertise with you. You can get in contact with us at 01225 764 444 or info@zirconsoftware.co.uk.

1 Vallance, C. (2016). Facewatch ‘thief recognition’ CCTV on trial in UK stores – BBC News.

[online] BBC News. Available at: http://www.bbc.co.uk/news/technology-35111363 [Accessed 22 Jun. 2016].