The SECURIFACE interphone is a video intercom system equipped with a novel face detection technology which distinguishes suspicious visitors based on their faces. The system uses face detection to judge whether a visitor is trying to hide their face or body by obstructing their face with a hat or hiding in a blind spot.

Face obstruction is processed by extracting the areas with human elements, extracting the head and finally determining the level of obstruction from the camera image. Creating an robust algorithm that can work in the adverse outdoor conditions such as weather and sunlight that an intercom is exposed to and still accurately distinguish a myriad of visitors was our principle goal.

In this article, face detection functionality and the underlying algorithm are introduced.

Background

Ringing a doorbell and running away before the door is answered is an innocent prank played by children all around the world. Recently, this prank is used for more nefarious purposes — would be burglars employing a similar method to whether anyone is home or forcing their way inside if the door is answered.

The SECURIFACE Interphone was developed by Secom to prevent this kind of crime and was made commercially available in November 2003.

Overview

In addition to detecting faces, the SECURIFACE Interphone System can also allow residents monitor the external premises using a separate camera or record the camera or visitor images, creating a proactive residential security video surveillance system.

In this article, we focus on the "Face Recognition" aspect of the system which is a central feature of "SECURIFACE" Interphone System.

Face Detection Technology

The newly developed face detection technology can be explained through the difference from current face detection/recognition technologies.

Face detection technology is locating the position of human faces in an arbitrary image. Recent models of digital cameras and printers now include this technology, which is used mainly to adjust brightness or focus.

On the other hand, face recognition is a technology to determine the individual that the face belongs to. Commercialization of this technology has been limited, mostly to applications in access control or detection in a crowd.

Our approach to face detection is unique, in that we determine whether a face in a given image is identifiable by a human.

Face Detection Function

By equipping an intercom with our face detection technology, it becomes possible to determine beforehand whether a visitor's face can be identified by the resident. In other words, it provides a way to determine whether the face or figure is obstructed.

Fig.1 SECURIFACE Interphone

SECURIFACE Operation Overview

The intercom can ring the doorbell in three different ways, depending on the face detection results of the visitor.

Fig.2 Face detection behaviors

Effectiveness of Face Detection

We can analyze the effectiveness of the SECURIFACE Interphone system by examining the behavior of both the residents and visitors.

Effects on Visitors

If a visitor's faces or body is obstructed while operating the intercom, the unit will play a pre-recorded message, with the following implications:

• The non-typical response (voice, instead of a bell) acts as a deterrent against visitors with nefarious intent.
• Ringing the doorbell cannot be used to determine whether the resident is present.

Effects on Residents

If the face of the visitor is not clearly visible, the doorbell will ring differently:

• Residents can exercise due caution when manually confirming the visitor on the monitor.

If configured to stay silent under such a scenario:

• The residents do not have to answer the door.
• Minimize the risk of opening the door out of concern about who is visiting.

Approach to Development

Assumptions about the Operating Environment

The camera unit is assumed to be located outside of the premises, affecting and varying the image quality due to sunlight or weather. High performance image processing under these demanding conditions is very difficult, and there are very few examples of commercialized products. The primary goal of this project was to the development of an algorithm for a commercial product that achieves a constant level of performance over many harsh conditions.

Algorithmic Requirements

A lightweight algorithm was required since all image processing must be completed without a noticeable lag after the doorbell is depressed.

Data Collection

Images were obtained from the outdoor camera unit under a variety of conditions, including environmental variations such as weather (sunny, cloudy, rain, snow) and lighting (daytime, dawn/dusk, night, street lighting), and also for variation in visitors (height, color/pattern of clothes, facial features, hair style, accessories (umbrella, bags)).

Developing the Algorithm

During development we repeatedly evaluated an algorithm, identified problems with it, and took corrective measures to improve the algorithm. To efficiently conduct development, we built an image database to run the algorithms against so that the performance of the algorithm can be simulated easily.

As we got closer to the desired level of performance, we implemented the algorithm in the actual unit and conducted field tests for final tuning.

Through this approach and by choosing the appropriate evaluation methodology at each step, we were able to achieve very high development efficiency.

Overview of the Face Detection Algorithm

When the visitor rings the doorbell, an image is acquired and analyzed to determine whether the face or body is obstructed. The algorithm is outlined below.

Fig.3 Outline of the Algorithm

STEP 1. Body Extraction

The first step is to determine whether a human is present in the image. By comparing the image captured when the doorbell is pressed with a background image stored earlier, the resulting silhouette used to determine whether a human is present. Our approach is based on edge detection and not brightness information as brightness changes throughout the day.

Edge detection only extracts the outlines of individuals, so another process is used to fill in the outline, resulting in a human candidate area..

STEP2. Head Extraction

Next, candidates for the head region are extracted from the human area candidate. The head regions are extracted by finding oval patterns in the area extracted in STEP 1. The oval patterns are also based on edge information, not brightness.

Obstruction Classification

To decide whether a face is obstructed, the algorithm calculates a "natural face" likelihood score for each head candidate area. If the highest score exceeds a certain threshold, the algorithm classifies the image as recognizable. If scores from all of the areas do not meet the threshold, the image is classified as obstructed

Unobstructed faces have certain features, such as multiple horizontal features which correspond to eyes or mouths and a large amount of skin colored areas. The likelihood score is calculated based on assessment of these features from the image. For high accuracy in classification, the features which are robust against environmental changes, such are vertical/horizontal edge strength, hue, average and distribution of brightness, are used. The likelihood score is calculated using a statistical method based on the similarity of features between images containing obstructed and unobstructed individuals. Applying a threshold to this score allows classification on whether the face is unobstructed (recognizable) or not.

In this article, we focused on "SECURIFACE" Interphone System and its face detection algorithm. In developing the algorithm, we made full use of face recognition and outdoor sensing technology, was well as technology developed in-house. This system is available as a commercial security product, and is used in locations where traditionally, image processing technology has proven to be difficult.