Persons or vehicles recognition and tracking for video surveillance systems
SentiVeillance SDK is designed for developing software that performs biometric face identification, detects moving pedestrians or vehicles or other objects and performs automatic license plate recognition using live video streams from digital surveillance cameras.
The SDK is used for passive identification – when passers-by do not make any efforts to be recognized. List of possible uses includes law enforcement, security, attendance control, visitor counting, traffic monitoring and other commercial applications.
The Face modality from the SentiVeillance component performs real-time detection, biometric identification and tracking of all detected faces:
- Facial identification reliability enables using large watchlist databases (i.e. a watch-list of suspects or a list of company employees).
- New faces may be enrolled to the database either automatically from video streams or manually as templates, which were created with VeriLook SDK.
- Persons wearing face masks or respirators can be recognized without separate enrollment.
SentiVeillance algorithms allows to perform these advanced actions with the detected faces:
- Face tracking, which is performed in all successive frames from the video source until they disappear from camera field of view. The face tracking algorithm uses dynamic face and motion prediction models that make it robust to occlusions like other objects or even other faces. The algorithm is able to continue tracking a face even when it re-appears after being fully covered by occlusions (like walls, furniture, posters etc).
- Gender classification (optional) for each person in the frame.
- Age determination (optional) for each person in the frame.
- Face mask, smile, open-mouth, closed-eyes, glasses, dark-glasses, beard and mustache attributes detection (configurable).
VH modality can be used together with the Face modality to perform further tracking of the identified persons even when their faces no longer visible, as well as associating previous tracking history with the recognized person. Also, the VH modality can be used to obtain additional data about person's clothes.
Programming samples from the SDK show how to use this modality.
A license for using the SentiVeillance Face modality on one video stream is included with SentiVeillance 8.0 SDK. Additional licenses or upgrade for the number of processed video streams in the existing licenses can be purchased any time by SentiVeillance 8.0 SDK customers.
Basic usage recommendations and constraints
Face recognition accuracy of SentiVeillance heavily depends on the quality of a face image in a frame. There are some basic recommendations and constraints when using face recognition applications based on SentiVeillance SDK.
Image quality during enrollment is important, as it influences the quality of the face template.
Enrollment from photo or video stream is possible.
- Several images during enrollment are recommended for better facial template quality which results in improvement of recognition accuracy and reliability.
- Additional enrollments may be needed when facial hair style changes, especially when beard or mustache is grown or shaved off.
- 32 pixels is the recommended minimal distance between eyes for a face on image or video stream to perform face template extraction reliably. 64 pixels or more recommended for better face recognition results. Note that this distance should be native, not achieved by resizing an image.
- 1 MegaPixel or better camera resolution is recommended for face enrollment and recognition. Make sure that native resolution is provided by a camera, as some cameras or webcams may scale up native images to higher resolution without image quality improvement.
Face posture tolerance:
- head roll (tilt) – ±15 degrees;
head pitch (nod) – ±15 degrees from frontal position.
- The head pitch tolerance can be increased up to ±25 degrees if several views of the same face that covered different pitch angles were used during enrollment.
head yaw (bobble) – ±90 degrees from frontal position (default value).
- Several views of the same face can be enrolled to the database to cover the whole ±90 degrees yaw range from frontal position.
Below are provided specifications for SentiVeillance facial recognition modality.
4 % of the frame's larger side (at least 32 pixels) is the minimal recommended distance between eyes for a face on video stream or image to perform reliable face tracking and template extraction. The speeds of face tracking, template extraction and matching against a watchlist database are dependent on actual size of a face in a frame, not on the size of the whole frame.
The performance specifications are provided for Intel Core i7-4771 processor, running at 3.5 GHz clock rate, and 1920 x 1080 pixels videos.
|SentiVeillance 8.0 biometric face recognition algorithm technical specifications|
|Frame rate when tracking up to 5 faces||More than 25 frames per second|
|Face watch-list database matching time (1)||Less than 0.5 second|
|Single face record size in a template (bytes)||194 or 322 (configurable)|
|Maximum face watch-list database size||Limited by amount of free RAM|
(1) up to 20,000 face records in the database; larger database yelds slower response time. Note that each person may be represented by several records in the database with different appearance variations, different capture angles etc.