Smart Eye [02]: Head and Gaze Tracking

The Smart Eye Pro system is a head and gaze tracking system, which measures the subject’s head pose and gaze direction in 3D.

Head Tracking

Head tracking estimates the 3D pose (position and orientation) of the subject’s head in the world coordinate system.

It uses a personal profile that is created automatically in the “Automatic Profile” mode.

A profile is a model of the subject’s head that contains the 3D coordinates and tracking templates of salient facial features such as eye corners, nostrils, mouth corners, and ears. By detecting the positions of some of the features in the profile while tracking, the system can calculate the pose of the head.

Head Position Quality

The head position quality (the HeadPositionQ value) indicates the current tracking state:

• Face Finder: Face Tracking is not initialized yet or was lost over a longer period of time. Quality is 0.1.
• Face Refinder: Face Tracking was temporarily lost and the face tracker is trying to recover. Quality is 0.2. Face Tracking: Face Tracking works normally. Quality is 1.0.

Automatic Profile

After starting tracking, the profile is initialized automatically when the face is visible in at least one camera. During tracking the profile is updated and improved continuously, i.e. more tracking templates are added to cover a wider range of head rotations and the 3D head geometry is recalculated to better fit the subject’s head.

The generated profile can be saved and will not be updated anymore thereafter. Turn off tracking and save the profile under Profile->Save as….

When a gaze calibration is performed, it can be saved to the profile, by turning off tracking and selecting Profile->Save.

Good tracking results are generally achieved using a profile which includes 20 seconds of tracking followed by a gaze calibration. It is thereafter recommended to save the profile.

When tracking is turned off, the previous profile will be active when tracking is started again.

Gaze Tracking

Gaze tracking calculates the 3D gaze measurements in the world coordinate system.

Gaze Measurement

A “Gaze Measurement” is defined by a “Gaze Origin” point in 3D space plus a “Gaze Direction” vector.

There is a separate gaze measurement for the left and the right eye. These are combined into a “Consensus” gaze measurement for a virtual eye positioned at the midpoint between the left and the right eye.

Gaze Direction Quality

The GazeDirectionQ value can take any floating point value between 0.0 and 1.0 and depends on a number of factors. Thus, it is not easy to set a general threshold, where all gaze data below that threshold should be considered unreliable, for all possible setups and subjects.

Gaze Caliberation

The gaze direction measurement is affected by several parameters such as the individual eye geometry, refracting eyeglasses, and individual characteristics such as strabismus. This means the gaze direction needs to be adjusted uniquely for each individual. The purpose of the gaze calibration is to determine the difference between the visual and the optical axis of the eye and other relevant eye parameters.

Gaze Calibration Points

Before a calibration can be started, a number of calibration points need to be added to the World Model. These points should be easy to distinguish and evenly spread over the area of interest. If they are not on a screen or a plane their 3D position needs to be defined in the world coordinate system.

Gaze Calibration

Gaze calibration is performed through the Gaze Calibration dialog. The red dots on the circles for the “Left Eye” and “Right Eye” shows where the current uncalibrated gaze intersects a plane, orthogonal to the current calibration point in theWorld Model with the vector direction towards the eye center. blue dots show all saved samples and green dots show the samples after the calibration has been processed. Ideally the green dots should be in the middle of the target. Each circle on the target corresponds to 2 degrees of accuracy.

The accuracy and standard deviation of the calibration can be checked for each point. If any point has a value that significantly differs from the others it should be removed and “trained” again.

Calibration procedures can be referenced to the post Getting Started.