AI tool helps customers monitor social distancing in the workplace
21 May 2020
Image courtesy of Landing AI/YouTube
Landing AI has developed an AI-enabled social distancing detection tool that can detect if people are keeping a safe distance from each other by analysing real time video streams from the camera.
For example, at a factory that produces protective equipment, technicians could integrate this software into their security camera systems to monitor the working environment with easy calibration steps. As the demo in the video shows, the detector could highlight people whose distance is below the minimum acceptable distance in red and draw a line between to emphasise this. The system will also be able to issue an alert to remind people to keep a safe distance if the protocol is violated.
As part of an ongoing effort to keep customers and others safe and understanding that the only way through this is with global collaboration, Landing AI wanted to share the technical methodology it used to develop this software. The video will help to visually explain the approach that consists of three main steps: calibration, detection, and measurement.
Calibration
As the input video may be taken from an arbitrary perspective view, the first step of the pipeline is computing the transform (more specifically, the homography) that morphs the perspective view into a bird’s-eye (top-down) view. Landing AI term this process calibration. As the input frames are monocular (taken from a single camera), the simplest calibration method involves selecting four points in the perspective view and mapping them to the corners of a rectangle in the bird’s-eye view. This assumes that every person is standing on the same flat ground plane. From this mapping, a transformation can be derived that can be applied to the entire perspective image. This method, while well-known, can be tricky to apply correctly. As such, Landing AI have built a lightweight tool that enables even non-technical users to calibrate the system in real time.
During the calibration step, the scale factor of the bird’s eye view is also estimated, e.g. how many pixels correspond to 6 feet in real life.
Detection
The second step of the pipeline involves applying a pedestrian detector to the perspective views to draw a bounding box around each pedestrian. For simplicity, Landing AI use an open-source pedestrian detection network based on the Faster R-CNN architecture. To clean up the output bounding boxes, it applies minimal post-processing such as non-max suppression (NMS) and various rule-based heuristics; rules should be chosen that are grounded in real-life assumptions (such as humans being taller rather than they are wide), so as to minimise the risk of overfitting.
Measurement
Now, given the bounding box for each person, Landing AI estimate their (x, y) location in the bird’s-eye view. Since the calibration step outputs a transformation for the ground plane, said transformation is applied to the bottom-centre point of each person’s bounding box, resulting in their position in the bird’s eye view. The last step is to compute the bird’s eye view distance between every pair of people and scale the distances by the scaling factor estimated from calibration. People are highlighted whose distance is below the minimum acceptable distance in red and draw a line between to emphasise this.
As medical experts point out, until a vaccine becomes available, social distancing is our best tool to help mitigate the coronavirus pandemic and as we open up the economy. The goal with creating this tool and sharing it at such an early stage, is to help customers and to encourage others to explore new ideas to keep us safe.
More information...