Video analytics for monitoring operator activity in a port

In a major port in China, the work of gantry crane operators managing ship unloading and loading processes is a critical link in the logistics chain.

Project background


In a major port in China, the work of gantry crane operators managing ship unloading and loading processes is a critical link in the logistics chain. Each operation is performed manually by an operator using joysticks installed in an office for remote crane control. Operator distraction, fatigue, or violation of work regulations can lead to serious consequences: cargo damage, ship downtime, injuries, and significant financial losses.
 

The existing control system was limited to periodic visual inspections, which did not allow for real-time detection of violations or incident prevention. An automated solution was required for continuous monitoring of operator activity, recording of distractions and violations, and prompt notification of responsible personnel.
 

Implementation process


Specialists from OSMOVIEW LLC developed and implemented a solution for monitoring gantry crane operator activity based on the OSMOCV platform. The platform was deployed on the client's server infrastructure within a local network. Video streams from cameras covering the operator and their workstation were integrated into the platform.
 

The solution included analytics scenarios developed in-house by OSMOVIEW LLC:

  • Operator posture detection — analysis of the operator's body position, recording deviations from the working posture, signs of fatigue, or improper seating;
  • Hand position detection — monitoring the operator's hands on the controls (joysticks); recording instances where the operator removes their hands from the controls during a critical operation;
  • Mobile phone detection — identifying the use of a phone during working hours;
  • Operator absence — recording instances where the operator leaves the workstation without necessity or for an extended period;
  • Sleep / inactivity detection — identifying signs of the operator falling asleep or prolonged inactivity during work.
     

Critical importance of operator monitoring

Ship loading and unloading are high-risk operations where the crane is controlled manually by an operator via joysticks. Even brief distraction can lead to cargo colliding with ship structures, container damage, dockworker injuries, or equipment downtime. The system continuously analyzes operator activity and instantly records any deviations: hands off the joysticks, phone use, closed eyes, or absence from the workstation. Each violation is accompanied by video evidence and an automatic notification to the shift supervisor, allowing intervention before the violation leads to an incident.
 

Additionally implemented:

  • automatic generation of alert events upon detection of deviations, with notifications sent to the shift supervisor (Telegram, email);
  • generation of daily and weekly reports on operator activity and violations;
  • accumulation of statistics for each operator to analyze efficiency and conduct targeted briefings.

Achieved results


  • Operator violations have been minimized — the system records distractions, phone use, and hands off the joysticks in real time, which allowed for a reduction in the number of violations and improved discipline.
  • Risks of cargo and equipment damage have been reduced — prompt notification of violations allows the shift supervisor to intervene before operator distraction leads to an incident.
  • Transparency of staff operations has been ensured — accumulated statistics for each operator allow for objective assessment of efficiency, identification of risk areas, and conduct of targeted briefings.