Autonomous multi-camera neural network computer vision system for manual operation monitoring

I. M. Klemyshev^a, S. M. Kolchin^b, S. S. Lebedev^b, S. O. Starkov<sup>c

a</sup> General Computer Systems Limited liability company, Obninsk, Russian Federation
^b COMPVI Limited liability company, Obninsk, Russian Federation
^c Obninsk Institute for Nuclear Power Engineering, Obninsk, Russian Federation

Abstract: This paper presents a prototype system for the automatic monitoring of manual production processes. The trained system observes the workplace from multiple angles and analyzes recorded events in real time using artificial intelligence. When abnormal employee actions or incorrect execution of technological processes are detected, the system alerts the security service operator, assisting in decision-making or facilitating a quick search through archived video surveillance records for incident investigation. A mechanism for the auto-calibration of the multi-angle video camera system is proposed, which simultaneously observes key points on the hands, determines their spatial location, and mathematically resolves the problem of combining images from multiple angles. The hardware component of the prototype consists of two Intel RealSense 435 stereo cameras and a computing module equipped with a Jetson AGX Xavier graphics processor. The software component includes several subsystems: video surveillance, data storage, and neural network analysis of the digital images.

Keywords: computer vision, pattern recognition, artificial neural networks, intelligent video camera, neural network detector.

DOI: 10.51790/2712-9942-2024-5-3-03