Methods and algorithms of computational mathematics and their applications

Performance analysis methodology of deep neural networks inference on the example of an image classification problem

M. R. Alibekov^a, N. E. Berezina^b, E. P. Vasiliev^a, I. B. Vikhrev^b, Yu. D. Kamelina^b, V. D. Kustikova^a, Z. A. Maslova^b, I. S. Mukhin^a, A. K. Sidorova^a, V. N. Suchkov<sup>a

a</sup> National Research Lobachevsky State University of Nizhny Novgorod, Nizhny Novgorod, Russia
^b YADRO, Nizhny Novgorod, Russia

Abstract: Deploying of deep neural networks requires inference performance analysis on the target hardware. Performance results are aimed to be used as motivation to evaluate a decision for deployment, find the best performing hardware and software configurations, decide is there's a need for optimization of DL model and DL inference software. The paper describes a technique for analyzing and comparing inference performance using an example of image classification problem: converting a trained model to the formats of different frameworks, quality analysis, determining optimal inference execution parameters, model optimization and quality reanalysis, analyzing and comparing inference performance for the considered frameworks. Deep Learning Inference Benchmark Tool is aimed to support the performance analysis cycle. The technique is implemented on the example of the MobileNetV2 model.

Keywords: deep learning, neural networks, inference, performance, MobileNetV2, Deep Learning Inference Benchmark.

UDC: 004.032.26; 004.048; 004.021

Received: 10.12.2023
Accepted: 05.03.2024

DOI: 10.26089/NumMet.v25r211