Part 1. General problems of the intellectual systems theory

Learning parameters rotation method

A. A. Khusaenov

Lomonosov Moscow State University, Faculty of Mechanics and Mathematics

Abstract: A method is proposed for improving the quality of training of convolutional artificial neural networks (ANN) by dividing the parameters according to their ability to expand the receptive field. For example, ResNet50 accuracy increase is achieved by only 4 layers freeze. It is shown that the model generalizing ability increase is achieved by eliminating the excessive contribution of individual significant (occlusive) image elements in the feature maps. In favor of these assumptions the results of experiments in the transfer learning task and reasoning about the existence of this problem are presented. The proposed approaches can be useful in training ANNs on small data or distillation of the training set, where the problems of overfitting on individual occlusive features are of high importance.

Keywords: convolutional artificial neural network, neuron receptive field, model overfitting problems, feature occlusion in convolutional artificial neural networks.