MATHEMATICS

Toward nanomagnetic implementation of energy-based machine learning

Igor S. Lobanov

Faculty of Physics, ITMO University, Lomonosova Str. 9, Saint Petersburg, 191002 Russia

Abstract: Some approaches to machine learning (ML) such as Boltzmann machines (BM) can be reformulated as energy based models, which are famous for being trained by minimization of free energy. In the standard contrastive divergence (CD) learning the model parameters optimization is driven by competition of relaxation forces appearing in the target system and the model one. It is tempting to implement a physical device having natural relaxation dynamics matching minimization of the loss function of the ML model. In the article, we propose a general approach for the design of such devices. We systematically reduce the BM, the restricted BM and BM for classification problems to energy based models. For each model we describe a device capable of learning model parameters by relaxation. We compare simulated dynamics of the models using CD, Monte-Carlo method and Langevin dynamics. Benchmarks of the proposed devices on generation and classification of hand-written digits from MNIST dataset are provided.

Keywords: machine learning, Boltzmann machine, energy based model, dissipative training.

Received: 10.10.2023
Revised: 11.11.2023
Accepted: 07.12.2023

DOI: 10.17586/2220-8054-2023-14-6-613-625

Bibliographic databases:

•