---
title: Smarter Training of Memristive Neural Networks
description: Our Advanced Science paper showing how to make memristive neural networks more robust.
author: Dovydas Joksas
url: https://dovydas.com/blog/smarter-training-of-memristive-neural-networks
html: https://dovydas.com/blog/smarter-training-of-memristive-neural-networks?format=html
---

May 5, 2022 · 1 min read

# Smarter Training of Memristive Neural Networks

Neural networks consume massive amounts of power, and memristive implementations may offer a solution. But although memristors are much less power-hungry, they are also stochastic and—like most analog devices—less precise. How do we deal with that?

We have been collaborating with [Imperial](https://www.imperial.ac.uk/electrical-engineering) to make memristive neural networks

- adapt to nonidealities
- consume even less energy
- be robust to uncertainty

As illustrated below, we did this by

- redefining neural network node functions so that they take into account potential nonlinearity and stochasticity
- rethinking how neural network weights are implemented using memristor conductances, so that regularization could act as a way of further tuning power consumption
- computing validation error multiple times at checkpoints to take stochastic nature of memristors into account

![Overview of our new techniques: taking into account nonlinearity and stochasticity, rethinking how weights are mapped onto conductances, and improving validation](https://media.dovydas.com/ce52467a54152121a74a95c96f58578d0f8ce3d12f52e0864703eda5eb666fa5.svg)

The resulting paper [just came out](https://doi.org/10.1002/advs.202105784) in Advanced Science, while the code can be found [here](https://github.com/joksas/nonideality-aware-mnn-training).

I want to thank my coauthors for all their contributions:

- [Erwei Wang](https://www.linkedin.com/in/erweiwang)
- [Nikolaos Barmpatsalos](https://www.linkedin.com/in/nikolaos-barmpatsalos-a199569a)
- [Wing Ng](https://www.ucl.ac.uk/electronic-electrical-engineering/people/dr-wing-ng)
- [Tony Kenyon](https://www.ucl.ac.uk/electronic-electrical-engineering/people/prof-tony-kenyon)
- [George Constantinides](https://cas.ee.ic.ac.uk/people/gac1/)
- [Adnan Mehonic](https://www.ee.ucl.ac.uk/~uceeadm/)

## Keep reading

1. [Memristive neural networks perform better when they work in teams](https://dovydas.com/blog/memristive-neural-networks-perform-better-when-they-work-in-teams) (2020-08-27)
2. [Analogue Solution to the Power-Hungry Field of Machine Learning](https://dovydas.com/blog/analogue-solution-to-the-power-hungry-field-of-machine-learning) (2020-02-28)
3. [The Power Crisis of Modern Computing](https://dovydas.com/blog/the-power-crisis-of-modern-computing) (2019-12-03)
