Aspects généraux du bruit dans les réseaux de neurones profonds sur une architecture matérielle analogique ; The general aspects of noise in analog hardware deep neural networks

We study and analyze the fundamental aspects of noise propagation in recurrent as well as deep, multilayer neural networks comprising noisy analogue neurons as well as noisy connections. The motivations of our study are therefore neural networks implemented in special purpose analog hardware; yet, the methodology provides insight into noisy networks in general. Considering noisy neurons, we investigate the signal-to-noise ratio at the network’s outputs, which determines the upper limit of computational precision. We study additive and multiplicative as well as correlated and uncorrelated noise and develop analytical methods that predict the noise level in any layer of symmetric feedforward neural networks or feedforward neural networks trained with back propagation. We find that noise accumulation is generally bounded, and adding additional network layers does not worsen the signal to noise ratio beyond this limit. Finally, we propose noise mitigation strategies for deep neural networks working for different noise types.The proposed analytical results are then extended for an application to trained feedforward neural networks. As an example, we show that it is in a good correspondence with networks trained for digit recognition, classification of clothing images and prediction of chaotic time series.In this thesis we also consider recurrent neural networks with linear neurons and apply the obtained analytical results to complex-valued recurrent neural networks with nonlinear neurons, whose topology is based on a hardware realization in integrated photonic circuits. For both networks we identify the parts that are most vulnerable to different kinds of noise, and demonstrate that some noise types can be suppressed by the system itself. ; Nous étudions et analysons les aspects fondamentaux de la propagation du bruit dans les réseaux neuronaux multicouches récurrents et profonds constitués de réseaux analogiques bruités y compris sur les connexions. Notre étude est donc concernée par la mise en oeuvre de réseaux de ...