Deep Learning Personalization Methods in Peer-to-peer Heterogeneous Systems ; Metode personalizacije dubokog učenja u istorazinskim heterogenim sustavima

Recent research has shown considerable interest in collaborative training of deep neural networks utilizing edge devices. Two predominant architectural paradigms for this training process include centrally orchestrated Federated Learning and fully decentralized peer-to-peer learning. Edge devices, termed agents, harbor local deep neural network models and distinct local datasets, composed of data collected specifically by each agent. While peer-to-peer learning techniques have been extensively investigated assuming independent and identically distributed (IID) data across agents, the learning efficacy significantly diminishes under non-IID assumptions, resulting in reduced model accuracy and slower convergence rates. The thesis aims to identify viable strategies for alleviating the impact of non-IID data on the overall learning process and to devise novel methodologies applicable in peer-to-peer deep learning contexts. These methodologies are subsequently evaluated using realistic non-IID datasets to assess their efficacy and applicability. The thesis will analyze autonomous personalized peer connection creation and present two methods of improving the peer-to-peer learning process in non-IID environments. The methods relate to improving peer-to-peer learning by enabling multi-task collaboration between agents learning two distinct tasks, and improving agent’s local model performance by a personalization technique. The results indicate a statistically significant increase of 11.6% in the mean relative accuracy for the proposed multi-task technique, and 16.9%-29.8% relative accuracy increase (depending on the topology) for the personalization technique. Compared to existing approaches, presented methods can be used to enhance the performance and scalability of peer-to-peer learning systems, and improve personalization resulting in greater model accuracy in diverse real-world scenarios. ; Suradničko obučavanje dubokih neuronskih mreža na rubnim (mobilnim i ugradbenim) uređajima izazvalo je znatan interes u ...