Federated Learning-Distillation Alternation for Resource-Constrained IoT

Federated learning (FL) faces significant challenges in Internet of Things (IoT) networks due to device limitations in energy and communication resources, especially when considering the large size of FL models. From an energy perspective, the challenge is aggravated if devices rely on energy harvesting (EH), as energy availability can vary significantly over time, influencing the average number of participating users in each iteration. Additionally, the transmission of large model updates is more susceptible to interference from uncorrelated background traffic in shared wireless environments. As an alternative, federated distillation (FD) reduces communication overhead and energy consumption by transmitting local model outputs, which are typically much smaller than the entire model used in FL. However, this comes at the cost of reduced model accuracy. Therefore, in this article, we propose FL-distillation alternation (FLDA). In FLDA, devices alternate between FD and FL phases, balancing model information with lower communication overhead and energy consumption per iteration. We consider a multichannel slotted-ALOHA (SA) EH-IoT network subject to background traffic/interference and compared FLDA to FL, FD, and SplitFed. In such a scenario, FLDA demonstrates higher model accuracy than both FL and FD, and achieves faster convergence than both FL and SplitFed. While SplitFed achieves a similar accuracy level to FLDA with no interference, the method is highly affected by interference, which also affects FL. Moreover, FLDA achieves target accuracies saving up to 98.02% in energy consumption relative to FL and up to 99.85% relative to SplitFed. © 2014 IEEE.