FedGS: A Federated Group Synchronization Framework Implemented by LEAF-MX.

Overview

FedGS: Data Heterogeneity-Robust Federated Learning via Group Client Selection in Industrial IoT

Preparation

  • For instructions on generating data, please go to the folder of the corresponding dataset. For FEMNIST, please refer to femnist.

  • NVIDIA-Docker is required.

  • NVIDIA CUDA version 10.1 and higher is required.

How to run FedGS

Build a docker image

Enter the scripts folder and build a docker image named fedgs.

sudo docker build -f build-env.dockerfile -t fedgs .

Modify /home/lizh/fedgs to your actual project path in scripts/run.sh. Then run scripts/run.sh, which will create a container named fedgs.0 if CONTAINER_RANK is set to 0 and starts the task.

chmod a+x run.sh && ./run.sh

The output logs and models will be stored in a logs folder created automatically. For example, outputs of the FEMNIST task with container rank 0 will be stored in logs/femnist/0/.

Hyperparameters

We categorize hyperparameters into default settings and custom settings, and we will introduce them separately.

Default Hyperparameters

These hyperparameters are included in utils/args.py. We list them in the table below (except for custom hyperparameters), but in general, we do not need to pay attention to them.

Variable Name Default Value Optional Values Description
--seed 0 integer Seed for client selection and batch splitting.
--metrics-name "metrics" string Name for metrics file.
--metrics-dir "metrics" string Folder name for metrics files.
--log-dir "logs" string Folder name for log files.
--use-val-set None None Set this option to use the validation set, otherwise the test set is used. (NOT TESTED)

Custom Hyperparameters

These hyperparameters are included in scripts/run.sh. We list them below.

Environment Variable Default Value Description
CONTAINER_RANK 0 This identify the container (e.g., fedgs.0) and log files (e.g., logs/femnist/0/output.0).
BATCH_SIZE 32 Number of training samples in each batch.
LEARNING_RATE 0.01 Learning rate for local optimizers.
NUM_GROUPS 10 Number of groups.
CLIENTS_PER_GROUP 10 Number of clients selected in each group.
SAMPLER gbp-cs Sampler to be used, can be random, brute, bayesian, probability, ga and gbp-cs.
NUM_SYNCS 50 Number of internal synchronizations in each round.
NUM_ROUNDS 500 Total rounds of external synchronizations.
DATASET femnist Dataset to be used, only FEMNIST is supported currently.
MODEL cnn Neural network model to be used.
EVAL_EVERY 1 Interval rounds for model evaluation.
NUM_GPU_AVAILABLE 2 Number of GPUs available.
NUM_GPU_BEGIN 0 Index of the first available GPU.
IMAGE_NAME fedgs Experimental image to be used.

NOTE: If you wish to specify a GPU device (e.g., GPU0), please set NUM_GPU_AVAILABLE=1 and NUM_GPU_BEGIN=0.

NOTE: This script will mount project files /home/lizh/fedgs from the host into the container /root, so please check carefully whether your file path is correct.

Visualization

The visualizer metrics/visualize.py reads metrics logs (e.g., metrics/metrics_stat_0.csv and metrics/metrics_sys_0.csv) and draws curves of accuracy, loss and so on.

Reference

  • This demo is implemented on LEAF-MX, which is a MXNET implementation of the well-known federated learning framework LEAF.

  • Li, Zonghang, Yihong He, Hongfang Yu, et al. "Data Heterogeneity-Robust Federated Learning via Group Client Selection in Industrial IoT." Submitted to IEEE Internet of Things Journal, (2021).

  • If you get trouble using this repository, please kindly contact us. Our email: [email protected]

Owner
Lizonghang
Intelligent Communication System, Distributed Machine Learning, Federated Learning
Lizonghang
A dataset for online Arabic calligraphy

Calliar Calliar is a dataset for Arabic calligraphy. The dataset consists of 2500 json files that contain strokes manually annotated for Arabic callig

ARBML 114 Dec 28, 2022
Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations. [2021]

Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations This repo contains the Pytorch implementation of our paper: Revisit

Wouter Van Gansbeke 80 Nov 20, 2022
Implementation of experiments in the paper Clockwork Variational Autoencoders (project website) using JAX and Flax

Clockwork VAEs in JAX/Flax Implementation of experiments in the paper Clockwork Variational Autoencoders (project website) using JAX and Flax, ported

Julius Kunze 26 Oct 05, 2022
Physics-informed Neural Operator for Learning Partial Differential Equation

PINO Physics-informed Neural Operator for Learning Partial Differential Equation Abstract: Machine learning methods have recently shown promise in sol

107 Jan 02, 2023
🕵 Artificial Intelligence for social control of public administration

Non-tech crash course into Operação Serenata de Amor Tech crash course into Operação Serenata de Amor Contributing with code and tech skills Supportin

Open Knowledge Brasil - Rede pelo Conhecimento Livre 4.4k Dec 31, 2022
Code for SyncTwin: Treatment Effect Estimation with Longitudinal Outcomes (NeurIPS 2021)

SyncTwin: Treatment Effect Estimation with Longitudinal Outcomes (NeurIPS 2021) SyncTwin is a treatment effect estimation method tailored for observat

Zhaozhi Qian 3 Nov 03, 2022
A modular PyTorch library for optical flow estimation using neural networks

A modular PyTorch library for optical flow estimation using neural networks

neu-vig 113 Dec 20, 2022
Multi-Scale Aligned Distillation for Low-Resolution Detection (CVPR2021)

MSAD Multi-Scale Aligned Distillation for Low-Resolution Detection Lu Qi*, Jason Kuen*, Jiuxiang Gu, Zhe Lin, Yi Wang, Yukang Chen, Yanwei Li, Jiaya J

DV Lab 115 Dec 23, 2022
A very impractical 3D rendering engine that runs in the python terminal.

Terminal-3D-Render A very impractical 3D rendering engine that runs in the python terminal. do NOT try to run this program using the standard python I

23 Dec 31, 2022
Python Auto-ML Package for Tabular Datasets

Tabular-AutoML AutoML Package for tabular datasets Tabular dataset tuning is now hassle free! Run one liner command and get best tuning and processed

Sagnik Roy 18 Nov 20, 2022
BasicRL: easy and fundamental codes for deep reinforcement learning。It is an improvement on rainbow-is-all-you-need and OpenAI Spinning Up.

BasicRL: easy and fundamental codes for deep reinforcement learning BasicRL is an improvement on rainbow-is-all-you-need and OpenAI Spinning Up. It is

RayYoh 12 Apr 28, 2022
PyTorch Live is an easy to use library of tools for creating on-device ML demos on Android and iOS.

PyTorch Live is an easy to use library of tools for creating on-device ML demos on Android and iOS. With Live, you can build a working mobile app ML demo in minutes.

559 Jan 01, 2023
Semi-supevised Semantic Segmentation with High- and Low-level Consistency

Semi-supevised Semantic Segmentation with High- and Low-level Consistency This Pytorch repository contains the code for our work Semi-supervised Seman

123 Dec 30, 2022
Mix3D: Out-of-Context Data Augmentation for 3D Scenes (3DV 2021)

Mix3D: Out-of-Context Data Augmentation for 3D Scenes (3DV 2021) Alexey Nekrasov*, Jonas Schult*, Or Litany, Bastian Leibe, Francis Engelmann Mix3D is

Alexey Nekrasov 189 Dec 26, 2022
Nest - A flexible tool for building and sharing deep learning modules

Nest - A flexible tool for building and sharing deep learning modules Nest is a flexible deep learning module manager, which aims at encouraging code

ZhouYanzhao 41 Oct 10, 2022
tf2-keras implement yolov5

YOLOv5 in tesnorflow2.x-keras yolov5数据增强jupyter示例 Bilibili视频讲解地址: 《yolov5 解读,训练,复现》 Bilibili视频讲解PPT文件: yolov5_bilibili_talk_ppt.pdf Bilibili视频讲解PPT文件:

yangcheng 254 Jan 08, 2023
An essential implementation of BYOL in PyTorch + PyTorch Lightning

Essential BYOL A simple and complete implementation of Bootstrap your own latent: A new approach to self-supervised Learning in PyTorch + PyTorch Ligh

Enrico Fini 48 Sep 27, 2022
Discriminative Region Suppression for Weakly-Supervised Semantic Segmentation

Discriminative Region Suppression for Weakly-Supervised Semantic Segmentation (AAAI 2021) Official pytorch implementation of our paper: Discriminative

Beom 74 Dec 27, 2022
Deep-learning X-Ray Micro-CT image enhancement, pore-network modelling and continuum modelling

EDSR modelling A Github repository for deep-learning image enhancement, pore-network and continuum modelling from X-Ray Micro-CT images. The repositor

Samuel Jackson 7 Nov 03, 2022
Unofficial Alias-Free GAN implementation. Based on rosinality's version with expanded training and inference options.

Alias-Free GAN An unofficial version of Alias-Free Generative Adversarial Networks (https://arxiv.org/abs/2106.12423). This repository was heavily bas

dusk (they/them) 75 Dec 12, 2022