FS-TFP/federatedscope/gfl

FederatedScope-GNN: Towards a Unified, Comprehensive and Efficient Package for Federated Graph Learning

FederatedScope-GNN (FS-G) is a unified, comprehensive and efficient package for federated graph learning. We provide a hands-on tutorial here, while for more detailed tutorial, please refer to FGL Tutorial.

Quick Start

Let’s start with a two-layer GCN on FedCora to familiarize you with FS-G.

Step 1. Installation

The installation of FS-G follows FederatedScope, please refer to Installation.

After installing the minimal version of FederatedScope, you should install extra dependencies (PyG, rdkit, and nltk) for the application version of FGL, run:

conda install -y pyg==2.0.4 -c pyg
conda install -y rdkit=2021.09.4=py39hccf6a74_0 -c conda-forge
conda install -y nltk

Now, you have successfully installed the FGL version of FederatedScope.

Step 2. Run with exmaple config

Now, we train a two-layer GCN on FedCora with FedAvg.

python federatedscope/main.py --cfg federatedscope/gfl/baseline/example.yaml

For more details about customized configurations, see Advanced.

Reproduce the results in our paper

We provide scripts (grid search to find optimal results) to reproduce the results of our experiments.

• Node-level tasks, please refer to federatedscope/gfl/baseline/repro_exp/node_level/:

  # Example of FedAvg
  cd federatedscope/gfl/baseline/repro_exp/node_level/
  bash run_node_level.sh 0 cora louvain
  
  # Example of FedAvg
  bash run_node_level.sh 0 cora random
  
  # Example of FedOpt
  bash run_node_level_opt.sh 0 cora louvain gcn 0.25 4
  
  # Example of FedProx
  bash run_node_level_prox.sh 0 cora louvain gcn 0.25 4
  

• Link-level tasks, please refer to federatedscope/gfl/baseline/repro_exp/link_level/:

  cd federatedscope/gfl/baseline/repro_exp/link_level/
  
  # Example of FedAvg
  bash run_link_level_KG.sh 0 wn18 rel_type
  
  # Example of FedOpt
  bash run_link_level_opt.sh 0 wn18 rel_type gcn 0.25 16
  
  # Example of FedProx
  bash run_link_level_prox.sh 7 wn18 rel_type gcn 0.25 16
  

• Graph-level tasks, please refer to federatedscope/gfl/baseline/repro_exp/graph_level/:

  cd federatedscope/gfl/baseline/repro_exp/graph_level/
  
  # Example of FedAvg
  bash run_graph_level.sh 0 proteins
  
  # Example of FedOpt
  bash run_graph_level_opt.sh 0 proteins gcn 0.25 4
  
  # Example of FedProx
  bash run_graph_level_prox.sh 0 proteins gcn 0.25 4
  

Advanced

Start with built-in functions

You can easily run through a customized yaml file:

# Whether to use GPU
use_gpu: True

# Deciding which GPU to use
device: 0

# Federate learning related options
federate:
  # `standalone` or `distributed`
  mode: standalone
  # Evaluate in Server or Client test set
  make_global_eval: True
  # Number of dataset being split
  client_num: 5
  # Number of communication round
  total_round_num: 400

# Dataset related options
data:
  # Root directory where the data stored
  root: data/
  # Dataset name
  type: cora
  # Use Louvain algorithm to split `Cora`
  splitter: 'louvain'
  # Use fullbatch training, batch_size should be `1`
  batch_size: 1

# Model related options
model:
  # Model type
  type: gcn
  # Hidden dim
  hidden: 64
  # Dropout rate
  dropout: 0.5
  # Number of Class of `Cora`
  out_channels: 7

# Criterion related options
criterion:
  # Criterion type
  type: CrossEntropyLoss

# Trainer related options
trainer:
  # Trainer type
  type: nodefullbatch_trainer

# Train related options
train:
  # Number of local update steps
  local_update_steps: 4
  # Optimizer related options
  optimizer:
    # Learning rate
    lr: 0.25
    # Weight decay
    weight_decay: 0.0005
    # Optimizer type
    type: SGD

# Evaluation related options
eval:
  # Frequency of evaluation
  freq: 1
  # Evaluation metrics, accuracy and number of correct items
  metrics: ['acc', 'correct']

Start with customized functions

FS-G also provides register function to set up the FL. Here we provide an example about how to run your own model and data to FS-G.

• Load your data (write in federatedscope/contrib/data/):

  import copy
  import numpy as np
  
  from torch_geometric.datasets import Planetoid
  from federatedscope.core.splitters.graph import LouvainSplitter
  from federatedscope.register import register_data
  
  
  def my_cora(config=None):
      path = config.data.root
  
      num_split = [232, 542, np.iinfo(np.int64).max]
      dataset = Planetoid(path,
                          'cora',
                          split='random',
                          num_train_per_class=num_split[0],
                          num_val=num_split[1],
                          num_test=num_split[2])
      global_data = copy.deepcopy(dataset)[0]
      dataset = LouvainSplitter(config.federate.client_num)(dataset[0])
  
      data_local_dict = dict()
      for client_idx in range(len(dataset)):
          data_local_dict[client_idx + 1] = dataset[client_idx]
  
      data_local_dict[0] = global_data
      return data_local_dict, config
  
  
  def call_my_data(config):
      if config.data.type == "mycora":
          data, modified_config = my_cora(config)
          return data, modified_config
  
  
  register_data("mycora", call_my_data)
  
  

• Build your model (write in federatedscope/contrib/model/):

  import torch
  import torch.nn.functional as F
  
  from torch.nn import ModuleList
  from torch_geometric.data import Data
  from torch_geometric.nn import GCNConv
  from federatedscope.register import register_model
  
  
  class MyGCN(torch.nn.Module):
      def __init__(self,
                   in_channels,
                   out_channels,
                   hidden=64,
                   max_depth=2,
                   dropout=.0):
          super(MyGCN, self).__init__()
          self.convs = ModuleList()
          for i in range(max_depth):
              if i == 0:
                  self.convs.append(GCNConv(in_channels, hidden))
              elif (i + 1) == max_depth:
                  self.convs.append(GCNConv(hidden, out_channels))
              else:
                  self.convs.append(GCNConv(hidden, hidden))
          self.dropout = dropout
  
      def forward(self, data):
          if isinstance(data, Data):
              x, edge_index = data.x, data.edge_index
          elif isinstance(data, tuple):
              x, edge_index = data
          else:
              raise TypeError('Unsupported data type!')
  
          for i, conv in enumerate(self.convs):
              x = conv(x, edge_index)
              if (i + 1) == len(self.convs):
                  break
              x = F.relu(F.dropout(x, p=self.dropout, training=self.training))
          return x
  
  
  def gcnbuilder(model_config, input_shape):
      x_shape, num_label, num_edge_features = input_shape
      model = MyGCN(x_shape[-1],
                    model_config.out_channels,
                    hidden=model_config.hidden,
                    max_depth=model_config.layer,
                    dropout=model_config.dropout)
      return model
  
  
  def call_my_net(model_config, local_data):
      # Please name your gnn model with prefix 'gnn_'
      if model_config.type == "gnn_mygcn":
          model = gcnbuilder(model_config, local_data)
          return model
  
  
  register_model("gnn_mygcn", call_my_net)
  
  

• Run with following command to start:

  python federatedscope/main.py --cfg federatedscope/gfl/baseline/example.yaml data.type mycora model.type gnn_mygcn
  

Publications

If you find FS-G useful for research or development, please cite the following paper:

@inproceedings{federatedscopegnn,
  title     = {FederatedScope-GNN: Towards a Unified, Comprehensive and Efficient Package for Federated Graph Learning},
  author    = {Zhen Wang and Weirui Kuang and Yuexiang Xie and Liuyi Yao and Yaliang Li and Bolin Ding and Jingren Zhou},
  booktitle = {Proc.\ of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'22)},
  year      = {2022}
}