About large model

[xiang-xiang-zhu]

I want to know how do you maintain the parameters of each large model (such as Bert) in the process of federated learning, such as the fedavg algorithm? Because before server aggregation, if you run federated learning locally, you need to save many model parameters in memory

[sauravpr]

From my understanding, they are held in CPU memory of server, and aggregations are carried out using the model state dictionaries