what if scheduler/server/worker failed?

[lmybill]

Can we restart this node and the training job still works

[eric-haibin-lin]

Not at the moment. https://github.com/awslabs/dynamic-training-with-apache-mxnet-on-aws supports adding/removing workers during training though.