Shared layers problem

[WJW02]

In case of models with shared layers, such as FOMO, server model inference could fail, as it does not limit which layers can be offloaded.