How to use NSRM in HR-Net

[Ericjiejie]

hi, @HowieMa
Thanks for sharing your code, I am very interested in your ideas. You said in readme.md that NSRM can be used for HR-Net, but the network has no intermediate supervision, so how to perform a cascaded multi-task architecture to learn the hand structure and keypoint representations jointly.Could you give some advice?Looking forward to your reply.

[HowieMa]

Thanks for your asking. That's an interesting question. Personally, you can still do it based on a Stacked Hourglass Network. In detail, you can replace each hourglass module to the HRNet module :)

[HowieMa]

That's a good one, but the parameters and computation of the stacked HR-Net will increase significantly compared to the original HR-Net. What do you think？

You are right. However, personally, you just need to stacked the HighResolutionModule in HRNet. Besides, you can custom the HRNet by decreasing the number of branches and number of blocks to reduce the number of parameters in one module.

[Ericjiejie]

Decreasing the number of blocks is indeed an effective way to reduce the number of parameters in one module.But decreasing the branch will also remove the low-resolution representation, I don’t know if this will reduce the accuracy of the network, but i can try to do so.The last very important question, HR-net is an asymmetric network, and the branches are gradually added. Do I still learn the hand structure in the previous part and the keypoint estimation in the latter part, similar to the CPM you use.

[HowieMa]

Decreasing the number of blocks is indeed an effective way to reduce the number of parameters in one module.But decreasing the branch will also remove the low-resolution representation, I don’t know if this will reduce the accuracy of the network, but i can try to do so.The last very important question, HR-net is an asymmetric network, and the branches are gradually added. Do I still learn the hand structure in the previous part and the keypoint estimation in the latter part, similar to the CPM you use.

Personally, maybe you can try a network with two stacked HRNet. Both have half the number of blocks as the default setting. For the first net, you can let it learn the structure representation, while for the second, you can let it learn the keypoints. Anyway, this is just my idea, I am not sure whether it is helpful or not :)

[Ericjiejie]

Thank you very much for your suggestion, I think it is feasible. Excuse me for your precious time, thanks again.

[HowieMa]

Thank you very much for your suggestion, I think it is feasible. Excuse me for your precious time, thanks again.

Thanks for proposing this interesting question. Actually, I haven't thought it carefully when writing the README.md file. Hope it is helpful to you :)