Question on validation loss not decreasing

[angerhang]

Thank you so much for this beautiful package allowing us to test it with PyTorch model directly.

I've been playing with this package for various disease risk prediction tasks.

It seems that the validation loss almost never decreases or only decreases for a few epochs.

We observed similar situation even with the provided example that depending on the random seed, the validation loss is difficult to reduce.

Not sure if this is a general issue with deep survival analysis or something else with the problem at hand.

If you have good tips on how to better train the network with torhcsurv that will be great!

[tcoroller]

Hi @angerhang thanks for the kind words and reaching out.

Your comment gave me an idea to better illustrate the link between loss and "data quality" using rho as true latent risk is explained by the covariates. I am simulating a synthetic data under several assumption (rho = 0. no to little signal, rho=1 strong survival survival) and see how the loss and Cindex varies: synthetic data notebook

Based on your comment, this could indicate either that your data contains little signal or that your model is converging very quickly (e.g., due to a small model).

As mentioned previously in response to another issue, we are not able to support general modeling questions unless they are directly related to TorchSurv’s internal mechanisms (such as bugs or unexpected behavior). Providing broader modeling support would not be manageable for us at scale.

Thank you!

[angerhang]

Hi @angerhang thanks for the kind words and reaching out.

Your comment gave me an idea to better illustrate the link between loss and "data quality" using rho as true latent risk is explained by the covariates. I am simulating a synthetic data under several assumption (rho = 0. no to little signal, rho=1 strong survival survival) and see how the loss and Cindex varies: synthetic data notebook

Based on your comment, this could indicate either that your data contains little signal or that your model is converging very quickly (e.g., due to a small model).

As mentioned previously in response to another issue, we are not able to support general modeling questions unless they are directly related to TorchSurv’s internal mechanisms (such as bugs or unexpected behavior). Providing broader modeling support would not be manageable for us at scale.

Thank you!

Ah this is a great notebook demonstration. I guess because deep survival analyses are still a relatively young field. Many people don't have a good intuition regarding its training specifics. Thank you so much for the helpful response!

I am closing this issue as this is indeed not a TorchSurv function issue but more to do with the method.