Skip to content

Develop learner multilabel#2

Open
bstellato wants to merge 5 commits intodevelop_learnerfrom
develop_learner_multilabel
Open

Develop learner multilabel#2
bstellato wants to merge 5 commits intodevelop_learnerfrom
develop_learner_multilabel

Conversation

@bstellato
Copy link
Owner

@bstellato bstellato commented Jul 13, 2019

Add option to have multiple labels in the PyTorch learner. This is helpful when the strategy for the integer variables corresponds to the actual integer variables vector. (Similar holds for active constraints).

This PR also adds the n_layers as parameter.

TODO

  • Implement onehot flag to initialize learner. This also adapts the learner cost function.
  • Fix issue with train/evaluation metric during the training!
  • Implement logic to choose from the problem size whether to use the onehot encoding or the multiple labels for active constraints and integer variables.
    • Split the predictors in one for the active constraints and one for the integer variables
    • Smartly choose between onehot (when there are many variables/constraints and few different combinations from data) and multiple labels (small number of variables/constraints and many different combinations data)

juliayyan and others added 5 commits July 12, 2019 13:07
- add optional argument `onehot` (default True) to indicate net type
- change loss function to MultiLabelSoftMarginLoss() if not `onehot`
- add argument to get_dataloader() to change y datatype if not `onehot`
- change predict() function to output y vectors if not `onehot`
@juliayyan
Copy link
Collaborator

So I was running this and got a runtime warning:

RuntimeWarning: overflow encountered in exp
  normalized_outputs = np.reciprocal(1 + np.exp(-outputs))  # Normalize using sigmoid

It's still running so I can't confirm, but I think the issue might be here:
https://github.com/bstellato/mlopt/blob/develop_learner_multilabel/mlopt/learners/pytorch/pytorch.py#L92
https://github.com/bstellato/mlopt/blob/develop_learner_multilabel/mlopt/learners/pytorch/pytorch.py#L104

Unlike the one-hot case, I need to wrap the sigmoid around the net outputs to get 0-1s:
https://github.com/bstellato/mlopt/blob/develop_learner_multilabel/mlopt/learners/pytorch/pytorch.py#L299
which is not being done before the calls to the MSE function.

@bstellato
Copy link
Owner Author

I think it is just a warning. A solution could be this little trick which needs to be adapted to the vectorized case. Another solution is to silence that working inside the function call. In any case it should not affect the output of your code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants