How to load T5 after fine tuning?

[majd-alhafi]

Good morning.

This repo is so great, but I have a little problem after the model is saved how can I load them and generate a sequence?

Thank you.

[samareshyadav55]

Hi Shivanandroy this was a great repo, please tell us the problem how to load the model to predict the text?

[timohromadka]

Hi Shivanand,

I'd also like to know how to load the model after training is done. Is it necessary to load it to HuggingFace? After I finish fine-tuning, my 'model' is no longer callable. Any advice is appreciated.

Thanks

[Shivanandroy]

Hi @timohromadka , @samareshyadav55 - I have packaged this script into a python library for ease of use: SimpleT5
pip install simplet5

# import
from simplet5 import SimpleT5

# instantiate
model = SimpleT5()

# load (supports t5, mt5, byT5 models)
model.from_pretrained("t5","t5-base")

# train
model.train(train_df=train_df, # pandas dataframe with 2 columns: source_text & target_text
            eval_df=eval_df, # pandas dataframe with 2 columns: source_text & target_text
            source_max_token_len = 512, 
            target_max_token_len = 128,
            batch_size = 8,
            max_epochs = 5,
            use_gpu = True,
            outputdir = "outputs",
            early_stopping_patience_epochs = 0,
            precision = 32
            )

# load trained T5 model
model.load_model("t5","path/to/trained/model/directory", use_gpu=False)

# predict
model.predict("input text for prediction")

[samareshyadav55]

thanks for your help shivanandroy 👍

[VishalPallagani]

Hi @Shivanandroy, thank you so much for this repo.
I have made some modifications to the T5 model and would like to use the fine-trained model for inference (instead of simpleT5). It would be really helpful if you can explain how I can use the fine-tuned model for inference and also is there a way I can generate some kind confidence percentage for a prediction during inference.