Works in local but not after deployed

[MattiooFR]

Hello again,

My code works in local, I get my summary after 12s, but when I try after deployment, I get 504 Gateway Timeout. Probably because my Lambda function need more than 29s to run and send back the answer. Therefore I tried to use the async:true in the serverless.yml config file but it is still not working and when I look ad AWS logs I dont really understand as I'm not used to debug this.

Maybe you have some insight on how I can make it works ?

[philschmid]

The API-Gateway has a max timeout limit of 29seconds. If you rant into multiple 504, even after the function has loaded your model AWS Lamdba with an API Gateway is probably not suited.

[XiaohanYa]

Hi, thank you for the tutorial. I also encountered the same issue when using BertTokenizer and BertModel ("bert-base-uncased"). It took me 1.836s to run my functions locally, which I think should be fine for deployment. As it's my first time using aws, I wonder if you can share some insights on debugging. Many thanks.

[philschmid]

@XiaohanYa i recommend taking a look at the newest developments: https://www.philschmid.de/serverless-transformers-sagemaker-huggingface
AWS added support for serverless Inference for Amazon SageMaker.