Problem with rast-process-genome: Gateway Time-out

[lzaramela]

Hi,
I am trying to annotate multiple genomes and I am getting a Gateway Time-out error. Sometimes after several attempts, it gets done, but in many cases, an empty file is created. See error below:

rast-process-genome < genome1.renamed.gto> genome1.renamed.gto.gto2
2019-04-04 09:03:06: Request failed with code=504 msg=Gateway Time-out, sleeping 1 and retrying
2019-04-04 09:04:13: Request failed with code=504 msg=Gateway Time-out, sleeping 2 and retrying
2019-04-04 09:05:19: Request failed with code=504 msg=Gateway Time-out, sleeping 5 and retrying
2019-04-04 09:06:27: Request failed with code=504 msg=Gateway Time-out, sleeping 10 and retrying
2019-04-04 09:11:51: Request succeeded after 4 retries

[jdwinkler-lanzatech]

I'm also having the same problem.

[cssanchez23]

I am also having this problem, how did you all resolve this issue?

[WallyL]

I'm having this same issue today as well. So I guess no one is going to address this?

[olsonanl]

Let me check the service. Are you running single genomes or did you submit a batch?

[WallyL]

I'm just running single genomes

[WallyL]

Two jobs have both failed after 8 requests (restart after previous failures). I'll kill it and wait until I hear something back from you.

[olsonanl]

Can you tell me your IP address so I can track errors in the service logs? I'm seeing running jobs on the service currently; how large are the genomes you are running (both in DNA size and number of contigs)?

[WallyL]

128.192.15.224
Both 5 MB, both single contigs

[WallyL]

Mb

[olsonanl]

I'm having the machine the service is running on rebooted; that should help. There were a bunch of stuck connections it looks like.

[WallyL]

Okay, thank you for getting on this. I've been using the PATRIC/RASTtk service for a few years now and this is the first time I've run into this issue. Perhaps it's getting too popular!
Walt

[WallyL]

I just tried toi run again. Same result....
2020-01-27 14:11:54: Request failed with code=502 msg=Bad Gateway, sleeping 1 and retrying

[olsonanl]

Yes, issue with the machine coming up. Waiting on systems folks.

[olsonanl]

Have you used the PATRIC genome annotation service for analyzing the genomes? It is more robust than the rasttk version (it is much more heavily used).

[WallyL]

I haven't simply because I've been using the rasttk pipeline for a long time and I have multiple gto and export scripts that I plug and play. Looks like I need to upgrade...