flush_by_bbox - invalid xml received

[ltclm]

sometimes the following error occurs. p.e. when you try to re-run a bbox delete a second time:

$ .venv/bin/python tool_aws/s3/rm.py --profile default --bucket-name bgdi-tod-cache-dev --prefix /1.0.0/ch.swisstopo.landeskarte-farbe-10/default/current/2056/* --bbox 2672500,1212000,2681250,1218000 --image-format png -n 5 -f
....
INFO:__main__:Deleting 1000 keys at a time
Traceback (most recent call last):
  File "tool_aws/s3/rm.py", line 350, in <module>
    main()
  File "tool_aws/s3/rm.py", line 343, in main
    deleteWithBBox(opts, S3Bucket, keys)
  File "tool_aws/s3/rm.py", line 298, in deleteWithBBox
    callback=callback)
  File "/home/ltclm/tool-aws/.venv/local/lib/python2.7/site-packages/poolmanager/__init__.py", line 57, in imap_unordered
    raise Exception(e)
Exception: Unable to parse response (no element found: line 2, column 0), invalid XML received:
<?xml version="1.0" encoding="UTF-8"?>

[loicgasser]

Seems like a aws response, maybe we can catch this exception and perform a retry. Does it stop the process?

[loicgasser]

With this tool you will reach aws limits in terms of deletion speed. I wonder if their is a way to increase this limit. I had a lot of slowDown response from S3 when testing it.

[ltclm]

the error is always in the end of the process.

[loicgasser]

Thanks it helps.

[loicgasser]

So, I added a bigger timeout when S3 asks to slow down and I am now catching this error and retrying the request.