Skip to content
This repository was archived by the owner on Jan 8, 2020. It is now read-only.
This repository was archived by the owner on Jan 8, 2020. It is now read-only.

--concurrency option seems to cause non-deterministic I/O errors #190

@hkeeler

Description

@hkeeler

I have not been able to confirm this bug, but through a fair amount of observation, it seems errors crop up when --concurrency or -c is set to more that 1. They are generally I/O errors, such as FTP download and file unzipping, and the errors happen for different files and FTP sites on subsequent runs of the process, but at least one usually occurs on a given run of the loader. I have attempted a couple runs of the loader with -c 1, and no errors occurred.

Also of note, I've only seen this while running in a Docker container. I don't see how that would cause this, but seemed worth mentioning.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions