Attempting to resolve problem where during 2hour runs the token for S…

[mwendigg]

…3 uploads expires

[AWS token expiration resolution]

Key Changes:

• changed aws auth logic for S3 bucket to fetch new token when old one expires during runs

Changed:

• aws code in span.py so that on failure it attempts to get a new token
• matched these changes over to main.py for compatibility

---

Generated Summary:

• Enhanced handling of user data credentials by storing them in an instance variable for easier access.
• Implemented a new _refresh_fs method to refresh the S3 filesystem with updated credentials, improving fault tolerance against expired tokens.
• Introduced a fs_refresh_callback parameter to RunSpan to allow for dynamic filesystem updates during operations.
• Improved error handling in the file writing logic to automatically refresh the filesystem if an "ExpiredToken" error occurs, ensuring smoother operation during interactions with cloud storage.
• Updated main.py to use the new methods and parameters, ensuring consistent retrieval and management of user data credentials across different functions.
• These changes enhance the reliability of the S3 filesystem integration within the application, particularly in scenarios involving credential expiration.

This summary was generated with ❤️ by rigging