Skip to content

⚡ Bolt: Parallelize folder deletion #96

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
google-labs-jules wants to merge 1 commit into from
Closed

⚡ Bolt: Parallelize folder deletion #96

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 18 additions & 2 deletions main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -646,11 +646,27 @@
existing_folders = list_existing_folders(client, profile_id)
if not no_delete:
deletion_occurred = False

# Identify folders to delete first
folders_to_delete = []
for folder_data in folder_data_list:
name = folder_data["group"]["group"].strip()
if name in existing_folders:
delete_folder(client, profile_id, name, existing_folders[name])
deletion_occurred = True
folders_to_delete.append((name, existing_folders[name]))

if folders_to_delete:
log.info(f"Deleting {len(folders_to_delete)} folders in parallel...")

Check notice

Code scanning / Pylintpython3 (reported by Codacy)

Use lazy % formatting in logging functions Note

Use lazy % formatting in logging functions

Check warning

Code scanning / Prospector (reported by Codacy)

Use lazy % formatting in logging functions (logging-fstring-interpolation) Warning

Use lazy % formatting in logging functions (logging-fstring-interpolation)
# Parallelize deletion of folders (I/O bound)
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as delete_executor:
futures = [
delete_executor.submit(delete_folder, client, profile_id, name, fid)
for name, fid in folders_to_delete
]
# Wait for all deletions to complete
for future in concurrent.futures.as_completed(futures):
future.result()

deletion_occurred = True

# CRITICAL FIX: Increased wait time for massive folders to clear
if deletion_occurred:
Expand Down
Loading