Skip to content

Sync Storage sometimes gives database locked error #9

Description

@bovlb

This happens when trying to sync a large number (2000+) of images. Sounds like SQLite database contention. Looks like it only affects one image. Unclear whether it retries.

[2024-06-27 14:32:52,318] [io_storages.base_models::sync::570] [INFO] Start syncing storage ApertureDBExportStorage object (2)
[2024-06-27 14:32:52,338] [io_storages.aperturedb.models::scan_and_create_links::265] [WARNING] Unhandled exception during ApertureDBImportStorage sync:  Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/sqlite3/base.py", line 423, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: database is locked

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/label-studio/label_studio/io_storages/aperturedb/models.py", line 263, in scan_and_create_links
    return self._scan_and_create_links(ApertureDBImportStorageLink)
  File "/label-studio/label_studio/io_storages/base_models.py", line 396, in _scan_and_create_links
    task = self.add_task(data, self.project, maximum_annotations, max_inner_id, self, key, link_class)
  File "/label-studio/label_studio/io_storages/base_models.py", line 320, in add_task
    task = Task.objects.create(
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 453, in create
    obj.save(force_insert=True, using=self.db)
  File "/label-studio/label_studio/tasks/models.py", line 445, in save
    super().save(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/base.py", line 739, in save
    self.save_base(using=using, force_insert=force_insert,
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/base.py", line 776, in save_base
    updated = self._save_table(
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/base.py", line 881, in _save_table
    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/base.py", line 919, in _do_insert
    return manager._insert(
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 1270, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/usr/local/lib/python3.10/dist-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.10/dist-packages/sentry_sdk/integrations/django/__init__.py", line 641, in execute
    result = real_execute(self, sql, params)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 79, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.10/dist-packages/django/db/backends/sqlite3/base.py", line 423, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: database is locked

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions