Missing requests dependency or needs refactor

[cgabriel]

The test_http_backend fails due to missing aiohttp and requests modules. I see that requests has been replaced with urllib to avoid external dependency, so it's either a missing dependency or the need for a refactor

[genro]

Hi @cgabriel,

I've tested the HTTP backend and all tests are passing successfully:

$ pytest tests/test_http_backend.py -v
========================= 4 passed, 1 warning in 1.53s =========================

The code correctly uses urllib (standard library) and doesn't require requests. The aiohttp dependency is already present in optional dependencies:

[project.optional-dependencies]
http = [
    "aiohttp>=3.8.0",
]

Could you provide more details on when/how the test fails?

• What command are you running?
• Are you running with specific optional dependencies installed?
• Full error traceback?

This might have been fixed in recent commits or could be environment-specific.

[cgabriel]

Requests is used here:

genro-storage/tests/test_additional_backends.py

Line 69 in f4b3130

import requests

Tests fails anyway they're invoked, by running pytest manually or using the provided Makefile.

All dependencies listed in the project are installed (clean env before each run, to test the dependencies).

Here's the full error traceback:

_________________________________________ TestReadOnlyBackendValidation.test_http_delete_fails _________________________________________

protocol = 'http'

    def get_filesystem_class(protocol):
        """Fetch named protocol implementation from the registry
    
        The dict ``known_implementations`` maps protocol names to the locations
        of classes implementing the corresponding file-system. When used for the
        first time, appropriate imports will happen and the class will be placed in
        the registry. All subsequent calls will fetch directly from the registry.
    
        Some protocol implementations require additional dependencies, and so the
        import may fail. In this case, the string in the "err" field of the
        ``known_implementations`` will be given as the error message.
        """
        if not protocol:
            protocol = default
    
        if protocol not in registry:
            if protocol not in known_implementations:
                raise ValueError(f"Protocol not known: {protocol}")
            bit = known_implementations[protocol]
            try:
>               register_implementation(protocol, _import_class(bit["class"]))
                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^

.env/lib/python3.13/site-packages/fsspec/registry.py:261: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.env/lib/python3.13/site-packages/fsspec/registry.py:296: in _import_class
    mod = importlib.import_module(mod)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
/usr/lib/python3.13/importlib/__init__.py:88: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    import asyncio
    import io
    import logging
    import re
    import weakref
    from copy import copy
    from urllib.parse import urlparse
    
>   import aiohttp
E   ModuleNotFoundError: No module named 'aiohttp'

.env/lib/python3.13/site-packages/fsspec/implementations/http.py:9: ModuleNotFoundError

The above exception was the direct cause of the following exception:

self = <tests.test_backend_permissions.TestReadOnlyBackendValidation object at 0x7f93b6fc95b0>

    def test_http_delete_fails(self):
        """HTTP backend cannot be configured with delete permission."""
        storage = StorageManager()
    
        with pytest.raises(StorageConfigError, match="read-only"):
>           storage.configure([{
                'name': 'http',
                'type': 'http',
                'base_url': 'http://example.com',
                'permissions': 'delete'
            }])

tests/test_backend_permissions.py:236: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
genro_storage/manager.py:226: in configure
    self._configure_mount(config)
genro_storage/manager.py:439: in _configure_mount
    backend = FsspecBackend('http', base_path=config['base_url'])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
genro_storage/backends/fsspec.py:80: in __init__
    self.fs = fsspec.filesystem(protocol, **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.env/lib/python3.13/site-packages/fsspec/registry.py:321: in filesystem
    cls = get_filesystem_class(protocol)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

protocol = 'http'

    def get_filesystem_class(protocol):
        """Fetch named protocol implementation from the registry
    
        The dict ``known_implementations`` maps protocol names to the locations
        of classes implementing the corresponding file-system. When used for the
        first time, appropriate imports will happen and the class will be placed in
        the registry. All subsequent calls will fetch directly from the registry.
    
        Some protocol implementations require additional dependencies, and so the
        import may fail. In this case, the string in the "err" field of the
        ``known_implementations`` will be given as the error message.
        """
        if not protocol:
            protocol = default
    
        if protocol not in registry:
            if protocol not in known_implementations:
                raise ValueError(f"Protocol not known: {protocol}")
            bit = known_implementations[protocol]
            try:
                register_implementation(protocol, _import_class(bit["class"]))
            except ImportError as e:
>               raise ImportError(bit.get("err")) from e
E               ImportError: HTTPFileSystem requires "requests" and "aiohttp" to be installed

.env/lib/python3.13/site-packages/fsspec/registry.py:263: ImportError

[cgabriel]

This is the error summary:

FAILED tests/test_backend_permissions.py::TestReadOnlyBackendValidation::test_http_readonly_is_ok - ImportError: HTTPFileSystem requires "requests" and "aiohttp" to be installed
FAILED tests/test_backend_permissions.py::TestReadOnlyBackendValidation::test_http_readwrite_fails - ImportError: HTTPFileSystem requires "requests" and "aiohttp" to be installed
FAILED tests/test_backend_permissions.py::TestReadOnlyBackendValidation::test_http_delete_fails - ImportError: HTTPFileSystem requires "requests" and "aiohttp" to be installed