2024.10.0: pytest fails in fsspec/implementations/tests/test_ftp.py units

[kloczek]

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix> using installer module
• run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
• build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-fsspec-2024.10.0-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-fsspec-2024.10.0-2.fc37.x86_64/usr/lib/python3.10/site-packages
+ /usr/bin/pytest -ra -m 'not network'
============================= test session starts ==============================
platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0
configfile: pyproject.toml
plugins: asyncio-0.23.8, mock-3.14.0, vcr-1.0.2
asyncio: mode=strict
collected 1737 items / 9 skipped

fsspec/implementations/tests/local/local_test.py ....................... [  1%]
........................................................................ [  5%]
........................................                                 [  7%]
fsspec/implementations/tests/memory/memory_test.py ..................... [  8%]
........................................................................ [ 13%]
..........................................                               [ 15%]
fsspec/implementations/tests/test_archive.py ........................... [ 17%]
..........................................................ssssssssssssss [ 21%]
sss                                                                      [ 21%]
fsspec/implementations/tests/test_cached.py ............................ [ 23%]
..............ssssssssssss................................               [ 26%]
fsspec/implementations/tests/test_common.py ssss                         [ 26%]
fsspec/implementations/tests/test_data.py ..                             [ 26%]
fsspec/implementations/tests/test_dirfs.py ............................. [ 28%]
........................................................................ [ 32%]
.................................                                        [ 34%]
fsspec/implementations/tests/test_ftp.py FFFFFFFF...........             [ 35%]
fsspec/implementations/tests/test_http.py .............................. [ 37%]
..........................                                               [ 38%]
fsspec/implementations/tests/test_libarchive.py s                        [ 38%]
fsspec/implementations/tests/test_local.py .s........................... [ 40%]
.s............................................................ss........ [ 44%]
ss.sssss.......sss....s..........................                        [ 47%]
fsspec/implementations/tests/test_memory.py ............................ [ 49%]
..                                                                       [ 49%]
fsspec/implementations/tests/test_reference.py .................s.....ss [ 50%]
..sss                                                                    [ 50%]
fsspec/implementations/tests/test_sftp.py ssssssssssss                   [ 51%]
fsspec/implementations/tests/test_tar.py .........................       [ 53%]
fsspec/implementations/tests/test_webhdfs.py sssssssssss                 [ 53%]
fsspec/implementations/tests/test_zip.py .................               [ 54%]
fsspec/tests/test_api.py ...............x......                          [ 55%]
fsspec/tests/test_async.py .........s...                                 [ 56%]
fsspec/tests/test_caches.py ............................................ [ 59%]
........................................................................ [ 63%]
.............................                                            [ 64%]
fsspec/tests/test_callbacks.py ........                                  [ 65%]
fsspec/tests/test_compression.py .....s                                  [ 65%]
fsspec/tests/test_config.py .......                                      [ 66%]
fsspec/tests/test_core.py .............................................. [ 68%]
...s.......s                                                             [ 69%]
fsspec/tests/test_file.py ...........                                    [ 70%]
fsspec/tests/test_generic.py ......                                      [ 70%]
fsspec/tests/test_mapping.py .................                           [ 71%]
fsspec/tests/test_parquet.py sssssssssssssssssssssssssssssssssssssssssss [ 73%]
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 78%]
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 82%]
sssss                                                                    [ 82%]
fsspec/tests/test_registry.py ......s                                    [ 82%]
fsspec/tests/test_spec.py ....................x......................... [ 85%]
........................................................................ [ 89%]
........................................................................ [ 93%]
.......................................                                  [ 96%]
fsspec/tests/test_utils.py ............................................. [ 98%]
......................                                                   [100%]

=================================== FAILURES ===================================
_____________________________ test_tls[False-FTP] ______________________________

ftp = ('localhost', 2121), tls = False, exp_cls = <class 'ftplib.FTP'>

    @pytest.mark.parametrize(
        "tls,exp_cls",
        (
            (False, FTP),
            (True, FTP_TLS),
        ),
    )
    def test_tls(ftp, tls, exp_cls):
        host, port = ftp
>       fs = FTPFileSystem(host, port, tls=tls)

/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:42:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__
    obj = super().__call__(*args, **kwargs)
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__
    self._connect()
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect
    self.ftp.connect(self.host, self.port)
/usr/lib64/python3.10/ftplib.py:158: in connect
    self.sock = socket.create_connection((self.host, self.port), self.timeout,
/usr/lib64/python3.10/socket.py:845: in create_connection
    raise err
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

address = ('localhost', 2121), timeout = 30, source_address = None

    def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
                          source_address=None):
        """Connect to *address* and return the socket object.

        Convenience function.  Connect to *address* (a 2-tuple ``(host,
        port)``) and return the socket object.  Passing the optional
        *timeout* parameter will set the timeout on the socket instance
        before attempting to connect.  If no *timeout* is supplied, the
        global default timeout setting returned by :func:`getdefaulttimeout`
        is used.  If *source_address* is set it must be a tuple of (host, port)
        for the socket to bind as a source address before making the connection.
        A host of '' or port 0 tells the OS to use the default.
        """

        host, port = address
        err = None
        for res in getaddrinfo(host, port, 0, SOCK_STREAM):
            af, socktype, proto, canonname, sa = res
            sock = None
            try:
                sock = socket(af, socktype, proto)
                if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
                    sock.settimeout(timeout)
                if source_address:
                    sock.bind(source_address)
>               sock.connect(sa)
E               ConnectionRefusedError: [Errno 111] Connection refused

/usr/lib64/python3.10/socket.py:833: ConnectionRefusedError
____________________________ test_tls[True-FTP_TLS] ____________________________

ftp = ('localhost', 2121), tls = True, exp_cls = <class 'ftplib.FTP_TLS'>

    @pytest.mark.parametrize(
        "tls,exp_cls",
        (
            (False, FTP),
            (True, FTP_TLS),
        ),
    )
    def test_tls(ftp, tls, exp_cls):
        host, port = ftp
>       fs = FTPFileSystem(host, port, tls=tls)

/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:42:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__
    obj = super().__call__(*args, **kwargs)
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__
    self._connect()
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect
    self.ftp.connect(self.host, self.port)
/usr/lib64/python3.10/ftplib.py:158: in connect
    self.sock = socket.create_connection((self.host, self.port), self.timeout,
/usr/lib64/python3.10/socket.py:845: in create_connection
    raise err
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

address = ('localhost', 2121), timeout = 30, source_address = None

    def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
                          source_address=None):
        """Connect to *address* and return the socket object.

        Convenience function.  Connect to *address* (a 2-tuple ``(host,
        port)``) and return the socket object.  Passing the optional
        *timeout* parameter will set the timeout on the socket instance
        before attempting to connect.  If no *timeout* is supplied, the
        global default timeout setting returned by :func:`getdefaulttimeout`
        is used.  If *source_address* is set it must be a tuple of (host, port)
        for the socket to bind as a source address before making the connection.
        A host of '' or port 0 tells the OS to use the default.
        """

        host, port = address
        err = None
        for res in getaddrinfo(host, port, 0, SOCK_STREAM):
            af, socktype, proto, canonname, sa = res
            sock = None
            try:
                sock = socket(af, socktype, proto)
                if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
                    sock.settimeout(timeout)
                if source_address:
                    sock.bind(source_address)
>               sock.connect(sa)
E               ConnectionRefusedError: [Errno 111] Connection refused

/usr/lib64/python3.10/socket.py:833: ConnectionRefusedError
_____________________________ test_basic[False--] ______________________________

ftp = ('localhost', 2121), tls = False, username = '', password = ''

    @pytest.mark.parametrize(
        "tls,username,password",
        (
            (False, "", ""),
            (True, "", ""),
            (False, "user", "pass"),
            (True, "user", "pass"),
        ),
    )
    def test_basic(ftp, tls, username, password):
        host, port = ftp
>       fs = FTPFileSystem(host, port, username, password, tls=tls)

/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:57:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__
    obj = super().__call__(*args, **kwargs)
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__
    self._connect()
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect
    self.ftp.connect(self.host, self.port)
/usr/lib64/python3.10/ftplib.py:158: in connect
    self.sock = socket.create_connection((self.host, self.port), self.timeout,
/usr/lib64/python3.10/socket.py:845: in create_connection
    raise err
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

address = ('localhost', 2121), timeout = 30, source_address = None

    def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
                          source_address=None):
        """Connect to *address* and return the socket object.

        Convenience function.  Connect to *address* (a 2-tuple ``(host,
        port)``) and return the socket object.  Passing the optional
        *timeout* parameter will set the timeout on the socket instance
        before attempting to connect.  If no *timeout* is supplied, the
        global default timeout setting returned by :func:`getdefaulttimeout`
        is used.  If *source_address* is set it must be a tuple of (host, port)
        for the socket to bind as a source address before making the connection.
        A host of '' or port 0 tells the OS to use the default.
        """

        host, port = address
        err = None
        for res in getaddrinfo(host, port, 0, SOCK_STREAM):
            af, socktype, proto, canonname, sa = res
            sock = None
            try:
                sock = socket(af, socktype, proto)
                if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
                    sock.settimeout(timeout)
                if source_address:
                    sock.bind(source_address)
>               sock.connect(sa)
E               ConnectionRefusedError: [Errno 111] Connection refused

/usr/lib64/python3.10/socket.py:833: ConnectionRefusedError
______________________________ test_basic[True--] ______________________________

ftp = ('localhost', 2121), tls = True, username = '', password = ''

    @pytest.mark.parametrize(
        "tls,username,password",
        (
            (False, "", ""),
            (True, "", ""),
            (False, "user", "pass"),
            (True, "user", "pass"),
        ),
    )
    def test_basic(ftp, tls, username, password):
        host, port = ftp
>       fs = FTPFileSystem(host, port, username, password, tls=tls)

/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:57:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__
    obj = super().__call__(*args, **kwargs)
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__
    self._connect()
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect
    self.ftp.connect(self.host, self.port)
/usr/lib64/python3.10/ftplib.py:158: in connect
    self.sock = socket.create_connection((self.host, self.port), self.timeout,
/usr/lib64/python3.10/socket.py:845: in create_connection
    raise err
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

address = ('localhost', 2121), timeout = 30, source_address = None

    def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
                          source_address=None):
        """Connect to *address* and return the socket object.

        Convenience function.  Connect to *address* (a 2-tuple ``(host,
        port)``) and return the socket object.  Passing the optional
        *timeout* parameter will set the timeout on the socket instance
        before attempting to connect.  If no *timeout* is supplied, the
        global default timeout setting returned by :func:`getdefaulttimeout`
        is used.  If *source_address* is set it must be a tuple of (host, port)
        for the socket to bind as a source address before making the connection.
        A host of '' or port 0 tells the OS to use the default.
        """

        host, port = address
        err = None
        for res in getaddrinfo(host, port, 0, SOCK_STREAM):
            af, socktype, proto, canonname, sa = res
            sock = None
            try:
                sock = socket(af, socktype, proto)
                if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
                    sock.settimeout(timeout)
                if source_address:
                    sock.bind(source_address)
>               sock.connect(sa)
E               ConnectionRefusedError: [Errno 111] Connection refused

/usr/lib64/python3.10/socket.py:833: ConnectionRefusedError
_________________________ test_basic[False-user-pass] __________________________

ftp = ('localhost', 2121), tls = False, username = 'user', password = 'pass'

    @pytest.mark.parametrize(
        "tls,username,password",
        (
            (False, "", ""),
            (True, "", ""),
            (False, "user", "pass"),
            (True, "user", "pass"),
        ),
    )
    def test_basic(ftp, tls, username, password):
        host, port = ftp
>       fs = FTPFileSystem(host, port, username, password, tls=tls)

/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:57:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__
    obj = super().__call__(*args, **kwargs)
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__
    self._connect()
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect
    self.ftp.connect(self.host, self.port)
/usr/lib64/python3.10/ftplib.py:158: in connect
    self.sock = socket.create_connection((self.host, self.port), self.timeout,
/usr/lib64/python3.10/socket.py:845: in create_connection
    raise err
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

address = ('localhost', 2121), timeout = 30, source_address = None

    def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
                          source_address=None):
        """Connect to *address* and return the socket object.

        Convenience function.  Connect to *address* (a 2-tuple ``(host,
        port)``) and return the socket object.  Passing the optional
        *timeout* parameter will set the timeout on the socket instance
        before attempting to connect.  If no *timeout* is supplied, the
        global default timeout setting returned by :func:`getdefaulttimeout`
        is used.  If *source_address* is set it must be a tuple of (host, port)
        for the socket to bind as a source address before making the connection.
        A host of '' or port 0 tells the OS to use the default.
        """

        host, port = address
        err = None
        for res in getaddrinfo(host, port, 0, SOCK_STREAM):
            af, socktype, proto, canonname, sa = res
            sock = None
            try:
                sock = socket(af, socktype, proto)
                if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
                    sock.settimeout(timeout)
                if source_address:
                    sock.bind(source_address)
>               sock.connect(sa)
E               ConnectionRefusedError: [Errno 111] Connection refused

/usr/lib64/python3.10/socket.py:833: ConnectionRefusedError
__________________________ test_basic[True-user-pass] __________________________

ftp = ('localhost', 2121), tls = True, username = 'user', password = 'pass'

    @pytest.mark.parametrize(
        "tls,username,password",
        (
            (False, "", ""),
            (True, "", ""),
            (False, "user", "pass"),
            (True, "user", "pass"),
        ),
    )
    def test_basic(ftp, tls, username, password):
        host, port = ftp
>       fs = FTPFileSystem(host, port, username, password, tls=tls)

/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:57:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__
    obj = super().__call__(*args, **kwargs)
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__
    self._connect()
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect
    self.ftp.connect(self.host, self.port)
/usr/lib64/python3.10/ftplib.py:158: in connect
    self.sock = socket.create_connection((self.host, self.port), self.timeout,
/usr/lib64/python3.10/socket.py:845: in create_connection
    raise err
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

address = ('localhost', 2121), timeout = 30, source_address = None

    def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
                          source_address=None):
        """Connect to *address* and return the socket object.

        Convenience function.  Connect to *address* (a 2-tuple ``(host,
        port)``) and return the socket object.  Passing the optional
        *timeout* parameter will set the timeout on the socket instance
        before attempting to connect.  If no *timeout* is supplied, the
        global default timeout setting returned by :func:`getdefaulttimeout`
        is used.  If *source_address* is set it must be a tuple of (host, port)
        for the socket to bind as a source address before making the connection.
        A host of '' or port 0 tells the OS to use the default.
        """

        host, port = address
        err = None
        for res in getaddrinfo(host, port, 0, SOCK_STREAM):
            af, socktype, proto, canonname, sa = res
            sock = None
            try:
                sock = socket(af, socktype, proto)
                if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
                    sock.settimeout(timeout)
                if source_address:
                    sock.bind(source_address)
>               sock.connect(sa)
E               ConnectionRefusedError: [Errno 111] Connection refused

/usr/lib64/python3.10/socket.py:833: ConnectionRefusedError
_______________________________ test_not_cached ________________________________

ftp = ('localhost', 2121)

    def test_not_cached(ftp):
        host, port = ftp
>       fs = FTPFileSystem(host, port)

/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:65:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__
    obj = super().__call__(*args, **kwargs)
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__
    self._connect()
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect
    self.ftp.connect(self.host, self.port)
/usr/lib64/python3.10/ftplib.py:158: in connect
    self.sock = socket.create_connection((self.host, self.port), self.timeout,
/usr/lib64/python3.10/socket.py:845: in create_connection
    raise err
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

address = ('localhost', 2121), timeout = 30, source_address = None

    def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
                          source_address=None):
        """Connect to *address* and return the socket object.

        Convenience function.  Connect to *address* (a 2-tuple ``(host,
        port)``) and return the socket object.  Passing the optional
        *timeout* parameter will set the timeout on the socket instance
        before attempting to connect.  If no *timeout* is supplied, the
        global default timeout setting returned by :func:`getdefaulttimeout`
        is used.  If *source_address* is set it must be a tuple of (host, port)
        for the socket to bind as a source address before making the connection.
        A host of '' or port 0 tells the OS to use the default.
        """

        host, port = address
        err = None
        for res in getaddrinfo(host, port, 0, SOCK_STREAM):
            af, socktype, proto, canonname, sa = res
            sock = None
            try:
                sock = socket(af, socktype, proto)
                if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
                    sock.settimeout(timeout)
                if source_address:
                    sock.bind(source_address)
>               sock.connect(sa)
E               ConnectionRefusedError: [Errno 111] Connection refused

/usr/lib64/python3.10/socket.py:833: ConnectionRefusedError
____________________________ test_ls_root_dircache _____________________________

ftp = ('localhost', 2121)

    def test_ls_root_dircache(ftp):
        host, port = ftp
>       fs = FTPFileSystem(host, port)

/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/tests/test_ftp.py:72:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/spec.py:81: in __call__
    obj = super().__call__(*args, **kwargs)
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:75: in __init__
    self._connect()
/home/tkloczko/rpmbuild/BUILD/filesystem_spec-2024.10.0/fsspec/implementations/ftp.py:91: in _connect
    self.ftp.connect(self.host, self.port)
/usr/lib64/python3.10/ftplib.py:158: in connect
    self.sock = socket.create_connection((self.host, self.port), self.timeout,
/usr/lib64/python3.10/socket.py:845: in create_connection
    raise err
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

address = ('localhost', 2121), timeout = 30, source_address = None

    def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
                          source_address=None):
        """Connect to *address* and return the socket object.

        Convenience function.  Connect to *address* (a 2-tuple ``(host,
        port)``) and return the socket object.  Passing the optional
        *timeout* parameter will set the timeout on the socket instance
        before attempting to connect.  If no *timeout* is supplied, the
        global default timeout setting returned by :func:`getdefaulttimeout`
        is used.  If *source_address* is set it must be a tuple of (host, port)
        for the socket to bind as a source address before making the connection.
        A host of '' or port 0 tells the OS to use the default.
        """

        host, port = address
        err = None
        for res in getaddrinfo(host, port, 0, SOCK_STREAM):
            af, socktype, proto, canonname, sa = res
            sock = None
            try:
                sock = socket(af, socktype, proto)
                if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
                    sock.settimeout(timeout)
                if source_address:
                    sock.bind(source_address)
>               sock.connect(sa)
E               ConnectionRefusedError: [Errno 111] Connection refused

/usr/lib64/python3.10/socket.py:833: ConnectionRefusedError
================================== XFAILURES ===================================
__________________________ test_multilevel_chained_fs __________________________

    @pytest.mark.xfail(reason="see issue #334", strict=True)
    def test_multilevel_chained_fs():
        """This test reproduces fsspec/filesystem_spec#334"""
        import zipfile

        d1 = tempfile.mkdtemp()
        f1 = os.path.join(d1, "f1.zip")
        with zipfile.ZipFile(f1, mode="w") as z:
            # filename, content
            z.writestr("foo.txt", "foo.txt")
            z.writestr("bar.txt", "bar.txt")

        # We expected this to be the correct syntax
>       with pytest.raises(IsADirectoryError):
E       Failed: DID NOT RAISE <class 'IsADirectoryError'>

fsspec/tests/test_api.py:252: Failed
__________________________________ test_find ___________________________________

    @pytest.mark.xfail
    def test_find():
        """Test .find() method on debian server (ftp, https) with constant folder"""
        filesystem, host, test_path = (
            FTPFileSystem,
            "ftp.fau.de",
            "ftp://ftp.fau.de/debian-cd/current/amd64/log/success",
        )
        test_fs = filesystem(host)
        filenames_ftp = test_fs.find(test_path)
>       assert filenames_ftp
E       assert []

fsspec/tests/test_spec.py:699: AssertionError
=============================== warnings summary ===============================
fsspec/tests/test_async.py::test_run_coros_in_chunks
  /usr/lib64/python3.10/asyncio/tasks.py:596: RuntimeWarning: coroutine 'test_run_coros_in_chunks.<locals>.runner' was never awaited
    await __sleep0()
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

fsspec/tests/test_async.py::test_async_streamed_file_write
  /usr/lib64/python3.10/functools.py:58: RuntimeWarning: coroutine 'test_run_coros_in_chunks.<locals>.runner' was never awaited
    getattr(wrapper, attr).update(getattr(wrapped, attr, {}))
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] fsspec/implementations/tests/test_arrow.py:5: could not import 'pyarrow.fs': No module named 'pyarrow'
SKIPPED [1] fsspec/implementations/tests/test_dask.py:5: could not import 'distributed': No module named 'distributed'
SKIPPED [1] fsspec/implementations/tests/test_dbfs.py:35: These tests need to be re-recorded.
SKIPPED [1] fsspec/implementations/tests/test_git.py:11: could not import 'pygit2': No module named 'pygit2'
SKIPPED [1] fsspec/implementations/tests/test_jupyter.py:10: could not import 'notebook': No module named 'notebook'
SKIPPED [1] fsspec/implementations/tests/test_smb.py:15: could not import 'smbprotocol': No module named 'smbprotocol'
SKIPPED [1] fsspec/tests/test_downstream.py:3: could not import 's3fs': No module named 's3fs'
SKIPPED [1] fsspec/tests/test_fuse.py:9: could not import 'fuse': No module named 'fuse'
SKIPPED [1] fsspec/tests/test_gui.py:3: could not import 'panel': No module named 'panel'
SKIPPED [18] fsspec/implementations/tests/test_archive.py:45: could not import 'libarchive': No module named 'libarchive'
SKIPPED [6] fsspec/implementations/tests/test_cached.py:878: could not import 'pyarrow.fs': No module named 'pyarrow'
SKIPPED [6] fsspec/implementations/tests/test_cached.py:892: could not import 'pyarrow.fs': No module named 'pyarrow'
SKIPPED [1] fsspec/implementations/tests/test_common.py:10: could not import 'pyarrow.fs': No module named 'pyarrow'
SKIPPED [3] fsspec/implementations/tests/test_common.py:21: could not import 'pyarrow.fs': No module named 'pyarrow'
SKIPPED [1] fsspec/implementations/tests/test_local.py:135: could not import 's3fs': No module named 's3fs'
SKIPPED [1] fsspec/implementations/tests/test_local.py:287: could not import 'cloudpickle': No module named 'cloudpickle'
SKIPPED [2] fsspec/implementations/tests/test_local.py:785: Windows only
SKIPPED [2] fsspec/implementations/tests/test_local.py:801: Windows only
SKIPPED [5] fsspec/implementations/tests/test_local.py:823: Windows only
SKIPPED [3] fsspec/implementations/tests/test_local.py:880: Windows only
SKIPPED [1] fsspec/implementations/tests/test_local.py:895: Windows only
SKIPPED [1] fsspec/implementations/tests/test_reference.py:433: could not import 'zarr': No module named 'zarr'
SKIPPED [1] fsspec/implementations/tests/test_reference.py:579: could not import 'fastparquet': No module named 'fastparquet'
SKIPPED [1] fsspec/implementations/tests/test_reference.py:625: could not import 'fastparquet': No module named 'fastparquet'
SKIPPED [1] fsspec/implementations/tests/test_reference.py:733: could not import 'zarr': No module named 'zarr'
SKIPPED [2] fsspec/implementations/tests/test_reference.py:766: could not import 'kerchunk': No module named 'kerchunk'
SKIPPED [1] fsspec/implementations/tests/test_sftp.py:71: docker run not available
SKIPPED [2] fsspec/implementations/tests/test_sftp.py:86: docker run not available
SKIPPED [2] fsspec/implementations/tests/test_sftp.py:104: docker run not available
SKIPPED [1] fsspec/implementations/tests/test_sftp.py:143: docker run not available
SKIPPED [1] fsspec/implementations/tests/test_sftp.py:152: docker run not available
SKIPPED [1] fsspec/implementations/tests/test_sftp.py:183: docker run not available
SKIPPED [2] fsspec/implementations/tests/test_sftp.py:205: docker run not available
SKIPPED [2] fsspec/implementations/tests/test_sftp.py:222: docker run not available
SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:43: htcluster not found
SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:49: htcluster not found
SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:57: htcluster not found
SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:67: htcluster not found
SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:84: htcluster not found
SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:100: htcluster not found
SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:121: htcluster not found
SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:140: htcluster not found
SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:153: htcluster not found
SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:163: htcluster not found
SKIPPED [1] fsspec/implementations/tests/test_webhdfs.py:177: htcluster not found
SKIPPED [1] fsspec/tests/test_async.py:134: only for windows
SKIPPED [1] fsspec/tests/test_compression.py:134: could not import 'snappy': No module named 'snappy'
SKIPPED [1] fsspec/tests/test_core.py:357: could not import 's3fs': No module named 's3fs'
SKIPPED [1] fsspec/tests/test_core.py:471: could not import 'adlfs': No module named 'adlfs'
SKIPPED [64] fsspec/tests/test_parquet.py:37: fastparquet not found
SKIPPED [64] fsspec/tests/test_parquet.py:37: pyarrow not found
SKIPPED [64] fsspec/tests/test_parquet.py:37: No parquet engine (fastparquet or pyarrow) found
SKIPPED [1] fsspec/tests/test_registry.py:129: could not import 's3fs': No module named 's3fs'
XFAIL fsspec/tests/test_api.py::test_multilevel_chained_fs - see issue #334
XFAIL fsspec/tests/test_spec.py::test_find
FAILED fsspec/implementations/tests/test_ftp.py::test_tls[False-FTP] - Connec...
FAILED fsspec/implementations/tests/test_ftp.py::test_tls[True-FTP_TLS] - Con...
FAILED fsspec/implementations/tests/test_ftp.py::test_basic[False--] - Connec...
FAILED fsspec/implementations/tests/test_ftp.py::test_basic[True--] - Connect...
FAILED fsspec/implementations/tests/test_ftp.py::test_basic[False-user-pass]
FAILED fsspec/implementations/tests/test_ftp.py::test_basic[True-user-pass]
FAILED fsspec/implementations/tests/test_ftp.py::test_not_cached - Connection...
FAILED fsspec/implementations/tests/test_ftp.py::test_ls_root_dircache - Conn...
= 8 failed, 1452 passed, 284 skipped, 2 xfailed, 2 warnings in 103.85s (0:01:43) =
/usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-970f0ce1-2bfe-4e93-9c9c-e0210285fe29/test_discovery_via_path_in_non0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_discovery_via_path_in_non0'
  warnings.warn(
/usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-970f0ce1-2bfe-4e93-9c9c-e0210285fe29
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-970f0ce1-2bfe-4e93-9c9c-e0210285fe29'
  warnings.warn(
/usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-eec6b57a-4bc2-40fa-8c57-7bef6d50de7c/test_discovery_via_path_in_non0
<class 'OSError'>: [Errno 39] Directory not empty: 'test_discovery_via_path_in_non0'
  warnings.warn(
/usr/lib/python3.10/site-packages/_pytest/pathlib.py:98: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-eec6b57a-4bc2-40fa-8c57-7bef6d50de7c
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-eec6b57a-4bc2-40fa-8c57-7bef6d50de7c'
  warnings.warn(
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x7ff14ad77ee0>

List of installed modules in build env:

Package                       Version
----------------------------- -----------
aiohappyeyeballs              2.4.3
aiohttp                       3.10.10
aiosignal                     1.3.1
alabaster                     0.7.16
async-timeout                 4.0.3
attrs                         24.2.0
babel                         2.16.0
bcrypt                        4.2.0
build                         1.2.2.post1
cffi                          1.17.1
charset-normalizer            3.4.0
cryptography                  42.0.8
defusedxml                    0.7.1
distro                        1.9.0
docutils                      0.21.2
exceptiongroup                1.1.3
frozenlist                    1.4.1
hatch-vcs                     0.4.0
hatchling                     1.25.0
imagesize                     1.4.1
importlib_metadata            8.5.0
iniconfig                     2.0.0
installer                     0.7.0
Jinja2                        3.1.4
lz4                           4.3.3
MarkupSafe                    3.0.2
multidict                     6.1.0
numpy                         1.26.4
numpydoc                      1.7.0
packaging                     24.0
pandas                        2.2.1
paramiko                      3.5.0
pathspec                      0.12.1
pluggy                        1.5.0
ply                           3.11
pycparser                     2.22
pyftpdlib                     1.5.7
Pygments                      2.18.0
PyNaCl                        1.5.0
pyproject_hooks               1.2.0
pytest                        8.2.2
pytest-asyncio                0.23.8
pytest-mock                   3.14.0
pytest-vcr                    1.0.2
python-dateutil               2.9.0.post0
pytz                          2024.2
PyYAML                        6.0.2
requests                      2.32.3
setuptools                    75.1.0
setuptools-scm                8.1.0
snowballstemmer               2.2.0
Sphinx                        8.1.3
sphinx_design                 0.6.1
sphinxcontrib-applehelp       2.0.0
sphinxcontrib-devhelp         1.0.6
sphinxcontrib-htmlhelp        2.1.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          2.0.0
sphinxcontrib-serializinghtml 2.0.0
tabulate                      0.9.0
tokenize_rt                   6.0.0
tomli                         2.0.1
tqdm                          4.66.5
trove-classifiers             2024.10.17
typing_extensions             4.12.2
urllib3                       2.2.2
vcrpy                         6.0.2
wheel                         0.44.0
wrapt                         1.16.0
yarl                          1.12.1
zipp                          3.20.2
zstandard                     0.22.0

Please let me know if you need more details or want me to perform some diagnostics.

[martindurant]

Do you know how to test for the effect of -m "not network" within the test run session? All of these tests can be skipped for such a run.

[kloczek]

I don't see any of use network' pytest mark to skip some units so that module should not be affected by this. not notwork` should skip anything which requires more than localhost access.
In this case looks like failing units uses access to localhost but nothing is listening.

[martindurant]

OK, so listening on localhost doesn't work; is there a way to make a pytest fixture that can know this restriction?