How to properly close (SSH) file systems?

[mxmlnkn]

I did open an SSH file system and used it successfully:

o = fsspec.open("ssh://[email protected]")
o.fs.listdir("/")

Then, I lost the SSH connection because I lost WLAN connectivity because I entered suspend mode on my notebook. After that, I got this error on o.fs.listdir:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.local/lib/python3.12/site-packages/fsspec/spec.py", line 1593, in listdir
    return self.ls(path, detail=detail, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/lib/python3.12/site-packages/fsspec/implementations/sftp.py", line 127, in ls
    stats = [self._decode_stat(stat, path) for stat in self.ftp.listdir_iter(path)]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/paramiko/sftp_client.py", line 278, in listdir_iter
    t, msg = self._request(CMD_OPENDIR, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/paramiko/sftp_client.py", line 821, in _request
    num = self._async_request(type(None), t, *arg)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/paramiko/sftp_client.py", line 846, in _async_request
    self._send_packet(t, msg)
  File "/usr/lib/python3/dist-packages/paramiko/sftp.py", line 198, in _send_packet
    self._write_all(out)
  File "/usr/lib/python3/dist-packages/paramiko/sftp.py", line 162, in _write_all
    n = self.sock.send(out)
        ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/paramiko/channel.py", line 801, in send
    return self._send(s, m)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/paramiko/channel.py", line 1198, in _send
    raise socket.error("Socket is closed")
OSError: Socket is closed

I was not able to get around this error without closing and restarting the Python interactive interpreter itself. Note that this error takes a while. Directly after loosing connectivity, or enabling VPN, the listdir call hangs multiple minutes before I loose patience. I tried unsuccessfully:

• o.close()
• o.fs.close() # no attribute "close"
• Simply reopening the same SSH connection: fsspec.open("ssh://[email protected]").fs.listdir('/')
• o.fs.client.close(). This fixes the hang, but now I get the Socket is closed error, which I also got after ~30 minutes (timeout?), with no way to reopen the same connection again.
• quit ipython3 and reopen it: works fine

I am especially stumped as to why the last one did not work. It seems that SSH connections are somehow cached ad some layer.

I also looked into the paramiko API specification and found this:

close()

Close this SSHClient and its underlying Transport.

This should be called anytime you are done using the client object.

Warning

Paramiko registers garbage collection hooks that will try to automatically close connections for you, but this is not presently reliable. Failure to explicitly close your client after use may lead to end-of-process hangs!

The fsspec SFTP implementation never calls this close method on the Client. As far as I can see, the client is only opened, never closed.

I would expect all other remote file systems to have similar issues, i.e., how can I ensure that the connection is properly closed?

Edit: I noticed that there is some kind of caching going on. This would explain why simply trying to reopen the SSH mount did not work.

[martindurant]

The .client object has a close() as you say, but I see no way to check if the connection is still live. In any case, calling connect() should first close any client that might already be there, and that would give you a way forward here.

Edit: I noticed that there is some kind of caching going on.

I don't think that condition does anything anymore. But actually, instances in fsspec are generally cached: if you instantiate with exactly the same arguments as previously, you get back the same object as before. You can pass skip_instance_cache=True to disable this, or .clear_instance_cache to discard stored instances (which will del them)