Add documentation, switch to pyproject.toml and ruff

[araffin]

Description

Add proper documentation because the README was a mess.

Also do the same as in DLR-RM/stable-baselines3#1361

Can be see here: https://rl-baselines3-zoo.readthedocs.io/en/feat-doc/index.html

@qgallouedec in case you can do a quick review that's nice, otherwise I'll merge it without as this is important to provide clearer doc.

You don't need to go though all files, there are too many changes.

Motivation and Context

• I have raised an issue to propose this change (required for new features and bug fixes)

See #299

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation (update in the documentation)

Checklist:

• I've read the CONTRIBUTION guide (required)
• I have updated the changelog accordingly (required).
• My change requires a change to the documentation.
• I have updated the tests accordingly (required for a bug fix or a new feature).
• I have updated the documentation accordingly.
• I have reformatted the code using make format (required)
• I have checked the codestyle using make check-codestyle and make lint (required)
• I have ensured make pytest and make type both pass. (required)

Note: we are using a maximum length of 127 characters per line

[qgallouedec]

Looks great! It's much better now!
I didn't go through all the files since it's unlikely to break anything.

[araffin]

I didn't go through all the files since it's unlikely to break anything.

The build is somehow broken now, but related to setuptools I think.

[qgallouedec]

It seems like enjoy is broken for PandaReach. I'm taking a look

[araffin]

It seems like enjoy is broken for PandaReach. I'm taking a look

The problem comes from VecNormalize which overrides somehow the observation space. Will see if I can push a fix soon.

[araffin]

I found the issue, it is a mix of DLR-RM/rl-trained-agents#10 and DLR-RM/stable-baselines3@12e9917 (where the observation space was set to the env one), I will push a fix for the VecNormalize stats.