The starter code stops with an error at the first step (download)

[fantauzzi]

Steps to reproduce:

• fork and clone the repository as per instructions in its README or in Udacity's course
• still following those instructions, run the first step in the pipeline, which is already implemented:
  mlflow run . -P steps=download
  the execution stops with an error, see below.
  As per Udacity's knowledge base, this issue has been outstanding and known for at least one year.

Here the command output, with the error message.

2022/09/07 09:37:24 INFO mlflow.utils.conda: === Creating conda environment mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a ===
Collecting package metadata (repodata.json): done
Solving environment: done

Downloading and Extracting Packages
pyyaml-5.3.1         | 201 KB    | ################################################################################################################################################################################# | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: | Ran pip subprocess with arguments:
['/home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt']
Pip subprocess output:
Collecting wandb==0.10.31
  Downloading wandb-0.10.31-py2.py3-none-any.whl (1.8 MB)
Requirement already satisfied: requests<3,>=2.0.0 in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (2.28.1)
Requirement already satisfied: six>=1.13.0 in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (1.16.0)
Requirement already satisfied: GitPython>=1.0.0 in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (3.1.27)
Requirement already satisfied: python-dateutil>=2.6.1 in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (2.8.2)
Requirement already satisfied: configparser>=3.8.1 in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (5.3.0)
Requirement already satisfied: PyYAML in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (5.3.1)
Requirement already satisfied: protobuf>=3.12.0 in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (4.21.5)
Requirement already satisfied: Click>=7.0 in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (8.1.3)
Requirement already satisfied: docker-pycreds>=0.4.0 in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (0.4.0)
Requirement already satisfied: gitdb<5,>=4.0.1 in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from GitPython>=1.0.0->wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (4.0.9)
Requirement already satisfied: smmap<6,>=3.0.1 in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from gitdb<5,>=4.0.1->GitPython>=1.0.0->wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (3.0.5)
Collecting promise<3,>=2.0
  Using cached promise-2.3-py3-none-any.whl
Collecting psutil>=5.0.0
  Using cached psutil-5.9.2-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 kB)
Requirement already satisfied: charset-normalizer<3,>=2 in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (2.1.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (1.26.11)
Requirement already satisfied: idna<4,>=2.5 in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in /home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages (from requests<3,>=2.0.0->wandb==0.10.31->-r /home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/condaenv.4q8c9b1s.requirements.txt (line 1)) (2022.6.15)
Collecting sentry-sdk>=0.4.0
  Using cached sentry_sdk-1.9.8-py2.py3-none-any.whl (158 kB)
Collecting shortuuid>=0.5.0
  Using cached shortuuid-1.0.9-py3-none-any.whl (9.4 kB)
Collecting subprocess32>=3.5.3
  Using cached subprocess32-3.5.4-py3-none-any.whl
Collecting pathtools
  Using cached pathtools-0.1.2-py3-none-any.whl
Installing collected packages: subprocess32, shortuuid, sentry-sdk, psutil, promise, pathtools, wandb
Successfully installed pathtools-0.1.2 promise-2.3 psutil-5.9.2 sentry-sdk-1.9.8 shortuuid-1.0.9 subprocess32-3.5.4 wandb-0.10.31
                                                                                                                                                                                                                          done
#
# To activate this environment, use
#
#     $ conda activate mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a
#
# To deactivate an active environment, use
#
#     $ conda deactivate

Retrieving notices: ...working... done
2022/09/07 09:38:47 INFO mlflow.projects.utils: === Created directory /tmp/tmpyq8ya6a9 for downloading remote URIs passed to arguments of type 'path' ===
2022/09/07 09:38:47 INFO mlflow.projects.backend.local: === Running command 'source /home/fanta/.local/miniconda3/bin/../etc/profile.d/conda.sh && conda activate mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a 1>&2 && python main.py main.steps=\'download\' $(echo '')' in run with ID '9f412ca11bdf416b9ceeb2c68c4720a2' === 
2022/09/07 09:38:48 INFO mlflow.projects.utils: === Fetching project from https://github.com/udacity/build-ml-pipeline-for-short-term-rental-prices/tree/main/components/get_data into /tmp/tmpinc_jjw1 ===
Traceback (most recent call last):
  File "/home/fanta/workspace/ml-pipeline-for-short-term-rental-prices/main.py", line 40, in go
    _ = mlflow.run(
  File "/home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages/mlflow/projects/__init__.py", line 293, in run
    submitted_run_obj = _run(
  File "/home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages/mlflow/projects/__init__.py", line 92, in _run
    submitted_run = backend.run(
  File "/home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages/mlflow/projects/backend/local.py", line 45, in run
    work_dir = fetch_and_validate_project(project_uri, version, entry_point, params)
  File "/home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages/mlflow/projects/utils.py", line 124, in fetch_and_validate_project
    work_dir = _fetch_project(uri=uri, version=version)
  File "/home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages/mlflow/projects/utils.py", line 158, in _fetch_project
    _fetch_git_repo(parsed_uri, version, dst_dir)
  File "/home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages/mlflow/projects/utils.py", line 185, in _fetch_git_repo
    origin.fetch()
  File "/home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages/git/remote.py", line 886, in fetch
    res = self._get_fetch_info_from_stderr(proc, progress,
  File "/home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages/git/remote.py", line 750, in _get_fetch_info_from_stderr
    proc.wait(stderr=stderr_text)
  File "/home/fanta/.local/miniconda3/envs/mlflow-39beb38945f7e5378bc314f1ee9f6f4b4376524a/lib/python3.9/site-packages/git/cmd.py", line 502, in wait
    raise GitCommandError(remove_password_if_present(self.args), status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
  cmdline: git fetch -v origin
  stderr: 'fatal: repository 'https://github.com/udacity/build-ml-pipeline-for-short-term-rental-prices/tree/main/components/get_data/' not found'

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
2022/09/07 09:38:49 ERROR mlflow.cli: === Run (ID '9f412ca11bdf416b9ceeb2c68c4720a2') failed ===

[fantauzzi]

EDIT as per comment below by @nakany15 I have posted here the solution to a different issue, in an error. I will leave it here for context to the comment below.

===========================================================================
I have been able to fix the problem by indicating jupyter instead of jupyterlab in the dependencies of src/eda/conda.yml. This is the content of my src/eda/conda.yml that fixed the problem

name: eda
channels:
  - conda-forge
  - defaults
dependencies:
  - jupyter=1.0.0
  - seaborn=0.11.2
  - pandas=1.4.4
  - pip=22.1.2
  - pandas-profiling=3.2.0
  - pyarrow=8.0.0
  - pip:
      - wandb==0.13.2

[nakany15]

@fantauzzi
I think the solution you provide is for an error in running mlflow run src/eda.

This is caused by possible misspecifications in main.py and config.yml.
At the line 40 in main.py You should add version = 'main' as an argument of mlflow.run.
In the config.yml, component_repositry should be "https://github.com/udacity/build-ml-pipeline-for-short-term-rental-prices#components".