Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error facing during Quickstart #307

Open
junu0723 opened this issue Oct 16, 2024 · 8 comments
Open

Error facing during Quickstart #307

junu0723 opened this issue Oct 16, 2024 · 8 comments
Labels
bug Something isn't working

Comments

@junu0723
Copy link

Version

0.9.4

Platform

Darwin 2033000841 23.5.0 Darwin Kernel Version 23.5.0: Wed May 1 20:13:18 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6030 arm64

Docker version

Docker version 27.0.3, build 7d4bcd8

Kubernetes version

No response

Describe the bug.

image

During the command $ domino platform run-compose, it failed with the below error.

no matching manifest for linux/arm64/v8 in the manifest list entries
Docker images pull failed.

What steps will reproduce the bug?

No response

What is the expected behavior? Why is that the expected behavior?

No response

What do you see instead?

No response

Additional information

No response

@junu0723 junu0723 added the bug Something isn't working label Oct 16, 2024
@luiztauffer
Copy link
Member

Hi @junu0723 , this looks like a problem in your local docker engine, maybe try this:

docker system prune --all
sudo service docker restart

warning: the first command will delete all previously cached images in your system

@junu0723
Copy link
Author

image

I tried using your command, but after that, it kept failing.

@junu0723
Copy link
Author

I'm trying this approach:

  • pull images separately as linux/amd64, and tag as latest
    • docker pull ghcr.io/tauffer-consulting/domino-frontend:k8s-dev@sha256:a2bc077f630b0e4fa1facc37cef6ac227cfa87fb5929bbbc0320864c99655311
    • docker pull ghcr.io/tauffer-consulting/domino-airflow-base:latest-dev@sha256:505bc59f8dd6398f6865a6c76eae4eba05a0159ce6184354adcb4b3983f4883c
    • docker pull ghcr.io/tauffer-consulting/domino-rest:latest-dev@sha256:762a0a05c29d563b0e7bcd4b0f71a1dd8f69aa75fe9b4677ca2b605d7a635d12
    • edit docker-compose.yaml file using pre-downloaded images
    • execute command docker-compose up to start server
    • it works!
      image

@junu0723
Copy link
Author

But I'm stuck at the login phase :/

image

@tdari
Copy link
Contributor

tdari commented Oct 17, 2024

hey @junu0723, is docker proxy service up and running?

@junu0723
Copy link
Author

junu0723 commented Oct 17, 2024

@tdari
I found docker-rest is dead.
Here are the logs I found.

(venv) 10/17/24|1:16:04  domino git:(main) ✗  $ docker ps -a
CONTAINER ID   IMAGE                                            COMMAND                  CREATED          STATUS                     PORTS                    NAMES
c9374b6747e7   apache/airflow:2.7.2-python3.9                   "/usr/bin/dumb-init …"   15 minutes ago   Up 7 minutes (healthy)     0.0.0.0:8080->8080/tcp   airflow-webserver
4d66eda90b0b   ghcr.io/tauffer-consulting/domino-airflow-base   "/usr/bin/dumb-init …"   15 minutes ago   Up 7 minutes (healthy)     8080/tcp                 airflow-domino-worker
ee472130dd5b   ghcr.io/tauffer-consulting/domino-airflow-base   "/usr/bin/dumb-init …"   15 minutes ago   Up 7 minutes (healthy)     8080/tcp                 airflow-domino-scheduler
32f3716d8c92   apache/airflow:2.7.2-python3.9                   "/usr/bin/dumb-init …"   15 minutes ago   Up 7 minutes (healthy)     8080/tcp                 airflow-triggerer
d770d3289f7a   ghcr.io/tauffer-consulting/domino-frontend       "sh /usr/share/nginx…"   15 minutes ago   Up 8 minutes               0.0.0.0:3000->80/tcp     domino-frontend
96669f70fa37   apache/airflow:2.7.2-python3.9                   "/bin/bash -c 'funct…"   15 minutes ago   Exited (0) 7 minutes ago                            airflow-init
e44f668232d1   ghcr.io/tauffer-consulting/domino-rest           "bash -c 'alembic up…"   15 minutes ago   Exited (1) 8 minutes ago                            domino-rest
51d1d9af2913   postgres:13                                      "docker-entrypoint.s…"   15 minutes ago   Up 8 minutes (healthy)     0.0.0.0:5433->5432/tcp   domino-postgres
b029a4d02576   postgres:13                                      "docker-entrypoint.s…"   15 minutes ago   Up 8 minutes (healthy)     5432/tcp                 airflow-postgres
b04bdb836a7a   redis:latest                                     "docker-entrypoint.s…"   15 minutes ago   Up 8 minutes (healthy)     6379/tcp                 airflow-redis
d66960605d8c   alpine/socat                                     "socat TCP4-LISTEN:2…"   15 minutes ago   Up 8 minutes               0.0.0.0:2376->2375/tcp   domino-docker-proxy
(venv) 10/17/24|1:16:07  domino git:(main) ✗  $ docker logs domino-rest
Traceback (most recent call last):
  File "/usr/local/bin/alembic", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/alembic/config.py", line 630, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/usr/local/lib/python3.10/site-packages/alembic/config.py", line 624, in main
    self.run_cmd(cfg, options)
  File "/usr/local/lib/python3.10/site-packages/alembic/config.py", line 601, in run_cmd
    fn(
  File "/usr/local/lib/python3.10/site-packages/alembic/command.py", line 399, in upgrade
    script.run_env()
  File "/usr/local/lib/python3.10/site-packages/alembic/script/base.py", line 578, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/usr/local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 109, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/rest/database/alembic/env.py", line 12, in <module>
    from database.interface import DBInterface
  File "/rest/./database/interface.py", line 4, in <module>
    from core.settings import settings
  File "/rest/./core/settings.py", line 164, in <module>
    settings: Settings = get_settings()
  File "/rest/./core/settings.py", line 156, in get_settings
    "local-k8s": LocalK8sSettings(),
  File "/usr/local/lib/python3.10/site-packages/pydantic_settings/main.py", line 71, in __init__
    super().__init__(
  File "/usr/local/lib/python3.10/site-packages/pydantic/main.py", line 164, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
pydantic_core._pydantic_core.ValidationError: 1 validation error for LocalK8sSettings
CREATE_DEFAULT_USER
  Input should be a valid boolean, unable to interpret input [type=bool_parsing, input_value='', input_type=str]
    For further information visit https://errors.pydantic.dev/2.4/v/bool_parsing
Traceback (most recent call last):
  File "/usr/local/bin/alembic", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/site-packages/alembic/config.py", line 630, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/usr/local/lib/python3.10/site-packages/alembic/config.py", line 624, in main
    self.run_cmd(cfg, options)
  File "/usr/local/lib/python3.10/site-packages/alembic/config.py", line 601, in run_cmd
    fn(
  File "/usr/local/lib/python3.10/site-packages/alembic/command.py", line 399, in upgrade
    script.run_env()
  File "/usr/local/lib/python3.10/site-packages/alembic/script/base.py", line 578, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/usr/local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
  File "/usr/local/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 109, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/rest/database/alembic/env.py", line 12, in <module>
    from database.interface import DBInterface
  File "/rest/./database/interface.py", line 4, in <module>
    from core.settings import settings
  File "/rest/./core/settings.py", line 164, in <module>
    settings: Settings = get_settings()
  File "/rest/./core/settings.py", line 156, in get_settings
    "local-k8s": LocalK8sSettings(),
  File "/usr/local/lib/python3.10/site-packages/pydantic_settings/main.py", line 71, in __init__
    super().__init__(
  File "/usr/local/lib/python3.10/site-packages/pydantic/main.py", line 164, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
pydantic_core._pydantic_core.ValidationError: 1 validation error for LocalK8sSettings
CREATE_DEFAULT_USER
  Input should be a valid boolean, unable to interpret input [type=bool_parsing, input_value='', input_type=str]
    For further information visit https://errors.pydantic.dev/2.4/v/bool_parsing

@tdari
Copy link
Contributor

tdari commented Oct 17, 2024

@junu0723 did you modify the compose env variables for the rest service? I was able to start without any problem. This is my compose file. It looks like there is something with the default user.

domino_rest:
# image: ghcr.io/tauffer-consulting/domino-rest:latest
build:
context: ./rest
dockerfile: Dockerfile
container_name: domino-rest
command: bash -c "alembic upgrade heads && uvicorn main:app --reload --workers 1 --host 0.0.0.0 --port 8000"
ports:
- 8000:8000
environment:
- DOMINO_DB_USER=postgres
- DOMINO_DB_PASSWORD=postgres
- DOMINO_DB_HOST=domino_postgres
- DOMINO_DB_PORT=5432
- DOMINO_DB_NAME=postgres
- DOMINO_DEFAULT_PIECES_REPOSITORY_VERSION=0.6.0
- DOMINO_DEFAULT_PIECES_REPOSITORY_TOKEN=${DOMINO_DEFAULT_PIECES_REPOSITORY_TOKEN}
- DOMINO_GITHUB_ACCESS_TOKEN_WORKFLOWS=${DOMINO_GITHUB_ACCESS_TOKEN_WORKFLOWS}
- DOMINO_GITHUB_WORKFLOWS_REPOSITORY=${DOMINO_GITHUB_WORKFLOWS_REPOSITORY}
- CREATE_DEFAULT_USER=true
- DOMINO_DEPLOY_MODE=local-compose
- AIRFLOW_ADMIN_USERNAME=airflow
- AIRFLOW_ADMIN_PASSWORD=airflow
networks:
- domino-postgres-network
- default
volumes:
- ./rest/:/rest # Enable hot reload for backend
- ${AIRFLOW_PROJ_DIR:-./airflow}/dags:/opt/airflow/dags
depends_on:
domino_postgres:
condition: service_healthy

@junu0723
Copy link
Author

junu0723 commented Oct 17, 2024

Finally success! Thanks @luiztauffer and @tdari. Now I can login, and create some workflow :)
image

I'm still curious about how to add more pieces and how to generate custom pieces, but I'll check on my side first on that issue.

Just focusing here on reporting the errors during Quickstart in M2 Mac.
I can easily reproduce the error situation.

(venv) 10/17/24|8:38:54  domino_test2  $ domino platform run-compose

Welcome to Domino! ❤️

Github token for access default pieces repositories.: my_token
Starting Domino Platform using Docker Compose.
Please wait, this might take a few minutes...

Pulling Docker images...
WARN[0000] The "AIRFLOW_UID" variable is not set. Defaulting to a blank string.
WARN[0000] The "AIRFLOW_UID" variable is not set. Defaulting to a blank string.
WARN[0000] /domino_test2/docker-compose.yaml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
[+] Pulling 9/32
 ✔ postgres Skipped - Image is already being pulled by domino_postgres                                                                                                                                                                          0.0s
 ✔ airflow-webserver Skipped - Image is already being pulled by airflow-triggerer                                                                                                                                                               0.0s
 ✔ airflow-scheduler Skipped - Image is already being pulled by airflow-worker                                                                                                                                                                  0.0s
 ✔ airflow-init Skipped - Image is already being pulled by airflow-triggerer                                                                                                                                                                    0.0s
 ✔ domino_postgres Pulled                                                                                                                                                                                                                       2.1s
 ⠸ domino_rest Pulling                                                                                                                                                                                                                          2.4s
 ⠸ airflow-triggerer [⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀] Pulling                                                                                                                                                                                            2.4s
   ⠦ 85e50d2242ce Pulling fs layer                                                                                                                                                                                                              0.6s
   ⠦ 2502a223284a Pulling fs layer                                                                                                                                                                                                              0.6s
   ⠦ df17d43e6f87 Pulling fs layer                                                                                                                                                                                                              0.6s
   ⠦ dabd0dc1d70c Waiting                                                                                                                                                                                                                       0.6s
   ⠦ 4f51c6350ea0 Waiting                                                                                                                                                                                                                       0.6s
   ⠦ 807262954edf Waiting                                                                                                                                                                                                                       0.6s
   ⠦ 03864fb53afc Waiting                                                                                                                                                                                                                       0.6s
   ⠦ 718050a58fab Waiting                                                                                                                                                                                                                       0.6s
   ⠦ 4fe5d6e407bc Waiting                                                                                                                                                                                                                       0.6s
   ⠦ 59529b3985e0 Waiting                                                                                                                                                                                                                       0.6s
   ⠦ ea7d43bcdae6 Waiting                                                                                                                                                                                                                       0.6s
   ⠦ 629e8c0dadb9 Waiting                                                                                                                                                                                                                       0.6s
   ⠦ 5ae19301a855 Waiting                                                                                                                                                                                                                       0.6s
   ⠦ e10d11442621 Waiting                                                                                                                                                                                                                       0.6s
   ⠦ 8c107f4a58e3 Waiting                                                                                                                                                                                                                       0.6s
   ⠦ 13bb6bb26492 Waiting                                                                                                                                                                                                                       0.6s
   ⠦ 108800cc3bbf Waiting                                                                                                                                                                                                                       0.6s
   ⠦ 2d1009b96222 Waiting                                                                                                                                                                                                                       0.6s
   ⠦ e703e6802e5f Waiting                                                                                                                                                                                                                       0.6s
   ⠦ 4f4fb700ef54 Waiting                                                                                                                                                                                                                       0.6s
   ⠦ 0f5124bb789f Waiting                                                                                                                                                                                                                       0.6s
 ✘ airflow-worker Error                 context canceled                                                                                                                                                                                        2.4s
 ✘ domino_frontend Error                context canceled                                                                                                                                                                                        2.4s
 ✔ redis Pulled                                                                                                                                                                                                                                 2.1s
 ✔ docker-proxy Pulled                                                                                                                                                                                                                          1.8s
no matching manifest for linux/arm64/v8 in the manifest list entries
Docker images pull failed.

Starting services...
(venv) 10/17/24|8:39:17  domino_test2  $ ls
airflow             docker-compose.yaml domino_data

And the domino_rest part in the docker-compose.yaml file that is automatically created is:

  # Domino REST Api
  domino_rest:
    image: ghcr.io/tauffer-consulting/domino-rest:latest${DOMINO_COMPOSE_DEV}
    container_name: domino-rest
    command: bash -c "alembic upgrade heads && python -m utils.populate_first_user && uvicorn main:app --reload --workers 1 --host 0.0.0.0 --port 8000"
    ports:
      - 8000:8000
    environment:
      - DOMINO_DB_USER=postgres
      - DOMINO_DB_PASSWORD=postgres
      - DOMINO_DB_HOST=domino_postgres
      - DOMINO_DB_PORT=5432
      - DOMINO_DB_NAME=postgres
      - DOMINO_DEFAULT_PIECES_REPOSITORY_TOKEN=${DOMINO_DEFAULT_PIECES_REPOSITORY_TOKEN}
      - DOMINO_DEPLOY_MODE=local-compose
      - AIRFLOW_ADMIN_USERNAME=airflow
      - AIRFLOW_ADMIN_PASSWORD=airflow
      - CREATE_DEFAULT_USER=${DOMINO_CREATE_DEFAULT_USER}
    networks:
      - domino-postgres-network
      - default
    volumes:
      - ${AIRFLOW_PROJ_DIR:-./airflow}/dags:/opt/airflow/dags
    depends_on:
      domino_postgres:
        condition: service_healthy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants