feat: #21 recreate cli

docs/overview/command.en.md

@@ -14,3 +14,101 @@ you can also use [wasim](https://docs.web.auto/developers-guides/wasim/introduct
 Please see the [wasim documentation site](https://docs.web.auto/developers-guides/wasim/use-cases/run-simulations-locally/) for an example of tool usage.
 
 Since wasim downloads and executes scenarios from Autoware Evaluator, it can only execute scenarios that are already registered in the cloud environment.
+
+## Run driving_log_replayer_v2 with driving-log-replayer-v2-cli
+
+Using CLI, multiple scenarios can be executed consecutively with a single command input.
+
+For example, suppose that multiple scenarios are placed in subdirectories (SCENARIO_DIR1, SCENARIO_DIR2...) under SCENARIO_ROOT as shown below.
+
+```shell
+SCENARIO_ROOT
+├── SCENARIO_DIR1
+│   ├── out # output directyory
+│   └── SCENARIO_DIR1_DATASET0 # t4_dataset
+│   │  ├── annotation
+│   │  ├── data
+│   │  ├── input_bag
+│   │  ├── map
+│   │  └── status.json
+│   └── SCENARIO_DIR1_DATASET1 # t4_dataset
+│   │  └── ...
+│   │  ...
+│   └── scenario.yaml  # scenario fixed name
+│
+├── SCENARIO_DIR2
+│   ├── out # output directyory
+│   └── SCNERIO_DIR2_DATASET0 # t4_dataset
+│   │  ├── annotation
+│   │  ├── data
+│   │  ├── input_bag
+│   │  ├── map
+│   │  └── status.json
+│   └── scenario.yaml  # scenario fixed name
+...
+```
+
+To run multiple scenarios in the above SCENARIO_ROOT with the ros2 launch command, you need to hit the command multiple times as follows.
+
+```shell
+source ${AUTOWARE_ROOT}/install/setup.bash
+ros2 launch driving_log_replayer_v2 driving_log_replayer_v2 scenario_path:=${SCENARIO_ROOT}/SCENARIO_DIR1/scenario.yaml dataset_index:=0 # If you have multiple datasets
+ros2 launch driving_log_replayer_v2 driving_log_replayer_v2 scenario_path:=${SCENARIO_ROOT}/SCENARIO_DIR1/scenario.yaml dataset_index:=1 # If you have multiple datasets
+ros2 launch driving_log_replayer_v2 driving_log_replayer_v2 scenario_path:=${SCENARIO_ROOT}/SCENARIO_DIR2/scenario.yaml
+...
+```
+
+If you use cli, you only need the following command
+
+```shell
+source ${AUTOWARE_ROOT}/install/setup.bash
+dlr2 simulation run ${SCENARIO_ROOT}
+```
+
+### cli installation
+
+You can install cli with the following command.
+
+```shell
+# install
+pipx install git+https://github.com/tier4/driving_log_replayer_v2.git
+
+# upgrade
+pipx upgrade driving-log-replayer-v2
+
+# uninstall
+pipx uninstall driving-log-replayer-v2
+```
+
+### cli limitation
+
+The following limitations apply because of the automatic search for scenario files in CLI.
+
+- The scenario file must be named scenario.yaml
+- scenario.yaml must exist in a subdirectory under SCENARIO_ROOT (no sub-sub-directories allowed)
+
+### cli output
+
+Using CLI not only shortens the command input, but also increases the number of files output.
+
+The following is an example of the output destination when using CLI.
+The simulation run command and the console log are output as files.
+This is used to run multiple tests in a row and debug only the tests that show errors later.
+
+```shell
+OUTPUT_LATEST
+├── 0 # result of Datasets[0]
+│   ├── result.jsonl
+│   ├── result_archive_path
+│   └── result_bag
+│       ├── metadata.yaml
+│       └── result_bag_0.db3
+├── 1 # result of Datasets[1]
+│   ├── result.jsonl
+│   ├── result_archive_path
+│   └── result_bag
+│       ├── metadata.yaml
+│       └── result_bag_0.db3
+├── console.log # Logs displayed in the console as a file
+└── run.bash # simulation run command
+```

docs/overview/command.ja.md

@@ -14,3 +14,101 @@ TIER IV が提供している[Autoware Evaluator](https://docs.web.auto/user-man
 使い方は[ドキュメントサイト](https://docs.web.auto/developers-guides/wasim/use-cases/run-simulations-locally/)を参照。
 
 wasim は Autoware Evaluator からシナリオをダウンロードして実行するので、クラウド環境に登録済みのシナリオしか実行出来ない。
+
+## driving-log-replayer-v2-cli による driving_log_replayer_v2 実行
+
+cliを利用すると、1回のコマンド入力で複数のシナリオを連続で実行できます。
+
+例えば以下のように、SCENARIO_ROOT配下に複数のシナリオがサブディレクトリ(SCENARIO_DIR1, SCENARIO_DIR2...)に置かれているとします。
+
+```shell
+SCENARIO_ROOT
+├── SCENARIO_DIR1
+│   ├── out # output directyory
+│   └── SCENARIO_DIR1_DATASET0 # t4_dataset
+│   │  ├── annotation
+│   │  ├── data
+│   │  ├── input_bag
+│   │  ├── map
+│   │  └── status.json
+│   └── SCENARIO_DIR1_DATASET1 # t4_dataset
+│   │  └── ...
+│   │  ...
+│   └── scenario.yaml  # scenario fixed name
+│
+├── SCENARIO_DIR2
+│   ├── out # output directyory
+│   └── SCNERIO_DIR2_DATASET0 # t4_dataset
+│   │  ├── annotation
+│   │  ├── data
+│   │  ├── input_bag
+│   │  ├── map
+│   │  └── status.json
+│   └── scenario.yaml  # scenario fixed name
+...
+```
+
+上記のSCENARIO_ROOTにある複数シナリオをros2 launchコマンドで実行するには以下のように複数回のコマンドを叩く必要があります。
+
+```shell
+source ${AUTOWARE_ROOT}/install/setup.bash
+ros2 launch driving_log_replayer_v2 driving_log_replayer_v2 scenario_path:=${SCENARIO_ROOT}/SCENARIO_DIR1/scenario.yaml dataset_index:=0 # 複数datasetある場合
+ros2 launch driving_log_replayer_v2 driving_log_replayer_v2 scenario_path:=${SCENARIO_ROOT}/SCENARIO_DIR1/scenario.yaml dataset_index:=1 # 複数datasetある場合
+ros2 launch driving_log_replayer_v2 driving_log_replayer_v2 scenario_path:=${SCENARIO_ROOT}/SCENARIO_DIR2/scenario.yaml
+...
+```
+
+cliを使う場合は以下のコマンドで済みます
+
+```shell
+source ${AUTOWARE_ROOT}/install/setup.bash
+dlr2 simulation run ${SCENARIO_ROOT}
+```
+
+### cli installation
+
+以下のコマンドでcliをインストールできる。
+
+```shell
+# install
+pipx install git+https://github.com/tier4/driving_log_replayer_v2.git
+
+# upgrade
+pipx upgrade driving-log-replayer-v2
+
+# uninstall
+pipx uninstall driving-log-replayer-v2
+```
+
+### cli limitation
+
+cliでシナリオファイルを自動で探すため以下の制約がある。
+
+- シナリオファイルの名前がscenario.yamlでなければならない
+- SCENARIO_ROOTの下のサブディレクトリにscenario.yamlが存在しなければならない(サブサブディレクトリは不可)
+
+### cli output
+
+cliを利用すると、コマンド入力を短くするだけでなく出力されるファイルが増える。
+
+以下が、cliを利用した場合の出力先の例である。
+simulationの実行コマンドと、コンソールのログがファイルとして出力されている。
+複数個のテストを連続で実行し、後でエラーが出たテストだけデバッグするといった場合に利用する。
+
+```shell
+OUTPUT_LATEST
+├── 0 # Datasets[0]の結果
+│   ├── result.jsonl
+│   ├── result_archive_path
+│   └── result_bag
+│       ├── metadata.yaml
+│       └── result_bag_0.db3
+├── 1 # Datasets[1]の結果
+│   ├── result.jsonl
+│   ├── result_archive_path
+│   └── result_bag
+│       ├── metadata.yaml
+│       └── result_bag_0.db3
+├── console.log # コンソールに表示されているログをファイル化したもの
+└── run.bash # simulationの実行コマンド
+```

docs/trouble_shooting/index.en.md

@@ -11,7 +11,7 @@ The sensor_model, vehicle_model, and vehicle_id specified in the scenario are no
 ### Example 1
 
 ```shell
-❯ ros2 launch driving_log_replayer_v2 driving_log_replayer_v2.launch.py scenrio_path:=$HOME/driving_log_replayer_v2/sample.yaml
+❯ ros2 launch driving_log_replayer_v2 driving_log_replayer_v2.launch.py scenario_path:=$HOME/driving_log_replayer_v2/sample.yaml
 [INFO] [launch]: All log files can be found below /home/hyt/.ros/log/2024-06-07-12-37-19-365597-dpc2405001-1360746
 [INFO] [launch]: Default logging verbosity is set to INFO
 1717731451.040883 [77]       ros2: determined eno1 (udp/10.0.55.137) as highest quality interface, selected for automatic interface.

docs/trouble_shooting/index.ja.md

@@ -11,7 +11,7 @@
 ### 例1
 
 ```shell
-❯ ros2 launch driving_log_replayer_v2 driving_log_replayer_v2.launch.py scenrio_path:=$HOME/driving_log_replayer_v2/sample.yaml
+❯ ros2 launch driving_log_replayer_v2 driving_log_replayer_v2.launch.py scenario_path:=$HOME/driving_log_replayer_v2/sample.yaml
 [INFO] [launch]: All log files can be found below /home/hyt/.ros/log/2024-06-07-12-37-19-365597-dpc2405001-1360746
 [INFO] [launch]: Default logging verbosity is set to INFO
 1717731451.040883 [77]       ros2: determined eno1 (udp/10.0.55.137) as highest quality interface, selected for automatic interface.

driving_log_replayer_v2_analyzer/README.md

@@ -17,5 +17,5 @@ This package is installed with driving_log_replayer_v2_cli
 ## How to use
 
 ```shell
-dlr-analyzer analysis ${use-case-name} ${result.jsonl_path} [-c ${config_path}]
+dlr2-analyzer analysis ${use-case-name} ${result.jsonl_path} [-c ${config_path}]
 ```

driving_log_replayer_v2_cli/README.md

@@ -0,0 +1,9 @@
+# driving_log_replayer_v2_cli
+
+## unit test
+
+```shell
+# unsource /opt/ros/humble/setup.bash
+rye sync
+pytest driving_log_replayer_v2_cli/test -svv
+```

driving_log_replayer_v2_cli/__init__.py

@@ -0,0 +1,41 @@
+from importlib.metadata import version
+
+import click
+
+from driving_log_replayer_v2_cli.simulation import simulation
+
+try:
+    __version__ = version("driving-log-replayer-v2")
+except Exception:  # noqa
+    __version__ = "0.0.0"
+
+CONTEXT_SETTINGS = {"help_option_names": ["-h", "--help"]}
+
+
+def main() -> None:
+    cmd.add_command(simulation)
+    cmd()
+
+
+def print_version(ctx, param, value):  # noqa
+    if not value or ctx.resilient_parsing:
+        return
+    click.echo(__version__)
+    ctx.exit()
+
+
+@click.group(context_settings=CONTEXT_SETTINGS)
+@click.option(
+    "--version",
+    "-v",
+    is_flag=True,
+    callback=print_version,
+    expose_value=False,
+    is_eager=True,
+)
+def cmd() -> None:
+    """
+    Command line tool to use driving_log_replayer_v2.
+
+    https://github.com/tier4/driving_log_replayer_v2
+    """

driving_log_replayer_v2_cli/core/result.py

@@ -0,0 +1,47 @@
+from pathlib import Path
+
+import simplejson as json
+import termcolor
+
+
+def load_result(result_path: Path) -> list[dict]:
+    if not result_path.exists():
+        return []
+    with result_path.open() as jsonl_file:
+        try:
+            return [json.loads(line) for line in jsonl_file]
+        except json.JSONDecodeError:
+            return []
+
+
+def load_last_result(result_path: Path) -> dict:
+    result_list = load_result(result_path)
+    if result_list == []:
+        return {}
+    return result_list[-1]
+
+
+def load_final_metrics(result_path: Path) -> dict:
+    return load_last_result(result_path).get("Frame", {}).get("FinalMetrics", {})
+
+
+def display(result_path: Path) -> None:
+    print("--------------------------------------------------")  # noqa
+    last_result = load_last_result(result_path)
+    if last_result:
+        # dict is not empty
+        if last_result["Result"]["Success"]:
+            result = "TestResult: Passed"
+            color = "green"
+        else:
+            result = "TestResult: Failed"
+            color = "red"
+        termcolor.cprint(result, color)
+        termcolor.cprint(last_result["Result"]["Summary"], color)
+
+
+def display_all(output_directory: Path, target_dir_name: str) -> None:
+    result_paths = output_directory.glob(f"**/{target_dir_name}/**/result.jsonl")
+    for result_path in result_paths:
+        termcolor.cprint(f"{result_path.as_posix()}", "white")
+        display(result_path)

driving_log_replayer_v2_cli/core/shell.py

@@ -0,0 +1,18 @@
+from pathlib import Path
+import subprocess
+import sys
+
+
+def run_with_log(cmd: list, log_path: Path) -> None:
+    proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+    f = log_path.open("w", encoding="utf-8")
+
+    try:
+        while True:
+            line = proc.stdout.readline().decode("utf-8")
+            sys.stdout.write(line)
+            f.write(line)
+            if not line and proc.poll() is not None:
+                break
+    finally:
+        f.close()

driving_log_replayer_v2_cli/simulation/__init__.py

@@ -0,0 +1,34 @@
+from pathlib import Path
+
+import click
+
+from driving_log_replayer_v2_cli.core.result import display_all
+from driving_log_replayer_v2_cli.simulation.run import run as sim_run
+
+CONTEXT_SETTINGS = {"help_option_names": ["-h", "--help"]}
+
+
+@click.group(context_settings=CONTEXT_SETTINGS)
+def simulation() -> None:
+    """Run simulation and check simulation log."""
+
+
+@simulation.command(context_settings=CONTEXT_SETTINGS)
+@click.argument(
+    "scenario_root_directory",
+    type=click.Path(exists=True, file_okay=False, resolve_path=True, path_type=Path),
+)
+@click.option("--launch_args", "-l", multiple=True, default=[])
+def run(scenario_root_directory: Path, launch_args: list[str]) -> None:
+    sim_run(scenario_root_directory, launch_args)
+
+
+@simulation.command(context_settings=CONTEXT_SETTINGS)
+@click.argument(
+    "log_root",
+    type=click.Path(exists=True, file_okay=False, resolve_path=True, path_type=Path),
+)
+@click.option("--target_dir_name", "-t", type=str, default="**")
+def show_result(log_root: Path, target_dir_name: str) -> None:
+    """Show summary of simulation results in log_root."""
+    display_all(log_root, target_dir_name)