Agents Not Acting Autonomously in MAAC and MAPPDG Implementations

[zwk9298]

Hello,I am greatly appreciative of your valuable work.While I have some questions.
I have successfully set up the environment following your documentation and am able to run the main.py files for MAAC and MAPPDG. However, while the program is running, I observe through the displayed window that the agents do not seem to be acting autonomously. I am wondering what could be the reason for this, and how I can enable the agents to act on their own.

Could you please provide any suggestions or guidance on this matter? I would greatly appreciate your assistance in troubleshooting this issue. Thank you in advance for your time and expertise.

[leehe228]

Hello, Thank you so much for your interest in my project.

By the way, when you say "the agent doesn't act autonomously", do you mean that the agent doesn't act "well" (correct behaviour, like picking up a box)?

If so, I'm wondering if you used a pre-trained model or if you started learning from without any pre-trained model.

If you started learning without pre-trained model, agents may not act well in the early epochs. In the early epochs, it is not trained yet, so it acts almost randomly (due to epsilon-greedy exploration).

If possible, could you please attach a picture or video, or write the parameters of training code you set for training?

[zwk9298]

@leehe228, I appreciate your prompt response. I realize that I may not have been entirely clear in my previous inquiry. When I open the LogisticsEnv-main\MAAC\main.py file and configure it to run in Build_Windows mode, a logistics.exe window pops up, but the drones remain stagnant, unresponsive to any movement commands. However, they do move when I attempt to control them using the keyboard, which suggests that the training has not been initiated properly. Could you kindly guide me on what might be causing this issue?

Additionally, when I set env = GymEnv(name="../Build_Windows/Logistics", ...) in the LogisticsEnv-main\MAAC\replay.py file and specify the model_path as 'E:\code\LogisticsEnv-main\MAAC\models\Logistics\MAAC\run2\model_MAAC_20k.pt', I face a similar challenge. It seems that the model_MAAC_20k.pt model is not being utilized as intended. Could you please elaborate on the potential reasons behind this? Your insights would be greatly appreciated.

[zwk9298]

I'm having trouble with your code and would appreciate your assistance. I did not find a .yaml file in your repository, which seems necessary for the agents to act autonomously. When I open the scene in Unity and run it, all the agents remain stationary and display a message indicating that they are not connected to port 5004. Furthermore, when I attempt to control the agents using the keyboard, they all perform the same action. Could you please explain what might be causing this? And, what steps should I take in Unity to enable the agents to act autonomously?

Additionally, when I run the main.py file in the MACC folder, I encounter a similar issue, suggesting that the control code is not functioning as expected. Could you guide me on how to make the agents act autonomously? Thank you for your time and assistance.