result question

[Electric-V]

Thanks for your code, I've got it working. But I have a question. Why is the DDPG algorithm so much better than PPO in your case, but the results I ran show that they are almost the same? Need to make some adjustments? Thanks!

[xing-ji63]

I ran it the same way. Not as obvious as in bloggers.

Have you encountered this problem（File "D:\py-projects\Chargym-Charging-Station\Chargym_Charging_Station\envs\Charging_Station_Enviroment.py", line 112, in reset
self.Invalues['ArrivalT'].append(self.Invalues['Arrival'][ii][0].tolist())） before？

[Electric-V]

Sorry, I don't remember encountering this problem. I had no problem running the entire project, but the result seemed wrong. I suggest you use chatGPT or deepseek to check it again. By the way, if you are using pycharm, you can try the Tongyi Lingma plugin, I think it is very useful. I use it to help solve similar problems when I encounter them while writing code.

[georkara]

Hi Electric-V, thanks for playing with Chargym-Charging-Station :) I am trying to think a logical reason... If you see the same PPO behaviour, same levels of achieved reward after a number of episodes, then please check the hyperparameters of DDPG that we report at the paper. You can play with the hyperparameters of any RL algorithm within stablebaselines3.
Thanks for your interest in using Chargym,
Giorgos

Thanks for your code, I've got it working. But I have a question. Why is the DDPG algorithm so much better than PPO in your case, but the results I ran show that they are almost the same? Need to make some adjustments? Thanks!

[Electric-V]

Hi Electric-V, thanks for playing with Chargym-Charging-Station :) I am trying to think a logical reason... If you see the same PPO behaviour, same levels of achieved reward after a number of episodes, then please check the hyperparameters of DDPG that we report at the paper. You can play with the hyperparameters of any RL algorithm within stablebaselines3. Thanks for your interest in using Chargym, Giorgos

Thanks for your code, I've got it working. But I have a question. Why is the DDPG algorithm so much better than PPO in your case, but the results I ran show that they are almost the same? Need to make some adjustments? Thanks!

Hi Giorgos, thank you very much for your advice! I will read the paper carefully and try again.

[georkara]

Oh, I just realized that you opened this issue 1 year ago. I deeply apologize. I just saw that today.
Giorgos

[Electric-V]

Oh, I just realized that you opened this issue 1 year ago. I deeply apologize. I just saw that today. Giorgos

No worries at all! Thank you for following up. I completely understand as I've been busy with other things as well and haven't run the program in almost a year. I’m glad to continue the conversation whenever you’re ready. Looking forward to future discussions!

[xing-ji63]

Is the result incorrect? Does it mean it runs differently from the blogger? Okay, thank you for your reply。运行的不会像博主那么明显 波动很大 奖励都很接近  1552237320 ***@***.***  

…

------------------ 原始邮件 ------------------ 发件人: "georkara/Chargym-Charging-Station" ***@***.***>; 发送时间: 2025年3月9日(星期天) 晚上9:48 ***@***.***>; ***@***.******@***.***>; 主题: Re: [georkara/Chargym-Charging-Station] result question (Issue #6) Sorry, I don't remember encountering this problem. I had no problem running the entire project, but the result seemed wrong. I suggest you use chatGPT or deepseek to check it again. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***> Electric-V left a comment (georkara/Chargym-Charging-Station#6) Sorry, I don't remember encountering this problem. I had no problem running the entire project, but the result seemed wrong. I suggest you use chatGPT or deepseek to check it again. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[georkara]

No I think that the question emerges from observing the same levels of reward between DDPG and PPO after the same number of epochs during training. I think that the reason is because Electric-V used the default hyperparameters, while someone can use different network size (check Table 2 at layer structure in the pdf), or different batch size, learning rate or eventually specify different set of hyperparameters than the default ones that the stablebaselines3 provides. You can play with the hyperparameters from problem to problem, because at the end of the day, you will need different size (number of neurons and hidden layers) of actor and critic networks from application to application. I hope this helps.
Giorgos

[xing-ji63]

No I think that the question emerges from observing the same levels of reward between DDPG and PPO after the same number of epochs during training. I think that the reason is because Electric-V used the default hyperparameters, while someone can use different network size (check Table 2 at layer structure in the pdf), or different batch size, learning rate or eventually specify different set of hyperparameters than the default ones that the stablebaselines3 provides. You can play with the hyperparameters from problem to problem, because at the end of the day, you will need different size (number of neurons and hidden layers) of actor and critic networks from application to application. I hope this helps. Giorgos

Okay, thank you very much. I'll come down and debug. I would like to ask about the rule-based controller (RBC) mentioned in the following text. There is no detailed explanation in this article. Can I understand it as the principle of first come, first served charging? Or is it the meaning represented by the program in RBC.py?

[georkara]

The RBC is a simple, conventional and logical way to have a baseline controller to test any implementation regarding the decisions made. You can have it as much complex as you want. Actually you can have different RBCs if you want. You could have a first come-first charged if you want but the RBC we used is described in Section 4.2 of the paper that is also uploaded here in this repo. It is described in equation 6. In practice it says that if the car leaves within the next 3 hours, then charge it with full charging rate, else charge it with a mean of the current solar radiation and the one hour forecast. The latter is practically, if we are gonna have sunny conditions and it is daylight, we can charge the car using the energy produced by the PV system. This is a logical thing in order to extrapolate the PV produced energy. If you want to follow this rationale you can modify RBC.py to include another 2 prediction steps of radiation (i think it is state[3] and state[4], thus to get the "(state[0]+state[2]+state[3]+state[4])/4" , or you can have whatever logical RBC you want. Keep that in general the RBC can be as complex as you want. Note that many heuristic approaches can be developed just to keep it simple and to avoid training efforts.
Giorgos

[xing-ji63]

The RBC is a simple, conventional and logical way to have a baseline controller to test any implementation regarding the decisions made. You can have it as much complex as you want. Actually you can have different RBCs if you want. You could have a first come-first charged if you want but the RBC we used is described in Section 4.2 of the paper that is also uploaded here in this repo. It is described in equation 6. In practice it says that if the car leaves within the next 3 hours, then charge it with full charging rate, else charge it with a mean of the current solar radiation and the one hour forecast. The latter is practically, if we are gonna have sunny conditions and it is daylight, we can charge the car using the energy produced by the PV system. This is a logical thing in order to extrapolate the PV produced energy. If you want to follow this rationale you can modify RBC.py to include another 2 prediction steps of radiation (i think it is state[3] and state[4], thus to get the "(state[0]+state[2]+state[3]+state[4])/4" , or you can have whatever logical RBC you want. Keep that in general the RBC can be as complex as you want. Note that many heuristic approaches can be developed just to keep it simple and to avoid training efforts. Giorgos

OK, thank you very much for your reply. I'm looking forward to further discussions with you on this project. I'll do some research first.