The optimization of Beamforming and Trajectory for reconfigurable intelligent surface assisted UAV communication system based on deep reinforcement learning

Aiming at the highly coupled design of phase shift matrix of reconfigurable intelligent surface(RIS) and unmanned aerial vehicle(UAV) trajectory in RIS-assisted UAV communication system, the paper apply a twin deep deterministic policy gradient(TDDPG) framework for RIS-assisted UAV communication. The method applies two deep deterministic policy gradient(DDPG) structures to decouple the two sub-problems of beamforming matrix design and UAV trajectory and a penalty related to energy consumption of UAV is added into reward function to jointly optimize system spectral efficiency(SE) and energy efficiency(EE). Simulation results show that it is effective for the improvement of system performance by jointly optimizing UAV trajectory and beamforming matrix and correct design of reward function could effectively guide the agent to learn correct UAV trajectory and beamforming policy in dynamic wireless environment. Compared to baseline methods, TDDPG structure achieves at least 12% SE improvement and 24% EE improvement.