Abstract: In a multi-node wireless sensor network under dynamic jamming environment, traditional reinforcement learning is difficult to converge with the increase of state-action space. To overcome this disadvantage, in this paper, we propose a fast convergence anti-jamming algorithm based on reinforcement learning. The proposed algorithm combines multi-agent Q-learning with value function transfer. Firstly, the multi node communication anti-jamming problem is modeled as a Markov game. Then, we introduce Bisimulation Relation to measure the similarity between different state action pairs. Finally, the multi-agent Q learning algorithm is used to learn the anti-jamming strategy, and after each step of Q-value updating, the value function is transferred according to the similarity between different state-action pairs. The simulation results show that the anti-jamming performance of the proposed algorithm is significantly better than that of the orthogonal frequency hopping and the random frequency hopping. When the same anti-jamming effect is achieved, the number of iterations required is much less than that of the traditional Q-learning algorithm.