Practical implementations of Continuous Variable (CV) Quantum Key Distribution (QKD) systems can be performed over the existing optical fiber telecommunication infrastructure. However, optical fibers suffer from random variations in the state of polarization affecting the performance of the system. In this work, we study the impact of the polarization drift on the performance of a CV-QKD setup with a polarization diversity receiver, in which the polarization drift is pre-compensated on Alice’s side. We show that pre-compensating for the polarization drift can successfully realign Alice’s and Bob’s raw signal, without affecting the excess noise, up to a polarization drift variance of 10^{−10}, compatible with most practical implementations.