Performance Analysis of Single Carrier Coherent and Noncoherent Modulation under I/Q Imbalance

In-phase/quadrature-phase Imbalance (IQI) is considered a major performance-limiting impairment in direct-conversion transceivers. Its effects become even more pronounced at higher carrier frequencies such as the millimeter-wave frequency bands considered for 5G systems. In this work, we quantify the effects of IQI on the performance of different modulations under multipath fading channels. This is realized by developing a comprehensive framework for the symbol error rate (SER) analysis of coherent phase shift keying (PSK), noncoherent differential phase shift keying (DPSK) and noncoherent frequency shift keying (FSK) under IQI effects. In this context, the moment generating function of the signal-to-interference-plus-noise-ratio is first derived for single-carrier systems suffering from transmitter (TX) IQI only, receiver (RX) IQI only and joint TX/RX IQI. Capitalizing on this, we derive analytic expressions for the SER of the different modulation schemes considered. These expressions are corroborated with simulation results and they provide insights into the dependence of IQI on the system parameters. We further demonstrate that, while in some cases, IQI can cause a slight degradation of the SER performance and, hence, it can be neglected, in other cases it should be compensated in order to achieve a reliable communication link.