Demodulation of Cochannel QAM Signals

This paper describes a technique for demodulating a quadrature amplitude modulated (QAM) signal in the presence of a second cochannel QAM interferor and intersymbol interference. The method consists of a two-step procedure whereby tentative estimates of both the desired and interfering signals are made and then converted into final estimates by detecting and correcting errors. The tentative decisions are formed using a structure consisting of a primary demodulator followed by a secondary demodulator incorporating an adaptive interference canceller. Errors in the tentative decisions are detected using an adaptive channel estimator and then corrected using a maximum likelihood sequence estimator. Signature curves quantifying the symbol error rate of the tentative demodulator as a function of signal-to-noise ratio and signal-to-interference ratio are presented. A simulation of the error detection/correction method is also presented.

Digital communication systems employing frequency reusage (e.g. cellular radio and dual-polarized microwave radio) often exhibit limitations in performance due to cochannel interference. In these systems, both the desired and interference signals are quadrature-amplitude-modulated (QAM). For such systems, development of a demodulator with reduced sensitivity to cochannel QAM interference would be highly beneficial.

The symbol error rate (SER) of a QAM signal in the presence of a single cochannel QAM interferor has been analyzed in the literature [1–4]. There has also been considerable work related to the mitigation of interference on dual-polarized systems using cross-polarization cancellers [4] requiring two orthogonally-polarized input channels. However, to the authors’ knowledge there has been no previous published work related to the development of a single-input demodulator specifically tuned to the QAM-on-QAM cochannel problem. The only related work in this area has focused on the improved demodulation of FM signals in the presence of cochannel FM interference using interlocking phase locked loops [5–7].

This paper proposes a method for improving the performance of a single-input QAM demodulator in the face of cochannel QAM interference by jointly estimating the desired (primary) signal and the interfering (secondary) signal. The method is shown to be a suboptimal realization of the optimal maximum likelihood estimator.

A discrete-time model of the QAM-on-QAM problem is shown in Figure 1. The primary data source generates a sequence of complex symbols designated by

. This symbol sequence is interleaved with zeros to double the sampling rate and then fed into an FIR filter with impulse response

. The FIR filter models the effects of transmitter spectral shaping and channel distortion or intersymbol interference (ISI). Zero interleaving is required to model the nonzero excess bandwidths typical of real modem signals.

Figure 1. Discrete-time model of cochannel QAM system.

The secondary (interfering) signal is generated in a similar manner and then added to the primary signal. The received signal consists of the sum of the primary and secondary signals plus white gaussian noise (AWGN). In addition to transmit filtering and ISI, the primary and secondary channel models include the effects of differences in symbol timing phase, carrier phase, and gain between the primary and secondary signals.

Slow variations in the differential carrier and/or symbol timing phase are modeled by a time-varying channel. While not shown in the figure, larger differences between the carrier frequencies or symbol rates of the primary and secondary signals can easily be included in the model and will not affect the generality of the results that follow.

The problem addressed in this paper is to jointly estimate both the primary and secondary data sequences given a received signal consisting of the sum of the two. For the moment, consider the case where the AWGN is zero and the SIR is poor enough that a conventional demodulator fails. Assume further that the primary and secondary channel impulse responses are known. Since the spectrum of the two signals overlap, a linear filter can do little to improve the demodulability of the received signal. The hypothetical estimation procedure described in the next paragraph addresses this problem.

Hypothesize primary and secondary symbol sequences,

and

. Zero interleave these two sequences, convolve them with the known channel impulse responses, sum the results, and form the error signal between the summed signal and the received signal. Out of all possible symbol sequences, there must exist at least one pair of sequences that results in zero error signal (recall that the AWGN is zero). This pair of sequences comprises the most likely estimates of the transmitted symbol sequences. In the case of nonzero AWGN, form the error power for all possible sequences and choose the pair of sequences that result in minimum error power. This procedure yields the maximum likelihood estimate of the primary and secondary transmitted sequences. The error probability depends solely upon the primary and secondary channel responses and the signal-to-noise ratio. (Calculation of this error probability is beyond the scope of this paper, but will be addressed in [8].) Obviously, this estimation procedure is unrealistic since it requires knowledge of the channel transfer functions and is exponentially complex. A realizable procedure based upon this optimal estimation technique is described in the sections that follow.

The proposed demodulation technique involves the generation of tentative decisions which are then passed into a maximum likelihood error detector/corrector to form the final decision outputs. Refer to Figure 2 for a block diagram of the method used to generate tentative estimates (decisions) of both the primary and secondary transmitted symbol sequences. The received signal r(k), sampled at exactly twice the primary symbol rate, is fed into a T/2-spaced adaptive equalizer. Primary symbol decisions are formed from the equalizer output (a simple hardlimiter for BPSK modulations). The decision error is formed and used to adapt the equalizer. If the symbol error rate is better than approximately one in ten, the adaptive equalizer will converge to the optimum Wiener filter. The decision element outputs a tentative estimate of the primary symbol sequence.

Figure 2. Method used to generate tentative estimates of both the primary and secondary symbol sequences.

Tentative primary decisions are also fed into an adaptive noise canceller which serves to remove the primary modulation from the received signal. Initially, with the secondary equalizer set to identity, the canceller is adapted to minimize the difference between the canceller output and the received signal. In this manner, the canceller converges to an estimate of the primary channel response. Once it reaches steady-state, the canceller and secondary T/2-spaced equalizer are jointly adapted to minimize the secondary decision error. Thus the canceller/equalizer combination removes the primary modulation from the received signal in addition to removing the ISI on the secondary signal. The canceller could have been adapted independently of the equalizer but it has been shown [8] that joint adaptation has better convergence and misadjustment properties. Tentative secondary symbol decisions are formed from the output of the secondary canceller/equalizer.

The above procedure is used to generate tentative decisions of both the primary and secondary transmitted symbol sequences. The symbol error rates (SER) of these tentative decisions will depend upon both the SNR and the SIR. In the discussion that follows, it is assumed that the SNR and SIR are good enough to support tentative SERs of at least one in ten.

The tentative decisions generated as described above are fed into the adaptive channel estimator shown in Figure 3. Both primary and secondary symbol sequences are zero interleaved and fed into FIR filters denoted by

, and

. These filter outputs are summed and compared to an appropriately-delayed version of the received signal. The error signal is used to jointly adapt both filters (using a least-squared error algorithm such as LMS or RLS). Assuming the tentative symbol sequences are independent and have low SER’s, the FIR filters will converge to unbiased estimates of the primary and secondary channel responses. As analyzed in [9], a nonzero SER causes a mild bias in the converged channel model.

Figure 3. Adaptive channel estimator.

For the sake of discussion, assume zero AWGN. At convergence, the error signal between the channel estimator and the received signal will be zero except when there are errors in either of the tentative symbol sequences. Note that a decision error can be modeled as an impulse added to the correct symbol sequence. Thus, the channel modeling error will consist of this impulsive signal convolved with the channel response. Given that tentative symbol errors are separated by an amount of time greater than the duration of the channel impulse response, each symbol error will exhibit itself in the channel modeling error signal as an error event consisting of a replica of the channel impulse response scaled by the magnitude and phase of the error impulse. For zero AWGN, these error events are readily distinguishable in the channel modeling error. This forms the basis of the error detection method.

Once an error event has been detected, it can be corrected using the procedure shown in Figure 4. It tests all possible perturbation sequences in the vicinity of the error event and finds the one that results in the minimum channel modeling error power. The optimum perturbation sequence is then added to a delayed version of the tentative decisions to generate the final output sequences. This procedure is tantamount to performing maximum likelihood sequence estimation localized to the vicinity of the tentative decision sequence where an error is known to have occurred. In this way the complexity is kept to a minimum.

Figure 4. Proposed error detection/correction method.

The error correction technique becomes suboptimal at low SNR’s due to the difficulty in detecting error events. In nonzero AWGN, the channel modeling error will consist of the AWGN plus error events. At low SNR’s, error events will not be distinguishable from the AWGN. Thus the correction algorithm will not be executed and the error will go uncorrected. The technique also becomes suboptimal when the SER of the tentative decisions is bad enough to induce bias in the channel estimates. A biased channel estimate causes an increase in the channel modeling error which masks the error event. This reduces the technique’s ability to find the optimum perturbation sequence.

This section demonstrates the performance of the proposed demodulation technique through Monte Carlo simulations. Results for a specific cochannel interference scenario are presented. The SER performance of the tentative decision estimation process is plotted and the impact of the error detection/correction technique is discussed.

The specific cochannel interference scenario under examination was generated by summing a simulated QPSK signal with a second cochannel QPSK signal and white Gaussian noise. The SIR of the primary (desired) signal with respect to the secondary (interference) signal was 7 dB and the SNR of the primary signal was 30 dB. Both signals were generated with the same symbol rate and carrier frequencies, but the differential carrier and baud phase was selected to cause the maximum possible degradation of the SER at the output of the primary demodulator. For QPSK signals this worst case condition corresponds to a carrier phase offset of

and a symbol phase offset of 50%.

The signal simulated as above was fed into the system of Figure 2. Upon convergence of the adaptive filters, the SER’s of the tentative symbol streams were measured to be

for the primary and

for the secondary. These tentative estimates of the symbol streams were then fed into separate channel modeling filters and summed to estimate the received signal. Since the SER in the two symbol streams was better than

, the channel models converged to reasonably accurate estimates of the two signal paths. Upon convergence, the difference between the channel model output and the received signal (the channel modeling error) should be approximately equal to the additive Gaussian noise except when a decision error occurs in either of the channel model input symbol streams. An incorrect symbol will result in poor cancellation of the signal in question within the neighborhood of that incorrect symbol and consequently an abrupt increase in the channel modeling error. This increase in the channel modeling error is not localized to a single output sample but is spread over several data samples as discussed in the previous section.

Figure 5(a) shows 1000 samples of the channel modeling error signal for the interference scenario described above. Note the sudden increases in the channel modeling error due to incorrect tentative primary and secondary decisions. For the SNR of this simulation, regions where tentative decision errors occur are easily detectable. These regions are known as error events. Error events are automatically detected using an algorithm which monitors the short term power level of the channel modeling error signal. Due to the tendency of primary symbol errors to induce secondary symbol errors, the increase in the channel modeling error is due to the sum of two or more error pulses originating from either symbol stream, convolved with the two channel responses. This makes it difficult to correct the error merely by observing the shape of the channel modeling waveform. Instead, the algorithm waits for a sudden increase in the channel modeling error power and then recalculates the channel modeling error with different tentative symbols being substituted for those symbols believed to be wrong. The algorithm selects the two symbol sequences which result in the lowest short term channel modeling error power in the immediate vicinity of an error event.

Figure 5. Channel modeling error signal (a) before error correction and (b) after error correction.

Due to the smearing of symbol error and the overlapping nature of primary and secondary errors, the exact error location cannot be precisely pinpointed but can be postulated within a few symbols. This forces a larger number of symbols to be retried each time an error event is detected to insure that all possible sources of the error are examined.

Figure 5(b) shows the channel modeling error after the three tentative symbol errors have been corrected. Notice that the power of the channel modeling error is more constant. The complete performance of this error detection/correction algorithm has not yet been fully explored. Future work will quantify the suboptimality of this method as compared to true joint maximum likelihood sequence estimation. For isolated simulations, the proposed error detection/correction method appears capable of completely eliminating errors in the tentative symbol streams.

The SER performance of the tentative estimation procedure (shown in Figure 2) has been extensively characterized through Monte Carlo simulations. Figure 6 shows contours of primary and secondary symbol error rates as a function of the primary SIR and SNR for a 20% square root raised cosine Nyquist channel. The figure characterizes the SER for cochannel QPSK signals assuming the worst case carrier and symbol phase offsets as described above. In order to achieve a primary SER better than

, the SIR and SNR must be greater than (above and to the right of) the primary contour labeled

in Figure 6. This curve shows that at high SNR, an SIR greater than 6.4 dB is required to achieve an SER of better than

. Similarly, to achieve this SER at high SIR, the SNR must be greater than 8.2 dB.

Figure 6. Performance contours for tentative primary and secondary symbol streams before error correction.

The performance contours for a given SER in the secondary symbol stream enclose a small subset of the corresponding region for the primary symbol stream. This is true for two reasons. First, the SNR of the secondary signal after primary cancellation will always be less than the primary SNR because both signals share the same noise floor and the secondary signal is (by definition) the lower power signal. This causes the secondary SER contours to approach a diagonal asymptote at high SIR instead of a vertical one as in the case of the primary. This diagonal asymptote corresponds to the conditions where perfect cancellation of the primary signal has occurred and the remaining secondary signal has an SNR which yields the SER of the contour.

At high SNR (and relatively low SIR) the secondary performance contour is controlled by a different factor. Here secondary symbol errors are mainly due to imperfect cancellation of the primary modulation resulting from primary symbol errors. Normally, each primary error will induce at least one secondary error and, in the worst case, two secondary errors in this region of the SNR-SIR space. The secondary performance contour will therefore parallel the primary contour at high SNR, but will remain slightly above it.

While the effect of the error detection/correction procedure on the above performance contours have not yet been extensively studied, they can be surmised as follows: The error correction procedure will expand the regions for achieving a given SER in those cases where the errors are mainly caused by the cochannel interference and not AWGN. Here, tentative symbol errors will cause jumps in the channel modeling error which will stand out clearly against the relatively low Gaussian noise floor. As the SNR becomes lower, the channel modeling error power will be dominated by Gaussian noise. Short term measurements of that error power will become less and less sensitive to the effects of wrong symbols. This means that the horizontal portions of each of the performance contours in Figure 6 can be lowered significantly by the above error correction scheme, but in the regions of the contours where the AWGN dominates, the error rate cannot be improved.

The error correction technique will also tend to fail when the error rates of the tentative symbol streams are high enough to affect the convergence of the channel modeling filters. This is expected to be a factor at symbol error rates of

and below. It is therefore expected that the error correction scheme will lower the

performance contour by a small amount, but that the performance contours for lower symbol error rates will be brought down within a few tenths of a dB of that contour. This amounts to a threshold effect with respect to the SIR. The error correction technique will offer no improvement until the SIR is reached which yields an SER of

. Then the SER will improve very quickly from

for any further increases in the SIR.

This paper has presented a method for demodulating a QAM signal in the presence of a second QAM interferor and intersymbol interference based upon joint maximum likelihood estimation of both the desired signal and the interferor. Monte Carlo simulations indicate the method will yield substantial improvement in performance relative to conventional demodulation. A future paper [8] will report on the use of the Viterbi algorithm as an improved method for implementing the joint maximum likelihood estimator, as well as error bounds for simple channels.