WO1991015900A1 - Trellis precoding for fractional bits/baud - Google Patents

Abstract

A digital data sequence is mapped into a signal point sequence e(D) for data transmission over a channel h(d) (40) to produce a channel output sequence y(d)=e(D)h(D). e(D) is chosen so that the signal points in the sequence y(D) belong to a class of possible sequences based on the digital data sequence, the class being based on a time-varying trellis code (Fig. 4) derived from an N-dimensional time-invariant trellis code by using different transformed versions of its underlying lattices for respectively different N-dimensional symbols.

Description

Trellis Precoding for Fractional Bits/Baud

Background of the Invention

This invention relates to modulation systems for sending digital data via a channel.

In a quadrature amplitude modulation (QAM) system, for example, two key parameters are the baud rate 1/T and the number of bits per baud β. The bit rate R of the QAM system is given by R=β/T, measured in bits per second (b/s).

Transmission equipment, such as a voiceband modem, operates at one of a number of possible bit rates (e.g., in the range 9.6 to 19.2 kb/s) depending on the line quality or the user's requirement. Therefore, for a given baud rate such a modem must be able to operate at different values of β, which may be a fraction, and the different values of β may not differ simply by integers. For example, if the baud rate is 2743 symbols/s, then to send at conventional modem rates of 19.2, 16.8, and 14.4 kb/s requires β=7, 6.125 and 5.25, respectively.

When β is a rational number of the form β=n+d/2^m, where n and 0<d<2^m are integers, a higher-dimensional (higher than two-dimensional ) QAM constellation, of dimension L=2^m with 2^βL/2 points can be used to represent βL/2 bits over L dimensions. Higher-dimensional constellations save in average power when their signal points are chosen to lie within

hyperspheres instead of hypercubes. This saving, called shaping gain, can be as high as 1.53 dB.

Because the size of the signal constellation increases rapidly with n and m, mapping from bits to signal points can be cumbersome. For example, in an uncoded system (one in which there is no interdependence between selected signal points), to send 7.5 bits/baud, a four-dimensional constellation with

2¹⁵=32,768 points is needed, which may be too large to

handle. However, such a 4D constellation can be formed by the Cartesian product of simple 128-point and 256-point two-dimensional constellations. More generally, a simple way of sending β=n+d/2^m bits per two-dimensions is to define a series of frames each encompassing the bits in a group of 2^m bauds and then for each frame send n bits/baud for (2^m-d) bauds and send (n+1) bits/baud for d bauds. The average power per two-dimensions for this scheme is approximately

P_β=P_n(1+d/2^m), where P_n is the average power of a

2ⁿ-point two-dimensional (2D) QAM constellation (The 2D

constellations are scaled to have the same minimum distance between adjacent points.). This approach is approximately as efficient as a conventional QAM system sending an integer number of bits/baud. However, the imbalance in size between the two 2D constituent constellations creates a large

peak-to-average ratio (PAR) in two-dimensions, making the system susceptible to other impairments such as nonlinear distortion or phase jitter.

Generalized cross-constellations, described by Forney and Wei, "Multidimensional Constellations - Part I:

Introduction, Figures of Merit, and Generalized Cross

Constellations," IEEE JSAC, pp 877-892, August, 1989, offer an alternative way of generating higher-dimensional constellations to represent fractional bits/baud. Here, the

higher-dimensional constellation is a subset of the signal points in a Cartesian product of identical expanded

two-dimensional constellations; simple coding rules are used to construct the subset. A generalized cross-constellation has a low 2D constellation expansion ratio (CER) and a low PAR in two-dimensions; however, it also generally has a low shaping gain.

Higher-dimensional constellations with simplified bit mapping and with significant shaping gain can be constructed using the Voronoi region of a lattice Λ' as the boundary of the signal point set, Forney, "Multidimensional Constellations - Part II: Voronoi Constellations," IEEE JSAC, pp 877-892, August, 1989. If signal points are chosen from some lattice Λ, then Λ and Λ' must be scaled such that the Voronoi

region of Λ' contains 2^βL/2 points from Λ; that means the order |Λ/Λ'| of the lattice partition Λ/Λ' must be

2^βL/2. The mapping of bits to signal points involves a decoding operation whose complexity is that of a

minimum-distance decoder for Λ'. The inverse mapping is accomplished by a simple hard-decision decoding operation. A Voronoi constellation generally is more complex to generate than a generalized cross constellation. For given lattices Λ and Λ', the Voronoi constellation can typically support only- certain fractional values of β, and that by using different versions of Λ'.

An important expansion of the Voronoi constellation technique is trellis shaping disclosed in United States patent application serial number 312,254, filed February 16, 1989, assigned to the same assignee as this application, and

incorporated herein by reference. In trellis shaping, the signal constellation is constructed on a sequence basis rather than on a block basis. Instead of the Voronoi region of a lattice, the decision regions associated with trellis codes are used to form the signal constellation boundary. In analogy to Voronoi constellations, in trellis shaping the mapping from bits to signal points involves a decoder for the trellis code. Trellis shaping offers a better performance/complexity

trade-off than Voronoi constellations; but, it also can support only certain fractional values of β.

An extension of Voronoi constellations and trellis shaping to non-ideal channels with linear distortion or

correlated noise has been disclosed in United States patent application serial number 351,199, filed May 12, 1989, assigned to the same assignee as this invention, and incorporated by reference. This extension, called lattice or trellis

precoding, provides the performance of ideal decision feedback equalization (DFE), preserves the coding gain of known coded modulation schemes designed for ideal channels, and, in

addition, provides shaping gain. Trellis precoding is similar to trellis shaping, however, it involves a decoder for a filtered trellis code rather than for the ordinary trellis code. Trellis precoding is a practically important scheme, but, like trellis shaping, is limited in the fractional values of β that it can support.

Summary of the Invention

The invention can support values of β (bits per baud) that differ by fractions without significantly modifying the underlying trellis codes used in the previously disclosed trellis precoding scheme. This is achieved by using a shaping trellis code C_s which is derived from some conventional

trellis code C_s(Λ/Λ',C) by using different versions of the underlying lattices Λ and Λ' at different times, in a

periodic fashion. For example, if C is an N-dimensional trellis code, to send (N/2)β=n+(d/2^m) (N/2) bits/N-dimensions, where 0≤d<2^m, we define a frame (period) of length

L/2=(N/2)2^m bauds, and use the code C for (N/2)(2^m-d)

bauds, and its scaled version RC for (N/2)d bauds.

In general, in one aspect, the invention features mapping a digital data sequence into a signal point sequence e(D) for data transmission over a channel h(D) to produce a channel output sequence y(D)=e(D)h(D). e(D) is chosen so that the signal points in the sequence y(D) belong to a class of possible sequences based on the digital data sequence, the class being based on a time-varying trellis code derived from an N-dimensional time-invariant trellis code by using different transformed versions of its underlying lattices for

respectively different N-dimensional symbols. Preferred embodiments include the following features. In some embodiments, the class comprises a signal space

translate of a time-varying trellis code C_s, where the

translation is a code sequence from a translate of a trellis code C_c determined based on the digital data sequence, and the time-varying trellis code C_s is derived from an

N-dimensional time-invariant trellis code C(Λ/Λ', C_s) by using different transformed versions of its underlying lattices Λ and Λ' for respectively different N-dimensional signal points.

In some embodiments, the class comprises a signal space translate of a trellis code C_s, where the translation is a code sequence from a translate of a time-varying trellis code C_c determined based on the digital data sequence and the time-varying trellis code C_c is derived from an N-dimensional time-invariant trellis code C(L/L', C_c) by using different transformed versions of its underlying lattices L and L' for respectively different N-dimensional signal points.

In some embodiments, the class comprises a signal space translate of a label translate of a time-varying trellis code C_s, where the label translate is based on a portion of the digital data sequence and the signal space translation is based on another portion of the elements of the digital data sequence, and the time-varying trellis code C_s is derived from an N-dimensional time-invariant trellis code C(Λ/Λ',

C_s) by using different transformed versions of its underlying lattices Λ and Λ' for respectively different N-dimensional signal points.

In some embodiments, the class comprises a signal space translate of a label translate of a trellis code C_s, where the label translate is based on a portion of the digital data sequence, where the signal space translation is based on another portion of the elements of the digital data sequence and is a code sequence from a time-varying trellis code C_c which is derived from an N-dimensional time-invariant trellis code C(L/L', C_c) by using different transformed versions of its underlying lattices L and L' for respectively different N-dimensional signal points.

In some embodiments, the class of possible sequences is identified by some sequence of subregions into which N-space has been partitioned, the sequence of subregions being

specified by a label sequence which belongs to a coset of a convolutional code C_s and the coset is determined based on a portion of the digital data sequence and the signal points in the subregions belonging to a coset of a time-varying trellis code C_c, where the sequences from the trellis code C_c are chosen based on another portion of the elements of the digital data sequence, and the time-varying trellis code C_c is

derived from an N-dimensional time-invariant trellis code

C(L/L', C_c) by using different transformed versions of its underlying lattices L and L' for respectively different

N-dimensional signal points.

In some embodiments, the class of possible sequences is identified by some sequence of subregions into which N-space has been partitioned, the sequence of subregions being

specified by a label sequence which belongs to a coset of a convolutional code C_s, the signal points in the subregions belonging to a coset of a trellis code C_c, where sequences from the trellis code C_c are chosen based on another portion of the elements of the digital data sequence and using

different approximately scaled versions of the subregions for respectively different N-dimensional signal points.

In general, in another aspect, the invention features choosing e(D) so that y(D) belongs to a signal space translate of a label translate of a trellis code C_s, the label

translate being based on a portion of the digital data sequence, and the signal space translation is specified based on another portion of the elements of the digital data sequenc and is a code sequence from a trellis code C_c.

In general, in another aspect, the invention features choosing x(D) so that the signal points in the sequence y(D) belong to some sequence of subregions into which N-space has been partitioned, the sequence of subregions being specified by a label sequence which belongs to a coset of a convolutional code C_s, the coset being determined based on a portion of the digital data sequence, the signal points in the subregions belonging to a coset of a trellis code C_c, and where

sequences from the trellis code C_c are chosen based on

another portion of the elements of the digital data sequence.

In preferred embodiments, the versions are

respectively used periodically (e.g., two different versions used alternatingly) wherein each period encompasses multiple

N-dimensional symbols. The transformed versions comprise rotated and/or scaled versions of the underlying lattices. The trellis code C_s is filtered to form a filtered trellis code

C_s' whose sequences are c_s' (D) = c_s(D)g(D) where g(D) is the formal inverse of a response related to the channel

response h(D), and c_s(D) is any code sequence in the trellis code C_s. The signal point sequence selection tends to

minimize the average power of the signal point sequence e(D) = y(D)g(D). The signal point sequence e(D) lies in a fundamental region of the filtered trellis code C_s'. The signal point sequence is selected based on reduced state sequence estimation with respect to the filtered trellis code C_s'. The reduced state sequence estimation uses no more states than the number of states of the original trellis code C_s'. The fundamental region of the filtered trellis code comprises approximately a Voronoi region of the filtered trellis code C_s'. The digital data sequence is recovered from a possibly noise-corrupted version of the signal point sequence, including decoding the signal point sequence to a sequence of estimated digital elements and forming a syndrome of fewer digital elements based on a portion of the estimated digital elements using a

feedback-free syndrome former H^T.

The signal point sequence is selected by mapping the digital data sequence into an initial sequence belonging to and representing a congruence class of the original trellis code

C_s, and choosing a signal point sequence belonging to and representing a congruence class of the filtered trellis code C_s' and which has no greater average power than the initial sequence, and wherein the mapping includes applying a portion of the elements of the digital data sequence to a coset

representative generator for forming a larger number of digital elements representing a coset representative sequence. The coset representative generator comprises a multiplication of a portion of the elements of the digital data sequence by a coset representative generator matrix (H^-1)^T which is inverse to a syndrome-former matrix H T for the code.

The trellis code C_s may be a linear trellis code or a non-linear trellis code. The linear trellis code C_s is a 4-state Ungerboeck code. The trellis code C_s is based on a partition of binary lattices, or a partition of ternary or quaternary lattices.

The step of selecting the signal point sequence is further constrained so as to reduce the peak power of the signal point sequence where the peak power represents the maximum energy of the signal point sequence in some number of dimensions N. In some embodiments, N = 2 or N = 4. The step of mapping the digital data sequence into the signal point sequence is arranged to ensure that the digital data sequence can be recovered from a channel-affected version of the signal point sequence which has been subjected to one of a number of predetermined phase rotations.

Other advantages and features will become apparent from the following description of the preferred embodiment and from the claims.

Description of the Preferred Embodiment

We first briefly describe the drawings.

Fig. 1 is a block diagram of trellis precoding.

Fig. 2 is a framing scheme.

Fig. 3 is a block diagram of an alternatively sealed trellis code transmitter.

Fig. 4 is a trellis diagram.

Fig. 5 is a reduced state trellis.

Fig. 6 is a diagram of the steps of a Viterbi algorithm.

Fig. 7 is a trellis for a Wei code.

Fig. 8 is a general block diagram of a transmitter and a receiver used in precoding.

Fig. 9 is a block diagram of a transmitter.

Fig. 10 is a block diagram of a receiver.

Fig. 11 is a block diagram of binary encoding for trellis precoding.

Fig. 12 is a block diagram of a Wei binary

convolutional encoder.

Figs. 13A, 13B are diagrams of quadrants of signal sets.

Figs. 14A, 14B are diagrams of quadrant shifting.

Fig. 15 is a diagram of portions of the trellis precoder of Fig. 9.

Fig. 16 is a block diagram of a binary syndrome former. Fig. 17 is a block diagram of the binary portion of the trellis decoder.

Fig. 18 is a block diagram of a binary inverse syndrome former. Fig. 19 is a block diagram of a constellation former.

Fig. 20 is a diagram of a quadrant labeling scheme.

Fig. 21 is a diagram of a phase-invariant labeling scheme.

Fig. 22 is a diagram of an encoder for trellis precoding based on label-translates of trellis codes.

Fig. 23 is a diagram of a decoder for trellis precoding based on label-translates.

Fig. 24 is a diagram of an encoder for trellis precoding based on regions.

Fig. 25 is a diagram of a decoder for trellis precoding based on regions.

Fig. 26 is a diagram of regions derived from a 4-way partition.

Trellis Precoding

First we will summarize the principles of trellis precoding. Certain terminology and principles which underlie the invention are set forth in Appendix A.

Referring to Fig. 1, trellis precoding is an encoding technique based on a discrete-time channel 40 operating at a baud rate 1/T. (If the actual channel is continuous-time, or discrete-time operating at a higher sampling rate, the

discrete-time channel at the baud rate can be constructed by appropriate filtering, sampling or sampling-rate conversion operations, as described in the trellis precoding patent application cited above).

We assume the following input-output relationship for the channel:

r(D) = e(D)h(D) + n(D)

where e(D) is the channel input, h(D) is a causal, stable channel response and n(D) is a noise sequence that can be modeled approximately as Gaussian and white.

Let C_s(Λ_s/Λ_s';C_s) be a time invariant trel l is code based on N-dimens ional l att ices Λ_s and Λ_s '

and a rate-k_s/(k_s+r_s) binary convolutional code C_s. As

in the trellis shaping patent application cited above

(Assumption A), we assume that C_s has the property that all possible symbols assigned to trellis branches leaving any state s_k of the encoder belong to a coset Λ_0s + a(s_k) of

some lattice Λ_0s, called the time-zero lattice of C_s.

The following theorem, which is proven in the Appendix, plays a central role in trellis precoding:

Theorem 1. Let R₀ be any fundamental region of the

N-dimensional time-zero lattice Λ_0s of C_s, and let

x(D)ε(R₀)^∞ and c_s(D)εC_s be unknown sequences

whose sum is y(D) = x(D) + c_s(D). Then x(D) and c_s(D) can be uniquely determined from y(D).

This important theorem implies that if x(D) is an information-bearing signal that lies in the region (R₀)^∞, then instead of transmitting x(D), one may transmit e(D) =

[x(D)+c_s(D)]g(D), where g(D) = 1/h(D), and still be able to recover x(D) at the receiver, provided that c_s (D)εC_s.

Any desired criterion may be used to select the code C_s and the code sequence c_s(D). First, we consider strictly an

average power criterion, in which case the code C_s and the code sequence c_s(D) are chosen to minimize the average power of the transmitted seqeunce e(D) (the mean-squared 'error').

In Fig. 1, the first step in trellis precoding is to map the incoming bit stream 42 into the information-bearing signal x(D) using a trellis code 44 to map to words and simple mapping 46 to map from words to x(D). For simplicity, the fundamental region R₀ in N-space can be chosen as the

Cartesian product of simple two-dimensional rectangular regions.

To obtain coding gain we choose x(D) from a coset of some trellis code C_c (Λ_c/Λ_c', C_c). Without much

loss of generality, we assume that Λ_c and Λ_c' are both N-dimensional binary lattices and that C_c is based on a binary rate-k_c/(k_c+r_c) convolutional encoder. It is

essential that C_s be a subcode of C_c, which is assured if

Λ_s is a sublattice of Λ_c'. This implies that if

c_s(D) is a sequence from C_s and x(D) is a sequence from

C_c, then c_s(D) + x(D) is also a sequence from C_c.

To send β bits per two dimensions, we will initially require that the fundamental volume of Λ_0s be 2^Nβ/2+rc

times larger than the fundamental volume of Λ_c, so that

2^Nβ/2+rc points from Λ_c can be placed in R₀ (to

accommodate the Nβ/2 information bits and r_c redundant bits);

i.e., |Λ_c/Λ_0s|=2^Nβ/2+rc. Now, given C_c and

C_s, we can determine the values of β that trellis preceding

can support. First, we write the partition Λ_c/Λ_0s more

explicitly in terms of the partition chain

Λ_c/Λ_0c/Λ_s/Λ_0s. We have

|Λ_c/Λ_0c|=2^rc

|Λ_c/Λ_0c|=2^rc

|Λ_c/Λ_0s|=2^rs^.

Furthermore , if we def ine | Λ_c' /Λ_s |=2^j , where j is

some integer, then it follows that

log₂ |Λ_c/Λ_0s |=r_c+k_c+r_s+j, and therefore

since also log₂ |Λ_c/Λ_0s| = Nß/2+r_c,Nβ/2=k_c+r_s+j;

that means trellis precoding can support β=(2/N)(k_c+r_s+j)

bits per two dimensions (i.e., per baud in a QAM system).

Suppose we replace the code C by its scaled and rotated version RC_s that is based on the partition

RΛ_s/RΛ_s', where R is the N-dimensional rotation

operator defined in G.D. Forney, Jr. "Coset codes--Part 1:

Introduction and Geometrical Classification," IEEE Inform. Vol.

Theory IT-34, pp. 1123-1151, Sept. 1, 1988. Then

|Λ_c'/RΛ_s |=2^j+N/2 and hence β=(2/N) (k_c+r_s+j)+1.

This illustrates that for binary lattices, trellis precoding can be scaled to support integer increments in β, by simply replacing the code C_s by its rotated, scaled versions. A similar effect can be achieved by replacing the code C_c by its scaled and rotated version RC_c.

Having discussed the scaling and the mapping of bits into the information bearing sequence x(D), we now proceed to describe the remaining operations in trellis precoding.

Referring again to Fig. 1, once x(D) is determined, the trellis precoder 48 tries to find a code sequence c_s(D)εC_s

such that the sequence

e(D)=[x(D)-c_s(D)]g(D)

has as small an average power as possible. The "error

sequence' e(D) is then transmitted.

The received sequence is given by

r(D) = e(D)h(D)+n(D)

= y(D)+n(D)

= x(D)-c_s(D)+n(D).

Since x(D) is chosen from a translate C_c+a(D) of C_c,

y(D)=x(D)-c (D) is also a code sequence in C_c +a(D).

Therefore, a conventional decoder 50 for C_c can estimate y(D) with essentially the same error probability as if x(D) were

detected with an ideal DFE which cancels the ISI due to the

tail of the impulse response. If the decoder can correctly

recover y(D)=x(D)-c_s(D), then it follows from Theorem 1 that x(D) can be correctly extracted using the simple hard-decision decoder 52 corresponding to the region (R₀)^∞. In order

to prevent an error in y_k from triggering error propagation, the syndrome-former methods, disclosed in the trellis precoding patent application cited above, may be used. The original bit stream 42 may then be recovered by inverse mapping 54.

Fractional bits per baud

As demonstrated above, using trellis codes based on binary lattices, trellis precoding can support values of β, the transmitted bits/baud, that differ by integers. In the

invention, trellis precoding can also support fractional differences in β without significantly changing the shaping code C_s.

This is achieved by deriving the shaping code C_s from some time-invariant code C by using different (rotated, scaled) versions of its underlying lattices, Λ and Λ', at different times in a periodic fashion. (Again, a similar effect can be achieved by deriving the coding code C from some time invariant code C(L/L', C_c) by using a different rotated, scaled version of its underlying lattices L and L'.) As we saw above, if C(Λ/Λ',C) can support k_c+r_s+j

bits/N-dimensions, then it can be made to support

k_c+r_s+j+(N/2) bits/N-dimensions, by replacing Λ and Λ'

by RΛ and RΛ', respectively. To send

(N/2)β=k_c+r_s+j+(d/2^m)(N/2) bits/N-dimensions, where

0≤d<2^m, we define a frame (period) of length L/2 = (N/2)2^m bauds, and use the code C for (N/2)(2^m-d) bauds, and its scaled version RC for (N/2)d bauds.

Using RC and C at different tynes will create an imbalance in 2D constellation sizes and result in a PAR that is higher than that of trellis precoding with time-invariant trellis codes. However, using constellation constraints, this imbalance will be substantially eliminated.

In the preferred embodiment, we use the 4D 16-state

Wei code (described in the trellis precoding patent

application) as the code for fundamental coding gain, and we use the 2D 4-state Ungerboeck code (described in the trellis precoding patent application) as the code for shaping gain. With this combination N=4, k_c=3, and r_s=2, and we can send at rates n+0.5 bits/baud using time-invariant scaling, where n>2 is an integer. To transmit at 19.2 kb/s using a baud rate of 1/T=2743 symbols/s, however, we need to send 7 bits/baud. This may be done as follows.

The 4D 16-state Wei code C_c is based on a rate-3/4 convolutional encoder and the four-dimensional partition

Z 4/2Z4. This code adds one redundant bit every four

dimensions (N=4, r_c=1). On the other hand, the code C_s is derived from the mod-2 trellis code C which is based on a rate-1/2 binary convolutional encoder and the two-dimensional partition Z²/2Z². Equivalently, C can be described by

a rate-2/4 convolutional encoder and the four-dimensional partition Z⁴/2Z⁴ (N=4, k_s=2). In this form, C has the

4D time-zero lattice RZ ⁴. To send 6.5 bits per symbol using these codes, we would pick C_s = 8C such that it is based on the partition 8Z⁴/16Z⁴, with time-zero

lattice 8RZ⁴. Then |Λ_C/Λ_0s|=|Z⁴/8RZ⁴|=2¹⁴=2^2x6.5+1=2^2β+rc. On the other hand, to send 7.5 bits/symbol, we would pick C

= 8 RC such that it is based on the partition

8RZ⁴/16RZ⁴, with time-zero lattice 16Z⁴. Then

|Λ_C/Λ_0s|=|Z⁴ /16Z⁴|=2^2x7.5+1=2¹⁶.

Thus, referring to Fig. 2, we can send an average of 7 bits/symbol (i.e., 7 bits/baud) by using a frame of 4 bauds or

L = 2x4=8 dimensions, and by sending 13 and 15 bits in

alternate 4D blocks using respectively the 8Z 4/16Z4 and

the 8RZ ⁴/16RZ⁴ based codes for shaping. Referring to

Fig. 3, the alternating of codes is achieved by including in the transmitter a framer-blocker 60 which assembles the bits into frames and organizes the frames into blocks 62.

Every two bauds, the coding code C_c will output the two symbols (x_2k and x_2k+1) and these would be processed by the precoder which will operate on a 2D trellis representation of the trellis code C_s. Referring to Fig. 4, for the first two bauds of the frame, this trellis corresponds to the trellis of the code 8C, whereas for the next two bauds, it corresponds to the trellis of the code 8RC. The decoder will search through this time-varying trellis to find a sequence c_s(D). Filtered trellis codes

The time varying trellis code discussed in the preceding section is subjected to a filtering process to form a filtered trellis code in the same manner as described in the trellis precoding patent application cited above. This is accomplished in the shaping/precoding operation 48 in Fig. 1. More specifically, we seek a c_s'(D) that minimizes

||e(D)||² = ||x'(D) - c_s'(D)||², where c_s'(D) (58

in Fig. 1) is any sequence of the form c_s(D)g(D),

c_s(D)εC_s. We call the set of all such c_s'(D) the

filtered trellis code C_s' = C_sg. To implement trellis

precoding, we need a minimum mean-squared error (MMSE) decoder for C_s'.

Even if C_s can be represented by a finite-state trellis, however, C_s' in general cannot. Indeed, if s_k is the state of an encoder that generates a code sequence

c_s(D)ε C_s at time k, then the state of the sequence

c_s'(D) = c_s(D)g(D) at time k is s_k' = [s_k;p_k], where

p.k = [c_s,k-1, c_s,k-2, ...] represents the state of the

'channel' whose response is g(D). The number of possible values of p_k is not finite, even if g(D) has finite degree, because the set of possible code symbols c_s,k-i, i=1,2, ..., is a coset of a lattice and thus has infinitely many elements. Therefore, in general, an MMSE decoder for C_s' cannot be realized by a trellis search.

Reduced-state decoders for filtered two-dimensional trellis codes

We now show how to construct a class of near-optimum 'reduced-state' trellis decoders for C_s', using reduced-state sequence estimation (RSSE) principles.

In this section, we shall assume that C_s is a two-dimensional trellis code derived from a scaled, rotated version of some 2D code C which is based on the partition

Λ/Λ' = Z²/R^ks^+rsZ²; these codes can be

represented by trellises that have 2^ks transitions per branch where each branch is associated with a unique coset of a scaled, rotated version of R^ks^+rsZ². In what

follows, we will denote the partition used at time k as

Λ_s,k/Λ'_{s , k} , k = 1_'..., L/2. In the next section, we

will show how the method may be extended to higher-dimensional trellis codes. It will be apparent that two-dimensional codes are the most natural ones to e when g(D) is a complex-valued response.

Two relevant principles of reduced-state sequence estimation are, first, to keep track only of the last K code symbols c_{s , k -i} ,1≤i≤K, even when the filter response g(D) has degree greater than K, and second, to keep track of c_{s ,k-i} only with respect to its membership in one of

| Λ_{s , k-i}/Λ_k( i ) I = 2^j i = J_i cosets

Λ_k ( i ) +a( c_{s ,k- i} ) of some lattice Λ_k( i ) . The sequence

of lattices, Λ_k(i) must satisfy the property that j_i,

i=1,...,K, is a non-increasing sequence so, as a code symbol c_{s , k-i} becomes less recent, the information that is kept about it becomes coarser and coarser, until it is forgotten altogether (at i = K).

A code symbol c_s,k-i, 1≤i≤K; in Λ_s,k-i belongs to one of J_i cosets of Λ_k(i). We denote this coset by

a(c_s,k-i). Then, the coset state' of a code sequence

c'_s(D) = c_s(D)g(D) at time k can be defined as t_k =

[a(c_s,k-1), ..., a(c_s,k-K)].

Next we concatenate coset states t_k with encoder states s_k to obtain 'super-states' v_k :

v_k = [s_k;t_k]

= [s_k;a(c_s,k-1), ...,a(c_s,k-K)].

Note that given the current state v_k and the code symbol c_s,k, the next state v_k+1 is uniquely determined. The

uniqueness is guaranteed by the condition that J_i is a

non-increasing sequence.

Referring to Fig. 5, the super-states define a reduced-state trellis, which we sometimes call a

super-trellis. In this trellis, signal points associated with all branches 27 leaving a super-state 25 belong to a coset of the time-zero lattice Λ_s0,k. If Λ_s,k' is a sublattice

of Λ_k (1), then as in the ordinary trellis there are 2^ks

branches leaving any super-state, each associated with one of the 2^ks cosets of Λ_s,k' whose union is the given coset of

Λ_s0,k' If Λ_k (1) is a sublattice of Λ_s,k', on the

other hand, then there is a branch for each of the 2^j1^-rs cosets of Λ_k(1) whose union is the given coset of Λ_s0,k.

The number of possible values that v_k can take is finite. In fact, if Λ_k(i) is a sublattice of

Λ_s,k' (j_i≥k_s+r_s) for all i < K, and Λ_k (K) is a

sublattice of Λ _{S 0 ,k} (j_k≥r_s), then the total number of

super-states is given by

S' = S π_1≤i≤K 2^ji^-rs.

Having described the reduced-state trellis, we now show how the Viterbi algorithm is used to search these

trellises. Let g(D) = g_K(D) + g_∞(D), where g_K(D) = 1 +

g₁D + ... + g_KD^K is the polynomial corresponding to the

first K+1 terms of g(D), and g_∞(D) is a residual component of possibly infinite span. Let g_∞(D) = g_N(D)/g_D(D),

where g_N(D) and g_D(D) are both finite-order polynomials.

Then g_N(D) has the same delay as g_∞(D), whose delay is at least K+1.

Suppose that the input to the reduced state decoder is x'(D). Referring to Fig. 6, let us define the Euclidean distance path metric of a code sequence c_s'(D) at a given time k as the cumulative sum P_k=∑_j≤kγ_j

of the branch metrics γ_j,j≤k, computed (31) according to

γ(D)=||x'(D)-c_s'(D)||²

=||x (D)-c_s(D)g_K(D)-w(D)||²

where w(D) = c_s(D)g_∞(D), which means that w(D) satisfies

w(D) =-w(D)[g_D(D)-1]+c_s(D)g_N(D),

with w_k=0, for k≤0. Note that [g_K(D) - 1], g_N(D), and [1

- g_D(D)] all have a delay of at least l; the Viterbi

algorithm (VA) can proceed in a recursive manner 37 by

comparing metrics (33) of merging paths and retaining 35 the surviving sequences with minimum path metrics. Therefore, the VA needs to store the w_k 's for every surviving path. It should be noted that even though only K terms of the filter response g(D) are taken into account in the trellis

construction, its entire memory is included in the branch metric computations.

Among these reduced-state decoders, there is one that particularly stands out in terms of its tradeoff of performance against complexity. This is the special case of parallel decision-feedback decoding (PDFD), where the reduced-state trellis is simply the trellis of the original code C_s. This is obtained by setting K=0. The PDFD decoder is the simplest in this class, and its performance is. the poorest, although it often performs close to the optimum decoder.

A practical VA has a finite decoding delay M. We can assume that it effectively makes a decision on the kth symbol c_s,k based on observations x_k, x_k+1' ..., X_k+M,

assuming the correct node at time k is v_k. (Under this

assumption the VA will always select a true code sequence.) For M = 0, such a decoder becomes a simple decision-feedback decoder (DFD), where in every symbol interval the decoder performs filtering using past decisions, and operates like a hard-decision decoder by selecting the closest signal point c_k in an appropriate coset of Λ_s0,k using a MMSE decoder for Λ_s0,k. Clearly in this case, regardless of the channel response, the error sequence will always lie in

[R_V(Λ_s0,k)]^∞, where R_V(Λ_s0,k) is the Voronoi

region of the time-zero lattice Λ_s0,k, and the average

power of these regions will determine the average power of the transmitted symbols. Since Λ_s0,k may be time-varying, the size of the signal constellation boundary may be time-varying also. This may translate into a higher peak-to-average ratio. However, this will be controlled by putting constraints into the decoder as will be explained later.

The error sequence e(D) = x(D) - c_s(D) will lie in some decision region of the RSSE decoder for the filtered trellis code C_s'. The average power of this region will essentially determine the shaping gain that can be achieved.

This region, and thus the shaping gain will generally depend on the interaction between the shaping code C_s and the filter g(D), as we have seen with MMSE decoders.

In practice, we can measure the shaping gain of an error region as follows. We take an input sequence x'(D), which is uniformly distributed in some simple fundamental region of C_s'. We decode it and obtain an error sequence e(D). This sequence will be uniformly distributed in the common error region of the decoder. The output mean-squared error (MSE) gives the shaping gain. For M = 0, we get the shaping gain of the regions R_V(Λ_s0,k)_, k = 1,..., L/2.

As M is increased towards infinity, the error region will change and the output MSE will monotonically decrease towards a limit which depends on the reduced-state trellis being used. Reduced-state Decoders for Filtered Multidimensional Trellis Codes

We now extend these reduced-state decoding methods to higher-dimensional (N > 2) trellis codes C_s, such as the dual Wei codes. The dual Wei codes are duals of the Wei codes of the kind disclosed in Wei, "Trellis-Coded Modulation with Multidimensional Constellations," IEEE Trans. Inform. Theory, Vol. IT-33, pp. 483-501, 1987, incorporated herein by

reference. These codes are 'mod 2' trellis codes, which can always be regarded as being based upon the lattice partition

Λ_s/Λ_s' =Z^N/2Z^N, and on some rate-k/N binary

convolutional code C that selects cosets of 2Z^N in Z^N.

These cosets can be specified by N/2 cosets of 2Z ² in

z².

It follows that the N-dimensional trellis code can be represented by a two-dimensional trellis, which is periodically time-varying with period N/2. At times nN/2, the trellis has S states and 2^m branches per state as in the original

N-dimensional trellis, whereas at intermediate times (kN/2) + j, j = 1, 2, ..., (N/2 - 1), the number of states may be larger, where the exact number of states depends on the

specific code in use.

As an example, consider the 4D 8-state dual Wei code, which can be viewed as a period-2, time-varying trellis code based on the partition Z ⁴/2Z⁴ and a rate-1/4

convolutional encoder. This trellis is illustrated in Fig. 7.

Note that there are 8 states at even times , with 2 distinct branches leaving each state , and there are 16 states at odd times, with one branch leaving each state.

Once a two-dimensional trellis is obtained,

reduced-state decoders can be defined as in the previous section. The operation of the VA is then similar, except it has to account for the additional time-varying nature of the trellis.

Construction of the Discrete-Time Channel Model :

So far, we have assumed a canonical discrete-time channel model described by r(D)=e(D)h(D)+n(D)

where e(D) is a complex input sequence, h(D) is a complex, stable, causal response with h₀=1 and n(D) is a white

Gaussian noise component which is independent of e(D). For best results we will choose h(D) to be minimum phase. In practice, the physical channel will rarely obey this canonical model; the channel response is often nonminimum-phase, the noise can be correlated and, furthermore, the physical channel is often continuous-time. Therefore, referring to Fig. 8, in practice, the physical channel 201 is often augmented by linear transmitter and receiver filters 202,204 in the transmitter and in the receiver, respectively, to construct a discrete-time channel 205 that obeys the canonical model.

In addition to providing the canonical discrete-time channel, it is desirable that these filters also help optimize the performance of the trellis precoder. It has been shown in the case of Tomlinson precoding that the optimum transmitter filter has a brickwall (or flat) spectrum within the Nyquist band (for continuous-time channels this result holds under certain mild conditions (Price, "Nonlinearly Feedback Equalized PAM versus Capacity for Noisy Filtered Channels," Proc. ICC-72, June, 1972, supra)); the optimum receiver filter, on the other hand, consists of a matched filter sampled at the baud rate (and at the optimum sampling phase), followed by a

discrete-time noise whitening filter that produces a

minimum-phase combined response h(D) (Price, supra). This noise-whitening filter can be described by the cascade of a zero-forcing linear equalizer and a minimum-phase linear prediction (error) filter for the residual noise sequence appearing at the output of the linear equalizer. The

prediction filter represents the channel model h(D).

In practice, it may sometimes be preferable to allow the overall discrete-time channel to deviate from the canonical model to arrive at a better trade-off between ISI and noise. For example, the selection of the filter may be based on the output MSE. Of course, due to the presence of ISI, minimum MSE does not guarantee minimum error probability. Nevertheless, this criterion is often used in practice because of ease in adaptive implementation, and because it is believed that often (if not always) it leads to lower error probability,

particularly at low SNR. Under the MSE criterion, the optimum transmitter spectrum is different; it has the 'water-pouring characteristic' found in information theory (J. Salz, "Optimum mean-squared error decision feedback equalization," BSTJ, vol. 52, pp. 1341-1373, 1973). For the moderate or high SNR's found on telephone lines, however, a 'water-pouring spectrum' can be closely approximated by a brickwall spectrum. The minimum MSE receiver filter also consists of a matched filter sampled at the baud rafe and a discrete-time filter that produces a white residual error sequence. The whitening filter can now be represented by the cascade of a MSE linear equalizer and a prediction error filter for the error sequence (ISI+noise) that appears at the output of that equalizer.

Under the MSE criterion, the received sequence can be written as r(D)=e(D)h(D)+n' (D), where the error sequence n'(D) is now signal-dependent and possibly non-Gaussian; the trellis precoder can still operate in the same manner. Here, it may be helpful to note that with an optimum transmitter filter, the overall response h(D) is independent of the SNR (in fact, it is the same filter that we obtain under the no-ISI criterion.)

In practice, the combination of the matched filter and the MSE linear equalizer can be implemented as a digital transversal equalizer with a fractional tap-spacing of T/M, where T is the baud interval and M is chosen sufficiently large to avoid aliasing. Since the characteristics of the physical channel is often unknown, an adaptive training algorithm is needed to learn this equalizer. This can be accomplished by transmitting a known training sequence such as a pseudo-noise (PN) sequence modulating a QPSK signal structure prior to data transmission, and then using a least-mean square (LMS)

algorithm (J. Proakis, "Digital Communications," McGraw-Hill, 1983). Once the equalizer is learned, an adaptive minimum MSE linear predictor can be realized. This predictor has a

tap-spacing of T and it whitens the residual error sequence. Its steady-state coefficients form the desired channel response h(D) of the model. So far, we have implicitly assumed that all filters are infinitely long. Of course, in practice, the linear equalizer and the predictor are implemented with finite length filters; nevertheless, when these are sufficiently long the general discussion presented above will approximately apply.

After a sufficiently long training period, the information about h(D) is passed back to the transmitter for use during trellis precoding. During data transmission an adaptive algorithm may continue to adjust the coefficients of the linear equalizer to track small variations in the channel response. However, the prediction filter is kept fixed. It is conceivable to update the predictor as well, provided that this information can be relayed back to the transmitter using a

'service channel' and proper synchronization between the

transmitter and the receiver can be maintained. Alternatively, the receiver may monitor the true current values of the

prediction filter coefficients and may request a new training signal if the discrepancy becomes excessive.

Modem Implementation

One application of trellis precoding is to a voiceband modem operating at a baud rate of 2743 symbol/s and a bit rate of L9.2 kb/s, thus sending 7 bits/baud. The transmitter and the receiver are implemented mostly in digital form.

The transmitter 102 and the receiver 104 of such a modem are shown respectively in Figs. 9, 10. B.ased on the optimality discussion above, the transmitter filter 106 is chosen to have a square-root-of-raised-cosine characteristics with a small excess bandwidth (≤ 12%), thus approximating a brickwall spectrum. During training, a known pseudo-random four-point QAM sequence 107 x(D) is transmitted via train/data switch 105 for a sufficiently long period of time to allow the receiver to learn first its adaptive equalizer 108 and

subsequently its adaptive prediction filter 110. The complex output of the transmitter filter is modulated to a carrier frequency w (radians/s) and the real part of the modulated signal is D/A converted, filtered by an analog filter, and transmitted over the channel, all by unit 107. In the receiver 104, the received signal is filtered by an analog filter 109 and then A/D converted at a sufficiently high nominal sampling rate M/T (the sampling phase is controlled by a timing recovery circuitry). The sequence of digital samples z is fed into an equalizer delay line 108 with a tap spacing of T/M, where n is the sampling index.

To describe the operation of the adaptive receiver, we denote the complex coefficients of the linear equalizer as c_n, -N₁≤n≤N₂ (N₁+N₂+1 taps). The output of the

equalizer is computed once every baud; then, it is demodulated to obtain the sequence r_k'. We assume that all coefficients are initially set to zero. The initial values may also be determined using a fast initialization method such as the scheme described in (Chevillat et al., "Rapid Training of a Voice-band Data Modem Receiver Employing an Equalizer with Fractional T-spaced Coefficients, IEEE Trans. Commun., vol.

COM-35, pp. 869-876, Sept. 1987). In any case, the

coefficients of the linear equalizer are adjusted according to c_n,k+1=c_n,k-α ε_k z_n*, n=-N₁,...,0,...,N₂ where

ε_k=r_k'-x_k is the error between the equalizer output r_k ' and the known transmitted symbol x_k , and k is the index for the baud interval. If the step size α is sufficiently small, then the coefficients will converge to values which minimize the MSE. Furthermore, if N₁ and N₂ are

sufficiently large, the filter converges to the optimum MSE linear equalizer that we described earlier.

Once the equalizer has converged, its outputs r_k ' are fed into a T-spaced prediction (error) filter with

coefficients h₀=1 (fixed), h₁, and h₂. This filter

produces r(D), the output of the channel model. Its

coefficients are updated according to

h_i , _k+1=h_{i , k}-βε *_k-iε'_k, i=1 , 2

where ε'_k=r_k-x_k - x_k-1h_1,k - x_k-2h_2,k is the

final error after prediction and β is another small step size. Again, when β is sufficiently small, these coefficients will converge to their values which minimize the MSE. If the overall channel model can be represented by a three-coefficient model as we assumed here, we expect the residual error sequence to be approximately white, as explained earlier.

After convergence, the coefficients h. and tu are sent back to the transmitter for use during trellis precoding. This can be accomplished, for example, by simply encoding the coefficients into binary words and transmitting these using 4-QAM signaling. These words are typically sent in a small frame which has a flag for recognizing the start of the frame and an error control check to detect transmission errors. In what follows, we denote the estimated channel response by h(D), and its formal inverse by g(D).

During transmission of actual data from a data terminal equipment (DTE) 120, the prediction filter 110 is kept fixed, but the linear equalizer 108 is continually adapted in a decision-directed mode to track small channel variations. (It is also possible to update the prediction filter along with the precoder coefficients. The latter would be updated using a side channel.) Because of trellis precoding, it is not possible to generate the error signal ε_k directly at the equalizer output, since decisions are available at the output of the prediction filter 110 h(D). Therefore, to reconstruct the equalizer error, the final prediction error

e'_k=r_k-y_k' between the output r_k of the prediction

error filter and the estimated channel symbol y_k ' is passed through g(D) 122. The estimates y_k ' can be obtained with no delay using a simple slicing operation, or they can be obtained from the Viterbi decoder, albeit after some delay. Experiments indicate that this structure operates reliably even in the presence of decision errors.

Referring again to Fig. 9, in the trellis precoder 124 we use a four-dimensional 16-state Wei code for coding (denoted as C_c) and a two-dimensional 4-state Ungerboeck code for shaping (denoted as C_s), and we send 7 bits per two

dimensions (or baud). The code C_s is derived from trellis code C which is based on a rate-1/2 4-state binary

convolutional encoder and the two-dimensional partition

Z ²/2Z². Equivalently, C can be described by a rate-2/4

convolutional encoder and the four-dimensional partition

Z ⁴/2Z⁴. We use a four baud frame to define C_s, where

in the first two bauds C_s = 8-C and in' the next two bauds C_s = 8RC, as described earlier. Thus, the time-zero lattice is

8RZ 4 during the first two bauds and 16Z⁴ during the

next two bauds of the frame. The code C_c, on the other hand, is based on a rate-3/4 binary convolutional encoder and the 4D partition Z 4/2Z4. This code adds one redundant bit

every four dimensions (or equivalently, every two bauds). Note that a fundamental region of the time-zero lattice 8RZ

will contain exactly 2^2x6.5+1 = 2¹⁴ points from any coset of the lattice Z⁴; i.e., |Z⁴/8RZ⁴|=2¹⁴, and a fundamental region of the time-zero lattice 16Z⁴ will

contain 2^2x7.5+1 = 2¹⁶ points from any coset of Z⁴.

A small buffer 130 is filled with bits received from the DTE at the rate of 7 bits per (2D) signaling interval.

Referring also to Fig. 2, during the two bauds, 4j and 4j+1

[the two bauds 4j+2, 4j+3] a scrambler 132 takes 7 [8] and then 6[7] bits from this buffer, respectively. The scrambled bits are delivered to a binary encoder in groups of 13 [15]

so-called I-bitε, labeled I6(4j) through I0(4j) and I5(4j+1) through I0(4j+1) [I7 (4j+2) through I0(4j+2) and I6(4j+3) through I0(4j+3)], at two successive times 4j and 4j+1 [4j+2 and 4j+3].

Referring to Fig. 11, the I-bits I2(4j)Il(4j) [I2(4j+2) I1(4j+2)] are taken to represent an integer mod 4 and are differentially encoded by a coding differential encoder 150 to produce two differentially encoded bits ID2(4j)ID1(4j)

[ID2(4j+2) ID1(4j+2)] according to the relation

ID2(4j)ID1(4j)=I2(4J)I1(4j) ⊕₄ ID2(4j-2) ID1(4j-2)

[ID2(4j+2)ID1(4j+2)=I2(4j+2)I1(4j+2)Θ₄ID2(4j)ID1(4j)] where⊕₄ represents mod-4 addition.

The bits ID1(4j) and I0(4j) [ID1(4j+2) and I0(4j+2)] enter a rate-2/3, 16-state binary convolutional encoder 152, which produces one redundant bit Y0(4j) [Y0(4j+2)]. Referring to Fig. 12, this encoder includes a sequential circuit 246 having four delay-2 shift registers 248 and three mod-2 adders 250. (This encoder can also be realized using other circuits.)

Referring again to Fig. 11, the bits ID2(4j)ID1(4j) I0(4j) [ID2(4j+2)ID1(4j+2)I0(4j+2)] and the redundant bit

Y0(4j) [Y0(4j+2)] enter a bit converter 154 that generates four output bits Zl(4j)Z0(4j)Zl(4j+1)Z0(4j+1)

[Z1(4j+2)Z0(4j+2)Z1(4j+3)Z0(4j+3)] according to the following table. Note that the corresponding bracketed headings for the table are

[ID2(4j+2)ID1(4j+2)I0(4j+2)Y0(4j+2)Z1(4j+3)ZO(4j+3)Z1(4j+2)Z0(4j+ 2)]:

ID2(4j) ID1(4j) I0(4j) Y0(4j) Z1(4j+1) Z0(4j+1) Z1(4j) Z0(4j)

These output bits are used to select one of 16 cosets of

2Z ⁴ whose union is the four-dimensional grid Z⁴+(.5,

.5, .5, .5). Each such coset is represented by a pair of

cosets of 2Z ² in the two-dimensional grid Z²+(.5, .5). These are selected individually by Z1(4j)Z0(4j) and

Z1(4j+1)Z0(4j+1)[Z1(4j+2)Z0(4j+2) and Z1(4j+3)Z0(4j+3)].

The remaining I-bits 13 (4j) through 16(4j) [I3(4j+2) through I7(4j+2)] pass through unchanged and are relabeled as Z2(4j) through Z5(4j) [Z2(4j+2) through Z6(4j+2)]. They are combined with Zl(4j)Z0(4j) [Zl(4j+2)Z0(4j+2)] to form the six-tuple Z5(4j)Z4(4j)...Z0(4j) [seven-tuple

Z6(4j+2)Z5(4j+2)...Z0(4j+2)]. Similarly, the I-bits I2(4j+1) through I5(2j+1) [I2(4j+3) through I6(4j+3)] together with Z1(4j+l)Z0(4j+1) [Z1(4j+3)Z0(4j+3)] form the six-tuple

Z5(4j+1)Z4(4j+1)...Z0(4j+1) [seven-tuple

Z6(4j+3)Z5(4j+3)...Z0(4j+3)].

The I-bits I1(4j+l)I0(4j+1) [I1(4j+3)I0(4j+3)] sequentially enter a rate-1/2 inverse syndrome former

(H ^-1)^T 156. H is a syndrome former of the binary code C defined above. Referring to Fig. 14, (H ^-1)^T produces two pairs of S-bits S1(4j)S0(4j) [S1(4j+2)S0(4j+2)] and

S1(4j+1)S0(4j+1) [S1(4j+3)S0(4j+3)]. As discussed earlier and disclosed in Forney and Eyuboglu, Trellis Shaping for

Modulation Systems, United States Patent Application cited above, (H ^-1)^T is the left inverse of (H)^T discussed

below, and is included to limit error propagation.

Referring to Figs. 13A, 13B during the two bauds 4j and

4j+1 [4j+2 and 4j+3], the 16-bits [18-bits] generated in this manner are mapped into two initial signal points r(4j) and r(4j+1) [r(4j+2) and r(4j+3)] chosen from a 256-point

[512-point] two-dimensional square signal set which lies on the grid Z²+(.5, .5). First, the six-tuple [seven-tuple]

Z-bits are used to select a signal point from the 64-point

[128-point] region-0 signal set 261. This signal set is divided into four subsets which are cosets of 2Z²

(indicated by different shadings in Figs. 13A and 13B). The signal points are labeled such that Zl and Z0 together select one of the cosets according .to the labeling shown in the lower left corner of Fig. 13A. The labeling of the remaining bits Z5,Z4,Z3,Z2 is shown only for coset a. For other cosets, the labeling can be obtained by the rule that 90 degree rotation of a point around the center (4,4) .[8,0] of the region-0 signal points result in the same labeling.

The Wei encoder 152 (Fig. 15) insures that the minimum distance between allowable sequences of region-0 points is kept large.

The S-bits S1, S0 determine the final region of the initial point according to the rule:

S1S0 Region

00 0

01 1

10 2

11 3

Referring to Figs. 14A, 14B, the signal points in region-0 are moved to the region chosen by the S-bits in such a way that they remain in the same coset of 8Z² [8RZ²].

This is done by offsetting the region-0 point by (0,0), (0,-8), (-8,-8), or (-8,0) [(0,0), (-8,-8), (-16,0), or (+8,-8)]. It follows that the translated final point is in the same coset of

Z ² as the region-0 point. Therefore, the minimum distance between allowable sequences remains unchanged. The signal points obtained in this manner form the initial sequence x(D). Thus the S-bits can be viewed as a coset representative

sequence which determines the coset of 16Z² in 8Z²

[16RZ² in 8RZ²] for the elements of initial sequence

X(D).

Referring to Fig. 15, the initial sequence x(D) is transformed into the transmitted sequence e(D) using parallel decision-feedback decoding (PDFD) 294 (which, along with the circuitry of Fig. 3, is part of the trellis precoder 124 of

Fig. 9). It is possible to replace PDFD by a more powerful

RSSE decoder, or by some other reduced-complexity decoder for the filtered trellis code C_s'.

As we saw earlier, the optimum decoder selects from the shaping code C_s the code sequence cto(D) which minimizes the mean-squared error between the sequences x' (D)=x(D)g(D) and c_s'(D)=c_s(D)g(D). The transmitted sequence is the error sequence e(D)=x'(D)-c_s'(D). The code sequence selected by PDFD will not generally result in minimum MSE, however, experiments indicate that the excess MSE will often be small. For example, for a channel with two coefficentε, PDFD can achieve 0.5-0.85 dB shaping gain, depending on the values of h.. and h₂.

Now, we consider the operation of PDFD in some detail. First, we describe the shaping code C_s.

Referring again to Figs. 14A and 14B, the

two-dimensional symbols of this code are chosen from the integer lattice 8Z ² [8RZ²] 299. These symbols are

partitioned into four subsets which correspond to the (four) cosets of 16Z² in 8Z² [16RZ² and 8RZ²] and are

labeled as A, B, C and D as shown in the lower-left corner of Fig. 14A.

All possible sequences c_s(D) in the shaping code C_s are represented by the trellis diagram of F-ϊg. 4. Here, from any state there are two branches 302, each branch representing a two-dimensional subset. For example, state 0 has two branches labeled A and B.

PDFD operates recursively using the Viterbi algorithm and in synchronism with the four-dimensional Wei encoder such that every other baud it releases two delayed error symbols e(4j-M) and e(4j+1-M) [e(4j+2-M) and e(4j+3-M)], where M (an even number) is the decoding delay measured in number of

bauds. Every baud interval, the VA has in storage four path metrics λ_i(4j+ℓ) [λ_i(4j+ℓ+2)], i=0, 1,2,3, and

1=0,1, one for each state in the trellis diagram. The VA

also has in storage a finite history of previously hypothesized error symbols e_i(4j+ℓ-m) [e_i(4j+2+ℓ-m)], i=0, 1,2,3,

ℓ=0,1 and m=1,2,...,M, one for each state. In each iteration

(every baud), the paths are extended by incrementing path

metrics by the branch metrics; the branch metric for the path branch p(p=1,2) from the i'th state is described by b_i,p( j+ℓ)=|-e_i(4j+ℓ-1) h₁-e_i(4j+i-2) h₂+x(4j+ℓ) - c_i,p(4j+ℓ)|²

[b_i,p(4j+ℓ+2)=|-e_i(4J+ℓ+1) h_i-e_i (4j+ℓ)h₂ +x(4j+ℓ+2) - c_i,p(4j+ℓ+2)|²] where c_i,p(4j+ℓ) [c_i,p(4j+ℓ+2)] is a symbol from the

coset of 16Z² [16RZ²] associated with this branch that

gives the minimum value for the branch metric. For each state i' after the transition, the accumulated metric of the two

merging paths are compared and the path with the smaller metric is declared a survivor; its path metric becomes the new path

metric for state i' and the error symbol for the surviving

transition (i*,p*), which is given by e_i,(4j+i)=-e_i*(4j+ℓ-1)h₁ - e_i*(4j+ℓ-2) h₂+x(4j+ℓ) - c_i*,p*(4j+ℓ), [e_i, (4j+ℓ+2)=-e_i*(4j+ℓ+1)h₁ - e_i*(4j+ℓ) h₂+x(4j+ℓ+2) - c_i*,p*(4j+ℓ+2 becomes its most recent error symbol stored in its path history for subsequent use. Every other baud, or when l=1, the

algorithm determines the state with the minimum path metric,

traces back its error symbol history for M symbols, and

releases the error symbols for times 4j-M and 4j-M+l [4j-M+2 and 4J -M+3 ] as outputs .

Note that the error symbols generated by the precoder

124 lie inside the Voronoi region of the sublattice 16Z ², [16RZ²] which is a square of side 16 [a rotated square of side 16 √2], centered at the origin, independent of the channel response h(D). When h₁ and h₂ are both non-integers and h(D) is

FIR, then the error symbols can take infinitely many values within that square.

In trellis precoding using trellis codes that are based on partitions of binary lattices, the elements e. of the

transmitted sequence belong to a 2D constellation that lies within a square boundary. Furthermore, trellis precoding

expands the size of the 2D constellation. Moreover, all these effects, and in addition, the further imbalance in the sizes of the Voronoi regions of the time zero lattices Λ_s0,k,

increase the peak-to-average ratio of the transmitted symbols, which may be undesirable. It is possible to reduce the PAR and obtain a more circular boundary by applying constraints to PDFD (Fig. 7). For example, PDFD can be constrained in such a way that it only selects code sequences c_s'(D) (from the filtered code C_s') which result in transmitted sequences e(D) whose elements e. have magnitudes no greater than some

predetermined radius R C; i.e., |e_j|≤R_c. When R_c is

not too small, the decoder can proceed without violating the constraint.

This type of constraint can be incorporated into any

RSSE-type decoder very easily by deemphasizing those branches of the trellis whose mean-squared error (MSE) metrics exceed

by multiplying them with an appropriately chosen

number Q . The larger Q gets, the harder the constraint

becomes. This reduces the probability of choosing points lying outside the constraint circle. When Q is sufficiently large, and R_c is not very small, points outside the constraint circle will never be selected. Note that if R_c is too small, one should not disallow branches, because this may force the decoder to choose an illegal sequence as a code sequence, and then the initial sequence x(D) cannot be correctly recovered in the receiver.

When the transmitted sequence e(D) is filtered before transmission over the channel, it may be beneficial to apply the constraint in higher dimensions (on groups of successive elements of the sequence e(D)), in order to minimize the PAR after the filtering. For example, the constraint may be applied in four dimensions by comparing the sum of the two most recent branch metrics against some squared radius The

decoder could force the condition |e_2j|²+|e_2j+1|²<R_C', R_c' being the radius of a 4D square. This constraint may be applied blockwise or sequentially in a sliding window fashion.

The VA is further slightly modified to insure that the code sequence c_s'(D) it selects is an allowable sequence from C_s', since otherwise the initial sequence x(D) cannot be correctly recovered. Whenever the VA traces back the path history to make a decision, it traces back other paths as well to determine whether they end in the same state; when they don't, those paths are pruned by setting their current path metric to a very large value. This insures that only allowable code sequences are selected.

The precoded sequence e( D ) is passed through the transmitter shaping filter 106 (Fig. 10), D/A converted, filtered by the analog front-end 107 and subsequently

transmitted over the channel.

At the receiver, the received signal r(t) is first filtered by the analog filter 109 (Fig. 10) to remove

out-of-band noise, and A/D converted into a digital stream (z_n) at a rate of M/T. This digital stream is input into the adaptive linear equalizer 108 with N₁+N₂+1 taps and a tap-spacing of T/M (not to be confused with the case of M for delay elsewhere). The output of this filter is sampled at the baud rate 1/T, then demodulated and subsequently filtered by the prediction filter h(D) 110 to approximately produce the received sequence r(D)=e(D)h(D) + n'(D)=x(D)-c_s(D)+n'(D)=y(D)+n'(D).

As we discussed earlier the elements of y(D) lie in the same coset of 2Z ⁴ in Z² as x(D), and thus the same

coset of 2Z ² in Z². Therefore, the minimum distance

between possible sequences y(D) is no smaller than the minimum distance between possible input sequences x(D) and the sequence y(D) can be decoded with a Viterbi decoder 133 for x(D), except the decoder must be slightly modified to rake into account the potentially different signal set boundaries for the elements y_j of y(D).

It can be shown that the shape and size of the signal set boundary for y_j's depend on the channel coefficients h₁ and h₂. For example, if h₁ and h₂ are both real, then

the boundary will be a square of side length 2(1+h₁+h₂), again centered at the origin. If the output constellation size is of concern, it may be beneficial to limit the magnitudes of the coefficients during training. This may be accomplished using a constrained adaptation algorithm.

It is possible to make the decoder 133 transparent to the constellation expansion that is caused by the channel filter. If we take the received sequence y(D) and translate it by some sequence a(D), where the elements of a(D) are elements of the lattice 16Z ² [16RZ²], then the modified sequence

y(D)-a(D)=x(D)-c_s(D)-a(D) is still equal to x(D) mod C_s, since the component c_s (D)+a(D) is an allowable code

sequence. It follows that the received sequence can be translated inside the Voronoi region of 16Z² [16RZ²].

The Viterbi algorithm for the (coding) code C_c can now

operate on this translated sequence assuming a square boundary of side 16 during the first two bauds of a frame and a rotated square of boundary of side 16 /2. during the next two bauds of the frame and output a delayed estimated sequence y"(D).

Every other baud, the VA will produce two delayed decisions y"(4j) and y"(4j+1) [y"(4j+2) and y"(4j+3)]. To extract the I-bits from thes we first obtain the Z-bits by translating the decisions in region-0, in such a way that the

2D elements remain in the same coset of 8Z ² [8RZ²].

Then, the Z-bits can be extracted using an inverse mapping according to Figs. 15A and 15B.

To extract the S-bits, we first label the regions (as before) according to

Region SQ1SQ2

0 00

1 01

3 11

2 10

Then, the regions of y"(4j) and y"(4j+l) determine the labels SQ1(4j)SQ0(4j) [SQ1(4j+2)SQ0(4j+2)] and

SQ1(4j+1)SQO(4j+1)[SQ1(4j+3)SQ0(4j+3)]. It is easy to show that SQl(i)SQ0(i)=S1(i)S0(i)⊕₂B1(i)B0(i), where i=4j or 4j+1 [4j+2 or 4j+3], ⊕₂ represents exclusive-or operation and

Bl(i)B0(i) is a binary two-tuple represeting the coset (A=00, B=11, C=01 and

D=10) of 16Z² [16RZ²] for the code symbol c_s(i)

chosen during precoding by PDFD.

Referring to Figs. 16 and 17, the labels SQ1(i)SQ0(i) are then sequentially passed through a feedback-free syndrome-former H^T 256. The syndrome-former has the property that any allowable sequence of coset labels B1(i)B0(i) will produce an all-zero sequence at its output. Furthermore,

(H ^-1)^TH^T=1, the identity matrix. Therefore, a sequence

which first passes through (H ^-1)^T and then through H^T

will remain unchanged. Thus, as shown in Fig. 17, at the output of the syndrome-former we obtain the transmitted bits

I1'(4j+1)I0' (4j+1) [I1'(4j+3) I0' (4j+3)], as long as the estimates y"(i) are correct. When there are occasional errors, they do not create catastrophic error propagation, because H^T is feedback-free.

Referring again to Fig. 17, the remaining information bits can be recovered by an inverse bit converter 286 and a coding differential decoder 288 which operates according to the relation:

I2'(4j)I1' (4j)=ID2' (4j)ID1' (4j) θ₄ ID2' (4j-2) ID1' (4j-2)

[I2' (4j+2)I1' (4j+2)=ID2' (4j+2)ID1' (4j+2)θ₄ID2' (4j)ID1' (4j)] where θ₄ is a modulo-4 subtraction. Referring again to

Fig. 11 the I-bits are then descrambled in a descrambler which forms part of unit 133 and delivered to the DTE via a buffer

135.

Differential Coding for Shaping

Suppose the channel introduces a phase rotation of 90k degrees, k=1, 2, or 3. The translation of the error point into regiont-1 is then rotated around the point (4,4,) [8,0] by the same amount. Since the Wei code is transparent to 90k degree phase rotations, the mapping used in the transmitter guarantees that the Z-bits can be correctly recovered. Unfortunately, this is not true for the region labels SQ1SQ0.

To remedy this situation, the label sequence SQ1SQ0 can be generated according to the phase-invariant labeling shown in Fig. 18. In order to guarantee that the relationship

SQ1SQ0=S1S0⊕₂B1B0 still holds, a differential encoding

operation is necessary. This is done as follows: Referring to Fig. 19, for each signal point obtained through region-0 mapping 258, we extract its subregion label SQ1SQ0 with a subregion extractor 306 according to Fig. 20. A shaping differential encoder 308 and a map into offsets 310 use the bits SQ1SQ0 and S1S0 to offset the region - 0 point into two new points x0(i) and x1(i), where i = 4j [4j+2] or i = 4j+1

[4j+3] such that they remain in the same coset of Z². This mapping can be described by the following two tables:

In the PDFD decoder, the point x0(i) is used for branches corresponding to cosets A and B, whereas x1(i) is used for branches corresponding to cosets C and D. It can be shown that if SQ1SQ0 is the subregion label of the received point, then SQ1SQ0=S1S0⊕₂B1B0. Therefore, by passing the

subregion label (which is rotationally invariant) through the syndrome-former H T we can recover the bits I0(4j+1) and

I1(4j+1) [I0(4j+3) and I1(4j+3)] even in the presence of 90k degrees phase offset.

Hexagonal 2D Constellations

If we choose the underlying shaping trellis code

C(Λ/Λ';C) as a code based on a partition Λ/Λ' of

so-called ternary or quaternary lattices whose constituent 2D lattice and sublattice Λ₂ and Λ₂' are versions of the

hexagonal 2D lattice A₂, then the elements e. of the

transmitted sequence will be bounded within a region

R_V(Λ₂') which is hexagonal rather than square. Such a

constellation has the advantage of being more nearly spherical. Alternative formulation based on label translates:

In the preferred embodiment of trellis precoding, we have used a linear (mod-2) code C_s. This allowed us to use the syndrome-former mapping methods for preventing error propagation. It is possible to give an alternative formulation of trellis precoding, based on label translates of trellis codes, such that the syndrome-former methods can be applied with nonlinear trellis codes, as well.

As before, for coding we use an N-dimensional trellis code C_c(Λ_c/Λ_c';C_c), where Λ_c/Λ_c' is a

binary lattice partition of order 2ⁿc and C_c is a binary, rate-kc/nc convolutional code (n_c=k_c+r_c). For

shaping and precoding, we use a trellis code

C_s(Λ_s/Λ_s'; C_s) based on a binary rate-k /n

convolutional code C_s(n_s=k_s+r_s). As shown in Figure 22 , a sequence of r_s-tuples s(D) enters an inverse syndrome former (H ^-1)^T for the

convolutional code Cto to produce a sequence of nto-tuples t(D), which specifies a label-translated code

C_s (Λ_s/Λ_s'; C_s⊕₂t(D)). The trellis precoder will

select the channel output sequence y(D) from a coset

C_s(Λ_s/Λ_s'; C_s⊕₂t(D))+d(D) of this

label-translated code, where d(D) is a coset representative sequence identifying cosets of Λto in some translate

Λ_c+a of Λ_c.

A sequence of k_c-tuples i(D) enters the

convolutional encoder C_c to produce a sequence of n_c-tuples b(D). The sequence b(D) together with an uncoded sequence of n_f-tuples v(D) specifies the coset representative sequence d(D). Then a channel output sequence y(D) that lies in

C_s (Λ_s/Λ_s'; C_s+t(D)) + d(D) can be identified by

d(D) which specifies a sequence of cosets of Λto in Λ_c + a, a sequence z(D) = c_s(D)+t(D) with c_s(D)εC_s, which

further specifies a sequence of cosets of Λ_s' in Λ_s +

d(D) and a sequence λ(D)εΛ_s' which selects a particular sequence in that coset. A decoder will find the sequences cto (D) and λ(D), such that the resulting y(D), after

filtering by the channel inverse g(D), has as small an average power as possible. The transmitted sequence e(D) is the filtered sequence: e(D)=y(D)g(D).

In the receiver, after appropriate front-end filtering, we obtain the observations r(D) = e(D)h(D) + n(D) = y(D) + n(D). Referring to Figure 23, the sequences i(D), v(D) and s(D) can be recovered as follows. First, since Λ_s is a sublattice of Λ_c', the sequence y(D) is a legitimate code sequence from C_c . Therefore, a conventional decoder for C_c can be used to find an estimate y(D), as before, with the same decoding complexity and probability of error as that of the code C_c on an ideal channel , h(D) = 1 . As before , it is possible to translate r (D) to a fundamental region of Λ_s', thus making the decoder essentially independent of the channel response h(D) without affecting the performance. In the absence of decoding errors, the sequence d(D), representing the cosets of Λ_s' in Λ_c + a, can be obtained directly from

y(D). This can be used to recover the sequences b(D) and v(D). The information sequence i(D) can be found using a decoder for the convolutional code C_c. The sequence z(D) = c_s (D)⊕t(D) representing the coset of Λ_s' in Λ_s can be

recovered also directly from y(D). The estimate of the

sequence s(D) is obtained by passing z(D) through a

syndrome-former H^T; i.e., s(D) = z(D)H^T = c_s(D)H^T⊕

t(D)H^T = s(D)(H^-1)^TH^T = s(D), where we used the

well-known fact that c_s(D)H ^T=0. If H^T is chosen to be

feedback-free, then s(D) can be recovered without catastrophic error propagation.

The selection of the sequence e(D) can be done, as before, using RSSE techniques, with the slight modification that trellis branches will now correspond to cosets of Λ_s' in Λ _s+d_k as identified by the label-translates z(D) =

c_s(D)⊕t(D), the coset representative sequence d(D) and the sequence λ(D), and the branch metrics are computed according to

|e_j|² = |y(z_k = t_k + c_s,k, d_k, λ_k) -

Σ_j≥1 e_k-j h_j|²

Here we have used the notation y(z_k, = t_k⊕ c_s,k, d_k,

λ_k) to indicate a channel output symbol y_k specified by z_k, d_k and λ_k.

With this method, we can send βN/2 = k_c + n_f + r_s bits per N-dimensions, where n_f = log₂|Λ_c'/Λ_s|. As before, we can increment n_f

by N/2 (or equivalently, by 1 bit per two dimensions) by replacing Λ_s by RΛ_s , because

log₂|Λ_c,/RΛ_s| = log₂|Λ_c'/Λ_s| +

N/2. (Or similarly, we can increment n_f by N/2 by replacing Λ_c by RΛ_c.) To send (N/2)ß = k_c+r_s+j+(d/2^m) (N/2)

bits/N dimensions, where 0≤d≤2^m, we can again define a frame of length (N/2)2^m bauds, and use the code C_s for (N/2)(2^m - d) bauds, and its scaled versions RC_s for (N/2)d bauds. (A similar effect can be achieved by replacing the coding code C_c by its time-varying versions based on the codes C_c ad

RC_c.) The implementation given above for the time-invariant case remains essentially unchanged, except one now has to take into account the variations in scaling. Again, constellatation constraints can be applied to control PAR.

B. Alternative formulation based on regions:

It is possible to formulate trellis precoding using convolutional codes and N-dimensional regions, instead of trellis codes. This formulation is more general in some respects. In this section, we describe this formulation, first using time-invariant regions, and then with regions having time-varying scaling to support fractional rates.

As before, for coding we use an N-dimensional trellis code C_c(Λc/Λ_c'; C_c), where Λ_c /Λ_c' is a

binary lattice partition of order 2ⁿc and C_c is a binary, rate-k_c/n_c convolutional code. For shaping arid precoding, instead of a trellis code, we use a binary, rate-k_s/n_s

convolutional code C_s and a set of N-dimensional regions

R(j), j=1,...,J, where J may be infinite. Each region is partitioned into 2ⁿs subregions R_n(j), j=1,..., J

such a way that every subregion contains an equal number of points from each of the 2ⁿc cosets of Λ_c' in a translate Λ_c + a of Λ_c. To transmit at a rate β, where β ≥ (2/N) (k_c

+ r_s), the regions must be properly scaled such that each subregion contains 2ⁿc⁺ⁿf points from Λ_c + a, where

n_f = βN/2 - k_c - r_s.

We select the elements of the channel output sequences y(D) from cosets of Λ_c' in Λ_c+a, as specified by the

sequence of n -tuples b(D) produced by the convolutional encoder C_c, from within one of the subregions R_n(j). Any signal point y_k from Λ_c + a, in any one of the regions

R(j), can be identified by a quadruplet: y(b_k, v_k , z_k ,

j_k), where an index j. selects a region R(j), an n_s-tuple z_k selects a subregion R_n(j). an n_c-tuple b_k specifies

one of 2ⁿc cosets of Λ_c' in Λ_c +a, and an n_f-tuple

v_k picks one of 2ⁿf points of that coset in R_n(j). As

illustrated in Figure 24, the convolutional encoder produces the sequence of n_c-tuples b(D), and uncoded symbols form the sequence of n_f-tuples v(D). The sequences j(D) and z(D) which identify the subregions are selected as follows. First, a sequence of r_s-tuples, s(D), is mapped into a sequence of n_s-tuples t(D) according to t(D) = s(D)(H ^-1)^T, where

(H^-1)^T is a right inverse to the nxr syndrome-former matrix for C_s. Then, in the key step of trellis precoding, a

decoder selects j(D) and z(D)εt(D)⊕₂C_s such that the

transmitted sequence e(D) = y(D)g(D) has as small an average power as possible.

Since y(D) is a code sequence from

C_c (Λ_c /Λ_c'; C_c), it can be decoded, as before, by a

decoder for C_c. Again the decoding complexity and the

probability of error will be the same as if the code C_c was used on an ideal channel. Referring to Fig. 25, if y(D) is the estimated channel output sequence, then z(D) can be extracted by identifying the subregions R_z(j) in which the elements y_k lie. The estimates b(D), v(D) can be found directly from the method described above reduces to trellis shaping on regions. Notice that in that case, the minimization step in the branch metric computations is absent.

Now let Λ_s /Λ_s' be an N-dimensional lattice partition of order 2ⁿs. Next, let R(l) be a fundamental region of Λ_s', and then form an exhaustive partition of N-space into regions R(j) - R(l) + λ_j, were λ_j is an

element of Λ_s', with Λ₁ = 0. Then partition each R(j) into 2ⁿs subregions R_n(j), such that each subregion is a fundamental region of Λ_s, and furthermore, each subregion contains an equal number of points from the 2ⁿc cosets of Λ_c' in Λ_c + a. Notice that the regions R(j) differ by points on the lattice Λ_s', and hence the minimizing step in the branch metric computations is now only as complex as decoding Λ_s'. An example is shown in Figure 26 for a 4-way partition of the type Z ²/2Z².

We may point out that although we have used the syndrome-former methods to generate the coset representative sequence t(D) in the extensions we described above, other mapping methods can be used also. However, in that case, to recover t(D) from z(D) a binary decoder for C_c associated with the set of coset representatives must be used to extract c_s (D) and then form t(D) = c_s(D)⊕z(D).

Finally, we should also point out that when Cto is a linear code, the original formulation based on fundamental regions of time-zero lattices is identical to the formulation based on label-translates, and they both can be posed as a special case of the formulation based on regions.

With this method, we can also send βN/2 = k_c + n_f + r_s bits per N dimensions. As before, we can increment n_f by N/2 by replacing Λ by RΛ_c, to send (N/2)β = k_c +

r_s + j + (d/2^m)(N/2) bits per N dimensions, where

0≤d<2^m. We can again define a frame of length (N/2)2^m bauds, and use the code C_c for (N/2)2^m - d₀ bauds, and y(D) with an inverse map. The sequence s(D) can be extracted directly from z(D) with a syndrome-former H T according to s(D) = z(D)H^T, because c_s(D)H^T = 0 and (H^-1)^TH^T =

I, where I is the rxr identity matrix. If H^T is chosen to be feedback-free, then s(D) can be recovered with no error

propagation.

Having discussed the operational principles, we now turn to the selection of the index sequences j(D) and z(D). To minimize the output average power, we need to search a trellis with states s_k' = [s_k; j^k-1, z^k-1], where j^k-1 = (j_k-1, ..., J_k-κ) and z^k-1 = (z_k-1, ...,z_k-κ). If J and K are botn finite, then this trellis will have a finite number of states. But even in that case, it is often necessary to reduce complexity to make the search feasible. Again, this can be accomplished by an RSSE-like shaping decoder. An RSSE trellis can be

constructed by applying set partitioning principles to the past values of j_k-i and z _k-1 , i=1,...L≤K. Then, the VA can

be applied to search the reduced-state trellis with the branch metrics | e_k|² = |y(b_k, v_k, z_k = t_k ⊕₂ c_{s , k}, -∑_{j≤ 1}e_k-jh_j|₂

where c_{s ,k} is an nto-tuple corresponding to the branches of the convolutional code, and the minimization is over all possible values of j_k. Here the minimization is over all possible indices j_k, 1≤j_k≤J, for the regions R(j). Notice that, for large values of J, this minimization can be very time-consuming, if one has to compute the above metric for all possible values of j_k. Now we will show how this problem is. avoided, when the regions are chosen as fundamental regions of lattices.

But first we should note that when the channel is ideal (h(D) = 1), and the number of regions is one (J=1), then its scaled version RC_c for (N/2)d bauds. A similar effect can be achieved by varying the scaling of the subregions. The implementation given above for the time-invariant case remains essentially unchaged except one now has to take into account the time-varying nature of the code or the scaling. Again, constraints can be applied to reduce the peak to average ratio.

Other embodiments are also within the following claims. For example, other reduced-complexity decoders (e.g., M algorithm) could be used.

Appendix A

Terminology and principles

We first discuss terminology and principles which underlie the invention.

Trellis Precoding

A lattice Λ is a collection of N-dimensional vectors c, which form a group under ordinary vector addition. A fundamental region R(Λ) of A is a region of N-space which contains one and only one point (a coset representative) from each distinct coset (translate) Λ + a of Λ. All

fundamental regions of a given lattice have the same volume, V(Λ), and their translates R(Λ) + c, cεΛ, form a

tesselation of N-space. This induces the following coset decomposition: Any N-vector y can be uniquely decomposed as y = c + e, where cεΛ and eεR(Λ); that means, y is

equivalent to a coset representative eεR(Λ) modulo Λ.

The translates R(Λ) + c may be viewed as the decision regions D(c) of a decoder D(Λ), whose error regions D(c)-c are all equal to R(Λ) (to show this correspondence explicitly, in what follows, we will sometimes write R(Λ, D) or D(Λ, R)).

Thus, if eεR(Λ) is a coset representative, but otherwise an unknown information-bearing vector, then it can be recovered from y = c + e, without any knowledge about c, by first finding the lattice point c using the decoder D(Λ, R) and then

forming the error e = y - c.

The Voronoi region R_V(Λ) of a lattice Λ is

defined as the set of points that are at least as close to the origin as to any other lattice point. The Voronoi region is a fundamental region corresponding to a minimum-distance decoder of Λ (we ignore the effect of ties at the boundary points).

A rate-k/n convolutional code C is the set of sequences c(D) of q-ary n-tuples, which can be generated by a kxn generator matrix G as its input ranges over all sequences of q-ary k-tuples. In analogy to the fundamental region of a lattice, we can define a set. R of sequences of q-ary n-tuples, which contains one and only one sequence (a coset

representative) from every coset C⊕₂a(D) of C, where ⊕

represents modulo-q addition. Then, any sequence of q-ary n-tuples z(D) has the unique decomposition z(D) = t(D)⊕₂c(D), where t(D)εR and c(D)εC. An unknown coset representative sequence t(D)εR can be recovered from z(D) = t(D)⊕₂c(D), by first finding the code sequence c(D) using the decoder D(C,R) for the set of coset representatives R, and then forming t(D) =z(D)⊕₂c(D).

If Λ' is a sublattice of Λ, then Λ is the union of |Λ/Λ'| = V(Λ')/V(Λ) cosets of Λ' in Λ. Then

we say Λ/Λ' is a lattice partition of order |Λ/Λ'),

and each fundamental region R(Λ') of Λ' will contain

|Λ/Λ'| points from Λ.

In general, a signal-space trellis code C(Λ/Λ';C) is based on an N-dimensional lattice partition Λ/Λ' of

order qⁿ and a rate-k/n, q-ary convolutional code C. Each coded output n-tuple b. selects one of the qⁿ cosets Λ'

+ ζ(b_k) of Λ' in Λ, or some translate of Λ,

according to some labeling function ζ(b_k). The trellis

code C(Λ/Λ';C) is the set of all sequences c(D) that belong to a sequence of cosets of Λ' that could be selected by the code sequences in C. To simplify exposition, we shall often consider two-dimensional Ungerboeck-type trellis codes based on 2ⁿ-way two-dimensional lattice partitions Z²/RⁿZ²,

where Z² is the lattice of integer pairs and R is the

two-dimensional rotation operator defined by the 2x2 matrix {1,1), (1,-1)}. In sequence space, a lattice Λ can also be viewed as a trellis code C(Λ/Λ; C) = (Λ)^∞, which

is based on the trivial lattice partition Λ/Λ and a trivial rate-1 convolutional code, whose generator matrix G is an identity. For all known codes C, given any encoder state s_k, the set of all possible next outputs is a coset of some lattice Λ₀, called the time-zero lattice. The fundamental regions R_n of the time-zero lattice play a central role in trellis precoding. For a two-dimensional code C(Z²/RⁿZ²; C),

the time-zero lattice is Λ₀ = R^n-kZ².

If C is a linear code, i.e., a group under sequence addition, then it will carry the key properties of lattices.

For example, we can define a fundamental region R(C) of C as a set of sequences that contains one and only one sequence (a coset representative) from each distinct coset C + a(D) of C.

Also say any sequence y(D) can be uniquely decomposed as y(D) = c(D) + e(D), where c(D)εC and e(D)εR(C). Thus, a coset

representative sequence e(D)ε R(C) can be recovered from y(D) - c(D) + e(D), without any knowledge about c(D), by first finding the sequence c(D) using the decoder D(C,R) for the fundamental region R(C), and then forming the error sequence e(D) = y(D) - c(D).

A trellis code is linear, if and only if the labeling function ζ(b) is linear; i.e., ζ(bθb') = ζ(b) +

ζ(b'), where b, b' are two q-tuples. Most known trellis codes, although nonlinear, exhibit sufficient regularity such that their geometric properties such as distance profile or the shape of their Voronoi regions are independent of the code sequence c(D). All trellis codes with an isometric labeling function exhibit such geometric uniformity. For such codes, the decision regions for some given decoder are not simple translates of each other. But they can be transformed into another by isometric transformations, which consist of

translations, rotations or reflections. The following theorems play an important role in trellis precoding.

Theorem 1 : Let C_s be a trellis code, possibly nonlinear, with a time-zero lattice Λ_0s, then any sequence y(D) can be uniquely decomposed as y(D) = e(D) + c_s(D), where c_s(D)εC_s and e(D) is a coset representative sequence

whose elements belong to some fundamental region R_0s of

Λ_0s. An unknown coset representative sequence e(D) can be recovered from y(D) by first finding the code sequence c_s(D) with a hard-decision decoder based on R_0s and then forming the error sequence e(D) = y(D) - c_s(D).

Proof: Suppose at time k we are in state s_k of C_s. Then the next allowable symbols c_{s, k} belong to some coset

Λ_0s+a(s_k) of the time-zero lattice Λ_0s. Hence, we

can use a decoder D(R_0s) for Λ_0s+a(s_k) to obtain the

decomposition y_k = e_k+c_s,k, where

c_s,kεΛ_0s+a(s_k) and e_kεR_0s. Next, we let c_s,k

determine the next-state s_k+1 and then repeat the same argument for time k+1, and so on. Thus, starting from a kown state s₀, the sequences e(D) and c (D) in the decomposition y(D) = e(D) + cto(D) will be uniquely determined. Q.E.D.

Theorem 1 implies that if e(D) is some

information-bearing signal whose symbols e_k are constrained to lie in the fundamental region R_0s, then we can add an unknown code sequence cto(D) from C_s to e(D), and we will be able to recover e(D) from the resulting sequence y(D) = e(D) + c_s(D) perfectly without the knowledge of c_s(D). In fact, the above proof shows that the symbols e. can be recovered with no delay from the symbols y_k up to time k by a simple 'hard-decision' decoder for C_s. Typically, R_0S is chosen as a simple rectangular region, and then decisions on c_c,k may be made coordinate by coordinate.

If C(Λ/Λ'; C) is a trellis code, then its label translate C(Λ/Λ'; C+a(D)) is obtained by replacing the

convolutional code C by its coset C⊕₂(D). When C(Λ/Λ';

C) is geometrically uniform, then C(Λ/Λ'; C⊕-₂a(D)) has

the same geometric properties as C(Λ/Λ';C) and it is

related to C(Λ/Λ'; C) by an isometric transformation (for linear codes that isometric transformation is a simple

translation). Theorem 2: If c_c (D) and c_s(D) are code sequences in

N-dimensional codes C_c and C_s, respectively, and Λ_s is a sublattice of Λ _C', then y(D) = c_c(D)±c_s(D) is a code sequence in C_c.

Proof : Every sequence c_s(D) is a sequence of elements of Λto and therefore of Λ_c', which means that y(D) =

c_c (D)+c_s(D) is a sequence with elements in the same sequence of cosets of Λ_c' as c_c(D), so it must be in

C_c. Q.E.D.

Claims

Claims

1. A method of mapping a digital data sequence into a signal point sequence e(D) for data transmission over a channel h(D) to produce a channel output sequence y(D)=e(D)h(D), comprising choosing e(D) so that the signal points in the sequence y(D) belong to a class of possible sequences based on said digital data sequence, the class being based on a

time-varying trellis code derived from an N-dimensional

time-invariant trellis code by using different transformed versions of its underlying lattices for respectively different N-dimensional symbols.

2. The method of claim 1 wherein said class comprises a signal space translate of a time-varying trellis code C_s, where the trans lation is a code sequence from a trans late of a trellis code C_c determined based on said digital data

sequence, and said time-varying trellis code C_s is derived from an N-dimensional time-invariant trellis code C(Λ/Λ',

C_s) by using different transformed versions of its underlying lattices Λ and Λ' for respectively different N-dimensional signal points.

3. The method of claim 1 wherein said class comprises a signal space translate of a trellis code C_s, where the translation is a code sequence from a translate of a

time-varying trellis code C_c determined based on said digital data sequence and said time-varying trellis code C_c is

derived from an N-dimensional time-invariant trellis code

C(L/L', C_c) by using different transformed versions of its underlying lattices L and L' for respectively different

N-dimensional signal points.

4. The method of claim 1 wherein said class comprises a signal space translate of a label translate of a time-varying trellis code C_s, where the label translate is based on a portion of said digital data sequence and the signal space translation is based on another portion of the elements of said digital data sequence, and the time-varying trellis

code C_s is derived from an N-dimensional time-invariant

trellis code C(Λ/Λ', C_s) by using different transformed

versions of its underlying lattices Λ and Λ' for

respectively different N-dimensional signal points.

5. The method of claim 1 wherein said class comprises a signal space translate of a label translate of a trellis code C_s, where the label translate is based on a portion of said digital data sequence, where the signal space translation is based on another portion of the elements of said digital data sequence and is a code sequence from a time-varying trellis code C_c which is derived from an N-dimensional time-invariant trellis code C(L/L', C_c) by using different transformed

versions of its underlying lattices L and L' for respectively different N-dimensional signal points.

6. The method of claim 1 wherein said class of possible sequences is identified by some sequence of subregions into which N-space has been partitioned, said sequence of subregions being specified by a label sequence determined based on a portion of the digital data sequence and said signal points in said subregions belonging to a coset of a

time-varying trellis code C_c, where the sequences from said trellis code C_c are chosen based on another portion of the elements of said digital data sequence, and the time-varying trellis code C_c is derived from an N-dimensional

time-invariant trellis code C(L/L', C_c) by using different transformed versions of its underlying lattices L and L' for respectively different N-dimensional signal points.

7. The method of claim 1 wherein said class of possible sequences is identified by some sequence of subregions into which N-space has been partitioned, said sequence of subregions being specified by a label sequence based on a portion of the elements .of the digital data sequence and using different scaled versions of said subregions for respectively different N-dimensional signal points, said ccset being

determined based on another portion of the digital data

sequence, where the signal points in the sequence y(D) are chosen based on a portion of the elements of said digital data sequence, and wherein said subregions are scaled versions of each other.

8. A method of mapping a digital data sequence into a signal point sequence e(D) for data transmission over a channel h(D) (h(D) not a constant) to produce a channel output sequence y(D)=e(D)h(D), comprising choosing e(D) so that y(D) belongs to a signal space translate of a label translate based on a portion of said digital data sequence, and said signal space translation is specified based on another portion of the elements of said digital data sequence and is a code sequence from a trellis code C_c.

9. A method of mapping a digital data sequence into a signal point sequence e(D) for data transmission over a channel h(D) (h(D) not a constant) to produce a channel output sequence y(D)=e(D)h(D), comprising choosing e(D) so that the signal points in the sequence y(D) belong to some sequence of

subregions into which N-space has been partitioned, said sequence of subregions being specified by a label sequence based on a portion of the digital data sequence, said signal. points in said subregions belonging to a coset of a trellis code C_c, and where sequences from said telliε code C_c are chosen based on another portion of the elements of said digital data sequence.

10. The method of claims 4, 5, 6, 7, 8, or 9 whrein said label sequence belongs to a coset of a convolutional code C_c.

11. The method of claim 1, 2, 3, 4, 5, 6, or 7 wherein said versions are respectively used periodically.

12. The method of claim 10 wherein each period encompasses multiple N-dimensional symbols.

13. The method of claim 1, 2, 3, 4, 5, 6, or 7 wherein only two said different versions are used alternatingly.

14. The method of claim 1, 2, 3, 4, or 5, 6, or 7 wherein said transformed versions comprise rotated and/or scaled versions of said underlying lattices.

15. The method of claim 1, 2, 3, 4, 5, 6, 7, 8, or 9 further comprising filtering said trellis code Cto to form a filtered trellis code C_c' whose sequences are c_s' (D) =

c_s(D)g(D) where g(D) is the formal inverse of a response

related to the channel response h(D), and cto (D) is any code sequence in the trellis code C_s.

16. The method of claim 15 in which the signal point sequence selection tends to minimize the average power of the signal point sequence e(D) = y(D)g(D).

17. The method of claim 15 or 16 wherein sais signal point sequence e(D) lies in a fundamental region of said

filtered trellis code C_s'.

18. The method of claim 15 in which the signal point sequence is selected based on reduced state sequence estimation with respect to said filtered trellis code C_s'.

19. The method of claim 18 in which the reduced state sequence estimation uses no more states than the number of states of the original trellis code C_s'.

20. The method of claim 17 wherein said fundamental region of said filtered trellis code comprises approximately a Voronoi region of said filtered trellis code C_s'.

21. The method of claim 1, 2, 3, 4, 5, 6, 7, 8 or 9 further comprising recovering the digital data sequence from a possibly noise-corrupted version of the signal point sequence, including decoding the signal point sequence to a sequence of estimated digital elements and forming a syndrome of fewer digital elements based on a portion of the estimated digital elements using a feedback-free syndrome former H T.

22. The method of claim 1, 2, 3, 4, 5, 6, 7, 8, or 9 wherein said signal point sequence is selected by

mapping said digital data sequence into an initial sequence belonging to and representing a congruence class of said original trellis code Cto , and

choosing a signal point sequence belonging to and representing a congruence class of said filtered trellis code C_s' and which has no greater average power than said initial sequence, and wherein

said mapping includes applying a portion of the elements of said digital data sequence to a coset

representative generator for forming a larger number of digital elements representing a coset representative sequence.

23. The method of claim 22 wherein said coset representative generator comprises a multiplication of a portion of the elements of said digital data sequence by a coset representative generator matrix (H ^-1)^T which is

inverse to a syndrome-former matrix H T for said code.

24. The method of claim 23 further comprising recovering the digital data sequence from a possibly filtered and noise-corrupted version of the signal point sequence, including decoding the signal point sequence to a sequence of estimated digital elements and forming a syndrome of fewer digital elements based on a portion of the estimated digital elements using a feedback-free syndrome former H ^T .

25. The method of claim 1, 2, 3, 4, 5, 6, 7, 8, or 9 wherein said trellis code C_s is a linear trellis code.

26. The method of claim 1, 2, 3, 4, 5, 6, 7, 8, or 9 wherein said trellis code C_s is a non-linear trellis code.

27. The method of claim 25 wherein said linear trellis code C_s is a 4-state Ungerboeck code.

28. The method of claim 1, 2, 3, 4, 5, 6, 7, 8, or 9 wherein said trellis code C_s is based on a partition of binary lattices.

29. The method of claim 1, 2, 3, 4, 5, 6, 7, 8, or 9 wherein said trellis code C_s is based on a partition of ternary or quaternary lattices.

30. The method of claim 1, 2, 3, 4, 5, 6, 7, 8, or 9 wherein the step of selecting said signal point sequence is further constrained so as to reduce the peak power of said signal point sequence where said peak power represents the maximum energy of said signal point sequence in some number of dimensions N.

31. The method of claim 29 wherein N = 2.

32. The method of claim 29 wherein N = 4.

33. The method of claim 1, 2, 3, 4, 5, 6, 7, 8, or 9 wherein said step of mapping said digital data sequence into said signal point sequence is arranged to ensure that said digital data sequence can be recovered from a channel-affected version of said signal point sequence which has been subjected to one of a number of predetermined phase rotations.