WO1994023495A1

WO1994023495A1 - Adaptive signal processing methods using information state models

Info

Publication number: WO1994023495A1
Application number: PCT/AU1994/000169
Authority: WO
Inventors: John Barratt Moore; Vikram Krishnamurthy; Iain Bruce Collings; Subhrakanti Dey
Original assignee: The Australian National University
Priority date: 1993-04-01
Filing date: 1994-04-05
Publication date: 1994-10-13

Abstract

Techniques for optimal processing of signals which fit continuous-state and discrete-state system models are known. This invention concerns the processing of signals which have characteristics that are not exclusive to either the continuous-state or the discrete-state system, but fit only a mixed-state model, using a decision feedback approach. After establishing the relevant mixed-state model, the received signal is processed simultaneously, and in parallel, in a Kalman (or extended Kalman) filter and in a Hidden Markov Model filter. The output of the Kalman filter is coupled to a stage of the Hidden Markov filter and the information state estimate of the Hidden Markov filter is coupled to the Kalman filter. With this arrangement, the desired signal estimate is derived from the information state estimate of the output of the Hidden Markov filter.

Description

TITLE: "ADAPTIVE SIGNAL PROCESSING METHODS USING INFORMATION STATE MODELS"

Technical Field

This invention concerns signal processing. More particularly, it concerns signal processing techniques for a variety of applications, based on novel signal model formulations which utilise information states derived using -Hidden Markov Model filter theory. The invention has particular application in a number of areas of telecommunication and signal processing. It is particularly useful in traditional communications applications (such as the demodulation of signals received via fading channels) and to blind equalization of telecommunication channels. It is also useful in speech coding and for radar pulse train de-interleaving, particularly with randomly varying periods. These indicated applications of the invention are not exhaustive.

Background to the Invention

A major problem associated with the processing of signals in communication systems and of radar signals is the presence of noise in signal channels. For example, when there is movement in a mobile telephone receiver or computer keyboard, obstructions such as buildings, cars or people may cause signal fading. The consequential relative strength of the channel noise can then be intolerable to currently available equipment. And in the case of radar, the time of arrival of pulses may not be accurately detected, with the result that so-called jitter noise occurs, or the normally periodic pulse may have randomly varying periods.

To improve the signal-to-noise ratio in communication channels, some techniques have been developed, which work reasonably well with two classes of signal model.

The first class of signal model involves variables ( states) such as signal amplitudes or phases which are in a continuous range (for example, from 0 to ∞ for the amplitude and from -π to π for the phase). Models in this class are termed "continuous-state models" or "linear models" .

The second class of signal models involves variables (states) in a finite set, such as 1, 0, -1, or 1, 2, ..., 10. These models, which are almost invariably linear models, are termed "discrete-state models". One special category of continuous-state variables, associated with discrete-state models, are conditional probabilities associated with discrete states. The conditioning is on the measurements and perhaps other variables, such as assumed signal model parameters. These conditional probabilities are non-negative and add to unity; in this specification they are termed "information states" or "conditional information states", as appropriate.

Hybrid models are also known. Hybrid models involve variables (states), some of which are continuous and others of which are discrete. These models are termed

"mixed-state models" . When the discrete states in mixed-state models are in terms of (conditional) information states, these models become continuous-state models (which in this specification are termed "information state models"). The constraints on the information states are that they are positive and sum to unity.

The variables exist in continuous-time, usually indicated by t, with t in the range 0 to ∞. However, the variables can be processed at discrete-time instants, indicated by k, with k being 0, 1, 2,....

The signal models can have parameters, such as transition probabilities and discrete-state values, which are fixed and either known or unknown. These parameters may also vary slowly with time relative to other dynamics in the signal model. Variables such as amplitudes and phases which change slowly can also be termed parameters. Parameters can also be regarded as states: if the parameterized models are linear models, then treating the parameters as states gives rise to nonlinear state models. When the parameters are conditional probabilities, these in turn can also be viewed as information states.

In the case of continuous-state models, linear stochastic estimation theory has been developed for processing signals in known linear dynamical systems driven by white Gaussian noise. For approximately 30 years, optimal processing has been achieved using what are known as "Kalman" filters. However, real world applications involve unknown signal models, which may be nonlinear, and there is measurement in non-white and non-Gaussian noise environments. Since there is no complete nonlinear estimation theory that can be implemented with finite-dimensional filters, except for relatively simple low order models, creative designs have been developed which achieve a fair compromise between performance and practical implementation. One sub-class of such nonlinear estimation schemes are termed "adaptive estimation schemes". These schemes involve estimating or tracking unknown (possibly slowly varying) signal model parameters as part of the estimation process. Recursive prediction error (RPE) schemes fall into this category when the parameters are constant. Extended Kalman filters also fall into this category when the parameters are changing slowly compared to the dynamics of the model.

Reliable processing of signals which arise from discrete-state models has also been achieved, and if the signals arise from what are widely known as discrete state, discrete-time "Hidden Markov Models" (HMMs), optimal processing is possible. Hidden Markov Models have variables, termed "states", which belong to a finite set, such as the values 1, 2, ..., 10. Transitions, governed by known transition probabilities, take place at each discrete-time instant between states. The states are not observed directly, but are observed contaminated by noise.

The HMM approach can be applied to the case when the signal model states are in a continuous range and there is quantization of the states. Nonlinear continuous-state models are then approximated by linear discrete-state models. The approximation can be arbitrarily good, depending on the fineness of the quantization. For suitably fine quantization, the signal processing for nonlinear continuous-state noisy models is the best that can be achieved, according to widely accepted measures of performance. However, there is a trade-off between computational effort and loss of performance due to quantization errors. (There is no loss of optimality in increasing noise environments, as in extended Kalman filtering. )

Disclosure of the Present Invention It is an objective of the present invention to provide a signal processing method or technique which significantly improves the signal-to-noise ratio of the signal input to a receiver when the signal being processed fits neither a linear (or continuous state) model nor a discrete state model, but contains elements of both models (that is, the signal fits a mixed-state model ) .

This objective is achieved by applying the signal being processed to two filters in parallel, after formulating the mixed state model for the signal being processed. One of the filters in parallel is a Kalman filter or an extended Kalman. The other filter is a Hidden Markov filter. The filtering, although performed in parallel, is coupled. That is, the output signal (state estimates) from the Kalman (or extended Kalman) filter is an input to the Hidden Markov filter, while the output from the Hidden Markov filter (the information state or its estimate) is input to the Kalman (or extended Kalman) filter. The present inventors have ascertained that this coupling between the filters, which is also termed "decision feedback", enables the output from the (coupled) Kalman (or extended Kalman) filter to provide an estimate of the channel fading parameters, while the output from the coupled Hidden Markov filter is the signal model information state, which gives an estimate of the signal, that is input to the receiver.

Thus, according to the present invention, there is provided a method of processing a signal which has the characteristics of a hybrid or a mixed-state signal model, the method comprising the steps of:

(a) formulating a mixed-state model for the signal;

(b) applying the signal to an optimal Kalman (or extended Kalman) filter and simultaneously applying the signal to an optimal Hidden Markov filter; (c) coupling the output of the Kalman filter to a stage of the Hidden Markov filter;

(d) coupling the output from the Hidden Markov filter information state estimates to the Kalman filter; and

(e) deriving the desired signal estimate (namely, the information state estimate) from the output of the

Hidden Markov filter.

The signal modelled by the discrete-time mixed state signal model may be an analogue or a digital signal.

For better understanding of the present invention, examples of its application to a number of real-life signals will now be described, by way of example only. In the following description, reference will be made to the accompanying drawings. Brief Description of the Drawings

Figure 1 is a block diagram which illustrates the adaptive signal processing arrangement of the present invention.

Figure 2 is a block diagram showing one form of an extended Kalman filter/Hidden Markov Model signal processing arrangement.

Figure 3 shows three radar pulse trains, the first two being from independent sources and the third being a received interleaved pulse train.

Figure 4 shows the corresponding evolution with pulse number of the sequence of active sources for the interleaved pulse train of Figure 3.

Detailed Description of Embodiments of the Invention

As noted above, the present invention is essentially a combination of Kalman filtering and HMM filtering, which allows effective tackling of real life signal processing problems with signal having mixed continuous and discrete states. To illustrate the value of the present invention, its application to quadrature amplitude modulated (QAM) signals, FM demodulation, blind equalization, speech coding, and radar pulse train de-interleaving will be described below.

The first step in processing any one of these examples is to formulate the appropriate mixed-state signal model. The second step is to reformulate the models using known HMM signal processing theory in terms of continuous states, including information states, to achieve what the present inventors call the "Information State" (IS) signal model. Such reformulations depend on known HMM signal processing algorithms and result in nonlinear IS models.

The third step is to present algorithms for estimating the parameters and states of the IS models, and hence achieve estimates of the states and parameters of the mixed-state HMM. The algorithms should be able to be implemented on-line, so that the parameters and signal estimates of the IS models can be updated at each time instant. This makes the algorithms suitable for real-time signal processing. When the signal models are discrete-state models with parameters that are unknown but constant, the on-line estimation schemes will be based on recursive prediction error (RPE) techniques. When the parameters are slowly time-varying (in a relative sense), the on-line estimation scheme can be based on extended Kalman filtering (EKF) methods.

In general, the first class of estimation algorithms that will be considered for the mixed-state HMMs will involve full tree searches over all possible discrete state sequences. Such algorithms are optimal. The computational cost of such optimal schemes increases exponentially with the data length. Thus, such optimal schemes are impractical except in very restricted cases. The next class of proposed estimation techniques that will be considered for the mixed-state HMMs will be sub-optimal schemes which are computationally tractable. Such algorithms are based on the optimal schemes and approach the performance of the optimal schemes under reasonable conditions.

The concept of coupled conditional filters involving Hidden Markov filters, which is an essential feature of the present invention, has not been used in signal processing techniques previously, although its use in other contexts is known. In the present invention, the coupling is used with the IS models. The reason for the coupling is as follows. If one knows the model parameters, for example, one can estimate the model states using standard optimal techniques to achieve a conditional optimal filter. Likewise, if one knows the model states, one could construct an optimal conditional filter to estimate the parameters. When parameter estimates and state estimates are used in lieu of actual values in the conditional filters in a coupling of the filters, a coupled conditional filter arrangement is produced. Such schemes can also loosely be called "decision feedback" schemes. A feature of the coupled conditional filters for the IS models of the present invention is that the coupling signals are estimates of the conditional information states rather than estimates of the states themselves. Sometimes there is an advantage in processing the conditional information state estimates before feeding to the next subsystem. For example, rather than the full information state estimate, one may feedback only the relatively more significant components of the information state or only the most significant component, which is essentially equivalent to the best estimate of the state.

Those skilled in this art will appreciate that when using recursive prediction error methods, it will usually be an advantage, when estimating transition probabilities of Markov chains in the present invention, to parameterize the transition probabilities in such a way that their estimates are positive and add to unity. However, an alternative approach is to view the transition probabilities as information states and model the actual transitions via a Markov chain.

It will be apparent from the above discussion that the preferred approach of the present invention is to first model the continuous-valued (conditional) "information state" of the discrete-state process instead of using the discrete-state process itself. Hence the mixed-state model is transformed into a continuous-state (possibly non-linear) model, so that standard signal processing techniques can be applied. The resulting RPE, EKF and coupled conditional filters will be novel for each application of the present invention. Specific examples of such applications will be considered in more detail below.

The arrangement of the present invention is illustrated in block diagram form in Figure 1. In Figure 1, the nomenclature used will be self-evident to persons of skill in signal processing. Example 1 - QAM Demodulation in Fading Noisy Channels The coding schemes used in communication systems which transmit quadrature amplitude modulation (QAM) signals result in the production of signals which can be reasonably approximated by hidden Markov models (HMM). The transition probabilities of the signal may be known, or it may require estimation. As such, QAM demodulation lends itself to on-line HMM filters in the known model case or to adaptive HMM filters when the model parameters are unknown or slowly changing. When the channels are fading with characteristics (gain and phase changes) which vary slowly with time relative to the message rate, then again the signals are suitable for processing using the adaptive HMM filters of the present invention.

Accordingly, a mixed-state model of the QAM signals received after transmission through noisy fading channels can be formulated, and particular adaptive HMM filters for QAM demodulation can be established. The QAM signals are then reformulated as conditioned information state models via HMM filter theory. Passing the received QAM signals through the adaptive HMM filter and ( in parallel) through a Kalman filter, with the filters coupled in accordance with the present invention, achieves the adaptive signal estimation. Figure 2, illustrates one such arrangement.

A key feature when processing QAM signals is that the channel model can be expressed in terms of rectangular co-ordinates involving real and imaginary parts of the channel complex gain, rather than in terms of polar co-ordinates involving the amplitude changes and phase changes of the channel.

The situations in which there are dynamics in the channel and/or the channel is subject to coloured noise can also be tackled with the same systematic approach to achieve effective signal estimation.

For a complete description of the use of the present invention to process QAM signals, including mathematical formulation and computer simulation studies, reference should be made to the paper by I B Collings and J B Moore, entitled "Adaptive Demodulation of QAM Signals in Noisy Fading Channels" which was included as an appendix to the specification of Australian provisional patent application No PL 8127, and which is to be published in the August 1994 issue of the International Journal of Adaptive Control and Signal Processing. The contents of that paper are incorporated into this specification by this reference to that paper. Figure 2 of this specification is included in that paper.

Example 2 - FM Demodulation in Fading Noisy Channels

The demodulation of standard frequency modulated (FM) signals in noisy channels involves a phase-locked loop ( PLL) which is virtually optimal for white Gaussian noise channels when the channel noise is small. Automatic gain control (AGC) is used to cope with channels with amplitude fading, and a second PLL filter is used to cope with changing phase. As the signal to noise ratio decreases in a fading situation, then the estimation arrangement is far from optimal, and phase tracking cycle skipping is experienced. There is a threshold of noise level above which the reception of the FM signals is intolerable. There are other optimal processing schemes available, which use banks of PLLs in the context of what is called Gaussian sum filtering. Those schemes are effective in high noise but are not designed, nor easily adapted, for fluctuating (fading) channel characteristics. Adaptive signal processing techniques are required in such circumstances to achieve good signal processing.

Thus FM signals and channels lend themselves to mixed-state modelling and conditioned information state models by application of HMM filter theory. Such modelling, in turn, lends itself to coupled, conditioned, information-state filters with a number of alternative arrangements depending on the application.

The effective processing of FM signals requires estimates to be made of the information states of the message (frequency), the associated phase, and the fading channel amplitude and phase change parameters. The most attractive estimation scheme from a computational performance point of view which emerges from the application of the present invention is one where there are three completed conditional filters. The first filter effectively estimates the message (frequency) information state conditioned on knowledge of the channel parameters and phase information state, the second filter estimates the phase information state conditioned on knowledge of the message (frequency) information state and channel parameters, and the third filter estimates the channel parameters conditioned on knowledge of the information states of the phase and (message) frequency. These three estimates are coupled using conditional estimates in lieu of actual information states and parameters.

To achieve robustness, there are various adjustments to the assumed model parameters that can be made, which crudely model uncertainty in the models or in the implementation of the suboptimal estimator arrangement.

The resulting novel estimators involve quantization of the signal phase to (typically) 16 or 32 levels, so that there is immediately a quantization error introduced. However, this error will have a less significant effect on the signal to noise ratio than channel noise.

The message itself can be modelled most simply as a first order Markov discrete-state chain that has discrete state values determined by the phase quantization. Alternatively, it can be modelled in a more sophisticated way as a finite dimensional linear system driven by white noise, perhaps even with unknown parameters. There are other alternatives, but the experience of the present inventors suggests that the estimation algorithms derived from these models are quite robust to the message model assumptions.

A new insight which has facilitated the approach to FM signal demodulation, even in the absence of channel fading, is that conditioned first order HMM filtering can be conveniently applied directly to the state equation for the phase, conditioned on knowledge of the message (frequency) or its information state. This is not obvious, for the quantized phase is known to be at least a second order Markov chain. Without this observation, the appropriate information state for the message (frequency) and phase is a two dimensional grid of order (say) 16 x 32 or 32 x 64, which renders computations prohibitive in most applications, since there are of order the square of the number of grid points. Thus, when using the present invention, the computational effort is of order 16². Using other approaches to FM demodulation, it is of order 16⁴.

Another important feature of the use of the present invention in the demodulation of FM signals is that it is known to be more advantageous, under certain conditions, to work with channel parameters in so-called rectangular co-ordinates than in so-called polar co-ordinates.

The case of coloured noise channels and/or signals with memory can also be tackled using the systematic algorithm design approach of the present invention.

For complete technical details of the application of the present invention to FM demodulation, including mathematical formulation and computer simulation studies, reference should be made to the paper by I B Collings and J B Moore, entitled "Adaptive Demodulation of FM signals in Noisy Fading Channels", which was included in the appendix to Australian provisional patent application No PL 8127, and which is to be published in April 1994 in the Proceedings of the International Conference on Acoustics, Speech and Signal Processing. The contents of that paper are incorporated into this specification by this reference to that paper.

Example 3 - Blind Egualization of Communication Channels Often in communication systems the source sends signals into an unknown and possibly time-varying channel. Additive noise corrupts the signal at the input and at the output of the channel. An equalizer may be used to reduce the effect of intersymbol interference introduced as a consequence of ( i ) limited channel bandwidth and ( ii ) the effect of additive white noise on the channel. Very often the equalizer is required to operate in the absence of information about the channel (which may be time-varying) and without knowledge of the transmitted signal. Given the noisy filtered output signal from the channel, the present invention can be used to estimate the channel parameters, and hence to reconstruct an estimate of the unknown transmitted signal. This task is termed "blind equalization" .

To use the present invention for blind equalization of channels, the signal generated by the source is modelled as a finite-state first-order Markov chain which can have unknown transition probabilities. The additive noise is assumed to be white and Gaussian. This can be reasonably used to model Frequency Shift Keyed (FSK) signals and Phase Shift Keyed (PSK) signals. Such a model can be reformulated in terms of information states using HMM filter theory and appropriate estimation algorithms applied, such as coupled conditional filters, or filters obtained by EKF techniques. In this manner, computationally efficient blind equalization algorithms for FIR (Finite Impulse Response), as well as IIR (Infinite Impulse Response), channels with time varying-coefficients can be formulated. Estimates of the channel parameters, transition probabilities, noise variance as well as input signal can be obtained.

This approach leads to improved estimates of the channel parameters, transition probabilities, noise variance as well as the input signal, and thus exhibits an enhanced performance over current blind equalization schemes.

For a complete description of the way in which the present invention may be used for blind equalisation of signal channels, reference should be made to the paper by V Krishnamurthy, entitled "Blind Equalization of IIR Channels using Hidden Markov Models and Extended Least Squares", which is to be published in July 1994 in the Proceedings of the International Symposium on Information Theory, Trondheim, Norway. The contents of that paper are incorporated into this specification by this reference to that paper.

Example 4 - De-interleaving of Radar Pulse Trains As noted above, the present invention can also be used to de-interleave radar pulse trains using discrete-time stochastic dynamic-linear models. Pulse trains from a number of different sources are often received on the one communication channel. It is then of interest to identify which pulses are from which source, assuming that the different sources have different characteristics. This sorting task is termed de-interleaving. It has applications in radar detection and potential applications in computer communications and neural systems. There are three possible types of sources which may cause interleaving, namely, periodic sources, independent sources and Markov sources. The periodic source example will be considered.

Figure 3 shows the pulse trains from two periodic sources with periods 11 and 17 and phases 3, 4 respectively. The interleaved pulse train is shown as the third pulse train of Figure 3. Figure 4 shows the corresponding evolution with pulse number of the sequence of active sources.

If the source sequence is unpredictable because of its stochastic nature, then two situations need to be considered, namely, (i) when the stochastic process is independent, and ( ii ) when the process is Markov.

The problem to be solved is the de-interleaving of time-interleaved pulse trains from a finite known number of periodic sources. It is assumed that observations of the time of arrival of the pulses are obtained in additive white Gaussian jitter noise without any information of the pulse amplitudes and phases. The aim of the signal processing technique is to de-interleave the received signal, (that is, to detect which source is responsible for each received pulse). With this information, it is a trivial exercise to estimate the periods and phases of the periodic pulse-train sources.

A key aspect of the use of the present invention as a de-interleaving approach is to first formulate the pulse-train de-interleaving problem as a stochastic discrete-time Dynamic Linear Model (DLM). A DLM is a time-varying linear system formulated in state space form with the state matrix and observation matrix at each time instant belonging to a finite set of possible values. In the de-interleaving case, the discrete-time instants are not the pulse times of arrival but rather integers indicating the pulse's number. Thus the "time" instant k indicates the arrival of the kth pulse. Then the state and observation matrices at each "time" instant k, termed here pulse instant k, depend on which source is active to generate the kth pulse. The state at each pulse instant consists of the periods of the sources and the last arrival time of each of the sources. If the pulses contain energy (for example, amplitude) information about the sources, this information can also be incorporated into the state vector.

If the actual source sequence is known, (for example, when there is only one periodic source), then it is known that optimal estimates of the state of the DLM and hence the periods of the sources can be obtained using a Kalman filter (KF) . When the actual source sequence is not known, it is possible to use the simpler Recursive Least Squares (RLS) parameter estimation algorithm when the pulse periods are constant. However, in general, when there is more than one source, because the actual source sequence is not known, the optimal solution involves evaluating the prediction error cost of each source sequence and choosing the sequence with the minimum cost. The number of possible source sequences increases exponentially with the data length and so this procedure is not computationally feasible for other than short data segments with few

-sources (typically about 20 data points and 3 sources). Clearly forward dynamic programming, in its simplest form cannot be used effectively to pick the optimal sequence because the costs at any stage of the multi-stage decision process are dependent on the history of the sequence

(path) .

Thus it is necessary to adopt sub-optimal solutions to the de-interleaving problem. Two possible sub-optimal solutions are based on the standard techniques proposed for estimating DLMs in the book by D M Titterington, A F M Smith and U E Makov, entitled Statistical Analysis of Finite Mizture Distributions, which was published by John Wiley in 1985. These sub-optimal solutions can be viewed as tree pruning algorithms which attempt to eliminate low probability paths in order to achieve a computationally feasible algorithm.

Forward Dynamic Programming (FDP) in its rudimentary form cannot be used to obtain the optimal path sequence. However, it is possible to adopt a scheme which combines the optimal full tree search algorithm over a short segment (look-ahead interval) to reject improbable paths and FDP to update the most likely sequences and costs, terminating at each source at each pulse arrival. That is, over the look-ahead interval, the KF prediction error of all sequences can be evaluated. For N sources with a look-ahead of Δ, the computational cost is 0(N^Δ+3). Typically, for a small number of sources (N < 10), simulations show that for satisfactory performance, the look-ahead requirement is about 3, so that the computational cost is not excessive. Of course, if the look-ahead interval is the length of the observation sequence, then the algorithm is the optimal full tree search algorithm mentioned above. The tree pruning algorithms presented in Chapter 2 of the dissertation by F Gustafsson, entitled "Estimation of discrete parameters in linear systems", which was published by the Department of Electrical Engineering of Linkoping University, Sweden, as Dissertation No 271 in Linkoping Studies in Science and Technology, are very similar to FDP with look-ahead.

Probabilistic Teacher (PT) algorithms have been proposed for estimating DLMs in the paper by A K Agrawala, entitled

"Learning with a Probabilistic Teacher", which was published in the IEEE Transactions on Information Theory.

Volume IT-16, No 4, pages 373 to 379, July 1970, and also in the paper by F S Chang and M D Srinath, entitled "Tracking over Fading Channels", which was published in the

IEEE Transactions on Communications, April 1975. If the a priori probabilities of the sources (related to the periods) are known exactly, then the estimates of the periods using PT asymptotically tend to the optimal estimates. In fact, as shown in simulation studies for

SUBSTTTUTB SHEET (Rule 26) sequences of reasonable length, PT using the correct a priori probabilities yields estimates of the periods which are as good as the estimates when the true source sequence is known. When the a priori probabilities are not known, it is necessary to compute a posteriori probabilities and use PT with these probabilities to obtain state estimates of the DLM. However, PT using a posteriori probabilities is prone to error propagation and not robust to initial conditions.

Using the Hidden Markov Model approach, it is assumed that the source sequence is a Markov chain. Thus the de-interleaving problem becomes a time-varying linear system with the system and observation matrices depending on the state of the Markov chain. Using the approach of the present invention, recursive prediction error based techniques are used to obtain estimates. Since the optimal filtered estimates of the Markov chain cannot be obtained recursively, decision feedback has to be used to derive sub-optimal filtering equations.

For complete technical details, including mathematical formulation and computer simulation studies, reference should be made to the paper by J B Moore and V Krishnamurthy, entitled "De-interleaving Pulse Trains Using Discrete-time Stochastic Dynamic-Linear Models" which was included in the appendix to the specification of Australian provisional patent application No PL 8127, and which was/is to be published in IEEE Transactions of Signal Processing. The contents of that paper are incorporated into this specification by this reference to that paper. Example 5 - Improved Speech Signal Prediction Speech signals can be reasonably modelled as linear dynamical systems driven by approximately periodic pulses, for voiced speech, and by white noise for unvoiced speech. The linear system parameters correspond to those of the vocal tract resonant cavities and change with the flow of the speech according to movement of the mouth, tongue and chest. These changes are relatively slowly varying compared to speech signal variations.

Speech signals can also be modelled, reasonably accurately, as mixed-state models. The periodic pulses at the pitch frequency can be modelled by a first order finite state Markov chain with states in a discrete set, and the channel (vocal tract) can be modelled with continuous range states. Such a model can be reformulated in terms of information states using HMM filter theory and appropriate estimation algorithms applied, using coupled conditional filters, or EKF based filters. The resulting new algorithms have the potential for improved speech signal prediction from one sampling instant to the next.

The one-step-ahead prediction error is often transmitted in adaptive pulse code modulation (ADPCM) and related schemes, at least suitably quantized.

Information-state adaptive signal processing based on EKF theory applied to information-state models derived from HMM filter theory should lead to improved pitch period prediction and improved prediction error quantization, and thus give performance over state-of-the-art speech coders. Those skilled in the art of signal processing will appreciate that although specific examples of the use of the present invention have been provided in this specification, the invention is not limited to those examples, and that variations to and modifications of the described techniques may be made without departing from the present inventive concept.

Claims

1. A method of processing a signal which has the characteristics of a hybrid or a mixed-state signal model, the method comprising the steps of:

(a) formulating a mixed-state model for the signal;

(b) applying the signal to an optimal Kalman filter or extended Kalman filter and simultaneously applying the signal to an optimal Hidden Markov filter;

(c) coupling the output of the Kalman filter to a stage of the Hidden Markov filter;

(e) deriving the desired signal estimate from the output of the Hidden Markov filter.

2. A method as defined in claim 1, in which the signal being processed is a quadrature amplitude modulated signal.

3. A method as defined in claim 1, in which the signal being processed is a frequency modulated signal.

4. A method as defined in claim 1, in which the received signal being processed is from an unknown channel, corrupted by white Gaussian noise, and there is no test transmission for channel identification.

5. A method as defined in claim 1, in which the signal being processed is an interleaved pulse train, and said processing is performed to de-interleave said pulse train.

6. A method as defined in claim 1, in which the signal being processed is a speech signal.

7. A method as defined in claim 2, substantially as described in the paper entitled "Adaptive Demodulation of QAM signals in Noisy Fading Channels", which is identified in the text of this specification.

8. A method as defined in claim 3, substantially as described in the paper entitled "Adaptive Demodulation of FM Signals in Noisy Fading Channels", which is identified in the text of this specification.

9. A method as defined in claim 4, substantially as described in the paper entitled "Blind Equalisation of IIR Channels using Hidden Markov Models and Extended Least Squares", which is identified in the text of this specification.

10. A method as defined in claim 5, substantially as described in the paper entitled "De-interleaving Pulse Trains using Discrete-time Stochastic Dynamic-Linear Models", which is identified in the text of this specification.

11. A method of signal processing as defined in claim 1, substantially as hereinbefore described.