Complex Valued Nonlinear Adaptive Filters: Noncircularity, Widely Linear and Neural Models -- Series Page -- Contents -- Preface -- Acknowledgements -- 1 The Magic of Complex Numbers -- 1.1 History of Complex Numbers -- 1.1.1 Hypercomplex Numbers -- 1.2 History of Mathematical Notation -- 1.3 Development of Complex Valued Adaptive Signal Processing -- 2 Why Signal Processing in the Complex Domain? -- 2.1 Some Examples of Complex Valued Signal Processing -- 2.1.1 Duality Between Signal Representations in R and C -- 2.2 Modelling in C is Not Only Convenient But Also Natural -- 2.3 Why Complex Modelling of Real Valued Processes? -- 2.3.1 Phase Information in Imaging -- 2.3.2 Modelling of Directional Processes -- 2.4 Exploiting the Phase Information -- 2.4.1 Synchronisation of Real Valued Processes -- 2.4.2 Adaptive Filtering by Incorporating Phase Information -- 2.5 Other Applications of Complex Domain Processing of Real Valued Signals -- 2.6 Additional Benefits of Complex Domain Processing -- 3 Adaptive Filtering Architectures -- 3.1 Linear and Nonlinear Stochastic Models -- 3.2 Linear and Nonlinear Adaptive Filtering Architectures -- 3.2.1 Feedforward Neural Networks -- 3.2.2 Recurrent Neural Networks -- 3.2.3 Neural Networks and Polynomial Filters -- 3.3 State Space Representation and Canonical Forms -- 4 Complex Nonlinear Activation Functions -- 4.1 Properties of Complex Functions -- 4.1.1 Singularities of Complex Functions -- 4.2 Universal Function Approximation -- 4.2.1 Universal Approximation in R -- 4.3 Nonlinear Activation Functions for Complex Neural Networks -- 4.3.1 Split-complex Approach -- 4.3.2 Fully Complex Nonlinear Activation Functions -- 4.4 Generalised Splitting Activation Functions (GSAF) -- 4.4.1 The Clifford Neuron -- 4.5 Summary: Choice of the Complex Activation Function -- 5 Elements of CR Calculus.

5.1 Continuous Complex Functions -- 5.2 The Cauchy-Riemann Equations -- 5.3 Generalised Derivatives of Functions of Complex Variable -- 5.3.1 CR Calculus -- 5.3.2 Link between R- and C-derivatives -- 5.4 CR-derivatives of Cost Functions -- 5.4.1 The Complex Gradient -- 5.4.2 The Complex Hessian -- 5.4.3 The Complex Jacobian and Complex Differential -- 5.4.4 Gradient of a Cost Function -- 6 Complex Valued Adaptive Filters -- 6.1 Adaptive Filtering Configurations -- 6.2 The Complex Least Mean Square Algorithm -- 6.2.1 Convergence of the CLMS Algorithm -- 6.3 Nonlinear Feedforward Complex Adaptive Filters -- 6.3.1 Fully Complex Nonlinear Adaptive Filters -- 6.3.2 Derivation of CNGD using CR calculus -- 6.3.3 Split-complex Approach -- 6.3.4 Dual Univariate Adaptive Filtering Approach (DUAF) -- 6.4 Normalisation of Learning Algorithms -- 6.5 Performance of Feedforward Nonlinear Adaptive Filters -- 6.6 Summary: Choice of a Nonlinear Adaptive Filter -- 7 Adaptive Filters with Feedback -- 7.1 Training of IIR Adaptive Filters -- 7.1.1 Coefficient Update for Linear Adaptive IIR Filters -- 7.1.2 Training of IIR filters with Reduced Computational Complexity -- 7.2 Nonlinear Adaptive IIR Filters: Recurrent Perceptron -- 7.3 Training of Recurrent Neural Networks -- 7.3.1 Other Learning Algorithms and Computational Complexity -- 7.4 Simulation Examples -- 8 Filters with an Adaptive Stepsize -- 8.1 Benveniste Type Variable Stepsize Algorithms -- 8.2 Complex Valued GNGD Algorithms -- 8.2.1 Complex GNGD for Nonlinear Filters (CFANNGD) -- 8.3 Simulation Examples -- 9 Filters with an Adaptive Amplitude of Nonlinearity -- 9.1 Dynamical Range Reduction -- 9.2 FIR Adaptive Filters with an Adaptive Nonlinearity -- 9.3 Recurrent Neural Networks with Trainable Amplitude of Activation Functions -- 9.4 Simulation Results.

10 Data-reusing Algorithms for Complex Valued Adaptive Filters -- 10.1 The Data-reusing Complex Valued Least Mean Square (DRCLMS) Algorithm -- 10.2 Data-reusing Complex Nonlinear Adaptive Filters -- 10.2.1 Convergence Analysis -- 10.3 Data-reusing Algorithms for Complex RNNs -- 11 Complex Mappings and Möbius Transformations -- 11.1 Matrix Representation of a Complex Number -- 11.2 The Möbius Transformation -- 11.3 Activation Functions and Möbius Transformations -- 11.4 All-pass Systems as Möbius Transformations -- 11.5 Fractional Delay Filters -- 12 Augmented Complex Statistics -- 12.1 Complex Random Variables (CRV) -- 12.1.1 Complex Circularity -- 12.1.2 The Multivariate Complex Normal Distribution -- 12.1.3 Moments of Complex Random Variables (CRV) -- 12.2 Complex Circular Random Variables -- 12.3 Complex Signals -- 12.3.1 Wide Sense Stationarity, Multicorrelations, and Multispectra -- 12.3.2 Strict Circularity and Higher-order Statistics -- 12.4 Second-order Characterisation of Complex Signals -- 12.4.1 Augmented Statistics of Complex Signals -- 12.4.2 Second-order Complex Circularity -- 13 Widely Linear Estimation and Augmented CLMS (ACLMS) -- 13.1 Minimum Mean Square Error (MMSE) Estimation in C -- 13.1.1 Widely Linear Modelling in C -- 13.2 Complex White Noise -- 13.3 Autoregressive Modelling in C -- 13.3.1 Widely Linear Autoregressive Modelling in C -- 13.3.2 Quantifying Benefits of Widely Linear Estimation -- 13.4 The Augmented Complex LMS (ACLMS) Algorithm -- 13.5 Adaptive Prediction Based on ACLMS -- 13.5.1 Wind Forecasting Using Augmented Statistics -- 14 Duality Between Complex Valued and Real Valued Filters -- 14.1 A Dual Channel Real Valued Adaptive Filter -- 14.2 Duality Between Real and Complex Valued Filters -- 14.2.1 Operation of Standard Complex Adaptive Filters -- 14.2.2 Operation of Widely Linear Complex Filters.

14.3 Simulations -- 15 Widely Linear Filters with Feedback -- 15.1 The Widely Linear ARMA (WL-ARMA) Model -- 15.2 Widely Linear Adaptive Filters with Feedback -- 15.2.1 Widely Linear Adaptive IIR Filters -- 15.2.2 Augmented Recurrent Perceptron Learning Rule -- 15.3 The Augmented Complex Valued RTRL (ACRTRL) Algorithm -- 15.4 The Augmented Kalman Filter Algorithm for RNNs -- 15.4.1 EKF Based Training of Complex RNNs -- 15.5 Augmented Complex Unscented Kalman Filter (ACUKF) -- 15.5.1 State Space Equations for the Complex Unscented Kalman Filter -- 15.5.2 ACUKF Based Training of Complex RNNs -- 15.6 Simulation Examples -- 16 Collaborative Adaptive Filtering -- 16.1 Parametric Signal Modality Characterisation -- 16.2 Standard Hybrid Filtering in R -- 16.3 Tracking the Linear/Nonlinear Nature of Complex Valued Signals -- 16.3.1 Signal Modality Characterisation in C -- 16.4 Split vs Fully Complex Signal Natures -- 16.5 Online Assessment of the Nature of Wind Signal -- 16.5.1 Effects of Averaging on Signal Nonlinearity -- 16.6 Collaborative Filters for General Complex Signals -- 16.6.1 Hybrid Filters for Noncircular Signals -- 16.6.2 Online Test for Complex Circularity -- 17 Adaptive Filtering Based on EMD -- 17.1 The Empirical Mode Decomposition Algorithm -- 17.1.1 Empirical Mode Decomposition as a Fixed Point Iteration -- 17.1.2 Applications of Real Valued EMD -- 17.1.3 Uniqueness of the Decomposition -- 17.2 Complex Extensions of Empirical Mode Decomposition -- 17.2.1 Complex Empirical Mode Decomposition -- 17.2.2 Rotation Invariant Empirical Mode Decomposition (RIEMD) -- 17.2.3 Bivariate Empirical Mode Decomposition (BEMD) -- 17.3 Addressing the Problem of Uniqueness -- 17.4 Applications of Complex Extensions of EMD -- 18 Validation of Complex Representations - Is This Worthwhile? -- 18.1 Signal Modality Characterisation in R.

18.1.1 Surrogate Data Methods -- 18.1.2 Test Statistics: The DVV Method -- 18.2 Testing for the Validity of Complex Representation -- 18.2.1 Complex Delay Vector Variance Method (CDVV) -- 18.3 Quantifying Benefits of Complex Valued Representation -- 18.3.1 Pros and Cons of the Complex DVV Method -- Appendix A: Some Distinctive Properties of Calculus in C -- Appendix B: Liouville's Theorem -- Appendix C: Hypercomplex and Clifford Algebras -- C.1 Definitions of Algebraic Notions of Group, Ring and Field -- C.2 Definition of a Vector Space -- C.3 Higher Dimension Algebras -- C.4 The Algebra of Quaternions -- C.5 Clifford Algebras -- Appendix D: Real Valued Activation Functions -- D.1 Logistic Sigmoid Activation Function -- D.2 Hyperbolic Tangent Activation Function -- Appendix E: Elementary Transcendental Functions (ETF) -- Appendix F: The O Notation and Standard Vector and Matrix Differentiation -- F.1 The O Notation -- F.2 Standard Vector and Matrix Differentiation -- Appendix G: Notions From Learning Theory -- G.1 Types of Learning -- G.2 The Bias-Variance Dilemma -- G.3 Recursive and Iterative Gradient Estimation Techniques -- G.4 Transformation of Input Data -- Appendix H: Notions from Approximation Theory -- Appendix I: Terminology Used in the Field of Neural Networks -- Appendix J: Complex Valued Pipelined Recurrent Neural Network (CPRNN) -- J.1 The Complex RTRL Algorithm (CRTRL) for CPRNN -- Appendix K: Gradient Adaptive Step Size (GASS) Algorithms in R -- K.1 Gradient Adaptive Stepsize Algorithms Based on ∂J/∂μ -- K.2 Variable Stepsize Algorithms Based on ∂J/∂ε -- Appendix L: Derivation of Partial Derivatives from Chapter 8 -- L.1 Derivation of ∂e(k)/∂wn(k) -- L.2 Derivation of ∂e*(k)/∂ε(k - 1) -- L.3 Derivation of ∂w(k)/∂ε(k - 1) -- Appendix M: A Posteriori Learning -- M.1 A Posteriori Strategies in Adaptive Learning.

Appendix N: Notions from Stability Theory.