Santa Clara University ELEN 299: Directed Research

Mathematical Development of the Least Squares, Wiener Filter, LMS, and RLS Algorithms

Contents

1 Introduction 2

2 Least Squares 5

2.1 LSSolution.............................. 5

2.1.1 CalculusApproach...................... 6

2.1.2 OrthogonalityPrincipleApproach . . . . . . . . . . . . . 6

2.1.3 OrthogonalityandUniqueness ............... 7

3 Wiener Filter 7

3.1 WienerSolution............................ 8

3.1.1 CalculusApproach...................... 8

3.1.2 OrthogonalityPrincipleApproach . . . . . . . . . . . . . 9

4 LMS Algorithm 9

4.0.1 Development ......................... 10

4.0.2 Convergence ......................... 10

5 RLS Algorithm 11

5.0.1 Development ......................... 12

5.0.2 InitializationConcerns.................... 14

5.0.3 ImplementationConcerns .................. 15

6 Conclusion ..................15