Error Analysis

Error Amplification

Introduction

In the previous section we discussed a basic error estimation. Here we consider the propagation and amplification of errors during data processing. The simplest procedure is the solution of a system of linear equations.

 (1a)

The inverse system is

 (2a)

Obviously this is also a linear system.

Numerical experiments

Experiment 1. Compare the solution for  b1 = 1 and b2 = -1 with the solution for the right hand side with 1% noise.

 b1 b2 x1 e1 x2 e2 1.0 -1.0 40.0 -20.0 1.01 -1.01 40.4 1 % -20.2 1 % 1.01 -0.99 40.002 0.005% -19.996 0.02%

Here ei are the relative errors for the r. h. s. The accuracy of the solutions is also 1% or less.

Experiment 2. Compare the solution for  b1 = 1 and b2 = 1 with the solution for the r. h. s. with 1% noise.

 b1 b2 x1 err1 x2 err2 1.0 1.0 0.2 0.4 1.01 1.01 0.202 1 % 0.404 1 % 1.01 0.99 0.6 300 % 0.2 50%

The result speaks for itself: a simple linear transformation can cause a catastrophic increase of an error.

Analysis

Assuming that the errors for b1and b2 are uncorrelated with the standard deviation σ, we get from (2) (see the appendix for details)

 var(x1) = 800.02 σ2 var(x2) =  200.08 σ2 (3) cov(x1, x2) = -399.96 σ2

The errors for x are not only much larger but also are strongly correlated. This can be represented graphically. Two independent errors of the magnitude 0.01 form a circle in the  b1, b2 plane

The corresponding solution errors form the following elongated ellipse in the  x1, x2 plane

Algebraic approach

Equations (1a) and (2a) can be rewritten as

 (1b) (2b)

Using the singular value decomposition (SVD) the matrices can be presented in the following form

 (4)

and

 (5)

where

 (6)

and

 (7)

Any vector in the x-space can be presented as a linear combination of the vectors v1 and v2:

 (8)

Similarly any vector in the b-space can be expressed in terms of u1 and u2:

 (9)

Eq. (5) can be interpreted in the following way. The first term transforms the u1 component of b into the v1 component of x, and is the transformation factor. The second term transforms the u2 component of b into the v2 component of x. However, the transformation factor in this case is 100 times larger. This explains noise amplification.

Filtering

The components corresponding to large noise amplification cause instability in the solution of linear systems. The filters based on the SVD can improve stability without essential loss of accuracy.

Appendix

Extended variant of Eq. (3).

 var(x1) = 404.01 var(b1) - 799.98 cov(b1, b2) + 396.01 var(b2) var(x2) =  96.04 var(b1) - 199.92 cov(b1, b2) + 104.04 var(b2) (1) cov(x1, x2) = -196.98 var(b1) + 400.04 cov(b1, b2) + 202.98 var(b2)

References

1. Singular value decomposition (SVD).
2. Linear algebra: a summary of vector and matrix operations is available here.

© Nikolai Shokhirev, 2001 - 2017

Count: