Numerical Math - Solving systems of equations utilizing matrices (data fitting)

A nice pointer from Laundau et al [1] in the opening of their chapter (Ch. 8) on Solving Systems of Equations with Matrices; Data Fitting.  Many physical systems are modeled utilizing matrices which consist of a system of simultaneous equations.  However, these sets can become quite large and complicated which is why computers are very good with these processes.  Usually the algorithms for solving these sets of equations that utilize matrix theory is that they require repetition of a small list of steps which have been written in an efficient method.

One additional technique for speed is to tune the algorithm to the actual architecture of the computer which Landau et al [1] discuss more in their Chapter 14 called High-Performance Computing Hardware, Tuning, & Parallel Computing.

Many libraries exist which are "industrial-strength" subroutines for solving these matrix systems.  A majority of these libraries are well established such as the IMSL Numerical Libraries by Rogue Wave Software, Inc., the GNU Scientific Library (GSL), and the Netlib Repository at UTK and ORNL which contains LAPACK — Linear Algebra PACKage.  Landau et al [1] note that these libraries are usually an order of magnitude faster than general methods in linear algebra texts.  The libraries are streamlined for minimal round-off error and are aimed to solve a large spectrum of problems with high success.  It is here and for the reasons just mentioned where Landau et al warn that it is best if you don't write your own matrix subroutines but retrieve them from one of these libraries.  The libraries also provide the advantage of allowing the user to run them on one machine/processor or many machines/processors by varying with the computer architecture.

Next, Landau et al [1] ask the question which proposes to the user what is considered a "large" matrix.  Before, a large matrix was based upon a fraction of the RAM available to the computer system.  However, Landau et al describe a "large" matrix as now based upon the numerical time it takes to obtain values.  That is, if any waiting time is required, then a library should be used.  Landau et al also comment that the libraries are beneficial for speed even when the matrices may be small (which might apply in graphics processing).

One negative side effect lies in the multiple languages that the libraries are written.  That is one library may be in Fortran while the user is a C coder.  However, today libraries might exist which are programmed in or for many different computer coding languages.

---

References:


[1] R. H. Landau, M. J. Páez, and C. C. Bordeianu. A Survey of Computational Physics - Introductory Computational Science, Princeton University Press, Princeton, New Jersey. 2008