Numerical Math - Solving systems of equations utilizing matrices (data fitting)

A nice pointer from Laundau et al [1] in the opening of their chapter (Ch. 8) on Solving Systems of Equations with Matrices; Data Fitting.  Many physical systems are modeled utilizing matrices which consist of a system of simultaneous equations.  However, these sets can become quite large and complicated which is why computers are very good with these processes.  Usually the algorithms for solving these sets of equations that utilize matrix theory is that they require repetition of a small list of steps which have been written in an efficient method.

One additional technique for speed is to tune the algorithm to the actual architecture of the computer which Landau et al [1] discuss more in their Chapter 14 called High-Performance Computing Hardware, Tuning, & Parallel Computing.

Many libraries exist which are "industrial-strength" subroutines for solving these matrix systems.  A majority of these libraries are well established such as the IMSL Numerical Libraries by Rogue Wave Software, Inc., the GNU Scientific Library (GSL), and the Netlib Repository at UTK and ORNL which contains LAPACK — Linear Algebra PACKage.  Landau et al [1] note that these libraries are usually an order of magnitude faster than general methods in linear algebra texts.  The libraries are streamlined for minimal round-off error and are aimed to solve a large spectrum of problems with high success.  It is here and for the reasons just mentioned where Landau et al warn that it is best if you don't write your own matrix subroutines but retrieve them from one of these libraries.  The libraries also provide the advantage of allowing the user to run them on one machine/processor or many machines/processors by varying with the computer architecture.

Next, Landau et al [1] ask the question which proposes to the user what is considered a "large" matrix.  Before, a large matrix was based upon a fraction of the RAM available to the computer system.  However, Landau et al describe a "large" matrix as now based upon the numerical time it takes to obtain values.  That is, if any waiting time is required, then a library should be used.  Landau et al also comment that the libraries are beneficial for speed even when the matrices may be small (which might apply in graphics processing).

One negative side effect lies in the multiple languages that the libraries are written.  That is one library may be in Fortran while the user is a C coder.  However, today libraries might exist which are programmed in or for many different computer coding languages.

---

References:


[1] R. H. Landau, M. J. Páez, and C. C. Bordeianu. A Survey of Computational Physics - Introductory Computational Science, Princeton University Press, Princeton, New Jersey. 2008

Matrices - General math

As I've stated before I am logging topics/subjects I come across in classes and research. I am taking a vector calculus mainly for review, but I am also understanding the fundamentals much better and learning new stuff.  On top of that, I am taking a mathematical methods for physicists and inviscid flow and both are giving added and large doses of vector analysis.  I always have the point of view that the more you hear and see something the more you remember and take away from it. Thus, it becomes "easier" or almost second nature.  I also encourage getting a wide range of views from various people as they have very different teaching techniques, experiences, and their own learning background.  So, while I may have tortured myself by taking a couple of extra classes which are not necessarily "needed" for my degrees, I am gaining much valuable insight to the fundamentals and basics.

So, one of the first things (or eventual topics) in vector courses that comes about appears as matrices.  Matrices are very important in physics and math and thus most sciences.  Matrices appear in many math courses such as linear/matrix algebra and numerical linear/matrix algebra.  Matrices are associated with linear algebra due to collecting systems of linear ordinary differential equations ODEs into sets which are easily converted into matrices.  In fact, in Shores [1] book, Applied Linear Algebra and Matrix Analysis, he opens by remarking that

"The two central problems about which much of the theory of linear algebra revolves are the problem of finding all solutions to a linear system and that of finding an eigensystem for a square matrix."



In progress...


[1] T. S. Shores. Applied Linear Algebra and Matrix Analysis. Springer Science+Business Media, LLC., New York, NY. 2007

Vector Analysis and Calculus - General math

This post highlights vector calculus and vector analysis. Such topics and discussions will cover ideas and concepts encountered when dealing with vector analysis. Vectors are a very important concept in science and math and appear very early in calculus and physics classes in college or in advanced high school classes. Vectors are used to describe a direction and magnitude of important concepts in our world such as gravity and other forces. Vectors can also group together many sets of equations into a compact form such as the three-dimensional equations of motion expressed as a single line of a vector equation instead of expanding for each of the three components. Vectors appear very often in fluid dynamics and are a fundamental basis for the subject. Many fluid dynamic books introduce the reader into and with vector analysis [1-5] or contain an appendix at the end of the book [5] while the manuscript by Aris [6] is based upon vectors and fluid dynamics altogether.

As mentioned previously, vectors make up the fundamental basis of mathematics and physics in the world. Many mathematical books which generalize math topics which scientist, engineers, physicists, etc, should be familiar with often place vectorial analysis in the beginning or at least include it in the book type encyclopedia (a collection of topics and methods) of math for applied uses [7].

Vectors not only make up fundamental basis for fluid dynamics, but also for math and physics.  In fact it is an underlying foundation in math and physics sciences. Einstein used vector notation to develop and describe his relativity theories. Many in the early stages of the study of math and physics fields generally see vectors in essential courses such as calculus and general physics. In calculus, vector analysis might appear in a II or III semester of the course as the concepts of divergence, curl, gradient, etc. are brought up and discussed. Vectors appear very early at the very beginning in general college physics courses because almost everything is based upon the vector theory.  Forces, acceleration, velocity, position, momentum, electrostatics, fluid mechanics and dynamics, etc. all utilize vector analysis.

Specifically, since my background is in fluid dynamics from a graduate aerospace engineering education, I will use my experiences to discuss vectors and other topics.  Vectors in fluid dynamics/mechanics can be very useful to describe many fluid concepts and phenomena.  Additionally, vectors are very useful as they are coordinate independent or in other words the user does not have to worry about a coordinates system.

A general representation of a vector can be

\[ \vec{A} = A_1 \vec{e}_1 + A_2\vec{e}_2 + A_3 \vec{e}_3 + \ldots + A_n \vec{e}_n = \sum_{i = 1}^n \]

where the coordinate system can be anything and the number of dimensions \( n \).  This simple relation is why vectors are so useful, powerful, and why they make up the foundations of our math, physics, and related sciences.

The vector may also be written without the arrow above where boldface font is available.

\[ \mathbf{A} = A_1 \mathbf{e}_1 + A_2 \mathbf{e}_2 + A_3 \mathbf{e}_3 + \ldots + A_n \mathbf{e}_n \]

Scalars according to Serre [ ],

"...are elements of some field \( k \) (or \( K \)), or sometimes of a ring \( R \)."



In progress...

---

References:

[1] K. Karamcheti. Principles of Ideal-Fluid Aerodynamics. John Wiley & Sons, Inc., New York, NY. 1966

[2] R. L. Panton. Incompressible Flow. (3rd ed.). John Wiley & Sons, Inc. Hoboken, NJ. 2005

[3] M. E. O’Neill & F. Chorlton. Viscous and Compressible Fluid Dynamics. Ellis Horwood Limited. Chichester, UK. 1989

[4] T. C. Papanastasiou, G. C. Georgiou, & A. N. Alexandrou. Viscous Fluid Flow. CRC Press. Boca Raton, FL. 2000

[5] M. T. Schobeiri. Fluid Mechanics for Engineers - A Graduate Textbook. Springer.

Berlin, Germany. 2010

[6] R. Aris. Vectors, Tensors and the Basic Equations of Fluid Mechanics. Dover Publications.

New York, NY. 1990

[7] G. B. Arfken & H.-J. Weber, Mathematical Methods for Physicists (6th ed.). Elsevier Academic Press. Burlington, MA. 2005

[ ] D. Serre. Matrices: Theory and Applications. Springer-Verlag New York, Inc., New York, NY. 2002

[ ] D. Serre. Matrices: Theory and Applications. 2nd ed. Springer Science+Business Media, LLC., New York, NY. 2010

Some references for vector calculus and analysis

Here are some good resources to look at for vector calculus and analysis including linear algebra and matrices.

[1] K. Karamcheti. Principles of Ideal-Fluid Aerodynamics. John Wiley & Sons, Inc., New York, NY. 1966

[2] W. Kaplan. Advanced Calculus (5th ed.). Addison-Wesley. 2002

[3] G. B. Arfken & H.-J. Weber, Mathematical Methods for Physicists (6th ed.). Elsevier Academic Press. Burlington, MA. 2005

[4] R. Aris. Vectors, Tensors and the Basic Equations of Fluid Mechanics. Dover Publications. New York, NY. 1990

[5] A. I. Borisenko and I. E. Tarapov. Vector and Tensor Analysis with Applications, (translated by R. A. Silverman). Dover Publications Inc., Mineola, NY. 1979 (originally published in 1968 by Prentice-Hall, Inc.

[6] H. M. Schey. Div, Grad, Curl, and All That: An Informal Text on Vector Calculus, 3rd ed. W. W. Norton \& Company, New York, NY. 1997

[7] H. M. Schey. Div, Grad, Curl, and All That: An Informal Text on Vector Calculus, 4th ed. W. W. Norton \& Company, New York, NY. 2005

[8] K. R. Reddy, S. Raghavan, and D.V.N. Sarma. Elements of Mechanics, Universities Press (India) Limited, Hyderabad, India. 1994.

[9] C.-T. Tai. General Vector and Dyadic Analysis: Applied Mathematics in Field Theory, 2nd ed. Wiley-IEEE Press, New York, NY. 1997.

[10] J. Betten. Creep Mechanics, 2nd ed. Springer, Berlin, Germany. 2005.

[11] J. Betten. Creep Mechanics, 3rd ed. Springer, Berlin, Germany. 2008.

[12] J. C. Slattery. Advanced Transport Phenomena, Cambridge University Press, Cambridge, UK. 1999.

[13] F. Irgens. Continuum Mechanics Springer, Berlin, Germany. 2008.

[14] P. J. Pahl and R. Damrath. Mathematical Foundations of Computational Engineering: A Handbook Springer, Berlin, Germany. 2001.

[15] M. T. Schobeiri. Fluid Mechanics for Engineers - A Graduate Textbook. Springer.

Berlin, Germany. 2010

In progress...to be continued.