Preface: Real life phenomena of engineering, natural science or medical problems are often formulated by means of a mathematical model with the goal to simulate the behaviour of the system in computerized form. Advantages of mathematical models are their cheap availability, the possibility to simulate extreme situations that cannot be handled by experiments, or to simulate real systems during the design phase before constructing a first prototype. Moreover, they serve to verify decisions, to avoid expensive and time consuming experimental tests, to analyze, understand and explain the behaviour of systems, or to optimize design and production.
As soon as a mathematical model contains differential dependencies from an additional parameter, typically the time, we call it a dynamical model. There are now two key questions that always arise in a practical environment:
To summarize, parameter estimation or data fitting, respectively, is extremely important in all practical situations, where a mathematical model and corresponding experimental data are available to describe the behaviour of a dynamical system.
The main goal of the book is to give an overview of numerical methods that
are needed to compute parameters of a dynamical model by a least squares fit.
The mathematical equations that must be provided by the system analyst, are
explicit model functions or steady state systems in the most simple situations,
or responses of dynamical systems defined by ordinary differential equations,
differential algebraic equations, or one-dimensional partial differential equations.
Many different mathematical disciplines must be combined, and
the intention is to present at least some fundamental ideas of the numerical
methods needed, so that available software can be applied successfully.
It must be noted that there are two alternative aspects that are not treated in this book. First we do not emphasize statistical analysis, which is also known as nonlinear regression or nonlinear parameter estimation. Moreover, we do not investigate the question whether parameters of a dynamical model can be identified at all, and under which mathematical conditions. Is is supposed that a user is able prepare a well-defined model, i.e., that the dynamical system is uniquely solvable and that the parameters can be identified by a least squares fit. For both topics there exist numerous very qualified text books that are recommended as secondary literature.
It is assumed that the typical reader is familiar with basic mathematical notation of linear algebra and analysis, as for example learned in elementary calculus lectures. Any additional knowledge about mathematical theory is not required. New concepts are presented in an elementary form and are illustrated by detailed analytical and numerical examples.
Extensive numerical results are included to show the efficiency of modern mathematical algorithms. But we also discuss possible pitfalls in form of warnings that even the most qualified numerical algorithms we know today, can fail or produce unacceptable responses. The practical progress of mathematical models and data fitting calculations is illustrated by case studies from pharmaceutics, mechanical, electrical or chemical engineering, and ecology.
To be able to repeat all numerical tests presented in the book, and to play with algorithms, data and solution tolerances, an interactive software system is included that runs under Windows 95/98/NT4.0/2000. The program contains the mathematical algorithms described in the book. The database contains 1,000 illustrative examples, to be used as benchmark test problems. Among them is large number of real life models (learning by doing).
The book is the outcome of my research in this area since the last 20 years with emphasis on the development of numerical algorithms
for solving optimization problems.
It would be impossible to design applicable mathematical algorithms and implement the
corresponding software without intensive discussions, contact and cooperation
with firms, e.g. Boehringer Ingelheim Pharma, BASF Ludwigshafen, Siemens
Munich, Schloemann-Siemag Hilchenbach, DaimlerChrysler Aerospace (DASA/MBB) Munich,
Bayer Sarnia, Dornier Satellite Systems Munich, and many research institutions at universities.
In particular I would like to thank Dr. M. Wolf from Boehringer
Ingelheim Pharma KG for providing many dynamical models describing pharmaceutical
applications, and for encouraging the investigation of models based on partial differential equations.
Part of my research was supported by projects funded by the BMBF research program Anwendungsbezogene Verbundprojekte in
der Mathematik and the DFG research program Echtzeit-Optimierung grosser Systeme.
Parameter estimation, also called parameter identification, nonlinear regression or data fitting, is extremely important for all practical situations, where a mathematical model and corresponding experimental data are available to analyze the behaviour of a dynamical system.
The main goal of the book is to introduce some numerical methods that can be used to compute parameters by a least squares fit in form of a tool box. The mathematical model that must be provided by the system analyst, has to belong to one of the following categories:
To understand at least some of the basic features of the algorithms presented, to apply available software, and analyze numerical results, we need some knowledge from many different mathematical disciplines, e.g.
The general mathematical model to be investigated, contains certain features to apply the presented methods for a large set of different practical applications. Some of the most important items are:
Only for illustration purposes we denote the first independent model variable the time variable of the system, the second one the concentration variable and the dependent data as measurement values of an experiment. These words describe their probably most frequent usage in a practical application. On the other hand, the terms may have any other meaning within a model depending on the underlying application problem.
Due to the practical importance of parameter estimation, very many numerical codes have been developed in the past and are distributed within software packages. However, there is no guarantee that the mathematical algorithm chosen is capable to solve the problem under investigation. Possible traps preventing a solution in the desired way, are
Thus, users of parameter estimation software are often faced with the situation that the algorithm is unable to get a satisfactory answer subject to solution parameters selected, and that one has to restart the solution cycle by changing tolerances, internal algorithmic decisions of the code or at least the starting values to get a better result. To sum up, a black box approach to solve a parameter estimation problem, does not exist and a typical life cycle of a solution process consists of stepwise redesign of solution data. In the subsequent chapters we summarize the mathematical and numerical tools that are needed to set up a feasible model, to select a suitable algorithm, to perform a parameter estimation run and to analyze results in a practical situation from the viewpoint of a user.
First we give a very compact introduction into the mathematical background in Chapter 2. Only the most important topics needed for the design of the numerical methods discussed and for interpreting results, are outlined. Any special mathematical knowledge for instance about optimization theory or the theory of ordinary differential equations is not required to understand the chapter. However, some basic ideas about calculus and linear algebra are desirable. Topics treated, are optimality criteria, sequential quadratic programming methods, nonlinear least squares optimization, ordinary differential equations, differential-algebraic equations, one-dimensional time-dependent partial differential equations, Laplace transforms, automatic differentiation and statistical analysis. Readers mainly interested in solving a parameter estimation problem and with minor interest in the mathematical background of algorithms, may skip this chapter.
The mathematical parameter estimation model, alternative phrases are data fitting or system identification, is outlined in Chapter 3. Is is shown, how the dynamical systems have to be formulated and adapted to fit into the least squares formulation required for executing an optimization algorithm. Moreover, we show that there is a large variety of different alternatives to formulate a parameter estimation model that must be considered separately because of their practical importance.
In Chapter 4, we present numerical experiments. First we show a couple of possible pitfalls to outline that even qualified numerical algorithms can fail or can compute unacceptable answers. Possible remedies are proposed to overcome numerical instabilities and deficiencies in the model formulation. Also we discuss the important question, how to check the validity of a mathematical model. Then we present numerical results of comparative test runs of some least squares codes. To analyze these questions in a quantitative way, a large number of test examples is collected, many of them with some practical background and experimental data.
A couple of case studies are included in Chapter 5 to motivate the approach and to show the practical impact of the methods investigated. The application models discussed, are for
The installation of the software system is outlined in Appendix A. After a successful installation, more organizational details on its usage can be retrieved directly from the interactive, context-sensitive help option. The user interface is developed in form of an MS-Access database, the numerical algorithms in form of stand-alone 32-bit-Fortran codes.
The availability of test examples is an important issue for implementing and testing numerical software. Thus, another appendix is included that collects some information about the test problems a unified format. The problems can be retrieved immediately from the database, and they serve also for a review on the complexity of the test examples used in Chapter 4.
Appendix C contains a brief description of the modeling language used for the implementation of the test examples. Especially all error messages are listed. A particular advantage is that derivatives can be evaluated automatically without approximation errors.
To be able to execute the test examples from own analysis software, a program
is documented in Appendix D that generates Fortran code for function and
gradient evaluation based on the backward mode of automatic differentiation.
The code generator is also attached on the CD-ROM.
|2.1.2||Convexity and Constraint Qualification|
|2.1.3||Necessary and Sufficient Optimality Criteria|
|2.2||Sequential Quadratic Programming Methods|
|2.2.1||The Quadratic Programming Subproblem|
|2.2.2||Line Search and Quasi-Newton Updates|
|2.2.4||Systems of Nonlinear Equations|
|2.3||Least Squares Methods|
|2.3.2||Gauss-Newton and Related Methods|
|2.3.3||Solution of Least Squares Problems by a SQP Method|
|2.3.4||Constrained Least Squares Optimization|
|2.4||Ordinary Differential Equations|
|2.4.1||Explicit Solution Methods|
|2.4.2||Implicit Solution Methods|
|2.4.4||Internal Numerical Differentiation|
|2.5||Differential Algebraic Equations|
|2.5.2||Index of a Differential Algebraic Equation|
|2.5.3||Index Reduction and Drift Effect|
|2.5.5||Consistent Initial Values|
|2.5.6||Implicit Solution Methods|
|2.6||Partial Differential Equations|
|2.6.1||The General Time-Dependent Model|
|2.6.2||Some Special Classes of Equations|
|2.6.3||The Method of Lines|
|2.6.4||Partial Differential Algebraic Equations|
|2.6.7||Upwind Formulae for Hyperbolic Equations|
|2.6.8||Essentially Non-Oscillatory Schemes|
|2.6.9||Systems of Hyperbolic Equations|
|2.9||Statistical Interpretation of Results|
|3.||Data Fitting Models||124|
|3.1||Explicit Model Functions|
|3.3||Steady State Equations|
|3.4||Ordinary Differential Equations|
|3.4.2||Differential Algebraic Equations|
|3.4.6||Boundary Value Problems|
|3.4.7||Variable Initial Times|
|3.5||Partial Differential Equations|
|3.5.2||Partial Differential Algebraic Equations|
|3.5.4||Coupled Ordinary Differential Algebraic Equations|
|3.5.5||Integration Areas and Transition Conditions|
|3.6||Optimal Control Problems|
|4.2.3||Badly Scaled Data and Parameters|
|4.2.4||Identifiability of Models|
|4.2.5||Errors in Experimental Data|
|4.2.7||Non-Differentiable Model Functions|
|4.2.8||Oscillating Model Functions|
|4.3||Testing the Validity of Models|
|4.3.1||Mass Balance and Steady State Analysis|
|4.4.1||Comparing Least Squares Algorithms|
|4.4.2||Overall Numerical Performance|
|5.2||Receptor-Ligand Binding Study|
|5.4||Multibody System of a Truck|
|5.5||Binary Distillation Column|
|5.7||Transdermal Application of Drugs|
|5.9||Cooling a Hot Strip Mill|
|5.10||Drying Maltodextrin in a Convection Oven|
|5.11||Fluid Dynamics of Hydro Systems|
|5.12||Horn Radiators for Satellite Communication|
|A.1||Hardware and Software Requirements||297|
|B.1||Explicit Model Functions|
|B.3||Steady State Equations|
|B.4||Ordinary Differential Equations|
|B.5||Differential Algebraic Equations|
|B.6||Partial Differential Equations|
|B.7||Partial Differential Algebraic Equations|
|C.||The PCOMP Language||349|
|D.||Generation of Fortran Code||359|
|D.1.1||Input of Explicit Model Functions|
|D.1.2||Input of Laplace Transformations|
|D.1.3||Input of Systems of Steady State Equations|
|D.1.4||Input of Ordinary Differential Equations|
|D.1.5||Input of Differential Algebraic Equations|
|D.1.6||Input of Time-Dependent Partial Differential Equations|
|D.1.7||Input of Partial Differential Algebraic Equations|
|D.2||Execution of Generated Code|
The book has 402 pages containing 493 references, 147 figures, 79 tables, and 1,000 test examples.
Kluwer Academic Publishers, Dordrecht,
Hardbound together with CD-ROM, ISBN 1-4020-1079-6,
EUR 148.00 / USD 145.00 / GBP 90.00
|Back to home page||Back to list of publications||Back to list of firstname.lastname@example.org|