Science 480 Research Methods in Science

Study Guide :: Unit 3

Mathematical Modelling

Unit Outcomes

After completing this unit, you should be able to

• explain the relevance on mathematical modelling in science.
• identify the different types of mathematical models.
• understand the importance of functions in the development of mathematical models.
• understand the description and prediction aspects of a mathematical model.
• derive basic population discrete and continuous models.
• solve the linear and logistic discrete models.
• fit the linear and logistic discrete models to a set of experimental data using Excel.
• understand the derivation of the linear and logistic continuous models.
• use the method of separation of variables to find the explicit solutions for the linear and logistic differential equations.
• carry out a nonlinear fitting of the solutions of the linear and logistic differential equations to a set of experimental data using Excel.
• summarize a scientific article that involves a mathematical model.

Introduction to Mathematical Modelling

Unit 3 introduces mathematical modelling, with particular emphasis on simple deterministic models derived with the use of difference equations and differential equations. Thus, the mathematical background needed for this unit is simple Algebra and basic knowledge of Calculus.

The concept of mathematical modelling becomes evident from both the meaning of the words “modelling” and “mathematical.” On the one hand, the word “modelling” invokes or brings about the idea of representing our surroundings, our physical world, or aspects of it. On the other hand, the word “mathematical” is the adjective that describes how we are going to carry out this representation, namely, using mathematics! One could, of course, represent a certain aspect of our physical world with tools other than mathematical ones, for example, diagrams or even artistic tools such as drawings or paintings.

At this point, two questions naturally arise: Why do we need to model aspects of our physical world? What is the advantage of using mathematics over other tools for modelling?

There are two main reasons to model our physical world. First, we want to understand its nature and describe it; and second, we want to be able to make predictions of a particular situation being modelled. Thus, description and prediction are two very important aspects in modelling. Now, if we want either the description or the prediction to have both qualitative and quantitative foundations, we need numbers. In other words, we need mathematics.

We begin with some basic concepts or ingredients of mathematical modelling and then introduce the three main tools used to formulate mathematical models: linear regression, difference equations (iterative maps), and differential equations.

3.1: Basic mathematical ingredients for modelling

The fundamental mathematical object needed in the mathematical modelling techniques introduced in this unit is simply a function. A function \[f:A\to B\]

is a map that takes an element "$a$" of the domain set $A$ to an element $f\left( a \right)$ in the codomain set $B$. For the purpose of this unit, we will be working solely with the set of real numbers $\mathbb{R}$ as the domain and the codomain sets. In other words, we will work with real functions \[f:\mathbb{R}\to \mathbb{R}\text{.}\]

Thus, to refresh our memory of this concept of a function, the textbook introduces the very basic mathematical concepts of numbers, mathematical operations, and Algebra, together with a set of common functions one encounters in a first Calculus course.

Reading

Study pages 112‑121 (to “Estimations”) of the textbook. These pages review the necessary concepts of numbers, operations, and mathematical labeling to introduce the very basic mathematical object of a function. Some of the most common functions are introduced in these pages, namely linear functions, power functions, polynomials, exponentials, logarithms, and trigonometric functions.

3.2: Linear regression

Linear regression is perhaps one of the simplest approaches to model the relationship between a dependent variable and independent variables. In the case of one independent variable (the scope of this section), we will call the approach simple linear regression, or linear regression, from now on.

In linear regression, we model the relationship between a dependent variable  and an independent variable  with a linear function \[y = a + bx\text{,}\] (3.2.1) where $a$ denotes the $y$‑intercept and $b$ denotes the slope of the linear function. When carrying out a linear regression, we work with a data set of pair points $({{x}_{i}}\text{,}\ {{y}_{i}})$ for $i=0\ldots n\text{.}$ The objective is then to fit the linear function (3.2.1) to the data. In other words we would like to estimate the values of $a$ and $b$ in Equation 3.2.1 that provide the best fit of the linear function to the data. In order to find the best fitting of the straight line to the data we will use the method of least squares, which is explained in detail in the textbook (pp. 126‑127). The method also provides a correlation coefficient, a number that measures of how well the data follows the linear relation.

Figure 3.2.1.Scattered plot of the data presented in Table 3.2.1.

To illustrate how the linear regression works consider, for example, the data provided in Table 3.2.1 at the end of this section, which provides year by year, from 1960 to 2011, the population (number of individuals) and the carbon dioxide (CO₂) emission in Canada, measured in kilotons $(kt)$. The idea is, given the experimental data, to find out if there is a linear relation between the population and the CO₂ emission in the country. For this purpose, we use linear regression with some of the basic tools provided by Excel.

Given this data, the main question motivating a linear regression can be posed in different ways. For example, does the carbon emission in Canada depend linearly on its total population? Or what is the linear correlation between the carbon emission in Canada and its total population? Or what are the best parameter estimates for $a$ and $b$ when fitting the linear model $y = a + bx$ to the given data, where $x$ denotes the total population in Canada in a given year and $y$ denotes the carbon emission in the country during that year?

To answer this question we first would like to have a first graphical idea of a possible linear relation between the data by plotting it in a scattered plot (Figure 3.2.1). From this plot, we notice that the greater the population the greater the carbon emission. Moreover, it seems that this increase of carbon emission due to an increase of the population could be linear. Thus, the next step is to fit a straight line $y = a + bx$ to the data. The mathematical procedure to carry out this fitting, called the method of least squares, is explained in detail in the textbook (pp. 126‑128) and implemented in Excel in a user friendly fashion just by simply adding a “Linear Trendline” to the data in the “Layout” option. The result of doing this linear fitting on the scattered plot in Figure 3.2.1 using Excel is illustrated in Figure 3.2.2. Notice in the plot that Excel also offers the option (under “More Trendline Options” of the “Linear Trendline”) to display the linear equation $y = a + bx$, with the estimated values for $a$ and $b$ together with the value of the residual squared $({{R}^{2}})$. The residual squared provides a measure of how high the values of $x$ and $y$ are linearly correlated. The closer it is to 1, the higher the correlation; the closer it is to zero, the lower the correlation. Thus, as indicated by the value of ${{R}^{2}}$ in Figure 3.2.2, there is a high linear correlation between the CO₂ emission and the size of the population in Canada between the years 1960 and 2011.

Figure 3.2.2. Linear fit to the data presented in Table 3.2.1.

Textbook

Study pages 125‑128 (from “Linear Regression” to “Error estimates for slope and intercept”). This reading provides all the necessary details of how the method of least squares is used to fit a given set of data with a straight line. In other words, it introduces the reader to the details on linear regressions needed for this course.

Table 3.2.1. Total population and CO₂ emissions in Canada (1969‑2011). Data taken from https://data.worldbank.org/

3.3: Difference Equations

(iterative maps)

In this section, we introduce the concept of difference equations, also called iterative maps. Because these models usually involve time as the independent and discrete variable, difference equations are also called discrete‑time models.

The idea in a discrete model is to describe the value of a variable at a future time $n+1$ given the present value of the variable at time $t$ and the change it undergoes at time $n\text{.}$ In other words, \[{{X}_{n+1}}={{X}_{n}}+\Delta {{X}_{n}}\text{,}\] (3.3.1) where ${{X}_{n}}$ is the value of the dependent variable at time n and $\Delta {{X}_{n}}$ is the change of the variable at time $n\text{.}$

To obtain a good start on how the dynamics of a discrete model can evolve, first do the textbook reading. Next, go through the worked example at the end of this section, which illustrates how basic discrete models can be applied to real data in the context of population dynamics.

Reading

Study pages 136‑140 (to “Differential Equations”) of the textbook. This reading introduces, in the context of population dynamics, the two main discrete models that are the focus of study in this section, namely the linear discrete model and the logistic discrete model.

The Discrete Linear and Logistic Models: An application

Having now the basic knowledge on iterative maps, we are ready to apply both the discrete linear and logistic equations to some real data. Specifically, consider the population data presented in Table 3.3.1 and plotted in Figure 3.3.1, which describes the urban population of Virgin Islands from the year 1960 to the year 1974.

The main objective in the discrete modeling process is to find an appropriate $\Delta {{X}_{n}}$ in Equation 3.3.2. We can gain a first intuitive idea of this change by looking at the plot in Figure 3.3.1. In particular, note that the population grows proportional to its size. In other words, the population seems to be described by the linear discrete model \[{{X}_{n+1}}={{X}_{n}}+r{{X}_{n}}\text{,}\] (3.3.2) where denotes ${{X}_{n}}$ the population at year n and r denotes the yearly rate of change of the population. This hypothesis tells us that the yearly change in the population is proportional to its size. In other words\[\Delta {{X}_{n}}={{X}_{n+1}}-{{X}_{n}}=r{{X}_{n}}\text{.}\] (3.3.3)

Table 3.3.1. Total urban population of Virgin Islands (USA) from 1960 to 1974. Data taken from https://data.worldbank.org/

Figure 3.3.1. Urban population growth of Virgin Islands (USA) from 1960 to 1974.

The goal at this point is to test our hypothesis, which is equivalent to finding a good estimate for the rate of change r. For this purpose, we plot $\Delta {{X}_{n}}={{X}_{n+1}}-{{X}_{n}}$ versus  $r{{X}_{n}}$ (Figure 3.3.2) and verify if they follow a linear relation. At this point, the importance of the previous section on Linear Regression becomes apparent. Notice from Figure 3.3.2 that the variables under investigation follow a linear relation and a straight line can be nicely fit to the data with a residual square ${{R}^{2}}=0.8092$ and a slope $r=0.1013.$ The value of the residual square confirms our hypothesis and the slope of the straight line provides a good estimate for the rate of change, $r\text{,}$ of the population. This rate tells us that the population grows approximately 10.1% yearly.

Finally, we can use the estimated value of the rate of change to compare the behaviour of the discrete model given by Equation 3.3.2 with the observed data for the urban population of Virgin Islands. This is shown in Figure 3.3.2. Notice that the simulated data coming from the linear discrete model is very close to the real data!

Note that we have successfully provided a model that offers a good description of the behaviour of the urban population in Virgin Islands from 1960 to 1974. This is clearly an example of how a model can be used to “describe” a particular behaviour in nature.

Figure 3.3.2 Observed and simulated urban population growth data in Virgin Islands (USA) from 1960 to 1974.

Figure 3.3.3 Simulated urban population growth data in Virgin Islands (USA) from 1960 to 2000.

One can go even further with this model and use it for the purpose of “prediction.” For example, let us place ourselves in the year 1974 and ask ourselves: On the basis of the data of the urban population in Virgin Islands, can we predict what the population will be in the year 2000? To answer this question we simply let our model run with simulated data up to the year 2000, as shown in Figure 3.3.3. The model predicts that the population size in the year 2000 is in the order of 800,000 people. This is a completely valid prediction based on the linear discrete model in Equation 3.3.2.

However, this prediction might or might not turn out to be true. In fact, if we now reveal the actual urban population size in Virgin Islands the years following 1974 up to the year 2014 (Table 3.3.2), we can see in Figure 3.3.4 that the predicted data is not in agreement at all with the actual data.

This shows that even though a model can describe the past and present behaviour of a phenomenon, it could fail to predict the future. There can be several reasons for this failure. One of them could be simply that there was not enough data available to clearly see a long term behaviour or pattern. As you can see, we were missing data in the years following 1974, which obviously shows a different pattern in the urban population growth of Virgin Islands. Another reason for the failure in the prediction could be that the model is not considering aspects of the system being modelled which are essential for its dynamics. For example, if we take a closer look at the actual population growth in Virgin Islands from 1960 to the year 2014 (Figure 3.3.4), we notice that the population starts to level off around year 1995, which shows that the population might be reaching itscarrying capacity. When we think of the place where the population is growing, namely an island, this behaviour makes sense. In an island, resources and space seem to be more limited than in inland territory. Clearly, we did not consider a carrying capacity when coming up with the linear model given by Equation 3.3.2.

Table 3.3.2. Total urban population of Virgin Islands (USA) from 1975 to 2014. Data taken from https://data.worldbank.org/

Figure 3.3.4 Observed and simulated urban population growth data in Virgin Islands (USA) from 1960 to 2014.

Thus, with this insight of a carrying capacity in mind together with the full set of data from 1960 to 2014, we can reformulate our discrete model. In particular, notice that the growth rate, $r\text{,}$ of the population given by the linear equation (3.3.2) seems to be constant up to 1974 but starts to decrease to zero as time passes. Therefore, as stated on p. 137 of the textbook when introducing the logistic discrete model, we could reformulate our model by replacing the growth rate $r$ in Equation 3.3.2 with \[r\to r\times (1-{{{X}_{n}}}/{{{X}_{\max }}}\;)\text{,}\] (3.3.4) where ${{X}_{max}}$ denotes the carrying capacity of the population. In other words, our new model to describe the population growth in the Virgin Islands is given by \[{{X}_{n+1}}={{X}_{n}}+r\times (1-{{{X}_{n}}}/{{{X}_{\max }}}\;){{X}_{n}}\text{.}\] (3.3.5) This model, called the logistic discrete model, assumes that the that the yearly population growth $\Delta {{X}_{n}}$ in Equation 3.3.1 is given by \[\Delta {{X}_{n}}=r\times (1-{{{X}_{n}}}/{{{X}_{\max }}}\;){{X}_{n}}\text{.}\] (3.3.6)

Testing this assumption will let us know how well the logistic discrete equation (3.3.5) describes/fits our data. Before carrying out a linear regression to confirm a linear relationship between $\Delta {{X}_{n}}$ and $(1-{{{X}_{n}}}/{{{X}_{\max }}}\;){{X}_{n}}\text{,}$ we first provide an empirical estimate for ${{X}_{max}}$. We notice that from approximately 1995 the population starts to level off. Thus, by the taking the average of the population from 1995 onwards we obtain an empirical value for ${{X}_{\max }}=100,119.$ Taking this value, we plot $\Delta {{X}_{n}}={{X}_{n+1}}-{{X}_{n}}$ versus $(1-{{X}_{n}}/{{X}_{\max }}){{X}_{n}}$ and verify if they follow a linear relation. Notice from Figure 3.3.5 that the variables under investigation can be assumed to follow a linear relation with a residual square ${{R}^{2}}=0.6165$ and a slope $r=0.15.$

Figure 3.3.5 Testing the discrete logistic equation (3.3.5) to the data.

Taking the estimated value for $r\text{,}$ we finish building our discrete logistic growth model, given by \[{{X}_{n+1}}={{X}_{n}}+0.15\times (1-{{{X}_{n}}}/{100,119}\;){{X}_{n}}\text{.}\] (3.3.7)

Finally, to visualize how the discrete logistic model given by Equation 3.3.7 fits the observed data for the urban population size in Virgin Islands we run the model with an initial value corresponding to the year 1960 and plot it together with the observed population data in Figure 3.3.6. Note that the discrete logistic model offers a nice fit to the observed population data.

Figure 3.3.6 A comparison of the observed urban population data in Virgin Islands with simulated data obtained from the discrete logistic equation (3.3.7).

To finish this section, it is worthwhile to mention that the discrete logistic equation (3.3.5) does not exhibit a logistic growth—namely an initial exponential growth followed by a leveling off to a carrying capacity—for all values of the parameter $r$. In fact, Equation 3.3.5 can exhibit chaos. The study of this type of dynamics and the corresponding mathematical theory behind it is beyond the scope of this course.

3.4: Differential Equations

In the previous section we modeled the dynamics of a population growth over time using difference equations or discrete models. Note that the evolution in time was discrete; for example, yearly as in the case of the urban population growth in Virgin Islands. Another way of modelling a population growth can be considering a continuous time, in which case we need differential equations. Here, some Calculus background will come in handy! If one thinks of time as a discrete variable with a time step from one time to the next, then we can make it infinitely small (i.e., infinitesimal) so that we can obtain a continuous time. In other words, a discrete model in time where the time step $\Delta t$ goes to zero will result in a model continuous in time, which is called a differential equation!

In this section, we introduce differential equations in the context of our example of population growth. However, it is advisable that you go through the textbook pages identified below.

Reading

Study pages 140‑146 (to “Assignments”). This reading introduces the concept of differential equations and will help you to better understand the following application.

The Continuous Linear Model: An application

We introduce the concept of a differential equation by illustrating with an example how a differential equation results in a modelling process. For example, let’s suppose we want to model a simple population growth where there are no limitations of resources and space. In other words, we can assume that a population $P(t)$ grows over time $t$ proportionally to its size with a growth rate $r$, which has units $[{1}/{\text{time}]}\;\text{.}$ Note that this dynamics has the same basic linear principle as the linear discrete model described by Equation 3.3.2. Thus, following the same modelling idea, if we denote the population size at time $t$ as $P(t)$ then the size of the population at time $t+\Delta t\text{,}$ where $\Delta t$ denotes a time step or increment is given by the size of the population at time $t$ plus the change of the population during the time step $\Delta t$: \[P(t+\Delta t)=P(t)+r\Delta tP(t)\text{.}\] (3.4.1)

We can rewrite Equation 3.4.1 as \[\frac{P(t+\Delta t)-P(t)}{\Delta t}=rP(t)\text{.}\] (3.4.2)

Now, if we make the time step infinitely small, i.e. if we take the limit $\Delta t\to 0$ in Equation 3.4.2, we obtain that \[\lim \Delta t\to 0\frac{P(t+\Delta t)-P(t)}{\Delta t}=rP(t).\] (3.4.3)

Noticing that the left hand side of Equation 3.4.3 is the derivative of $P(t)$ with respect to $t$, we rewrite the Equation as \[\frac{dP}{dt}=rP(t)\text{.}\] (3.4.4)

Equation 3.3.4 is a differential equation that describes the dynamics of the population growth. It is called differential equation because is an equation that involves an unknown function (in this case, $P(t)$), and some of its derivatives (in this case ${dP}/{dt}\;$).

As one can intuitively see the linear differential equation (3.4.4) is the continuous analogue of the discrete linear equation (3.3.2). One main difference is that the discrete equation (3.3.2) already provides an expression for the population size over time whereas the differential equation needs to be solved to find an expression for $P(t)$. Thus, our main task now is to solve the differential equation (3.4.4). One way of finding the solution is simply by inspection. The solution function $P(t)$ must satisfy that its derivative equals the function itself times a constant $r$ (see Equation 3.4.4). Thus, we think of a solution of the form \[P(t)=K{{e}^{rt}},\] (3.4.5) where $K$ is an arbitrary constant. Note that the function $P(t)$ satisfies Equation 3.4.4. By taking the derivate of $P(t)$, we obtain \[\frac{dP}{dt}=rK{{e}^{rt}}=rP(t)\text{.}\] (3.4.6)

Notice that the solution (3.4.5) of the linear differential equation (3.4.6) is nonlinear (exponential).

Another way of solving Equation 3.4.4 is first to use Leibniz notation to rewrite the equation as \[\frac{dP}{P}=rdt\text{.}\] (3.4.7)

This technique is called separation of variables since we grouped the terms with the dependent variable $P$ on one side and the terms with the independent variable $t$ on the other side.

Noting that $\int{\tfrac{dP}{P}}=\ln (P)+{{C}_{1}}\text{,}$ where ${{C}_{1}}$ is an arbitrary constant, and  $\int{rdt}=rt+{{C}_{2}}\text{,}$ where ${{C}_{2}}$ is also an arbitrary constant, we then integrate both sides (the left hand side with respect to $P$ and the right hand side with respect to $t$) of Equation (3.4.7) to obtain \[\ln (P)+{{C}_{1}}=rt+{{C}_{2}}\text{.}\]

Now, taking the exponential on both sides, the last expression becomes \[{{e}^{\ln (P)}}{{e}^{{{C}_{1}}}}={{e}^{rt}}{{e}^{{{C}_{2}}}}\text{,}\] which can be rewritten as \[{{K}_{1}}P={{K}_{2}}{{e}^{rt}}\text{,}\] or simply (introducing and arbitrary constant $K={{K}_{2}}/{{K}_{1}}$) as \[P(t)=K{{e}^{rt}}\text{,}\] which is the same expression as the solution given by Equation 3.4.5 obtained by inspection. Thus, the linear differential equation (3.4.4) describes a population that is growing exponentially in time.

A natural question that arises now is can we use the linear differential equation (3.4.4) and its solution given by Equation 3.4.5 to describe and quantify the observed growth of a population? If a population is observed to be growing exponentially the answer is yes! For example, if we go back to the urban population size in Virgin Islands from 1960 to 1974 (Figure 3.3.1) we clearly see that the population is growing exponentially. How can we then fit the solution (3.4.5) of our linear differential equation to the observed data? In other words, can we find the values of $K$ and $r$ such that we obtain a good fit of the solution (3.4.5) to the observed data?

We will see that given our data we only need to estimate the growth rate $r$. For example, if our data started at $t=0$, then the value of $K$ will be equal to the population at time $t=0$, i.e. $P(0)\text{,}$ and equation (3.4.5) will become \[P(t)=P(0){{e}^{rt}}\text{.}\] (3.4.8)

This is not the case for our set of data since it starts at $t=1960.$ However, we can also find an expression for the exponential growth that only involves the parameter $r$. In particular, if we evaluate equation (3.4.5) at $t=1960\text{,}$ we obtain that \[P(1960)=18073=K{{e}^{rt}}\text{,}\] and therefore, \[K=18073{{e}^{-rt}}\text{.}\]

By substituting this expression for $K$ in equation (3.4.5), we obtain that \[P(t)=18073{{e}^{r(t-1960)}}\text{,}\] (3.4.9) which is the expression for the exponential growth we want to fit to the data in order to estimate the growth rate $r$. To carry out the nonlinear fitting of expression (3.4.9) using Excel one can follow, step by step, the procedure in this YouTube video on fitting nonlinear models to a given set of data.

After watching this video, one should be able to use solver function in Excel for our particular case of fitting the exponential function (3.4.9) to the population growth in Virgin Islands from 1960 until 1974 (Table 3.3.1), and obtain an estimate $r=0.092$ for the growth rate and a Excel spread sheet as in Figure 3.4.1.

Figure 3.4.1  Sample Excel spreadsheet obtained from fitting the exponential growth to the observed urban population data in Virgin Islands from 1960 until 1974 using the solver function in Excel.

As we would have expected the estimated value 0.092 for the growth rate $r$ is very close to the value of the growth rate obtained from fitting the linear discrete model (3.3.2) to the data, $r=0.1013.$

We already know that the exponential growth will not describe the dynamics of the population in Virgin Islands up to the year 2014. Thus, the natural question arises, is there a continuous version of the logistic equation that we could use to fit the data? The answer is yes! We will study the continuous logistic model applied to the population growth data in Virgin Islands in the assignment for this unit.

Assignment 3

Complete and submit Assignment 3.