4 Intertemporal Optimization
4.1 Introduction
In this chapter, we study techniques applied to dynamic optimizations. Optimization in dynamics economic problems, which are problems in which variables change over time, does not requires new principles vis-à-vis static problems but possesses a specific structure needed to take care about.
The most important part of this specific structure is the relation between stocks and flows. Some variables, which we will denote by \(y\), have the form of stocks, changing gradually over time. Other variables, which we will denote by \(x\), have the form of flows, which can change freely at any instant. Mathematically, stocks are called state variables and flows are called control variables.1
Stock variables evolves according both stocks and flows, control variables control changes in state variables. For instance, savings in period \(t\) determine the change in households wealth from period \(t\) to period \(t+1\). A general form of the volution of state variables is:
\[ y_{t+1}-y_t=Q(y_t,x_t,t ) \tag{4.1}\] where \(t,t+1,...\) are discrete time periods, \(y\) is a state variable (stocks), \(x\) is a control variable (flows) and \(Q\) is a vector function. There might be additional restrictions that we summarize under the form:
\[ G(y_t,x_t,t)\leq 0 \tag{4.2}\]
Note that restrictions (4.1) and (4.2) have different structure: the former involves directly a dynamic restriction while the latter is a static.
Furthermore, in most dynamic economic problems, agents have to optimize an objective function of the following additively-separable form:
\[ \sum_{t=0}^T F(y_t,x_t,t) \tag{4.3}\]
subject to restrictions (4.1) and (4.2). Periods start at \(t=0\) and end at \(T\), which is potentially infinite. For instance, the function \(F(y_t,x_t,t)\) can represent some households’ utility function or firms’ profit at date \(t\).2 Hence, we seek, in such examples, to maximize the sum of stream of instantaneous utilities (or profits) over time. The value of the initial stock at time 0 is taken as given but also at date T+1: such problems have therefore both initial and terminal conditions, the terminal condition being called transversality conditions.
We are going to present two widely used techniques in economics: optimal control and dynamic programming. Both are going to be applied in discrete and continuous time.
4.2 Optimal Control Methods
4.2.1 The Optimal Control Method in Discrete Time
In the optimal control problem, we want to select the variables \(y_t\) and \(x_t\) for \(t=0,1,2...,T\) to find the optimal solution of (4.3) subject to the constraints (4.1) and (4.2). To say it differently, we want to find the sequence \(x_0,x_1,x_2...x_T\) and \(y_1,y_2,...,y_T\) (remember that \(y_0\) and \(y_{T+1}\) are given) satisfying this problem.
To do so, we define multipliers (or shadow values) and construct the Lagrangian function. Define as \(\mu_t\) the multiplier for the constraints (4.2). These have the usual interpretation of shadow values for the constraints in period \(t\). The multipliers for the constraints (4.1) are different since they define the first-order change in the objective function if the constraint in the change of the stock is loosened (i.e. if we have a marginal increase in the stock variable \(y_{t+1}\)). They are therefore shadow values of the stock variables in period \(t+1\) and we denote them \(\lambda_{t+1}\).
We define as \(\mathcal{L}\) the Lagrangian function of the full intertemporal problem:
\[ \mathcal{L}= \sum_{t=0}^{T}\{F(y_t,x_t,t)+\lambda_{t+1}\left[y_t+Q(y_t,x_t,t)-y_{t+1}\right]-\mu_tG(y_t,x_t,t) \} \]
Note that because of the time additive structure of the problem, multipliers are also dated at some period.
The first-order conditions for the optimization of the Lagrangian function with respect to the control variables \(x\) are:
\[ \frac{\partial \mathcal{L}}{\partial x_t}=0 : \quad F_x(y_t,x_t,t)+\lambda_{t+1}Q_x(y_t,x_t,t)-\mu_tG_x(y_t,x_t,t)=0 \] for \(t=0,1,...,T\) and where \(F_x\), \(Q_x,\) and \(G_x\) are the partial first-order derivatives with respect to \(x\). Note again that we have \(T+1\) first-order conditions with respect to \(x\) but the only real differences between them is their date.
With respect to \(y\), the first-order conditions are a bit more complex because \(y\) appears in two consecutive periods, and therefore two terms of the sum. Let’s rearrange the Lagrangian function to see it clearly.
\[\begin{aligned} \mathcal{L}= \sum_{t=1}^{T}\{F(y_t,x_t,t)+\lambda_{t+1}Q(y_t,x_t,t)+y_t(\lambda_{t+1}-\lambda_{t})-\mu_tG(y_t,x_t,t) \}\\ +F(y_0,x_0,0)+\lambda_1Q(y_0,x_0,0)+y_0\lambda_1-y_{T+1}\lambda_{T+1} \end{aligned}\]
The final four terms in the previous equation refer to given value of \(y\) in period \(0\) and \(T+1\). The first order-condition for the optimum of \(\mathcal{L}\) for \(y_t\), \(t=,1,2...,T\) are:
\[\begin{aligned} \frac{\partial \mathcal{L}}{\partial y_t}=0 : & \quad F_y(y_t,x_t,t)+\lambda_{t+1}Q_y(y_t,x_t,t)+\lambda_{t+1}-\lambda_t-\mu_tG_y(y_t,x_t,t)=0\\ & \Rightarrow \lambda_{t+1}-\lambda_t=-\left[ F_y(y_t,x_t,t)+\lambda_{t+1}Q_y(y_t,x_t,t)-\mu_tG_y(y_t,x_t,t)\right] \end{aligned} \tag{4.4}\]
These conditions can be written in a more comprehensive and economically useful way. Define a new function, the Hamiltonian function \(\mathcal{H}\) as follows:
\[ \mathcal{H}(y_t,x_t,\lambda_{t+1},t)= F(y_t,x_t,t)+\lambda_{t+1}Q(y_t,x_t,t) \tag{4.5}\]
Equation (4.5) suggests that the control variable \(x\) must be selected to optimize \(\mathcal{H}(y_t,x_t,\lambda_{t+1},t)\) under the constraint \(G(y_t,x_t,t)\leq 0\) which can be written into a new Lagrangian:
\[ \tilde{\mathcal{L}}(y_t,x_t,\lambda_{t+1},t)= \mathcal{H}(y_t,x_t,\lambda_{t+1},t)-\mu_tG(y_t,x_t,t) \]
Then equation (4.4) can be written more simply as:
\[ \lambda_{t+1}-\lambda_t=-\tilde{\mathcal{L}}_y(y_t,x_t,\lambda_{t+1},t) \tag{4.6}\]
Finally, from (4.1) and (4.5), we get (using the Enveloppe Theorem):
\[ y_{t+1}-y_t=\tilde{\mathcal{H}}_{\lambda}(y_t,x_t,\lambda_{t+1},t)=Q(y_t,x_t,t) \tag{4.7}\] These properties of the Hamiltonian function are known as the Maximum Principle:
The necessary first-order conditions for the optimization of (4.3) under the constraints (4.1) and (4.2) are the following: (1) for each \(t\), the control variables \(x_t\) optimize the Hamiltonian function (4.5) under the static constraints (4.2). (2) The changes of \(y_t\) and \(\lambda_t\) over time are determined by the difference equations (4.6) and (4.7).
The Maximum Principle, proposed by Pontryagin et al. (1962), facilitates the determination of the first-order conditions for intertemporal optimization problems. It also gives easier interpretations of the first-order conditions of dynamic economic problems we have just seen.
In particular, changes in the decision variables \(x_t\) directly impact the objective function (4.3) but also on \(y_{t+1}\) through the impact on \(Q\). Hence, the change in the objective function is found by multiplying the impact of \(x\) on Q with the shadow value \(\lambda_{t+1}\) of \(y_{t+1}\). The Hamiltonian provides a simple way of converting the one-period objective function \(F\) to account for the future impact of the current choice of the control variable \(x\). A similar economic interpretation can be obtained to the first-order conditions for the state variable \(y\). A marginal change in \(y\) in period \(t\) gives the marginal change \(F_y-\mu G_y\) in period \(t\), given the shadow value of \(\lambda_{t+1}\) The right-hand side of (4.4) may be interpreted as a dividend. The change \(\lambda_{t+1}-\lambda_t\) is like a capital gain. Equation (4.4) tells us that he dividend plus the capital gain should be equal to zero: at the optimim, there can be no excess return from \(y\).
4.2.2 The Optimal Control Method in Continuous Time
In many applications, it is more convenient to treat time as a continuous variable. In such case, Equations (4.1) and (4.5) become:
\[ \dot{y}(t)=\frac{dy}{dt}=Q(y(t),x(t),t) \tag{4.8}\] \[ G(y(t),x(t),t)\leq 0 \tag{4.9}\]
For the objective function, Equation (4.3) in continuous time becomes:
\[ \int_0^{T}F(y(t),x(t),t)dt \tag{4.10}\]
We can use the Hamiltonian function as before:
\[ \mathcal{H}=F(y(t),x(t),t)+\lambda(t)Q(y(t),x(t),t) \tag{4.11}\]
The FOC for the optimization of the Hamiltonian function (4.11) under the static constraints (4.9) are:
\[ \frac{\partial \mathcal{H}}{\partial x(t)}-\mu(t)\frac{\partial G}{\partial x(t)}=0 \Rightarrow F_x+\lambda(t)Q_x=\mu(t)G_x \tag{4.12}\]
\[ \frac{\partial \mathcal{H}}{\partial y(t)}=-\dot\lambda(t) \Rightarrow F_y+\lambda(t)Q_y=-\dot{\lambda}(t) \tag{4.13}\]
\[ \frac{\partial \mathcal{H}}{\partial \lambda(t)}=\dot{y}(t) \Rightarrow \dot{y}(t)=Q(y(t),x(t),t) \tag{4.14}\]
These three equations are the continuous-time equivalent of the discrete time first-order conditions previously obtained.
4.3 Dynamic Programming and the Bellman Equation
Dynamic programming is an alternative method of solving the problem at the beginning of this chapter. It is an extremely useful in problems that combine time and uncertainty as ofent happens in economics.
Our problem is the optimization of:
\[ \sum_{t=0}^{T}F(y_t,x_t,t) \]
under the constraints:
\[ y_{t+1}-y_t=Q(y_t,x_t,t) \]
and
\[G(y_t,x_t,t) \leq 0\]
for \(t=0,1,2,..,T\). Again, the vectors of initial and final stocks \(y_0\) and \(y_{T+1}\) are taken as given. We can define the optimal value that comes out of this problem as a function of the initial stocks \(y_0\). Denote this function as \(V(y_0)\). The vector of the first derivatives of this function \(V_y(y_0)\) is the vector of the shadow values of these initial stocks.
The additive separability of the objective function and the constraints allow us to make an important generalization of the above ide. Instead of starting off at time zero, let us assume that we start off at time \(t=\tau\). For the decisions thaty start at \(\tau\), the only thing that matters from the past is the vector of stocks \(y_{\tau}\), which is the result of past decisions. Our problem is to optimize an objective function such as (4.3) and the associated constraints, with time starting from \(\tau\) and not 0. We define \(V(y_{\tau},\tau))\) as the optimal value that emerges as a function of stocks \(y_{\tau}\) and period \(\tau\). The vector of the first derivatives \(V_y(y_{\tau},\tau)\) denotes the marginal increase in the optimal value for a small increase of stocks in period \(\tau\), which is the vector of shadow values of the initial stocks for the optimization problem that starts in period \(\tau\). This applies at all \(t\).
Let us then select any \(t\) and exime the decision of choosing the values of the control variables for thaht period. Any choice of the control viarable \(x_t\) will lead to stocks \(y_{t+1}\) through ([-#eq-generaldynamics]). What remains is to solve the subproblem for period \(t+1\) and to find the optimal value \(V(y_{t+1},t+1)\). The total value in period \(t\) of a choice for the control variables \(x_t\) starting oof with stocks \(y_t\), can be separated into two terms: \(F(y_t,x_t,t)\) which occurs in the current period; and \(V(y_{t+1},t+1)\), which comes about in future periods. The choice of \(x_t\) must optimize the sum of these two terms under the relevant constraints. In other words, we have:
\[ V(y_t,t)= \max_{x_t}\left[F(y_t,x_t,t)+V(y_t,t+1) \right] \tag{4.15}\]
under the constraints (4.1) and (4.2) for the specific \(t\).
This method of intertemporal optimization, as a succession of static optimization problems, was proposed by Richard Bellman and is called dynamic programming. The idea that whatever the choice in period \(t\), the choices for the subproblem that begins in period \(t+1\) should be optimal, is known as Bellman’s principle of optimality. The optimal value function \(V(y_t,t)\) is called the Bellman’s value function and equation (4.16) the Bellman equation.
The Bellman equation gives us a recursive method for solving the original optimization problem. The idea is to start from the end and go backward. In period \(T\), there is no future, only the requirement for a given final stock \(Y_{T+1}\). Therefore:
\[ V(y_T,T)=\max_{x_T}F(Y_t,x_T,T) \]
under the constraints:
\[\begin{aligned} Y_{T+1}=Y_T+Q(y_T,x_T,T) \\ G(y_T,x_T,T) \leq 0 \end{aligned}\]
This is a simple probem of static optimization, which gives us the optimal value function \(V(Y_T,T)\). This function can in turn be used in the right-hand side of (4.16) for \(t=T-1\). This equation is then another static problem, which gives us the optimal value function \(V(y_{T-1},T-1)\). We can continue iun this way until we reach period \(0\). In pratice, this process provides results for the simplest problems. Analytical solutions exist when the functions \(F\), \(G\), and \(Q\) have a very simple form. Where analytical solutions do not exist, we can use numerical solutions, acknowledging that, for many economic applications, we have a better methods than the recursive method to find or characterize the solution.
Note that in the presence of uncertainty, the Bellman equation takes the form:
\[ V(y_t,t)= \max_{x_t}\left[F(y_t,x_t,t)+E_tV(y_t,t+1) \right] \tag{4.16}\] where \(E\) is the mathematical expectations operator. To find the Bellamn equations in continuous time, note that from (4.16) takes the form:
\[ V(y(t),t)= \max_{x(t)}\left[F(y(t),x(t),t)\Delta t+V(y(t+\Delta ty,t+\Delta t) \right] \]
where \(\Delta t\) is a small time interval. using a Taylor expansion of the last right-hand side term of the equation above, we get:
\[ V(y(t+\Delta ty,t+\Delta t)=V(y(t),t)+V_y(y(t),t)\left[y(t+\Delta t)-y(t)\right]+V_t(y(t),t)\Delta t \] and where: \(y(t+\Delta t)-y(t)=Q(y(t),x(t),t)\Delta t\) from (4.1). Plug this expression and the previous to obtain:
\[ V(y(t),t)= \max_{x(t)}\left[F(y(t),x(t),t)\Delta t+ V(y(t),t)+V_y(y(t),t)Q(y(t),x(t),t)\Delta t+V_t(y(t),t)\Delta t \right] \]
Divide both side by \(\Delta t\) and cancelling \(V(y(t),t)\) results in:
\[ 0= \max_{x(t)}\left[F(y(t),x(t),t)+V_y(y(t),t)Q(y(t),x(t),t)+V_t(y(t),t) \right] \] Which is a Bellman Equation in continuous time.
4.4 Present and Current Value Problems
Before going through additional details, we need to precise two fundamental points:
- Dynamic economic problems often discount future values of the objective function
- if \(T=+\infty\), there is no guarantee that the objective function (4.3) (or the continuous time version (4.10 ) is converging. Discounting the objective function is one way to guarantee such convergence.
When discounted, the objective functions write:
\[\begin{aligned} \sum_{t=0}^T \beta^t F(y_t,x_t,t), \quad \text{in discrete time} \\ \int_{t=0}^T \exp^{-\rho} F(y_t,x_t,t) \quad \text{in continuous time} \end{aligned} \tag{4.17}\]
with \(\rho>0\) the rate of time preference and \(\beta=\frac{1}{1+\rho} \in(0,1)\), the discount factor.3 This two parameters defines the degree of impatience of the economic agents, or how much they subjectively value the present versus the future. In particular, a low \(\beta\) (and a high \(\rho\) ) means that the agents is relatively impatient (she puts less weight on future value of the objective function).
From there, there are two different (although equivalent) way to frame the problem. Either build the present-value Hamiltonian or the current-value Hamiltonian. In continuous time, the present-value Hamiltonian is:4
\[ \mathcal{H}=F(y(t),x(t),t)\exp^{-\rho t}+\tilde{\lambda}(t)Q(y(t),x(t),t) \] The first-order conditions do not merely change in comparison to Section 4.2.2:
\[\begin{aligned} \frac{\partial \tilde{\mathcal{H}}}{\partial x(t)}=0 &:& \quad F_x\exp^{-\rho t}-\tilde{\lambda}(t)Q_x=0\\ \frac{\partial \tilde{\mathcal{H}}}{\partial y(t)}=-\dot{\tilde{\lambda}} &:& \quad F_y\exp^{-\rho t}-\tilde{\lambda}(t)Q_y=-\dot{\tilde{\lambda}}(t) \\ \frac{\partial \tilde{\mathcal{H}}}{\partial \lambda(t)}=\dot{y}(t)&:& \quad Q(y(t),x(t),t)=\dot{y}(t) \end{aligned} \tag{4.18}\]
Using an appropriate change of variable, the current-value Hamiltonian is:
\[ \mathcal{H}=\tilde{\mathcal{H}}\exp^{\rho t}= F(y(t),x(t),t)+\lambda (t)Q(y(t),x(t),t) \] and where \(\lambda(t)=\tilde{\lambda}(t) \exp^{-\rho t}\). Noting that \(-\dot{\tilde{\lambda}}(t)=-(\dot{\lambda}(t)-\rho\lambda (t))\). The FOC becomes:
\[\begin{aligned} \frac{\partial \mathcal{H}}{\partial x(t)}=0 & : & F_x-\lambda(t)Q_x=0 \\ \frac{\partial \mathcal{H}}{\partial y(t)}=-\dot\lambda(t)-\rho \lambda(t) \equiv -\dot{\tilde{\lambda}}\exp^{-\rho t} &:& F_y-\lambda(t)Q_y-\rho \lambda(t)=-\dot{\lambda}(t) \\ \frac{\partial \mathcal{H}}{\partial \lambda(t)}=\dot{y}(t) &:& Q(y(t),x(t),t)=\dot{y}(t) \end{aligned} \tag{4.19}\]
Note that using the two first equations in (4.18) (in particular by taking the time derivative of the first equation to eliminate the terms in \(\tilde{\lambda}\)) leads to the same expression for the second equation of (4.19). Hence, starting with the current-value Hamiltonian may save some computations.
A similar approach can be done in the discrete time problem. The present-value Lagrangian writes:
\[ \mathcal{\tilde{L}}=\sum_{t=0}^{T}\beta^{t} F(y_,x_t,t)+\tilde{\lambda}_{t+1}[Q(y_t,x_t,t)+y_t-y_{t+1}] \] with FOC:
\[\begin{aligned} \frac{\partial \mathcal{\tilde{L}}}{\partial x_t}=0 & : & \beta^t F_x+\tilde{\lambda}_{t+1}Q_x=0 \\ \frac{\partial \mathcal{\tilde{L}}}{\partial y_{t}}=0 &: & \beta^t F_y+\tilde{\lambda}_{t+1}Q_y+\tilde{\lambda}_{t+1}-\tilde{\lambda}_t=0 \\ \frac{\partial \mathcal{\tilde{L}}}{\partial \tilde{\lambda}_{t+1}}=0 &: & y_{t+1}-y_t=Q(y_t,x_t,t) \end{aligned} \tag{4.20}\]
Using the change of variable \(\tilde{\lambda}=\beta^{t}\lambda_t\), the current-value Lagrangian is now:
\[ \mathcal{\tilde{L}}=\sum_{t=0}^{T}\beta^{t} F(y_,x_t,t)+\lambda_{t+1}[Q(y_t,x_t,t)+y_t-y_{t+1}] \] with the following FOCs:
\[\begin{aligned} \frac{\partial \mathcal{\tilde{L}}}{\partial x_t}=0 & : & F_x+\lambda_{t+1}Q_x=0 \\ \frac{\partial \mathcal{\tilde{L}}}{\partial y_{t}}=0 &: & F_y+\lambda_{t+1}Q_y+\lambda_{t+1}-\beta^{-1}\lambda_t=0 \\ \frac{\partial \mathcal{\tilde{L}}}{\partial \lambda_{t+1}}=0 &: & y_{t+1}-y_t=Q(y_t,x_t,t) \end{aligned} \tag{4.21}\]
The main differences with the present-value FOCS are that the \(\beta^t\) in the first equation eliminates themselves while in the second equation, \(\beta^{t+1}\) factorizes the term in \(t+1\) and the \(\beta^t\) factorizes the term in \(t\), cancelling each other up to one factor \(\beta\). As in the continuous time case, the two formulation are equivalent once solved, although the current-value approach saves additional computations.
Let us look at discounted problem with the dynamic programming approach. Define:
\[ V(y_t,t)=\sum_{t=0}^{T}\beta^t F(y_t,x_t,t) \tag{4.22}\]
subject to:
\[ y_{t+1}-y_t=Q(y_t,x_t,t) \] Note that the constraint implies \(x_t \in H(y_t,y_{t+1},t)\). Problem (4.22) writes:5
\[\begin{aligned} V(y_t,t)= \max_{x_t \in H(y_t,y_{t+1})}\sum_{t=0}^{T}\beta^t F(y_t,x_t,t) & =\max_{x_t \in H(y_t,y_{t+1})}\left[F(y_t,x_t,t) + \beta \sum_{t=0}^{T}\beta^t F(y_{t+1},x_{t+1},t+1)\right]\\ & = \max_{x_t \in H(y_t,y_{t+1})}\left[F(y_t,x_t,t) + \beta V(y_{t+1},t+1)\right] \end{aligned} \tag{4.23}\]
To maximize the Bellman equation, write the static lagrangian:
\[ \mathcal{L}=F(y_t,x_t,t) + \beta V(y_{t+1},t+1)+\lambda\left[ Q(y_t,x_t,t)+y_t-y_{t+1}\right] \]
The first-order conditions are:
\[\begin{aligned} \frac{\partial \mathcal{L}}{\partial x_t}=0 & :& F_x+\lambda Q_x=0\\ \frac{\partial \mathcal{L}}{\partial y_{t+1}}=0 & :& -\lambda +\beta V_{y}=0 \\ \frac{\partial \mathcal{L}}{\partial \lambda}=0 & : & y_{t+1}-y_t=Q(y_t,x_t,t) \end{aligned}\]
Which gives the system to be solved recursively.
The Benveniste-Scheinkman condition gives using (4.23):
\[ \frac{\partial V(y_t,t)}{\partial y_t}=F_y+\lambda\left[Q_y+1\right] \Rightarrow \frac{\partial V(y_{t+1},t+1)}{\partial y_{t+1}}=F_y+\lambda\left[Q_y+1\right] \] which can be plugged in the FOCs above.
In continuous time, the discounted Bellman Equation is:6
\[ -\dot{V}(y(t),t)+\rho V(y(t),t)=\max_{x(t) \in H(y(t),t)}\left[F(y(t),x(t),t)+v_y(y(t),t)Q(y(t),x(t),t) \right] \]
4.5 Transversality Conditions
We have mentioned that initial conditions are usually given a dynamic economic problem, notably because this is part of history. However, we have not deal with terminal conditions, also called transversality conditions.
In the case of finite horizon problem where \(T<+\infty\), the terminal condition implies that the stock variables at period \(T+1\) are nil. This means that the Lagrangian has additional constraints of the form:
\[ \omega_{T+1}y_{T+1} \] whose associated FOCs are:7
\[\begin{aligned} \frac{\mathcal{L}}{\partial y_{T+1}}=0 : -\beta^{T}\lambda_{T+1}+\omega_{t+1}\beta^{T+1}=0 \end{aligned}\] Together with a complementary slackness condition:
\[\begin{aligned} \frac{\mathcal{L}}{\partial \beta^{T+1}\omega_{T+1}}=y_{t+1}\geq 0 \\ \beta^{T+1}\omega_{T+1}\frac{\mathcal{L}}{\partial \beta^{T+1}\omega_{T+1}}=\beta^{T}\lambda_{T+1}y_{t+1}=0 \end{aligned}\]
Since \(\lambda_{T+1}\) is not nil (because this would imply an infinite marginal benefits in the objective function), this means that \(y_{t+1}=0\).8
In continuous time, the equivalent condition at point in time \(T\) is:
\[ \lambda(T)y(T)=0 \] But what happens when we are looking at an infinite horizon problem ? The latter saves a lot of complication (basically, everything is equivalent as a two-period problem with “today” and “tomorrow” carried over and over) and is often assumed in dynamic problems. Then, we need to prevent the agents to shift indefinitely its decisions on the stocks \(y\) over and over.Hence, the equivalent transversality condition in discrete and continuous time in infinite horizon problems are:
\[\begin{aligned} \lim_{t\rightarrow +\infty}\lambda_ty_t=0 & \quad \text{in discrete time} \\ \lim_{t\rightarrow +\infty}\lambda(t)y(t)=0 & \quad \text{in continuous time} \\ \end{aligned}\]
This is an abusive interpretation as Kamihigashi (2008) highlights. Transversality conditions and no-Ponzi conditions, which is what we just described, have different roles, in particular transversality conditions are necessary in any infinite horizon problem. Be sure to check that there are satisfied (In the next chapters, they are.)
4.6 General Procedures
To summarize and give a “cookbook” approach, we present a general strategy to solve dynamic optimization problem.
For the sequence problem:
- Write the Lagrangian/Hamiltonian
- Find the first-order conditions
- Obtain the difference/differentials equations in control and state variables
- Use the techniques in Chapter 2 or in Chapter 3 to characterize the solution.
For the recursive approach, using the Bellman equation, things may differ as you can return to a sequence problem using the Benveniste-Scheinkman (and thus start at the third bullet-point of the previous list after defining the Bellman equation and find the first-order conditions). Otherwise You may also use a “guess and verify approach”, either analytically or numerically, of the value function or the policy function to obtain the solution.
Although we do not discuss second-order conditions in this chapter, one needs to check if the Lagrangian/Hamiltonian or the Bellman equation is concave (respectively convex) in order to maximize (respectively minimize) the original problem.
4.7 Applications
To be completed( you can refer to the next part though). One can check Dixit’s manual on optimization.
Not all state variables are stock variable. For instance, past decisions on controls can also be considered as state.↩︎
In such problems, we may see \(x_t\) and \(y_t\) as respectively the consumption level and the wealth owned at date \(t\) in the case of a household, or for the problem of a firm, \(x_t\) as the investment decision and \(y_t\) as the capital stock installed. ↩︎
These two parameters are related as follows: \(\beta^t=e^{t \ln \beta}=e^{\left(\ln \frac{1}{1+\rho}\right) t}=e^{[\ln 1-\ln (1+\rho)] t}=e^{[0-\ln (1+\rho)] t}=e^{-\rho t}\)↩︎
Without loss of generality, we exclude the static constraint \(G(y(t),x(t),t)\).↩︎
To obtain the different equalities, just decompose the sum with the terms in period \(t\) and the remaining terms and recognizing that: \(\sum_{t=1}^{T}\beta^t F(y_{t},x_{t},t)=\sum_{t=0}^{T}\beta^{t+1} F(y_{t+1},x_{t+1},t+1)=\beta \sum_{t=1}^{T}\beta^t F(y_{t},x_{t},t)=\beta V(y_{t+1},t+1)\).↩︎
The sketch of the proof being more complex here, it is left as an exercise.↩︎
remember that \(y_{T+1}\) comes also in the dynamic constraint involving \(Q\).↩︎
For instance, in the example of a household with wealth \(y\) and consumption level \(x\), this means that at the final period \(T\), the household consume all its wealth since it has no use for the periods after.↩︎