Exponential turnpike property for particle systems and mean-field limit

Abstract

This work is concerned with the exponential turnpike property for optimal control problems of particle systems and their mean-field limit. Under the assumption of the strict dissipativity of the cost function, exponential estimates for both optimal states and optimal control are proven. Moreover, we show that all the results for particle systems can be preserved under the limit in the case of infinitely many particles.

1. Introduction

For optimal control problems of time-dependent differential equations, the exponential turnpike property states that the optimal solution remains (exponentially) close to a reference solution. Usually, this reference solution is taken as the optimal solution to the corresponding static problem. The concept of turnpike was first introduced for discrete-time optimal control problems [15, 31]. Since then, many turnpike results have been established, and there has been recent interest in the mathematical community [14, 21–25, 29, 34, 35].

In the present work, we focus on the exponential turnpike phenomenon for optimal control problems of a class of interacting particle systems and their mean-field limit equations. Important applications for these systems occur in the fields of swarm robotics [13], crowd dynamics [3], traffic management [32] or opinion dynamics [5], to name but a few.

The original formulation of the interacting particle system is usually at the so-called microscopic level and given by a coupled system of Ordinary differential equations (ODEs). Alternatively, one can also focus on the collective behaviour by considering the probability density distribution of the particles and investigating the corresponding McKean–Vlasov or mean-field equation (see, e.g. [1, 8, 9] for results involving control actions). The control of large-scale interacting particle systems has gained recent interest (see, e.g. [5, 10, 12]). The control of high-dimensional system is challenging, and current approaches resort to, for example, using Riccati-based [2, 28], moment-driven control [4] or model predictive control approaches [5, 6, 33]. Motivated by this, we aim to utilise the turnpike property to control those high-dimensional systems [7, 30, 37]. More precisely, we prove the exponential turnpike estimate for ODE systems with an arbitrary particle number and show that the property also holds in the mean-field limit. Here, we utilise the particular structure of interacting particle systems to derive the turnpike property.

The topic of turnpike property for mean-field optimal control problems has been studied recently in [27]. At this point, we would compare [27] with the present paper and point out our main contributions. (1) In [27], the authors prove the turnpike property with interior decay [26], which is a time integral property [18]. In the present paper, under similar assumptions (with a minor modification), we present a point-wise exponential estimate, which is more quantitative. (2) In addition to the estimate of the optimal solution, we also prove the exponential decay for the optimal control.

As in [27], our basic assumption is that the optimal control problems satisfy a strict dissipativity inequality. By considering a feedback control, we obtain the cheap control inequality. Then, we use this inequality iteratively to prove the exponential estimate for the optimal solution. This iteration technique has also been used to prove the turnpike property for other optimal control problems (see, e.g. [16]). Note that all the estimates for particle systems are independent of the particle number $N$ . Thus, all results are also expected in the mean-field level as $N\rightarrow \infty$ . By using convergence in the Wasserstein distance and the lower semi-continuity of the cost function, we prove the corresponding exponential decay property for the solution of the mean-field optimal control problem. In order to establish the exponential decay for the optimal control, we design a specific feedback control (see also [17]). In this way, the optimal control can be bounded by the optimal solution. Combining with the estimate for solutions, we also prove the exponential decay property for the optimal control with respect to time $t$ .

The paper is organised as follows. In Section 2, we state the problem and present some basic assumptions. In Section 3, we prove the cheap control property for the optimal control problem of the particle system. By considering the limit $N\rightarrow \infty$ , we prove the same property in the mean-field level. Based on these results, we prove the exponential turnpike property for both the particle system and the mean-field problem in Section 4. At last, the auxiliary estimate in the Wasserstein distance is given in Appendix A. The main results are Theorem4.3 on the exponential turnpike property for the particle system and Theorems4.4–4.5 for the mean-field problem.

2. Preliminaries

Consider the optimal control problem $\mathcal {Q}(0,T,\mu _0)$ :

(2.1) \begin{align} \mathcal {V}\!\left(0,T,\mu _0 \right) =&\,\min _{u\in \mathcal {F}} \int _{0}^T \int L(x)d\mu (t,x) dt + \int _{0}^T \int \Psi (u(x,t))d\mu (t,x) dt \nonumber \\[3pt] \,:\!=\,&\,\min _{u\in \mathcal {F}} \int _{0}^T f\!\left(\mu (t,x),u(t,x)\right) dt. \end{align}

Here, $\mu (t,\cdot )\in P_2\!\left(\mathbb {R}^d \right)$ is a probability measure on $\mathbb {R}^d$ defined for $t\in [0, T]$ , and it satisfies the following equation in a distributional sense:

(2.2) \begin{align} & \partial _t\mu + \nabla _x\cdot \Big ((P\ast \mu + u)\mu \Big ) =0,\qquad 0\lt t\lt T,\quad x\in \mathbb {R}^d, \nonumber\\ & \mu (0,x) = \mu _0(x). \end{align}

Here, $P(x)\in \mathbb {R}^d$ is a vector-valued function and

\begin{align*} (P \ast \mu )(x,t) = \int _{\mathbb {R}^d} P(x-y)d\mu (t,y). \end{align*}

As that in [20], we take the control $u(t,x)\in \mathcal {F}$ satisfying

Definition 2.1. Fix a control bound $0\lt C_B\lt \infty$ . Then $u(t,x)\in \mathcal {F}$ if and only if

1. (i) $u\,:\,[0, T] \times \mathbb {R}^d \rightarrow \mathbb {R}^d$ is a Carathéodory function.

2. (ii) $u(t,\cdot )\in W^{1,\infty }_{loc}(\mathbb {R}^d)$ for almost every $t \in [0, T]$ .

3. (iii) $|u(t,0)| + Lip(u(t,\cdot ),\mathbb {R}^d) \leq C_B$ for almost every $t \in [0, T]$ . Here, $Lip(u(t,\cdot ),\mathbb {R}^d)$ is the Lipschitz constant for $u(t,\cdot )$ such that $|u(t,x)-u(t,y)|\leq Lip(u(t,\cdot ),\mathbb {R}^d) |x-y|$ for all $x,y\in \mathbb {R}^d$ .

Remark 2.1. In [20], the control bound can be chosen as an integrable function $l(t)\in L^q(0,T)$ for $1\leq q \lt \infty$ . For simplicity, we take the bound to be constant.

Next, we show assumptions for the optimal control problem (2.2).

Assumption 2.1. The cost function $f$ satisfies the following assumptions:

1. (i) Strict dissipativity: there exists a constant $C_D$ such that for any $b\geq a\geq 0$ and any pairs $(\mu (t,x),u(t,x))\in P_2(\mathbb {R}^d)\times \mathcal {F}$ , the following inequality holds

   \begin{align*} \int _a^b f(\mu (t,x),u(t,x)) dt \geq C_D \int _a^b \int _{\mathbb {R}^d}\Big (|x-\bar {x}|^2 +|u(t,x)|^2 \Big ) d\mu (t,x)dt . \end{align*}

2. (ii) There exists a constant $C_{L}$ such that $L(x)\leq C_L |x-\bar {x}|^2$ for all $x\in B(\bar {x}, R) \,:\!=\, \{x \in \mathbb {R}^d \,:\, |x-\bar {x}| \lt R\}$ . Moreover, there exists a constant $C_{\Psi }$ such that $\Psi (u)\leq C_{\Psi }|u|^2$ for all $u\in B(0,R)= \{u \in \mathbb {R}^d \,:\, |u| \lt R\}$ .

3. (iii) The interaction function $P(x)$ satisfies $P(0)=0$ and the following Lipschitz property:

   (2.3) \begin{equation} |P(x)-P(y)|\leq C_p|x-y|,\qquad \forall \,x\in \mathbb {R}^d \end{equation}
   with $C_P\gt 0$ a constant.

Remark 2.2. These assumptions are also used in [27] except for condition (ii). Here, we need to assume that both $\Psi$ and $L$ can be bounded by quadratic functions. Note that this assumption is also satisfied for the example of [27].

For further discussion of the optimal control problem, we consider the empirical measure on $[0,T] \times \mathbb {R}^d$ :

(2.4) \begin{equation} \mu _{N}(t,x) = \frac {1}{N}\sum _{i=1}^N \delta \left (x-x_i(t)\right ). \end{equation}

Here, $x_i(t)$ $(i=1,2,\ldots, N)$ is the solution to the optimal control problem $\mathcal {Q}_N(0,T,x_0)$ :

(2.5) \begin{align} \mathcal {V}_N(0,T,x_0) & = \,\min _{u_N\in \mathcal {F}} \frac {1}{N}\sum _{i=1}^N\int _{0}^T L(x_i(t)) + \Psi (u_N(t,x_i(t))) dt, \nonumber \\ \frac {dx_i(t)}{dt} & = \frac {1}{N}\sum _{j=1}^NP(x_i(t)-x_j(t)) + u_N(t,x_i(t)), \nonumber \\ x_i(0) & = x_{i0}. \end{align}

Here, $x(t)=(x_1(t),x_2(t),\ldots, x_N(t))$ represents $N$ particles, $x_0=(x_{10},x_{20},\ldots, x_{N0})$ is the initial data and $u_{N}(t,x_i(t))$ is the control. We use the subscript $N$ to emphasise the dependence of the optimal control $u_N$ of (2.5) on the number of particles $N$ .

Remark 2.3. Problem $\mathcal {Q}_N(0,T,x_0)$ can be formally derived from the original optimal control problem. For any $N$ , we have

\begin{align*} f(\mu _{N},u_N) & = \int L(x)d\mu _{N}(t,x) + \int \Psi (u_N(t,x))d\mu _{N}(t,x) \nonumber \\ & = \frac {1}{N}\sum _{i=1}^N \Big [L(x_i(t)) + \Psi (u_N(t,x_i(t)))\Big ]. \end{align*}

which implies that the cost function in (2.5) is given by

\begin{align*} \mathcal {V}_N(0,T,x_0) =\min _{u_N\in \mathcal {F}} \int _{0}^T f(\mu _{N},u_N) dt. \end{align*}

As outlined in the remark, the optimal control problem (2.5) and the original problem are intertwined. Under Assumption2.1, the existence and uniqueness of the problems (2.1)–(2.2) has been established in [20]. To recall the theorem, the definition of the $p-$ Wasserstein distance between two probability measures $\mu$ and $\nu$ is given:

\begin{align*} \mathcal {W}_p(\mu, \nu ) = \inf _{\gamma \in \Gamma (\mu, \nu )} \left ( \int _{\mathbb {R}^{2d}} |x-y|^p d\gamma (x,y) \right )^{1/p}. \end{align*}

Here, $\Gamma (\mu, \nu )$ denotes the set of transport plans, that is, collection of all probability measures with marginals $\mu$ and $\nu$ (see also [36]). Having these preparations, we state the existence theorem in [20], which gives the unique solution to the optimal control problems (2.1)–(2.2) as a mean-field limit of the $N$ -particles problem (2.5).

Theorem 2.1. Assume that the initial data $\mu _0$ in (2.2) is compactly supported; that is, there exists $R \gt 0$ such that $\text {supp}\, \mu _0 \subset B(0, R)$ . Moreover, we assume that the empirical measure $\mu _N(0,x)=\frac {1}{N}\sum _{i=1}^N \delta \left (x-x_{i0}\right )$ converges to $\mu _0$ in $\mathcal {W}_1$ distance. Let

\begin{align*} \mu _N(t,x) = \frac {1}{N}\sum _{i=1}^N \delta (x - x_i(t)) \end{align*}

be supported on the phase space trajectories $x_i(t) \in \mathbb {R}^d$ , for $i = 1,\ldots, N,$ defining the solution of (2.5) with the optimal control $u_N$ . Then, there exists a subsequence $(\mu _{N_k},u_{N_k})$ such that $u_{N_k}$ converges to $u$ in $\mathcal {F}$ as $k\rightarrow \infty$ and

\begin{align*} \lim _{k\rightarrow \infty }\mathcal {W}_1(\mu _{N_k}(t,\cdot ), \mu (t)) = 0 \end{align*}

uniformly with respect to $t \in [0, T]$ . Here, $\mu (t,x)$ is the weak equi-compactly supported solution to the mean-field problems (2.1)–(2.2) with the optimal control $u(t,x)$ . Namely, for all $t\in [0,T]$ , the distribution $\mu (t,x) \in C([0, T];\,P_1(\mathbb {R}^d))$ satisfies $\text {supp}\, \mu (t,\cdot ) \subset B(0, R)$ and

(2.6) \begin{align} & \int \phi (t,x)d\mu (t,x) -\int \phi (0,x)d\mu _0(x) \nonumber \\ & = \int _{0}^{t} \int \Big [\partial _t \phi (s,x) + \nabla _x \phi (s,x) \cdot ((P\ast \mu )(s,x) + u(s,x)) \Big ]d\mu (s,x)ds,\quad \forall \,\phi \in C_0^{\infty }([0,T]\times \mathbb {R}^d). \end{align}

Furthermore, we have the following lower semi-continuous property:

(2.7) \begin{equation} \int _0^T f(\mu (t,\cdot ),u(t,\cdot ))dt\leq \liminf _{k\rightarrow \infty }\int _0^T f(\mu _{N_k}(t,\cdot ),u_{N_k}(t,\cdot ))dt. \end{equation}

Here, $f(\mu (t,\cdot ),u(t,\cdot ))=\int L(x)d\mu (t,x)+ \int \Psi (u(x,t))d\mu (t,x)$ is the time-dependent functional defined in (2.1).

For the exponential stability later, we discuss solutions $\mu (t,x)$ in $C([0,T];\,P_2(\mathbb {R}^d))$ with metric $\mathcal {W}_2$ . By adapting the method in [11, 20], we have

Lemma 2.2. For fixed control $u(t,x)$ , if $\mu (t,x)$ and $\nu (t,x)$ are solutions to (2.2) with initial data $\mu _0$ and $\nu _0$ satisfying the assumption in Theorem 2.1, then there is a constant $C \gt 0$ such that

\begin{align*}\mathcal {W}_2(\mu (t,\cdot ),\nu (t,\cdot )) \leq e^{Ct}\,\mathcal {W}_2(\mu _0,\nu _0)\quad \text {for} \quad t \in [0, T]. \end{align*}

Some remarks are in order. The proof is similar to [11, 20] for the stability in $\mathcal {W}_1$ and deferred to Appendix A. Hence, the optimal solution is unique in $C([0,T];\,P_2(\mathbb {R}^d))$ if the initial data $\mu _0\in P_2\!\left(\mathbb {R}^d \right)$ . Due to this argument, we assume that the optimal solution $\mu (t,x)$ also satisfies

(2.8) \begin{equation} \lim _{k\rightarrow \infty }\mathcal {W}_2(\mu _{N_k}(t,\cdot ), \mu (t,\cdot )) = 0 \end{equation}

uniformly with respect to $t \in [0, T]$ . The assumption is justified since we have the convergence in $\mathcal {W}_1$ and the uniform boundness of the second-order moment for $\mu _N(t,\cdot )$ with respect to $N$ (see, e.g. Theorem4.3).

3. Cheap control property

The cheap control property of the optimal control problem shows that the optimal values are bounded by the distance between the initial state and the desired static state. Combining the cheap control property with the strict dissipativity, we provide a bound on the second-order moments of the probability density. More specifically, for the $N$ -particles system (2.5), we prove:

Lemma 3.1. Suppose $u_N$ is an optimal control to the problem $\mathcal {Q}_N(0,T,x_0)$ and $x(t)$ is the corresponding solution, then $u_N|_{t\in [a,T]}$ is also an optimal control to the sub-problem $\mathcal {Q}_N(a,T,x(a))$ for any $0\leq a\lt T$ . Moreover, the following inequality holds under Assumption 2.1:

(3.1) \begin{align} \frac {1}{N} \sum _{i=1}^N \int _a^T |x_i(t)-\bar {x}|^2 + |u_N(t,x_i(t))|^2dt \leq C_0 \frac {1}{N}\sum _{i=1}^N |x_i(a)-\bar {x}|^2. \end{align}

Here, $C_0$ is a positive constant independent of $N$ and $T$ .

Proof. Suppose there exists a control $\tilde {u}_N$ , defined on $t\in [a,T]$ , such that the corresponding solution $\tilde {x}(t)$ satisfies $\tilde {x}(a)=x(a)$ and

\begin{align*} \int _a^T f(\tilde {\mu }_{N},\tilde {u}_N) dt \lt \int _a^T f_N(\mu _N,u_N) dt. \end{align*}

Here, $\tilde {\mu }_{N}$ is the empirical measure given by

\begin{align*} \tilde {\mu }_{N} = \frac {1}{N}\sum _{i=1}^N \delta \left (x-\tilde {x}_i(t)\right ). \end{align*}

Then, we construct a control

\begin{align*} \hat {u}_N(t,x) = \left \{ \begin {array}{ll} u_N(t,x), & \qquad t\in [0,a) \\[3pt] \tilde {u}_N(t,x), & \qquad t\in [a,T]. \end {array} \right .\end{align*}

In this case, the cost satisfies

\begin{align*} \int _0^T f(\hat {\mu }_{N},\hat {u}_N) dt = \int _0^a f(\mu _{N},u_N) dt + \int _a^T f(\tilde {\mu }_{N},\tilde {u}_N) dt \lt \int _0^T f(\mu _{N},u_N) dt .\end{align*}

This contradicts to the fact that $(x(t),u_N(t))$ is an optimal solution on $[0,T]$ . Therefore, $u_N|_{t\in [a,T]}$ is an optimal control for the sub-problem $\mathcal {Q}_N(a,T,x(a))$ .

Thanks to the strict dissipativity, we have

\begin{align*} \int _a^T f(\mu _N,u_N) dt \geq &\, C_D \int _a^T \int _{\mathbb {R}^d}\Big (|x-\bar {x}|^2 +|u_N(t,x)|^2 \Big ) d\mu _N(t,x)dt \\[3pt] =&\,C_D \frac {1}{N}\sum _{i=1}^N\int _a^T |x_i(t)-\bar {x}|^2 + |u_N(t,x_i(t))|^2 dt. \end{align*}

By Remark2.3, we obtain the estimate (3.1) once we prove the following cheap control inequality:

(3.2) \begin{equation} \int _a^T f(\mu _N,u_N) dt = \frac {1}{N}\sum _{i=1}^N\int _{a}^T L(x_i(t)) + \Psi (u_N(t,x_i(t))) dt \leq C_DC_0 \frac {1}{N}\sum _{i=1}^N |x_i(a)-\bar {x}|^2 \end{equation}

for a constant $C_0\gt 0$ independent of $N$ and $T$ .

Next, we focus on the proof of (3.2). To this end, we consider the feedback control for the problem (2.5):

\begin{align*} \tilde {u}_N(t,\tilde {x}_i(t))=-\beta (\tilde {x}_i(t)-\bar {x})-\frac {1}{N}\sum _{j=1}^NP(\tilde {x}_i(t)-\tilde {x}_j(t)),\qquad i=1,2,,\ldots, N,\quad t\in [a,T]. \end{align*}

Note that $\tilde {u}_N\in \mathcal {F}$ holds. Indeed, due to assumption (2.3), we have that

\begin{align*} |\tilde {u}_N(t,x)-\tilde {u}_N(t,y)| =&\, \Big |\beta (x-y)+\frac {1}{N}\sum _{j=1}^N\big [ P(x-\tilde {x}_j(t)) -P(y-\tilde {x}_j(t)) \big ]\Big |\\[3pt] \leq &\, \beta |x-y| + C_P \frac {1}{N} \sum _{j=1}^N |x-y| = (\beta + C_P)|x-y|, \end{align*}

which gives a Lipschitz constant for $\tilde {u}_N(t,\cdot )$ . Based on this feedback control, $\tilde {x}_i(t)$ satisfies the equation

\begin{align*} \frac {d\tilde {x}_i(t)}{dt} = -\beta (\tilde {x}_i(t)-\bar {x}),\qquad \tilde {x}_i(a) = x_i(a). \end{align*}

It follows that

(3.3) \begin{equation} |\tilde {x}_i(t)-\bar {x}|^2 = e^{-2\beta (t-a)}|\tilde {x}_i(a)-\bar {x}|^2 = e^{-2\beta (t-a)}|x_i(a)-\bar {x}|^2. \end{equation}

In the next paragraph, we estimate $|\tilde {u}_N(t,\tilde {x}_i(t))|^2$ . By definition, we have

\begin{align*} |\tilde {u}_N(t,\tilde {x}_i(t))|^2 \leq 2\beta ^2 |\tilde {x}_i(t)-\bar {x}|^2 + 2\Big |\frac {1}{N}\sum _{j=1}^NP(\tilde {x}_i(t)-\tilde {x}_j(t))\Big |^2. \end{align*}

Using Jensen’s inequality, we have

(3.4) \begin{equation} \Big |\frac {1}{N}\sum _{j=1}^NP(\tilde {x}_i(t)-\tilde {x}_j(t))\Big |^2 \leq \frac {1}{N}\sum _{j=1}^N\Big |P(\tilde {x}_i(t)-\tilde {x}_j(t))\Big |^2. \end{equation}

Due to the assumption of $P(x)$ , we have

\begin{align*} \frac {1}{N}\sum _{j=1}^N\Big |P(\tilde {x}_i(t)-\tilde {x}_j(t))\Big |^2 \leq &\,\frac {C_P^2}{N}\sum _{j=1}^N|\tilde {x}_i(t)-\tilde {x}_j(t)|^2\\[3pt] \leq &\,2C_P^2|\tilde {x}_i(t)-\bar {x}|^2+ \frac {2C_P^2}{N}\sum _{j=1}^N|\tilde {x}_j(t)-\bar {x}|^2. \end{align*}

Then, it follows that

\begin{align*} |\tilde {u}_N(t,\tilde {x}_i(t))|^2 \leq &\,(2\beta ^2+4C_P^2)|\tilde {x}_k(t)-\bar {x}|^2 + \frac {4C_P^2}{N}\sum _{j=1}^N|\tilde {x}_j(t)-\bar {x}|^2. \end{align*}

We sum $i$ from $1$ to $N$ and get

\begin{align*} \frac {1}{N}\sum _{i=1}^N|\tilde {u}_N(t,\tilde {x}_i(t))|^2 \leq & \,C(\beta, C_P) \frac {1}{N}\sum _{i=1}^N |\tilde {x}_i(t)-\bar {x}|^2 \end{align*}

with $C(\beta, C_P)=2\beta ^2+8C_P^2$ . Since $u_N$ is optimal in (2.5), we have

\begin{align*} \frac {1}{N}\sum _{i=1}^N\int _{a}^T L(x_i(t)) + \Psi (u_N(t,x_i(t))) dt \leq &\, \frac {1}{N}\sum _{i=1}^N\int _a^T L(\tilde {x}_i(t))+\Psi (\tilde {u}_N(t,\tilde {x}_i(t)))dt \\[3pt] \leq &\, (C({\beta },C_P)C_{\Psi }+C_L)\frac {1}{N}\sum _{i=1}^N \int _a^T |\tilde {x}_i(t)-\bar {x}|^2 dt. \end{align*}

Note that the last inequality is due to Assumption2.1 (ii). Substituting (3.3) into the last inequality, we have

\begin{align*} &\,\frac {1}{N}\sum _{i=1}^N\int _{a}^T L(x_i(t)) + \Psi (u_N(t,x_i(t))) dt \\[3pt] \leq &\,(C({\beta },C_P)C_{\Psi }+C_L)\left (\int _a^Te^{-2\beta (t-a)}dt\right )\frac {1}{N}\sum _{i=1}^N |x_i(a)-\bar {x}|^2. \end{align*}

It is easy to show that

\begin{align*} \int _a^Te^{-2\beta (t-a)}dt=\frac {1}{2\beta }e^{-2\beta (t-a)}\Big |_T^a\leq \frac {1}{2 \beta }. \end{align*}

Then, we conclude

(3.5) \begin{align} \frac {1}{N}\sum _{i=1}^N\int _{a}^T L(x_i(t)) + \Psi (u_N(t,x_i(t))) dt \leq \frac {C({\beta },C_P)C_{\Psi }+C_L}{2\beta } \frac {1}{N}\sum _{i=1}^N |x_i(a)-\bar {x}|^2. \end{align}

Note that the inequality (3.2) holds if we take the constant

\begin{align*} C_0 = \frac {C({\beta },C_P)C_{\Psi }+C_L}{2\beta C_D}, \end{align*}

which is independent of $N$ and $T$ .

The estimate (3.1) is independent of $N$ . We consider $N\rightarrow \infty$ to get the corresponding result for the mean-field problem. To this end, we also need to use the lower semi-continuity of the cost function (2.1). Namely, we prove the following property for the mean-field problem.

Lemma 3.2. Suppose $(\mu (t,x),u(t,x))$ is the solution to the optimal control problems (2.1)–(2.2), then the following inequality holds under Assumption 2.1:

(3.6) \begin{align} \int _a^T \int _{\mathbb {R}^d}\Big (|x-\bar {x}|^2 +|u(t,x)|^2 \Big ) d\mu (t,x)dt \leq &\, C_0 \int |x-\bar {x}|^2 d\mu (a,x). \end{align}

Proof. Due to lower semi-continuity, we have

\begin{align*} \int _a^T f(\mu (t,x),u(t,x))dt \leq &\, \liminf _{k\rightarrow \infty }\int _a^T f(\mu _{N_k}(t,x),u_{N_k}(t,x))dt \\[3pt] = &\, \liminf _{k\rightarrow \infty }\frac {1}{N_k}\sum _{i=1}^{N_k}\int _a^T L(x_i(t))+\Psi (u_{N_k}(t,x_i(t)))dt. \end{align*}

On the other hand, since $u_{N_k}$ is the optimal solution to (2.5), it follows from (3.2) that

\begin{align*} \int _a^T f(\mu (t,x),u(t,x))dt \leq &\,\liminf _{k\rightarrow \infty }\frac {1}{N_k}\sum _{i=1}^{N_k}\int _a^T L(x_i(t))+\Psi (u_{N_k}(t,x_i(t)))dt\\[3pt] \leq &\, \liminf _{k\rightarrow \infty } C_DC_0 \frac {1}{N_k}\sum _{i=1}^{N_k} |x_i(a)-\bar {x}|^2 \\[3pt] =&\, C_DC_0 \int |x-\bar {x}|^2 d\mu (a,x). \end{align*}

Here, $C_0$ is the constant introduced in Lemma 3.1. Using the strict dissipativity shows that

\begin{align*} C_D \int _a^T \int _{\mathbb {R}^d}\Big (|x-\bar {x}|^2 +|u(t,x)|^2 \Big ) d\mu (t,x)dt \leq \int _a^T f(\mu (t,x),u(t,x))dt \leq C_DC_0 \int |x-\bar {x}|^2 d\mu (a,x). \end{align*}

This is the relation (3.6), and we conclude the result.

We conclude this section with the following remarks:

• • The inequality (3.6) is the mean-field limit of relation (3.1).

• • The right-hand side of (3.6) is independent of $T$ . As in other turnpike results, this shows an integral turnpike property. Namely, the second-order moments $\int _{\mathbb {R}^d}\Big (|x-\bar {x}|^2 +|u(t,x)|^2 \Big ) d\mu (t,x)$ must be small along the largest part of the time-horizon provided that $T$ is sufficiently large.

• • The cheap control idea was also used in [27] to prove the integral turnpike property with interior decay. Different from the results in [27], the present work uses the second-order moment $\int _{\mathbb {R}^d} |x-\bar {x}|^2 d\mu (a,x)$ as the bound in (3.6) instead of the first-order moment. This is important for the proofs in the next section.

4. Exponential turnpike property

In this section, we will prove that the optimal solution to (2.1)–(2.2) converges to the optimal static state exponentially fast. In general, the optimal static state $(\bar {\mu }(x),\bar {u}(x))$ is a solution to the problem:

\begin{align*} &\,\min _{\bar {u}\in \mathcal {F}} f(\bar {\mu }(x),\bar {u}(x)) \,:\!=\, \min _{\bar {u}\in \mathcal {F}} \int L(x)d\bar {\mu }(x) + \int \Psi (\bar {u}(x))d\bar {\mu }(x),\\[3pt] s.t. &\,\qquad \nabla _x\cdot \Big ((P\ast \bar {\mu } + \bar {u})\bar {\mu }\Big ) =0,\qquad x\in \mathbb {R}^d. \end{align*}

In the present work, we focus on the case where $\bar {\mu }(x)=\delta (x-\bar {x})$ and $\bar {u}(x)\equiv 0$ . We check that $\bar {\mu }(x)$ satisfies the equation in the weak sense: for all $\bar {\phi }\in C_0^{\infty }(\mathbb {R}^d)$ ,

\begin{align*} \int \nabla _x \bar {\phi } \cdot (P\ast \bar {\mu } + \bar {u}) d\bar {\mu }(x) = \int \nabla _x \bar {\phi } \cdot P(x-\bar {x}) d\bar {\mu }(x) = 0. \end{align*}

Thus, it is not difficult to see that $(\bar {\mu }(x),\bar {u}(x))=(\delta (x-\bar {x}),0)$ is an optimal static state.

The estimates on the inequalities for the optimal solution $\mu (t,x)$ and the optimal control $u(t,x)$ are given separately (see Theorems4.4 and 4.5 below). To this end, we derive the estimate for the optimal solution $x_i(t)$ of the $N$ -particles system. Then, we consider the mean-field limit $N \rightarrow \infty$ to obtain an estimate for $\mu (t,x)$ . At last, we prove that the optimal control $u(t,x)$ can be bounded in terms of the solution $\mu (t,x)$ .

4.1 Estimate for the solution

For the solution $x_i(t)$ of (2.5), we use Gronwall’s inequality to derive

Lemma 4.1. Suppose (3.1) holds, there exists a constant $C_1\geq 1$ , independent of $N$ and $T$ , such that

(4.1) \begin{equation} \frac {1}{N} \sum _{i=1}^N |x_i(t_2)-\bar {x}|^2 \leq C_1 \frac {1}{N}\sum _{i=1}^N |x_i(t_1)-\bar {x}|^2,\qquad \forall \,0 \leq t_1 \leq t_2 \leq T. \end{equation}

Proof. We estimate $y_i(t)=x_i(t)-\bar {x}$ by computing:

(4.2) \begin{align} \frac {1}{2}\int _{t_1}^{t_2}\frac {d}{dt}\langle y_i(t), y_i(t) \rangle dt=&\,\int _{t_1}^{t_2}\langle y_i(t),y_i'(t) \rangle dt \nonumber \\[3pt] = &\, \frac {1}{N}\sum _{j=1}^N \int _{t_1}^{t_2} \langle y_i(t), P(y_i(t)-y_j(t)) \rangle dt + \int _{t_1}^{t_2} \langle y_i(t), u_i(t) \rangle dt. \end{align}

For the second term, we have

(4.3) \begin{equation} \int _{t_1}^{t_2} \langle y_i(t), u_i(t) \rangle dt \leq \frac {1}{2} \int _{t_1}^{t_2} |u_i(t)|^2 dt + \frac {1}{2} \int _{t_1}^{t_2} |y_i(t)|^2 dt, \end{equation}

and for the first term, we have

(4.4) \begin{align} &\frac {1}{N}\sum _{j=1}^N \int _{t_1}^{t_2} \langle y_i(t), P(y_i(t)-y_j(t)) \rangle dt \leq \, \frac {1}{N}\sum _{j=1}^N C_P \int _{t_1}^{t_2} | y_i(t)||y_i(t)-y_j(t)| dt \nonumber \\[3pt] \leq \,& \frac {1}{N}\sum _{j=1}^N C_P \int _{t_1}^{t_2} | y_i(t)|^2+|y_i(t)||y_j(t)| dt \leq \, \frac {3C_P}{2} \int _{t_1}^{t_2} | y_i(t)|^2dt + \frac {C_P}{2N}\sum _{j=1}^N \int _{t_1}^{t_2} |y_j(t)|^2 dt. \end{align}

Combining (4.2)–(4.4) yields

\begin{align*} \frac {1}{2}\int _{t_1}^{t_2}\frac {d}{dt}\langle y_i(t), y_i(t) \rangle dt \leq \Big (\frac {1}{2}+\frac {3C_P}{2}\Big ) \int _{t_1}^{t_2} |y_i(t)|^2 dt + \frac {C_P}{2N}\sum _{j=1}^N \int _{t_1}^{t_2} |y_j(t)|^2 dt + \frac {1}{2}\int _{t_1}^{t_2} |u_i(t)|^2 dt. \end{align*}

We sum $i$ from $1$ to $N$ and multiply $1/N$ to obtain

\begin{align*} \frac {1}{N}\sum _{i=1}^N|y_i(t_2)|^2 \leq \frac {1}{N}\sum _{i=1}^N|y_i(t_1)|^2 + \left (1+4C_P\right ) \frac {1}{N} \sum _{i=1}^N \int _{t_1}^{t_2} |y_i(t)|^2 dt + \frac {1}{N} \sum _{i=1}^N \int _{t_1}^{t_2} |u_i(t)|^2 dt. \end{align*}

Combining this with (3.1), we obtain

\begin{align*} \frac {1}{N} \sum _{i=1}^N |x_i(t_2)-\bar {x}|^2 \leq C_1 \frac {1}{N}\sum _{i=1}^N |x_i(t_1)-\bar {x}|^2,\qquad \forall \,0 \leq t_1 \leq t_2 \leq T. \end{align*}

with $C_1= (2+4C_P)C_0+1$ . Note that $C_1$ is independent of $N$ and $T$ .

Combining this lemma with the inequality (3.1), we prove:

Lemma 4.2. Under Assumption 2.1, the following inequality holds for any $t \in [n\tau, T]$ with a given constant $\tau \gt 0$ and an integer $1\leq n\leq \frac {T}{\tau }$ :

\begin{align*} \frac {1}{N}\sum _{i=1}^N|x_i(t)-\bar {x}|^2 \leq \left (\frac {C_0C_1}{\tau }\right )^n \frac {1}{N}\sum _{i=1}^N|x_i(0)-\bar {x}|^2. \end{align*}

Proof. We first prove the case $n=1$ . There exists a point $t_1\in [0,\tau ]$ such that

\begin{align*} \frac {1}{N}\sum _{i=1}^N|x_i(t_1)-\bar {x}|^2 \leq \frac {1}{\tau } \int _{0}^{\tau } \frac {1}{N}\sum _{i=1}^N|x_i(t)-\bar {x}|^2 dt \leq \frac {C_0}{\tau } \frac {1}{N}\sum _{i=1}^N|x_i(0)-\bar {x}|^2. \end{align*}

Note that the last inequality follows by (3.1). For any $t\geq \tau \geq t_1$ , we obtain by Lemma 4.1

\begin{align*} \frac {1}{N}\sum _{i=1}^N|x_i(t)-\bar {x}|^2 \leq C_1\frac {1}{N}\sum _{i=1}^N|x_i(t_1)-\bar {x}|^2 \leq \frac {C_0C_1}{\tau } \frac {1}{N}\sum _{i=1}^N|x_i(0)-\bar {x}|^2. \end{align*}

Then we suppose the inequality holds for $n\geq 1$ and prove the result for $n+1$ . There exists $t_n\in [n\tau, (n+1)\tau ]$ such that

\begin{align*} \frac {1}{N}\sum _{i=1}^N|x_i(t_n)-\bar {x}|^2 \leq &\, \frac {1}{\tau } \int _{n\tau }^{(n+1)\tau }\frac {1}{N}\sum _{i=1}^N|x_i(t)-\bar {x}|^2dt\\[3pt] \leq &\,\frac {C_0}{\tau } \frac {1}{N}\sum _{i=1}^N|x_i(n\tau )-\bar {x}|^2\leq \frac {C_0}{\tau } \left (\frac {C_0C_1}{\tau }\right )^n \frac {1}{N}\sum _{i=1}^N|x_i(0)-\bar {x}|^2. \end{align*}

Thus, for any $t\in [(n+1)\tau, T]$ , we obtain by Lemma 4.1

\begin{align*} \frac {1}{N}\sum _{i=1}^N|x_i(t)-\bar {x}|^2 \leq C_1\frac {1}{N}\sum _{i=1}^N|x_i(t_n)-\bar {x}|^2\leq \left (\frac {C_0C_1}{\tau }\right )^{n+1} \frac {1}{N}\sum _{i=1}^N|x_i(0)-\bar {x}|^2 \end{align*}

and this completes the proof.

Thanks to the above lemmas, we are in the position to state the main result for the optimal solution $x_i(t)$ of the particle system (2.5):

Theorem 4.3. Suppose Assumption 2.1 holds. Then there exist constants $C_2\gt 0$ and $\alpha \gt 0$ , which are independent of $N$ and $T$ , such that for all $T\gt C_0C_1$ , the optimal solution of $\mathcal {Q}_N(0,T,x_0)$ satisfies the exponential turnpike property:

\begin{align*} \frac {1}{N}\sum _{i=1}^N|x_i(t)-\bar {x}|^2 \leq C_2 e^{-\alpha t} \frac {1}{N}\sum _{i=1}^N|x_i(0)-\bar {x}|^2 \end{align*}

for any $t\in (0,T)$ . Here, $C_0$ and $C_1$ are two constants given in Lemma 3.1 and Lemma 4.1, which are independent of $N$ and $T$ .

Proof. In this proof, we need to fix the constant $\tau$ in Lemma 4.2 such that $\tau \gt C_0C_1$ . Since $T\gt C_0C_1$ , we choose the constant $\tau$ satisfying $0\lt \tau \lt T$ . Next, we discuss the cases $t\in (0,\tau )$ and $t\in [\tau, T)$ separately.

For any $t\in [\tau, T)$ , we take the integer $n=\lfloor t/\tau \rfloor$ . Then, $1\leq n \leq \frac {T}{\tau }$ and $t\in [n\tau, T)$ , and we obtain by Lemma 4.2:

\begin{align*} \frac {1}{N}\sum _{i=1}^N|x_i(t)-\bar {x}|^2 \leq &\, \left (\frac {C_0C_1}{\tau }\right )^n \frac {1}{N}\sum _{i=1}^N|x_i(0)-\bar {x}|^2. \end{align*}

Due to the definition of $n$ , we have $n\gt t/\tau -1$ . Also, the constant $\tau$ is chosen such that $\tau \gt C_0C_1$ . Thus, we have

\begin{align*} &\, \left (\frac {C_0C_1}{\tau }\right )^n = \left (\frac {\tau }{C_0C_1}\right )^{-n} \leq \left (\frac {\tau }{C_0C_1}\right )^{1-t/\tau }. \end{align*}

The exponential estimate is then given by

\begin{align*} \frac {1}{N}\sum _{i=1}^N|x_i(t)-\bar {x}|^2 \leq \hat {C}_2 e^{-\alpha t} \frac {1}{N}\sum _{i=1}^N|x_i(0)-\bar {x}|^2,\quad \forall \,t\in [\tau, T) \end{align*}

with

\begin{align*} \hat {C}_2 = \frac {\tau }{C_0C_1},\qquad \alpha = \frac {1}{\tau }\log \left (\frac {\tau }{C_0C_1}\right )\gt 0. \end{align*}

On the other hand, for $t\in (0,\tau )$ , we have

\begin{align*} \hat {C}_2 e^{-\alpha t} \geq \hat {C}_2 e^{-\alpha \tau } = 1. \end{align*}

By Lemma 4.1, we have

\begin{align*} \frac {1}{N} \sum _{i=1}^N |x_i(t)-\bar {x}|^2 \leq C_1 \frac {1}{N}\sum _{i=1}^N |x_i(0)-\bar {x}|^2 \leq C_1 \hat {C}_2 e^{-\alpha t}\frac {1}{N}\sum _{i=1}^N |x_i(0)-\bar {x}|^2. \end{align*}

Recall that due to the proof of Lemma 4.1, $C_1\geq 1$ holds. To combine the results of $t\in (0,\tau )$ and $t\in [\tau, T)$ , we take $C_2=C_1 \hat {C}_2$ and obtain

(4.5) \begin{equation} \frac {1}{N} \sum _{i=1}^N |x_i(t)-\bar {x}|^2 \leq C_2 e^{-\alpha t}\frac {1}{N}\sum _{i=1}^N |x_i(0)-\bar {x}|^2,\quad \forall \,t\in (0,T). \end{equation}

This theorem implies that the empirical measure has equi-compact support and bounded second-order moments for any number of particles $N$ . Moreover, we know that the empirical measure $\mu _N(t,x)$ defined in (2.4) satisfies

\begin{align*} \mathcal {W}_2(\mu _N(t,\cdot ),\delta (x-\bar {x})) \leq \sqrt {C_2} e^{-\alpha t/2} \mathcal {W}_2(\mu _N(0,\cdot ),\delta (x-\bar {x})). \end{align*}

We established the exponential decay property for the second-order moment of the empirical measures $\mu _N(t,\cdot )$ with respect to $t$ :

\begin{align*} \int |x-\bar {x}|^2 d\mu _N(t,x) \leq C_2 e^{-\alpha t} \int |x-\bar {x}|^2 d\mu _N(0,x). \end{align*}

The constant $C_2$ is independent of $N$ . Thus, we can use the uniform $\mathcal {W}_2$ convergence to obtain the exponential turnpike property in the mean-field limit. Namely, we have

Theorem 4.4. Suppose Assumption 2.1 holds. For problem $\mathcal {Q}(0,T,\mu _0)$ with $T\gt C_0C_1$ , the optimal solution $\mu (t,x)\in C([0,T];P_2(\mathbb {R}^d))$ satisfies the exponential turnpike property in the sense that

\begin{align*} \int |x-\bar {x}|^2 d\mu (t,x) \leq C_2 e^{-\alpha t} \int |x-\bar {x}|^2 d\mu _0(x) \end{align*}

for any $t\in (0,T)$ . Here, the constants $C_2$ and $\alpha$ are the same as those in Theorem 4.3.

Remark 4.1. Alternatively, the result of the mean-field problem can be also proven by a direct estimate of (2.2). Namely, we may take a test function $\phi (t,x)=|x-\bar {x}|^2\chi _R(x)$ with $\chi _R(x)$ being a mollified characteristic function $\chi _R(x)=\psi _\delta \ast \chi _{[-R-\delta, R+\delta ]}$ , such that $\chi _R(x)=1$ for $|x|\leq R$ .

Then by the same argument as in Lemma 4.1, we have

\begin{align*} &\int |x-\bar {x}|^2 d\mu (t_2,x) \leq C_1 \int |x-\bar {x}|^2 d\mu (t_1,x),\qquad \forall \,0\leq t_1 \leq t_2 \leq T. \end{align*}

Similarly, the inequalities analogue to those in Lemma 4.2 and Theorem 4.3 can be also obtained.

4.2 Estimate on the control

In this subsection, we estimate the optimal control $u(t,x)$ in the mean- field problem. The idea is to construct a novel feedback control and take advantage of the strict dissipativity.

We divide the time interval $[0,T]$ into three parts:

\begin{align*} [0,T] = [0,s)\cup [s,s+mh] \cup (s+mh,T]. \end{align*}

Here, $s\in (0,T)$ is a fixed time point, $m\gt 0$ is a scale parameter, which will be given later (see (4.22)), and $h$ is a sufficiently small constant such that $s+mh\leq T$ . We construct a feedback control $\hat {u}(t,x)$ by

(4.6) \begin{equation} \hat {u}(t,x) = \left \{ \begin {array}{ll} u(t,x), & \qquad t\in [0,s) \\[3mm] \dfrac {1}{m}u\left (s+\dfrac {t-s}{m},x\right )-\dfrac {m-1}{m} (P\ast \hat {\mu })(t,x) & \qquad t\in [s,s+mh] \\[5mm] u\Big (t-(m-1)h,x\Big ), & \qquad t\in (s+mh,T], \end {array} \right . \end{equation}

where $u(t,x)$ is the optimal control to the problems (2.1)–(2.2) on the time interval $[0,T]$ and $\hat {\mu }(t,x)$ is the solution of (2.2) associated with the new control $\hat {u}(t,x)$ ,

Next, we discuss the solution $\hat {\mu }(t,x)$ on the different time intervals.

For $t\in [0,s)$ , we know that $\hat {u}(t,x)=u(t,x)$ and the initial data satisfies

\begin{align*} \hat {\mu }(0,\cdot ) = \mu _0(\cdot )\quad \text {in}\, P_2(\mathbb {R}^d). \end{align*}

According to the uniqueness of the solution to the mean-field equation (2.2), it is easy to see that

\begin{align*} \hat {\mu }(t,\cdot ) = \mu (t,\cdot )\quad \text {in}\,P_2(\mathbb {R}^d),\qquad \forall \,t\in [0,s]. \end{align*}

Here, $\mu (t,x)$ is the solution associated with the optimal control $u(t,x)$ .

On the other hand, for $t\in [s,s+mh]$ , we use the expression of $\hat {u}(t,x)$ to compute the equation of $\hat {\mu }$ (for simplicity in the strong form). A similar computation holds in the weak form.

\begin{align*} 0=&\,\partial _t \hat {\mu }(t,x) + \nabla _x \cdot \Big ( \big [(P\ast \hat {\mu })(t,x)+\hat {u}(t,x)\big ]\hat {\mu }(t,x)\Big ) \\[3pt] =&\, \partial _t \hat {\mu }(t,x) + \nabla _x \cdot \Big ( \Big [(P\ast \hat {\mu })(t,x)+\frac {1}{m} u\left (s+\dfrac {t-s}{m},x\right ) - \frac {m-1}{m}(P\ast \hat {\mu })(t,x) \Big ]\hat {\mu }(t,x)\Big ) \\[3pt] =&\, \partial _t \hat {\mu }(t,x) + \frac {1}{m} \nabla _x \cdot \Big ( \Big [(P\ast \hat {\mu })(t,x)+u\left (s+\dfrac {t-s}{m},x\right )\Big ]\hat {\mu }(t,x)\Big ). \end{align*}

Moreover, by the first step, we have

\begin{align*} \hat {\mu }(s,\cdot ) = \mu (s,\cdot )\quad \text {in}\,P_2(\mathbb {R}^d). \end{align*}

Thus, the equation for $\hat {\mu }$ reads (in weak form)

(4.7) \begin{align} &\int \phi (t,x)d\hat {\mu }(t,x) -\int \phi (s,x)d\mu (s,x) \nonumber \\[3pt] = &\int _{0}^{t} \int \Big [\partial _t \phi (r,x) + \frac {1}{m} \nabla _x \phi (r,x) \cdot \Big ((P\ast \hat {\mu })(r,x) + u\left (s+\dfrac {r-s}{m},x\right )\Big ) \Big ]d\hat {\mu }(r,x)dr \nonumber \\[3pt] &\quad \forall \,\phi (t,x)\in C_0^{\infty }\left ([s,s+mh]\times \mathbb {R}^d\right ). \end{align}

Since the map that maps $t\in [s,s+mh]$ to $t_1\in [s,s+h]$ by

\begin{align*} t\,\,\mapsto \,\,t_1 = s+\dfrac {t-s}{m} \end{align*}

is bijective, we consider the test function

\begin{align*} \phi (t,x) = \hat {\phi }(t_1,x) =\hat {\phi }\left (s+\dfrac {t-s}{m},x\right )\quad \text {with}\quad \hat {\phi }\in C_0^{\infty }\left ([s,s+h]\times \mathbb {R}^d\right ) \end{align*}

and the formula (4.7) is equivalent to

(4.8) \begin{align} &\int \hat {\phi }(t_1,x) d\hat {\mu }(t,x) -\int \hat {\phi }(s,x)d\mu (s,x) \nonumber\\[3pt] =& \int _{0}^{t} \int \Big [\partial _{t} \hat {\phi }(r_1,x) + \nabla _x \hat {\phi }(r_1,x) \cdot \big ((P\ast \hat {\mu })(r,x) + u(r_1,x)\big ) \Big ]d\hat {\mu }(r,x)dr_1, \nonumber\\[3pt] &\forall \,\hat {\phi }\in C_0^{\infty }\left ([s,s+h]\times \mathbb {R}^d\right ). \end{align}

Here, we use the relation

\begin{align*} r_1=s+\dfrac {r-s}{m},\qquad dr_1=\frac {1}{m}dr, \end{align*}

and obtain that

\begin{align*} \mu (t_1,x) = \mu \left (s+\dfrac {t-s}{m},x\right ) \end{align*}

is a solution to (4.8). Again, $\mu (t,x)$ is the solution associated with the optimal control $u(t,x)$ . Since the solution for (2.2) is unique in $P_2(\mathbb {R}^d)$ , we have

(4.9) \begin{equation} \hat {\mu }(t,\cdot ) = \mu (t_1,\cdot ) = \mu \left (s+\dfrac {t-s}{m},\cdot \right ) \quad \text {in}\,P_2(\mathbb {R}^d),\qquad \forall \,t\in [s,s+mh]. \end{equation}

In the last interval, for $t\in (s+mh,T]$ , the control is $\hat {u}(t,x)=u\big (t-(m-1)h,x\big )$ , and the equation for $\hat {\mu }$ reads (in strong form):

\begin{align*} 0=&\,\partial _t \hat {\mu }(t,x) + \nabla _x \cdot \Big ( \big [(P\ast \hat {\mu })(t,x)+\hat {u}(t,x)\big ]\hat {\mu }(t,x)\Big ) \\[3pt] =&\, \partial _t \hat {\mu }(t,x) + \nabla _x \cdot \Big ( \Big [(P\ast \hat {\mu })(t,x)+ u\big (t-(m-1)h,x\big ) \Big ]\hat {\mu }(t,x)\Big ). \end{align*}

Considering $t=s+mh$ , we have

\begin{align*} \hat {\mu }(s+mh,\cdot ) = \mu (s+h,\cdot )\quad \text {in}\,P_2(\mathbb {R}^d). \end{align*}

Thus, the weak form in the time interval $(s+mh,T]$ reads as

(4.10) \begin{align} &\int \phi (t,x)d\hat {\mu }(t,x) -\int \phi (s+mh,x)d\mu (s+h,x) \nonumber\\[3pt] = &\int _{0}^{t} \int \Big [\partial _t \phi (r,x) + \nabla _x \phi (r,x) \cdot \Big ((P\ast \hat {\mu })(r,x) + u\big (t-(m-1)h,x\big ) \Big ]d\hat {\mu }(r,x)dr \nonumber\\[3pt] &\quad \forall \,\phi (t,x)\in C_0^{\infty }\left ((s+mh,T]\times \mathbb {R}^d\right ). \end{align}

In the new variable $ t_2 = t-(m-1)h$ and for the test function

\begin{align*} \phi (t,x) = \hat {\phi }(t_2,x) =\hat {\phi }\left (t-(m-1)h,x\right )\quad \text {with}\quad \hat {\phi }\in C_0^{\infty }\left ((s+h,T-(m-1)h]\times \mathbb {R}^d\right ), \end{align*}

equation (4.10) reads

(4.11) \begin{align} &\int \hat {\phi }(t_2,x) d\hat {\mu }(t,x) -\int \hat {\phi }(s+mh,x)d\mu (s+h,x) \nonumber\\[3pt] = &\int _{0}^{t} \int \Big [\partial _{t} \hat {\phi }(r_2,x) + \nabla _x \hat {\phi }(r_2,x) \cdot \big ((P\ast \hat {\mu })(r,x) + u(r_2,x)\big ) \Big ]d\hat {\mu }(r,x)dr_2, \nonumber\\[3pt] &\quad \forall \,\hat {\phi }\in C_0^{\infty }\left ((s+h,T-(m-1)h]\times \mathbb {R}^d\right ) \end{align}

for $ r_2=r-(m-1)h$ and $dr_2=dr.$ It is easy to see that $ \mu (t_2,x) = \mu \left (t-(m-1)h,x\right )$ satisfies (4.11). At last, we use the uniqueness of (2.2) in $P_2(\mathbb {R}^d)$ to conclude that

(4.12) \begin{equation} \hat {\mu }(t,\cdot ) = \mu (t_2,\cdot ) = \mu \left (t-(m-1)h,\cdot \right ) \quad \text {in}\,P_2(\mathbb {R}^d),\qquad \forall \,t\in (s+mh,T]. \end{equation}

Summarising, we have

(4.13) \begin{equation} \hat {\mu }(t,\cdot ) = \left \{ \begin {array}{ll} \mu (t,\cdot ), & \qquad t\in [0,s), \\[4mm] \mu \left (s+\dfrac {t-s}{m},\,\cdot \right ), & \qquad t\in [s,s+mh],\\[5mm] \mu \left (t-(m-1)h,\,\cdot \right ), & \qquad t\in (s+mh,T]. \end {array} \right . \end{equation}

4.3 The turnpike estimate

Having the feedback control $\hat {u}(t,x)$ and its associated solution $\hat {\mu }(t,x)$ , we proceed to estimate the optimal control $u(t,x)$ :

Theorem 4.5. Suppose Assumption 2.1 holds. Then there exists a constant $C_{3}\gt 0$ such that the optimal control $u(t,x)\in \mathcal {F}$ for $\mathcal {Q}(0,T,\mu _0)$ with $T\gt C_0C_1$ satisfies the exponential turnpike property:

\begin{align*} \int |u(t,x)|^2 d\mu (t,x) \leq C_{3} e^{-\alpha t} \int |x-\bar {x}|^2 d\mu _0(x) \mbox { for a.e. } t\in (0,T). \end{align*}

Proof. Since $u(t,x)$ is optimal, we have

(4.14) \begin{align} &\int _{0}^Tf(\mu (t,x),u(t,x))dt \leq \int _{0}^Tf(\hat {\mu }(t,x),\hat {u}(t,x))dt \nonumber \\[3pt] =&\, \int _{0}^sf(\hat {\mu }(t,x),\hat {u}(t,x))dt + \int _s^{s+mh}f(\hat {\mu }(t,x),\hat {u}(t,x))dt + \int _{s+mh}^Tf(\hat {\mu }(t,x),\hat {u}(t,x))dt. \end{align}

According to (4.6) and (4.13), we have

(4.15) \begin{equation} \int _{0}^sf(\hat {\mu }(t,x),\hat {u}(t,x))dt = \int _{0}^sf(\mu (t,x),u(t,x))dt \end{equation}

and

(4.16) \begin{align} \int _{s+mh}^Tf(\hat {\mu }(t,x),\hat {u}(t,x))dt =\, \int _{s+mh}^Tf(\mu \left (t-(m-1)h,x\right ),u\left (t-(m-1)h,x\right ))dt \nonumber \\[3pt] =\, \int _{s+h}^{T-(m-1)h}f(\mu (t,x),u(t,x))dt \leq \, \int _{s+h}^{T}f(\mu (t,x),u(t,x))dt. \end{align}

Therefore, it follows by (4.14)–(4.16)

(4.17) \begin{align} \int _s^{s+h}f(\mu (t,x),u(t,x))dt \leq &\, \int _s^{s+mh}f(\hat {\mu }(t,x),\hat {u}(t,x))dt \nonumber \\[3pt] \leq &\, C_4 \int _s^{s+mh}\int |x-\bar {x}|^2 + | \hat {u}(t,x)|^2 d\hat {\mu }(t,x)dt \end{align}

with $C_4=\max \{C_{\Psi },C_L\}$ . Notice that the last inequality is due to Assumption2.1. Moreover, we use (4.6) and (4.13) to obtain

\begin{align*} &\, C_4 \int _s^{s+mh}\int |x-\bar {x}|^2 + | \hat {u}(t,x)|^2 d\hat {\mu }(t,x)dt \\[3pt] = &\, C_4 \int _s^{s+mh} \int | x-\bar {x}|^2 + | \frac {1}{m} u(t_1,x)- \frac {m-1}{m}(P\ast \mu )(t_1,x)|^2 d\mu (t_1,x)dt \end{align*}

with $t_1=s+\dfrac {t-s}{m}$ . By change of variables, the above inequality yields

(4.18) \begin{align} &\, C_4 \int _s^{s+mh}\int |x-\bar {x}|^2 + | \hat {u}(t,x)|^2 d\hat {\mu }(t,x)dt \nonumber \\[3pt] \leq &\, m C_4 \int _s^{s+h} \int | x-\bar {x}|^2 + | \frac {1}{m} u(t,x)- \frac {m-1}{m}(P\ast \mu )(t,x)|^2 d\mu (t,x)dt \nonumber \\[3pt] \leq &\, m C_4 \int _s^{s+h} \int | x-\bar {x}|^2 + \frac {3}{2}\frac {1}{m^2} |u(t,x)|^2 + 3\Big | \frac {m-1}{m}(P\ast \mu )(t,x)\Big |^2 d\mu (t,x)dt. \end{align}

Note that the last inequality follows from the basic inequality

\begin{align*} |a+b|^2 \leq \frac {3}{2}|a|^2 + 3|b|^2. \end{align*}

Using Jensen’s inequality and Assumption2.1, we have

(4.19) \begin{align} |(P \ast \mu )(t,x)|^2 \leq \int |P(x-y)|^2 d \mu (t,y) \leq C_P^2 |x-\bar {x}|^2 + C_P^2 \int |y-\bar {x}|^2 d\mu (t,y). \end{align}

By (4.17)–(4.19), there exists a constant $C_5\gt 0$ depending on $C_P,C_\Psi, C_L$ and $m$ , such that

(4.20) \begin{align} \int _s^{s+h}f(\mu (t,x),u(t,x))dt \leq \frac {3}{2} \frac {C_4}{m}\int _s^{s+h} \int |u(t,x)|^2 d \mu (t,x)dt + C_5 \int _s^{s+h} \int |x-\bar {x}|^2 d \mu (t,x) dt. \end{align}

On the other hand, by the strict dissipativity, we obtain

(4.21) \begin{align} \int _s^{s+h}f(\mu (t,x),u(t,x))dt \geq &\, C_D \int _s^{s+h} \int |x-\bar {x}|^2 + | u(t,x)|^2 d\mu (t,x) dt \nonumber \\ \geq &\, C_D \int _s^{s+h} \int | u(t,x)|^2 d\mu (t,x) dt. \end{align}

By equation (4.20)–(4.21), we conclude that

\begin{align*} \int _s^{s+h} \int | u(t,x)|^2 d\mu (t,x) dt \leq \frac {3}{2} \frac {C_4}{m C_D}\int _s^{s+h} \int |u(t,x)|^2 d \mu (t,x)dt + \frac {C_5}{C_D} \int _s^{s+h} \int |x-\bar {x}|^2 d \mu (t,x) dt. \end{align*}

Set

(4.22) \begin{equation} m = \max \left \{2, \frac {2C_4}{C_D}\right \}, \end{equation}

and hence, $ \frac {3}{2} \frac {C_4}{m C_D} \leq \frac {3}{4}.$ Therefore,

\begin{align*} \int _s^{s+h} \int | u(t,x)|^2 d\mu (t,x) dt \leq &\, \frac {3}{4} \int _s^{s+h} \int |u(t,x)|^2 d \mu (t,x)dt + \frac {C_5}{C_D} \int _s^{s+h} \int |x-\bar {x}|^2 d \mu (t,x) dt. \end{align*}

Since $m$ is given, we know that the constant $C_5\gt 0$ depends only on $C_P,C_\Psi, $ and $C_L$ , respectively. This holds for any $h$ satisfying $s+mh\leq T$ . By Lebesgue’s differentiation theorem [19], we obtain $ \int | u(s,x)|^2 d\mu (s,x) \leq \frac {4C_5}{C_D} \int | x-\bar {x}|^2 d\mu (s,x) \mbox { for a.e. } t\in (0,T).$ Combining this estimate with the results of Theorem4.4, the proof is completed for $ C_3 = \frac {4C_2C_5}{C_D}.$

Remark 4.2. By Theorem 4.4 and Theorem 4.5, the function $f$ in (2.1) decreases exponentially in the sense that for any $t\in (0,T)$ ,

\begin{align*} f(\mu (t,x),u(t,x)) \leq &\,C_L \int |x-\bar {x}|^2 d\mu (t,x) + C_{\Psi } \int |u(t,x)|^2 d\mu (t,x) \\[3pt] \leq &\,(C_L C_2 + C_{\Psi } C_3) e^{-\alpha t} \int |x-\bar {x}|^2 d\mu _0(x). \end{align*}

Remark 4.3. In the proof, we adapt the technique in [17] by considering a new feedback control and introducing an adaptive parameter $m$ in 4.22. If the cost function in equation (2.1) is of quadratic form,

\begin{align*} f(\mu (t,x),u(t,x))=\int |x-\bar {x}|^2 d\mu (t,x) + \int |u(t,x)|^2 d\mu (t,x), \end{align*}

then we have $C_\Psi = 1$ , $C_L=1$ and $C_D=1$ . It follows that $C_4=1$ and $m=2$ .

Remark 4.4. The exponential turnpike property for the optimal control problem of the $N$ -particles system (2.5) can also be proved by considering the feedback control

\begin{align*} \tilde {u}_N(t,\tilde {x}_i(t)) = \left \{ \begin {array}{ll} u_N(t,\tilde {x}_i(t)), & \qquad t\in [0,s) \\[3pt] \dfrac {1}{m}u_N(t_1,\tilde {x}_i(t))-\dfrac {m-1}{m}\dfrac {1}{N}\sum _{j=1}^NP(\tilde {x}_i(t)-\tilde {x}_j(t)) & \qquad t\in [s,s+mh] \\[4mm] u_N(t_2,\tilde {x}_i(t)), & \qquad t\in (s+mh,T], \end {array} \right . \end{align*}

where $t_1$ and $t_2$ are taken as those in the proof of Theorem 4.5.

5. Conclusion

In this work, we prove the exponential turnpike property for optimal control problems of both particle systems and their mean-field limit. The main assumptions include the strict dissipativity of the cost function and the Lipschitz property of the interaction function. Compared to the previous work [27] in this direction, our main contribution is a more quantitative exponential estimate for both the optimal solution and the optimal control. More specifically, for the $N$ -particles system, we prove the exponential decay property of the optimal solution by employing a feedback control and basic estimates. Then, by considering the limit $N\rightarrow \infty$ , we establish the same property at the mean-field level. At last, we design a novel feedback control to prove the exponential decay property for the optimal control. Possible future work includes the extension to the following cases: (1) second-order models in the microscopic level and (2) other types of cost function (e.g. $L^1$ -regularisation for the control).