Fractional interpolation inequality and radially symmetric ground states

Abstract

In this paper, we establish a new fractional interpolation inequality for radially symmetric measurable functions on the whole space $R^{N}$ and a new compact imbedding result about radially symmetric measurable functions. We show that the best constant in the new interpolation inequality can be achieved by a radially symmetric function. As applications of this compactness result, we study the existence of ground states of the nonlinear fractional Schrödinger equation on the whole space $R^{N}$. We also prove an existence result of standing waves and prove their orbital stability.

1. Introduction

Recently, a great attention has been focussed on the study of inhomogeneous fractional nonlinear Schrödinger equation

(1.1)\begin{equation} \left\{\begin{array}{@{}ll} i\dot u-(-\Delta)^{\alpha} u-m|x|^{\sigma} u+\varepsilon |x|^{\gamma}|u|^{p-1}u=0, & x\in R^{N}, \ t>0\\ u|_{t=0}=u_0\in H^{\alpha}(R^{N}), & N\geq 2, \ \alpha\in (0,1), \ p>1, \end{array}\right. \end{equation}

where $m>0$ and $\varepsilon >0$ are real physical constants, $\sigma$ and $\gamma$ are real constants, $1< p<\frac {N+2\alpha }{(N-2\alpha )_+}$, and this equation comes from various physical contexts in the description of nonlinear waves such as propagation of a laser beam and plasma waves. When $\sigma =2$, the potential term is often called the harmonic potential and occurs in the condensed states. When $\alpha =1$, there is a strong physical background related to the study of (1.1) and one may prefer to the papers [3, 34, 37]. When $\sigma =0=\gamma$, to understand the ground state of (1.1) (see proposition 3.1 in [14] or [33]), one may use the well-known fractional Gagliardo–Nirenberg inequality that for some $\theta \in (0,1)$ such that $\frac 1{q}=\frac 12-\frac {\theta \alpha }{N}$, it holds that

\[ \|u\|_{q}\leq C \|u\|_2^{\theta}\|u\|_{\dot{H}^{\alpha}}^{1-\theta} \]

(see proposition A.3 in [35]). When $\sigma =-1$, a new fractional Gagliardo–Nirenberg inequality has been obtained in [20] to prove the existence of the ground state. In the above papers, the Schwartz symmetrization method (so called re-arrangement argument) has been used. However, for $\sigma >0$ and $m>0$, one can not directly use the re-arrangement argument to get the ground state to (1.1). The natural question is if one may get the ground state in broad class of the powers $\sigma,\gamma >0$ without using re-arrangement argument, which is one subject of this paper. In [25], the author has found the ground state in the case when $\sigma =2$ and $\alpha \in \left (\frac 12,\frac {N}{2}\right )$ by restricting the working space to the class of radially functions. In case when $m=0$, the ground states have been obtained [26]. The range of $\alpha$ also plays a crucial role in the existence result of ground states. In this paper, we first set up a new fractional interpolation inequality related to (1.1) and prove that the best constant in the new inequality can be obtained by a radially measurable solution function. We then get the ground states of the fractional Schrödinger equation on the whole space. For $\alpha \in (0,\frac 12]$, the power of the nonlinearity will be restricted into $p\in \left (1,\frac {N}{N-\alpha }\right )$ and the compactness result about radially symmetric functions obtained by Lions [23] will be used. Even in this case, we need to have more conditions about $\gamma$ and $\sigma$. The precise results will be stated (see theorems 1.2 and 1.3 below) after we briefly report some related results obtained by others. In the recent very interesting paper [2], the authors have introduced the ODE approach to nonlinear equations with the fractional Laplacian and their approach may be useful to the problem (1.1).

In [5], for $\alpha =1$ in (1.1), Chen and Guo have given a criterion for the global existence of solutions of the corresponding Cauchy problem for an inhomogeneous nonlinear Schrödinger equation with harmonic potential:

(1.2)\begin{equation} \left\{\begin{array}{@{}ll} i\psi_t={-}\Delta\psi+|x|^{2} \psi-|x|^{b}|\psi|^{p-1}\psi, & b>0,\\ \psi(x,0)=\psi_0(x), & \end{array}\right. \end{equation}

with radial initial data

\[ \psi_0 \in \widetilde{\Sigma}:=\{u\in H^{1}(R^{N}): u\ is\ radial\ and \ \int_{R^{N}}|x|^{2}|u(x)|^{2}\,{\rm d}\kern0.06em x<\infty\}. \]

Their result may be stated as the following. Let $b>0, N>1+\frac {b}{2},1+\frac {4+2b}{N}< p<\frac {N+2}{N-2}+\frac {2b}{N-1}$ $(1+\frac {4+2b}{N}< p<\infty, if N\leq 2), A=\frac {N(p-1)-2b}{2},B=\frac {2(p-1)-(N(p-1)-2b)}{2}$ and let $V(\lambda )=\left (\frac {A-2}{p+1-A}\right )^\frac{A-2}{2B}||Q||^\frac{p-1}{B}_{L^{2}}\lambda ^{-\frac {A-2}{2B}}$, where $Q$ is the ground state solution to the equation

\[ -\Delta Q+Q-|x|^{b}|Q|^{p-1}Q=0. \]

Define $\mathscr {S}=\{\phi \in \widetilde {\Sigma }:||\phi ||_{L^{2}}< V(||\nabla \phi ||^{2}_{L^{2}}+||x\phi ||^{2}_{L^{2}})\}$. If $\varphi _0\in \mathscr {S}$, then the corresponding solution to (1.2) is global in time and uniformly bounded in $\widetilde {\Sigma }$. Furthermore, $\mathscr {S}$ is unbounded in $\widetilde {\Sigma }$. The proof relies on a Gagliardo–Nirenberg inequality with best constant and an energy method. Moreover, a similar result holds true for the critical power $p=1+\frac {4+2b}{N}$, where the assumption $\varphi _0\in \mathscr {S}$ is replaced with $\varphi _0\in \widetilde {\Sigma }$ and $||\varphi _0||_{L^{2}}<||Q||_{L^{2}}$. In addition, this criterion is sharp in the following sense. For any $\lambda >0$ and $c\in \mathbb {C}$ with $|c|\geq 1$, if $\varphi _0(x)= c\lambda ^{\frac {N}{2}}Q(\lambda x)$, then $\varphi _0\in \widetilde {\Sigma }, ||\varphi _0||_{L^{2}}\geq ||Q||_{L^{2}}$ and the corresponding solution to (1.2) blows up in finite time. See also [21, 34] for related results. For $\sigma =-s$ and $\gamma =-t$ with $0\leq t<2s$, via a detailed study of a fractional version Hardy–Sobolev–Maz'ya inequality, with particular attention to the optimality of the constants involved, Mallick [28] has studied the existence of positive solutions of the nonlinear equation involving fractional powers of the Laplacian with cylindrical potentials. Moreover, he has also considered the symmetric and asymptotic properties of the positive solution. The study of the problem (1.1) is also closely related to weighted Hardy type inequalities [13, 20]. See also [7, 9, 15–19, 24, 29, 32] for related results of nonlocal problems.

We use the following notations: $A\lesssim B$ denotes an estimate of the form $A\leqslant CB$ for some absolute constant $C>0$; for $q\geq 1$, $L^{q}:=L^{q}({R}^{N})$ is the Lebesgue space endowed with the usual norm $\|\,\cdot \,\|_q:=\|\,\cdot \,\|_{L^{q}}$; $\|\,\cdot \,\|:=\|\,\cdot \,\|_2$, and $\|(-\Delta )^\frac{\alpha }{2}\cdot \|=\|\cdot \|_{\dot H^{\alpha }}$. We also use the brief notation that $\int \cdot {\rm d}\kern0.06em x:=\int _{R^{N}} \cdot {\rm d}\kern0.06em x$.

Recall that for $s\in (0,1)$, the fractional Hilbert space $H^{s}=H^{s}(R^{N})$ is defined by [1, 6, 11, 12, 30]

\[ H^{s}(R^{N})=\{u\in L^{2}(R^{N}); (1+|\xi|^{2})^{s/2}\mathcal{F}(u(\xi))\in L^{2}(R^{N})\} \]

where $\mathcal {F}(u)$ denotes the Fourier transformation of $u$, with norm

\[ \|u\|_{H^{s}}=\sqrt{\int_{R^{N}}(1+|\xi|^{2})^{s}|\mathcal{F}(u(\xi))|^{2}\,{\rm d}\xi}. \]

We also denote by

\[ \|(-\Delta)^\frac{s}{2} u\|^{2}=\int_{R^{N}}|\xi|^{2s}|\mathcal{F}(u(\xi))|^{2}\,{\rm d}\xi, \quad u\in C_0^{\infty}(R^{N}). \]

We introduce, for $\sigma \in (0,2)$, $\alpha \in (0,1)$, a fractional Sobolev space $D^{\alpha }$ with the norm

\[ \|u\|_{D^{\alpha}}:=\left(\||x|^{\frac\sigma 2}u\|^{2}+\|(-\Delta)^\frac\alpha 2 u\|^{2}\right)^\frac 12, \]

such that under this norm $D^{\alpha }$ is the completion of $C_0^{\infty }(R^{N})$. We also let $\dot H^{\alpha }$ be the completion of $C_0^{\infty }(R^{N})$ under the usual norm $\|(-\Delta )^\frac\alpha 2 u\|$. For $\alpha >1$, we may use Fourier transform to define $H^{\alpha }$ and we refer to [8]. Then we may define $D^{\alpha }$ as above.

For $Z$ being any space of functions on $R^{N}$, we denote by $Z_{rd}$ the space of radial functions in $Z$. Here and hereafter, we denote, for $\phi \in D_{rd}^{\alpha },$ the mass and energy functionals for (1.1):

\[ M(\phi)=\int |\phi|^{2}\,{\rm d}\kern0.06em x:= \|\phi\|^{2}, \]

and

\[ E(\phi)=\frac{1}{2}\|(-\Delta)^\frac\alpha 2 \phi\|^{2} +\frac m2\int |x|^{\sigma}|\phi|^{2}\,{\rm d}\kern0.06em x -\frac{1}{p+1}\int |x|^{\gamma}|\phi|^{p+1}\,{\rm d}\kern0.06em x, \]

where the integration is over the whole space $R^{N}$. We remark that the ground state to (1.1) may be considered as the mountain pass critical point of the energy functional $E(\cdot )$ on the working space $D_{rd}^{\alpha }$ [10].

We define the functional

\[ J(u):=\frac{\|(-\Delta)^\frac\alpha 2u\|^{B}\||x|^{\frac\sigma 2}u\|^{A}}{\int|x|^{\gamma}|u|^{1+p}\,{\rm d}\kern0.06em x} \]

on the space $D_{rd}^{\alpha }$, where

\begin{align*} & A:=(p+1)\theta,\\ & B:=(p+1)(1-\theta), \end{align*}

where $\theta =\frac {N+\sigma -\frac {2N+2\gamma }{p+1}}{2\alpha +\sigma }$.

We now consider the minimization problem

(1.3)\begin{equation} d:=\inf_{0\neq \phi \in D^{\alpha}_{rd}}\left\{ J(u); u\in D^{\alpha}_{rd}, u\neq 0 \right\}. \end{equation}

There is no previous result about this problem with $\sigma >0$. We now set up the interpolation inequality as below.

Proposition 1.1

1. (1). Let $\alpha \in (0,\frac 12]$, $1< p<\frac {N}{N-\alpha }$, $\gamma '=\frac {\gamma }{p+1}>0$, and $\sigma >0$ with $\frac {2\alpha }{(1-p)N+\alpha (p+1)}\gamma \leq \sigma$. Then

   \[ \||x|^{\gamma'}u\|_{p+1}\lesssim\||x|^{\frac\sigma 2}u\|^{1-\theta}\|u\|_{\dot H^{\alpha}}^{\theta}, \]
   for any $u\in D_{rd}^{\alpha }({R^{N}})$.

2. (2). Let $\alpha \in \left (\frac 12,\frac {N}{2}\right )$, $\frac {N+2\gamma -2\alpha -4\sigma }{N-2\alpha }\leq p\leq \frac {2(N+\gamma )}{N-2\alpha }-1$, and $\theta :=\frac {N+\sigma -\frac {2N}{p+1}-2\gamma '}{2\alpha +\sigma }.$ Then

   \[ \||x|^{\gamma'}u\|_{p+1}\lesssim\||x|^{\frac\sigma 2}u\|^{1-\theta}\|u\|_{\dot H^{\alpha}}^{\theta}, \]
   for any $u\in D_{rd}^{\alpha }({R^{N}})$.

For $\alpha \in (0,1)$, a more general result than proposition 1.1 is proven in theorem 1.1 in [31]. Since we have different purpose, we shall present a simpler and direct proof of proposition 1.1 for completeness. We now explain why $\theta$ can be chosen in this way. So we do the dimension analysis. Note that for $p>1$, $\gamma '\geq 0$, $a>0,b>0$, we let the scaling $\psi = u_{a,b}:= au\left (\frac {x}{b}\right )$ with $\frac {x}{b}=z$:

\begin{align*} \||\cdot|^{\gamma'}u_{a,b}\|^{p+1}_{L^{p+1}}& =\int|x|^{\gamma'(p+1) }u^{p+1}_{a,b}(x)\,{\rm d}\kern0.06em x\\ & =a^{p+1}\int{|x|^{\gamma'(p+1)}u^{p+1}\left(\frac{x}{b}\right){\rm d}\kern0.06em x}\\ & =a^{p+1} b^{N+\gamma'(p+1) }\int{|x|^{\gamma'(p+1) }u^{p+1}(z)\,{\rm d}z}\\ & =a^{p+1} b^{N+\gamma'(p+1)}\||\cdot|^{\gamma'}u\|^{p+1}_{L^{p+1}}. \end{align*}

Note that for $p=1$ and $\gamma '=\frac \sigma 2$, we have

\[ \||\cdot|^{\frac{\sigma}{2}}u_{a,b}\|^{2}_{L^{2}}=a^{2} b^{N+\sigma}\||\cdot|^{\frac{\sigma}{2}}u\|^{2}_{L^{2}}. \]

We also have

\[ \|(-\Delta)^{\frac{\alpha}{2}}u_{a,b}\|^{2}_{L^{2}}=a^{2} b^{N-2\alpha}\|(-\Delta)^{\frac{\alpha}{2}}u\|^{2}_{L^{2}}. \]

If proposition 1.1 is true, then for any constant $c$, we have

\[ ab^{\frac{N}{p+1}+\gamma'}\leq c(ab^{\frac{N+\sigma}{2}})^{1-\theta}(ab^{\frac{N-2\alpha}{2}})^{\theta}. \]

By this we have

\[ b^{\frac{N}{p+1}+\gamma'}\leq cb^{\left(\frac{N+\sigma}{2}\right)(1-\theta)+\left(\frac{N}{2}-\alpha\right)\theta}, \]

which implies that

\[ 1\leq cb^{\left(\frac{N+\sigma}{2}\right)(1-\theta)+\left(\frac{N}{2}-\alpha\right)\theta-\frac{N}{p+1}-\gamma'}. \]

Then we get

\[ \left(\frac{N+\sigma}{2}\right)(1-\theta)+\left(\frac{N}{2}-\alpha\right)\theta-\frac{N}{p+1}-\gamma'=0, \]

which implies that

\[ \theta=\frac{N+\sigma-\frac{2N}{p+1}-2\gamma'}{2\alpha+\sigma}. \]

As the consequences, we may obtain some of our main results (and the very new part is when $\alpha \in \left (0,\frac 12\right ]$, which has not been treated before).

Theorem 1.2 Let $\gamma \geq 0$. Assume (1) $\alpha \in (0,\frac 12]$, $1< p<\frac {N}{N-\alpha }$, and $\frac {2\alpha }{(1-p)N+\alpha (p+1)} \gamma \leq \sigma$; or (2) $\alpha \in \left (\frac 12,\frac {N}{2}\right )$, and $\frac {2\gamma +2\alpha -N-4\sigma }{N-2\alpha }\leq p\leq \frac {2(N+\gamma )}{N-2\alpha }-1$. We have three conclusions below.

1. (1). There exists a positive constant $C(N,p,\gamma,\alpha )$, such that for any $u\in D^{\alpha }_{rd}$, it holds

   (1.4)\begin{equation} \int|x|^{\gamma}|u|^{1+p}\,{\rm d}\kern0.06em x\leq C(N,p,\gamma,\alpha)\||x|^\frac{\sigma}{2}u\|^{A}\|u\|_{\dot H^{\alpha}}^{B}. \end{equation}

2. (2). Moreover, if $\frac {2\gamma +2\alpha -N-4\sigma }{N-2\alpha }< p< \frac {2(N+\gamma )}{N-2\alpha }-1$, then the minimization problem

   (1.5)\begin{equation} \beta:=\inf\left\{J(u),\quad u\in D^{\alpha}_{rd}\right\} \end{equation}
   is attained in some $\psi \in D^{\alpha }_{rd}$ (that is, $\beta =(\int |x|^{\gamma }|\psi |^{p+1}\,{\rm d}\kern0.06em x)^{-1}$, $\||x|^\frac{\sigma }{2}\psi \|=\|\psi \|_{\dot H^{\alpha }}=1$) and $\psi$ satisfies
   (1.6)\begin{equation} B(-\Delta)^{\alpha}\psi+A|x|^{\sigma}\psi-\beta(p+1)|x|^{\gamma}\psi|\psi|^{p-1}=0. \end{equation}

3. (3). Furthermore,

   (1.7)\begin{equation} C(N,p,\gamma,\alpha)=\frac{1+p}{{A}}\left(\frac AB\right)^{\frac{B}2}\|\phi\|^{-(p-1)}, \end{equation}
   where $\phi$ is a ground state solution to the following equation on $R^{N}$:
   (1.8)\begin{equation} (-\Delta)^{\alpha} \phi +|x|^{\sigma}\phi -|x|^{\gamma}\phi|\phi|^{p-1}=0 ,\quad 0\neq \phi \in D^{\alpha}_{rd}. \end{equation}

Using this proposition, we shall prove that there is a ground state of (1.9) below.

Theorem 1.3 Take $\epsilon =1$, $\gamma \geq 0$, $\frac 12<\alpha <\frac N2$, and $\frac {2\gamma +2\alpha -N-4\sigma }{N-2\alpha }\leq p\leq \frac {2(N+\gamma )}{N-2\alpha }-1$. Then, there is a ground state solution to (1.1) in the following meaning

(1.9)\begin{equation} (-\Delta)^{\alpha} \phi +|x|^{\sigma}\phi -|x|^{\gamma}\phi|\phi|^{p-1}=0 ,\quad 0\neq \phi \in D_{rd}^{\alpha}. \end{equation}

To prove the result above, we need to set up a compactness result, which is parallel to the classical Sobolev injection (see [5, 23]). We state here such result in case $\frac 12<\alpha <\frac N2$.

Lemma 1.4 Let $\gamma \geq 0$, $\frac 12<\alpha <\frac N2$, and $\frac {2\gamma +2\alpha -N-4\sigma }{N-2\alpha }\leq p\leq \frac {2(N+\gamma )}{N-2\alpha }-1$. Then, the following injection

(1.10)\begin{equation} D^{\alpha}_{rd}(R^{N})\hookrightarrow\hookrightarrow\Sigma:=\{u\in L^{1}_{loc}(R^{n}); \int_{R^{n}} |u|^{p+1}|x|^{\gamma}\,{\rm d}\kern0.06em x<\infty\} \end{equation}

is compact, and we may simply write $\Sigma :=L^{p+1}(|x|^{\gamma }\,{\rm d}\kern0.06em x)$.

We may also show the existence of other standing states to (1.1) in the following way. We now define on $D^{\alpha }_{rd}$ the action for (1.1) by

\[ S(u)=E(u)+\frac 12 M(u) \]

and let $Q(u)=< S'(u),u>$. Set

\[ M=\{u\in D^{\alpha}_{rd}; u\not=0, Q(u)=0 \} \]

and

\[ m_0=\inf\{S(u),u\in {M}\}. \]

It is clear that when restricted on $M$, the action $S$ is simplified to

\[ S(u)=a\int |x|^{\gamma} |u|^{p+1}=a\int |(-\Delta)^\frac\alpha 2 u|^{2} +m|x|^{\sigma}|u|^{2}+|u|^{2}\,{\rm d}\kern0.06em x \]

where $a=\left (\frac 12-\frac {1}{p+1}\right )$. Hence $m_0\geq 0$.

We shall show in next section that

Lemma 1.5 $m_0=\inf \{S(u),u\in {M}\}>0.$

The proof of lemma 1.5 replies on theorem 1.2 and this result will play an important role in the argument of theorem 1.6 below.

Then, we can show the following result about the existence of standing waves and their orbital stability for (1.1).

Theorem 1.6 Take $\varepsilon =1, \gamma \geq 0$, $\alpha \in \left (\frac 12,\frac N2\right )$, and $\frac {2\gamma +2\alpha -N-4\sigma }{N-2\alpha }\leq p\leq \frac {2(N+\gamma )}{N-2\alpha }-1$. We have the following conclusions.

1. (1) There is a standing wave solution to (1.1) in the following sense

   \[ (-\Delta)^{\alpha}\phi+|x|^{\sigma}\phi+\phi-|x|^{\gamma}\phi|\phi|^{p-1}=0,\ 0\neq\phi\in D^{\alpha}_{rd},\ m_0=S(\phi). \]

2. (2) Let $B_1=(1-\theta _0)(q+1)+\frac {2\gamma }{N-2\alpha }$, where $q+1=p+1-\frac {2\gamma }{N-2\alpha }$ and $\theta _0=\frac {N}{\alpha }\left (\frac 12-\frac 1{q+1}\right )$. For $B_1<2$, this standing wave is orbitally stable.

As usual, in the orbitally stable part, we always suppose that the global existence of solutions to (1.1) with initial datum near to the standing waves.

The rest of the paper is organized as follows. In §2, we prove lemmas 1.4 and 1.5. We recall some useful inequalities and some tools like compactness result needed in the sequel. In §3, we prove proposition 1.1. In §4, we prove theorem 1.2. We prove theorem 1.3 in § 5. We establish the existence of standing states of (1.1) and the orbital stability in §6. Thus, we prove theorem 1.6 in the last section.

2. Preliminary results

In this section, we collect some well-known facts about properties about radially symmetric function on $R^{N}$. An estimate similar to Strauss's inequality [36] in the fractional case is as follows [8]:

Lemma 2.1 Let $N\geq 2$ and $\frac 12<\alpha <\frac N2$. Then, for any $u\in \dot H_{rd}^{\alpha }(R^{N})$,

(2.11)\begin{equation} \sup_{x\neq 0}|x|^{\frac N2-\alpha}|u(x)|\leq C(N,\alpha)\|(-\Delta)^\frac\alpha 2u\|, \end{equation}

where

\[ C(N,\alpha)=\left(\frac{\Gamma(2\alpha-1)\Gamma\left(\frac N2-\alpha\right)\Gamma\left(\frac N2\right)}{2^{2\alpha}\pi^{\frac N2}\Gamma^{2}(\alpha)\Gamma\left(\frac N2-1+\alpha\right)}\right)^\frac 12 \]

and $\Gamma$ is the Gamma function.

To facilitate the proofs of our latter results, we show that $\|u\|_{D^{\alpha }}$ is stronger than $\|u\|_{H^{\alpha }}$.

Proposition 2.2 Fix $\alpha >0$. There exists a uniform constant $c_1>0$ depending only on $\alpha$ and $n$ such that

(2.12)\begin{equation} c_1{\|u\|^{2}_{H^{\alpha}}}\leq {\|u\|^{2}_{D^{\alpha}}}, \end{equation}

for any $u\in D_{rd}^{\alpha }$.

Proof. We argue by contradiction. Assume that (2.12) is not true. Then there exists a sequence $(u_j)$ in $D_{rd}^{\alpha }$ such that $\|u_j\|_{H^{\alpha }}=1, \|u_j\|_{D^{\alpha }}\rightarrow 0$. We may assume that $u_j \rightarrow u$ in $L^{2}_{loc}$, $u_j \rightarrow u \ a.e.$, $(-\Delta )^\frac\alpha 2 u_j \rightarrow 0$ in $L^{2}$. Since

\[ 1=\|u_j\|^{2}_{H^{\alpha}}=\|(-\Delta)^{\frac{\alpha}{2}}u_j\|^{2}_{L^{2}}+\|{u_j}\|^{2}_{L^{2}}, \]

we have

(2.13)\begin{equation} 1+{\circ}(1)= \|{u_j}\|^{2}_{L^{2}}. \end{equation}

For any $R>0$, we know that $\|{u_j}\|^{2}_{L^{2}}=\|u_j\|^{2}_{L^{2}(B_R)}+\|u_j\|^{2}_{L^{2}(B^{c}_R)}$ and

\[ R^{\sigma} \|u_j\|^{2}_{L^{2}(B^{c}_R)} \leq {{\||x|^{\frac{\sigma}{2}}}{u_j}\|}^{2}_{L^{2}(B^{c}_R)}={\circ}(1). \]

This implies that $u=0$ almost everywhere and

\[ {\|u_j\|}_{L^{2}(B^{c}_R)}\leq{\circ}(1)\left(\frac{1}{R}\right)^\frac{\sigma}{2}. \]

By the local compactness imbedding theorem we conclude that $\|{u_j}\|_{L^{2}_{loc}}\rightarrow 0.$ Then we have $\|u_j\|^{2}_{L^{2}(B_R)}+\|u_j\|^{2}_{L^{2}(B^{c}_R)}= \circ (1)$, which leads to a contradiction with the equality (2.13). This completes the proof.

Taking into account proposition 2.2, we get $D_{rd}^{\alpha } \hookrightarrow H^{\alpha } \hookrightarrow L^{q}$ for any $q\in \left [2,\frac {2N}{N-2\alpha }\right ]$.

Lemma 2.3 Let $N\geq 2$, $p\in (1,\infty )$ and $\alpha \in (0,1)$. Then $D_{rd}^{\alpha } \hookrightarrow H^{\alpha } \hookrightarrow L^{q}$ for any $q\in \left [2,\frac {2N}{N-2\alpha }\right ]$.

This result will be useful in the proof of theorem 1.2.

We may prove lemma 1.4 by using the imbedding $D_{rd}^{\alpha } \hookrightarrow H^{\alpha }$. However, we prefer to give a direct proof below.

Proof. (Proof of lemma 1.4). Recall that

\[ \Sigma_{rd}=\{u\in L^{1+p}(|x|^{\gamma}\,{\rm d}\kern0.06em x); u=\text{radial and measurable in}\ R^{N} \}, \]

which is a Banach space endowed with the norm

\[ \|u\|_{\Sigma_{rd}}:=\left(\int|x|^{\gamma}|u(x)|^{1+p}\,{\rm d}\kern0.06em x\right)^\frac 1{1+p}. \]

We divide the proof into three steps. Take ${(u_j)}$ a bounded sequence of ${D_{rd}^{\alpha }}$ and let $\varepsilon >0$. We may assume that $u_j$ converges weakly in $D_{rd}^{\alpha }$ and almost everywhere to zero. We write

\[ \int |x|^{\gamma}|u_j|^{p+1}\,{\rm d}\kern0.06em x=\left(\int_{|x|\leq\varepsilon}+\int_{\varepsilon \leq |x|\leq \frac{1}{\varepsilon}}+\int_{|x|\geq \frac{1}{\varepsilon}}\right) |x|^{\gamma} |u_j|^{p+1}\,{\rm d}\kern0.06em x:=I+II+III. \]

We should only concern $\gamma > 0$. Lemma 2.1 will play an important role below.

Step I. We consider the integral in the region $|x|\leq \varepsilon.$

Since $|u(x)|\lesssim |x|^{\alpha -\frac {N}{2}}\|(-\Delta )^{\frac {\alpha }{2}} u\|$ for $x\neq 0$ and $\gamma +\left (\alpha -\frac {N}{2}\right )(p+1)+N > 0$, that is,

(2.14)\begin{equation} p+1 < \frac{N+\gamma}{\frac{N}{2}-\alpha}=\frac{2N+2\gamma}{N-2\alpha}, \end{equation}

we have

\begin{align*} I& =\int_{|x|\leq\varepsilon}|x|^{\gamma} |u_j|^{p+1}\,{\rm d}\kern0.06em x \\ & \lesssim \int_{|x|\leq\varepsilon}|x|^{\gamma} |x|^{\left(\alpha-\frac{N}{2}\right)(p+1)}\|(-\Delta)^{\frac{\alpha}{2}} u_j\|^{p+1}\\ & \lesssim {\int_0}^{\varepsilon} r^{\gamma+\left(\alpha-\frac{N}{2}\right)(p+1)+N-1}\,{\rm d}r \|(-\Delta)^{\frac{\alpha}{2}}u_j\|^{p+1}\\ & \lesssim \frac{1}{\gamma+\left(\alpha-\frac{N}{2}\right)(p+1)+N} \ r^{\gamma+\left(\alpha-\frac{N}{2}\right)(p+1)+N}|_0^{\varepsilon}\ \|(-\Delta)^{\frac{\alpha}{2}}u_j\|^{p+1}\\ & \lesssim \varepsilon^{\gamma+\left(\alpha-\frac{N}{2}\right)(p+1)+N}\|(-\Delta)^{\frac{\alpha}{2}}u_j\|^{p+1}\\ & \lesssim \varepsilon^{\gamma+\left(\alpha-\frac{N}{2}\right)(p+1)+N}\to 0, \end{align*}

as $\epsilon \to 0$.

Step II. We consider the integral in the region $O_\epsilon :=\{\varepsilon \leq |x|\leq \frac {1}{\varepsilon }\}.$

Since $\int _{O_\epsilon }|u_j|^{2}dx\rightarrow 0$ and $|x|^{2\sigma }$ is bounded in $O_\epsilon$, we obtain

\[ \int_{O_\epsilon} |x|^{2\sigma}|u_j|^{2}\,{\rm d}\kern0.06em x\rightarrow 0. \]

Then,

\begin{align*} II& =\int_{\varepsilon\leq |x|\leq\frac{1}{\varepsilon}}|x|^{\gamma}|u_j|^{p+1}\,{\rm d}\kern0.06em x\\ & =\int(|x|^{\frac{N}{2}-\alpha}|u_j|)^{p-1}|x|^{\gamma-(p-1)\left(\frac{N}{2}-\alpha\right)}|u_j|^{2}\,{\rm d}\kern0.06em x\\ & \leq C\|u_j\|_{\dot H^{\alpha}}^{p-1} \int|x|^{\gamma-(p-1)\left(\frac{N}{2}-\alpha\right)}|u_j|^{2}\,{\rm d}\kern0.06em x\\ & \leq C\|u_j\|_{\dot H^{\alpha}}^{p-1}\varepsilon^{\gamma-(p-1)\left(\frac{N}{2}-\alpha\right)-2\sigma}\int|x|^{2\sigma}|u_j|^{2}\,{\rm d}\kern0.06em x\\ & \leq C\varepsilon^{\gamma-(p-1)\left(\frac{N}{2}-\alpha\right)-2\sigma}\int|x|^{2\sigma}|u_j|^{2}\,{\rm d}\kern0.06em x\to 0 \end{align*}

as $j$ tends to infinity.

Step III. We consider the integral in the region $|x|\geq \frac {1}{\varepsilon }.$

For $\gamma <(p-1)\left (\frac {N}{2}-\alpha \right )+2\sigma$, we have

\begin{align*} III=\int |x|^{\gamma}|u_j|^{p+1}\,{\rm d}\kern0.06em x& =\int \left(|x|^{\frac{N}{2}-\alpha}|u_j|\right)^{p-1} |x|^{\gamma-(p-1)\left(\frac{N}{2}-\alpha\right)}|u_j|^{2}\,{\rm d}\kern0.06em x\\ & \lesssim c{\|(-\Delta)^{\frac{\alpha}{2}}u_j\|}^{p-1}\int|x|^{\gamma-(p-1)\left(\frac{N}{2}-\alpha\right)}|u_j|^{2}\,{\rm d}\kern0.06em x\\ & \lesssim c{\|(-\Delta)^{\frac{\alpha}{2}}u_j\|}^{p-1}\varepsilon^{-\gamma+(p-1)\left(\frac{N}{2}-\alpha\right)+2\sigma}\int |x|^{2\sigma}|u_j|^{2}\,{\rm d}\kern0.06em x\\ & \lesssim c\varepsilon^{-\gamma+(p-1)\left(\frac{N}{2}-\alpha\right)+2\sigma}\int |x|^{2\sigma}|u_j|^{2}\,{\rm d}\kern0.06em x\\ & \lesssim c\varepsilon^{-\gamma+(p-1)\left(\frac{N}{2}-\alpha\right)+2\sigma}\to 0 \end{align*}

as $\epsilon \to 0$. Thus, the proof is completed.

We now give the proof of lemma 1.5.

Proof. By theorem 1.2, for $u\in {M}$, we have

\begin{align*} |||x|^\frac{\gamma}{p+1}u||_{p+1}& \leq C||(-\Delta)^\frac{\alpha}{2}u||^\frac{A}{p+1}|||x|^\frac{\sigma}{2}u||^\frac{B}{p+1}\\ & \leq C_1(||(-\Delta)^\frac{\alpha}{2}u||^{2}+|||x|^\frac{\sigma}{2}u||^{2})^\frac{1}{2}, \end{align*}

and then

\begin{align*} S(u)|_{\textrm{M}}& =a\int{|x|^{\gamma}|u|^{p+1}}\\ & =a\int|(-\Delta)^\frac{\alpha}{2}u|^{2}\,{\rm d}\kern0.06em x+\int||x|^\frac{\sigma}{2}u|^{2}\,{\rm d}\kern0.06em x+\int u^{2}\,{\rm d}\kern0.06em x\\ & \geq a\int|(-\Delta)^\frac{\alpha}{2}u|^{2}\,{\rm d}\kern0.06em x+\int||x|^\frac{\sigma}{2}u|^{2}\,{\rm d}\kern0.06em x\\ & \geq\frac{1}{C_1}a\left(\int{|x|^{\gamma}|u|^{p+1}}\right)^\frac{2}{p+1}. \end{align*}

Hence,

\[ \left(\int|x|^{\gamma}|u|^{p+1}\,{\rm d}\kern0.06em x\right)^{1-\frac{2}{p+1}}\geq\frac{1}{C_1}a, \]

which implies that

\[ m_0\geq a\left(\frac{a}{C_1}\right)^\frac{1}{1-\frac{2}{p+1}}>0. \]

3. Proof of proposition 1.1

We now prove proposition 1.1 via arguing by contradiction.

Proof. (proposition 1.1). We prove the result by dividing $\alpha$ into two cases: case (1): $\alpha \in (0,1/2]$ and case (2): $\alpha \in (1/2,N/2)$.

Case (1): In this case we have

\[ \alpha\in (0,1/2], \ 1< p<\frac{N}{N-\alpha},\quad \text{and}\quad \frac{2\alpha}{(1-p)N+\alpha (p+1)}\gamma\leq \sigma. \]

If the conclusion of proposition 1.1 is not true, up to a scaling, there exists a sequence $(u_j)\subset D_{rd}^{\alpha }$ such that $\||x|^{\frac {\gamma }{p+1}} u_j\|_{L^{p+1}}=1$, $\|(-\Delta )^{\frac {\alpha }{2}}u_j\|_{L^{2}}=1$, and

(3.15)\begin{equation} \||x|^{\frac{\sigma}{2}}u_j\|_{L^{2}}\rightarrow 0. \end{equation}

By $D_{rd}^{\alpha } \hookrightarrow H^{\alpha }$, we may assume that $\|u_j\|_{H^{\alpha }}\leq C$ for some constant $C>0$. So for any $\epsilon >0$, we have $\||x|^{\frac {\gamma }{p+1}}u_j\|_{L^{p+1}(|x|\leq 1/\epsilon )}\rightarrow 0$. Let $\lambda =\frac {\frac {2N}{N-\alpha }-2p}{\frac {2N}{N-\alpha }-(p+1)}$. Let $B^{R}=\{|x|\geq R\}$ for $R=1/\epsilon$. Then, applying Cauchy–Schwartz inequality for $u=u_j$ and omitting the integration domain for moment,

\begin{align*} \||x|^{\frac{\gamma}{p+1}}u\|_{L^{p+1}(|x|\geq 1/\epsilon)}^{p+1}& = \int_{B^{R}} |x|^{\gamma-\frac{\sigma}{2}}|u|^{p}\cdot |x|^{\frac{\sigma}{2}}|u|\\ & \leq \left(\int_{B^{R}} |x|^{2\gamma-\sigma}|u|^{2p}\right)^{1/2}\left(\int_{B^{R}} |x|^{\sigma}|u|^{2}\right)^{1/2}. \end{align*}

Note that $2p=\lambda (p+1)+(1-\lambda )\frac {2N}{N-\alpha }$,

\[ \int |x|^{2\gamma-\sigma}|u|^{2p}=\int |x|^{2\gamma-\sigma}|u|^{\lambda (p+1)+(1-\lambda)\frac{2N}{N-\alpha}} \]

which is bounded by

\[ \left(\int |x|^{(2\gamma-\sigma)\frac{1}{\lambda}}|u|^{p+1}\right)^{\lambda}\left(\int |u|^{\frac{2N}{N-\alpha}}\right)^{1-\lambda} \]

and further bounded by

\[ \left(\int |x|^{\gamma}|u|^{p+1}\right)^{\lambda}\left(\|u\|_{\dot{H}^{\alpha}}^{\frac{2N}{N-\alpha}}\right)^{1-\lambda}. \]

In the last step we have used the assumption that

\[ \frac{2\alpha}{(1-p)N+\alpha (p+1)}\gamma\leq \sigma, \]

which is equivalent to $(2\gamma -\sigma )\lambda ^{-1}\leq \gamma$.

Combining all these together, we get for some uniform constant $C>0$,

\[ \||x|^{\frac{\gamma}{p+1}}u_j\|_{L^{p+1}(|x|\geq 1/\epsilon)}^{p+1}\leq C\left(\int_{B^{R}} |x|^{\sigma}|u_j|^{2}\right)^{1/2}\to 0, \]

which gives a contradiction to the assumption that

\[ \||x|^{\frac{\gamma}{p+1}} u_j\|_{L^{p+1}}=1. \]

Case (2): Again we argue by contradiction and take the sequence $(u_j)$ as above. Note that

\[ 1=\int |x|^{\gamma} |u_j|^{p+1} = \int \left(|x|^{\frac{N}{2}-\alpha}|u_j|\right)^{\frac{2\gamma}{N-2\alpha}}|u_j|^{p+1-\frac{2\gamma}{N-2\alpha}}. \]

Let $q+1=p+1-\frac {2\gamma }{N-2\alpha }$, By lemma 2.1, we know that

\[ 1\leq C\int |u_j|^{q+1}. \]

Using the well-known fractional Gagliardo–Nirenberg inequality (proposition A.3 in [35]) that

\[ \|u_j\|_{q+1}\leq C \|u_j\|_2^{\theta_0}\|u_j\|_{\dot{H}^{\alpha}}^{1-\theta_0}, \]

where $\theta _0=\frac {N}{\alpha }\left (\frac 12-\frac 1{q+1}\right )$, we get

\[ \|u_j\|_{L^{2}}\geq c > 0. \]

That is,

(3.16)\begin{equation} \|u_j\|_{L^{2}(B_\varepsilon)}+\|u_j\|_{L^{2}(B_\varepsilon^{c})}\geq c > 0. \end{equation}

However,

\[ \int_{B_\varepsilon^{c}} {|u_j(x)|^{2}\,{\rm d}\kern0.06em x}\leq \varepsilon^{-\sigma}\int_{B_\varepsilon^{c}} |x|^{\sigma}{|u_j(x)|^{2}\,{\rm d}\kern0.06em x}\leq \varepsilon^{-\sigma}\int_{R^{N}} |x|^{\sigma}{|u_j(x)|^{2}\,{\rm d}\kern0.06em x}\rightarrow 0. \]

For any $\varepsilon > 0$ small, $\|u_j\|_{L^{2}(B_\varepsilon )}\rightarrow \|u\|_{L^{2}(B_\varepsilon )}= \circ (\varepsilon )$. Then

\[ \|u_j\|_{L^{2}(R^{N})}\rightarrow \|u\|_{L^{2}(B_\varepsilon)}={\circ}(\varepsilon), \]

which lead to a contradiction with (3.16) that $\|u_j\|_{L^{2}}\geq c$.

4. Proof of theorem 1.2

The proofs of theorem 1.2 in the two groups of assumptions are almost the same. So we present the full proof only in case $\alpha \in (1/2,N/2)$ (but in case (1), we need to use the compactness result theorem II.1 of [23]). We divide the proof into three parts.

A. Proof of the interpolation inequality (1.4):

First, using lemma 2.1, we get

\begin{align*} \int|x|^{\gamma}|u(x)|^{1+p}\,{\rm d}\kern0.06em x & =\int(|x|^{\frac N2-\alpha}|u(x)|)^{\frac{2\gamma}{N-2\alpha}}|u(x)|^{1+p-\frac{2\gamma}{N-2\alpha}}\\ & \lesssim\|u\|_{\dot H^{\alpha}}^{\frac{2\gamma}{N-2\alpha}}\int|u(x)|^{1+p-\frac{2\gamma}{N-2\alpha}}. \end{align*}

Now, thanks to lemma 1.1, it yields

\begin{align*} \int|x|^{\gamma}|u(x)|^{1+p}\,{\rm d}\kern0.06em x & \lesssim\|u\|_{\dot H^{\alpha}}^\frac{2\gamma}{N-2\alpha}\|u\|^{1+p-\frac{2\gamma}{N-2\alpha}}_{1+p-\frac{2\gamma}{N-2\alpha}}\\ & \lesssim\|u\|_{\dot H^{\alpha}}^\frac{2\gamma}{N-2\alpha}(\|u\|^{1-\theta}\|u\|_{\dot H^{\alpha}}^{\theta})^{1+p-\frac{2\gamma}{N-2\alpha}}\\ & \lesssim\|u\|^{(1-\theta)\left(1+p-\frac{2\gamma}{N-2\alpha}\right)}\|u\|_{\dot H^{\alpha}}^{\theta\left(1+p-\frac{2\gamma}{N-2\alpha}\right)+\frac{2\gamma}{N-2\alpha}}. \end{align*}

The proof of (1.4) is complete.

B. Proof of the equation (1.6):

Recall that $\beta =\frac {1}{C(N, p,\gamma,\alpha )}$ and

\[ J(u)=\frac{\|(-\Delta)^\frac\alpha 2u\|^{B}\||x|^{\frac\sigma 2}u\|^{A}}{\int|x|^{\gamma}|u|^{1+p}\,{\rm d}\kern0.06em x} \]

on $D_{rd}^{\alpha }$, where

\begin{align*} & A:=(p+1)\theta,\\ & B:=(p+1)(1-\theta), \end{align*}

where $\theta =\frac {N+\sigma -\frac {2N+2\gamma }{p+1}}{2\alpha +\sigma }$.

Using the definition of $\beta$ in (1.5), there exists a sequence $(u_j)$ in $D^{\alpha }_{rd}$ such that $J(u_j)\rightarrow \beta.$ Denoting for $a_j, b_j>0$ such that

\[ \|{u_j}_{a_j b_j}\|=1\quad \text{and}\quad\||x|^{\frac{\sigma}{2}}{u_j}_{a_j b_j}\|=1. \]

Let $v_j={u_j}_{a_jb_j}$, $J(u_j)=J(v_j)$ and $\|v_j\|_{\dot H^{\alpha }}=1, \||x|^{\frac {\sigma }{2}}v_j\|=1.$

It follows that

\[ J(v_j)=\frac{1}{\int{|x|^{\gamma} |v_j|^{p+1}}}\to \beta. \]

Then we may assume that $v_j\rightarrow v$ a.e., $v_j\rightharpoonup v$ in $D^{\alpha }_{rd}$.

Since the injection $D_{rd}^{\alpha }\hookrightarrow L^{p+1}(|x|^{\gamma }\,{\rm d}\kern0.06em x)$ is compact (by lemma 1.4), we obtain

\[ \int{|x|^{\gamma} |v_j|^{p+1}}\rightarrow \int{|x|^{\gamma} |v|^{p+1}}, \]

which means that

\[ J(v_j)\rightarrow J(v). \]

Hence

\[ J(v)=\beta. \]

The minimizer $\psi :=v$ satisfies the Euler–Lagrange equation,

\[ \frac{{\rm d}}{{\rm d}\varepsilon}J(\psi+\varepsilon\eta)|_{\varepsilon=0}=0,\quad \forall \eta\in C^{\infty}_0\cap D^{\alpha}_{rd}. \]

Since

\[ lnJ(\psi)=\frac{B}{2}ln\|(-\Delta)^{\frac{\alpha}{2}}\psi\|^{2}+\frac{A}{2}ln\||x|^{\frac{\sigma}{2}}\psi\|^{2}-ln\int{|x|^{\gamma} \psi^{p+1}}\,{\rm d}\kern0.06em x, \]

we have from

\[ \frac{{\rm d}}{{\rm d}\varepsilon}J(\psi+\varepsilon\eta)|_{\varepsilon=0}=0, \]

that for any $\eta \in D^{\alpha }_{rd}$,

\[ B\frac{\int{(-\Delta)^{\frac{\alpha}{2}}\psi(-\Delta)^{\frac{\sigma}{2}}\eta}\,{\rm d}\kern0.06em x}{\|(-\Delta)^{\frac{\alpha}{2}}\psi\|^{2}}+A\frac{\int{|x|^{\sigma}\psi\eta}\,{\rm d}\kern0.06em x} {\||x|^{\frac{\sigma}{2}}\psi\|^{2}}-(p+1)\frac{\int{|x|^{\gamma}\psi^{p}\eta}\,{\rm d}\kern0.06em x}{\int{|x|^{\gamma}\psi^{p+1}}\,{\rm d}\kern0.06em x}=0. \]

That is, $\psi$ satisfies

\[ \frac{B}{\|(-\Delta)^{\frac{\alpha}{2}}\psi\|^{2}}(-\Delta)^{\alpha}\psi+\frac{A}{\||x|^{\frac{\alpha}{2}}\psi\|^{2}}|x|^{\sigma} \psi-\frac{p+1} {\int{|x|^{\gamma}\psi^{p+1}}\,{\rm d}\kern0.06em x}|x|^{\gamma}\psi^{p}=0 \]

in the weak sense, i.e., $\psi$ satisfies (1.6) in the weak sense.

C. Proof of the equation (1.7):

We use the scaling property of the functional $J$. By the fact that

\[ B(-\Delta)^{\alpha}\psi+A|x|^{\sigma}\psi-\beta(p+1)|x|^{\gamma}|\psi|^{p-1}\psi=0, \]

letting

\[ b=\left(\frac AB\right)^\frac 1{2\alpha}\quad\text{and}\quad a=\left((\frac{A}B)^\frac\gamma{2\alpha}\frac A{\beta(1+p)}\right)^\frac 1{p-1}. \]

and $\psi =\phi ^{a,b}:=a\phi (bx)$, we have

\[ Aa\left(\frac{A}{B}b^{2\alpha}(-\Delta)^{\alpha}\phi+|x|^{\sigma}\phi-\frac{A}{B}(p+1)a^{p-1}b^{-\gamma}|x|^{\gamma}\phi|\phi|^{p-1}\right)=0. \]

It follows that

\[ (-\Delta)^{\alpha}\phi+|x|^{\sigma}\phi-|x|^{\gamma}\phi|\phi|^{p-1}=0. \]

Since

\[ \|\psi\|=1=ab^{-\frac N2}\|\phi\|, \]

we get

\[ \beta=\frac{A}{1+p}(\frac AB)^{-\frac B2}\|\phi\|^{p-1}, \]

which is the desired equation (1.7). The proof is complete.

5. Proof of theorem 1.3

The plan of proving theorem 1.3 is to use the Nehari method (see [10, 25, 26]). We denote by $X:=D_{rd}^{\alpha }$.

Proof. We now consider the existence of the ground state $\phi$, that is, it is the positive solution with minimal energy to

\[ (-\Delta)^{\alpha} \phi +|x|^{\sigma}\phi -|x|^{\gamma}|\phi|^{p-1}\phi=0 ,\quad 0\neq \phi \in X. \]

For this, we let

\[ K(u)=\frac{1}{2}\int(|(-\Delta)^{\frac{\alpha}{2}}u|^{2}+|x|^{\sigma} u^{2})\,{\rm d}\kern0.06em x-\frac{1}{p+1}\int |x|^{\gamma} u^{p+1}\,{\rm d}\kern0.06em x,\quad u\in X. \]

Note that

\[ K'(u)\varphi=\int \left((-\Delta)^{\frac{\sigma}{2}}u(-\Delta)^{\frac{\sigma}{2}}\varphi+u\varphi\right)\,{\rm d}\kern0.06em x-\int|x|^{\gamma}|u|^{p-1}u\varphi\,{\rm d}\kern0.06em x, \]

We define the Nehari functional

\[ N(u)=K'(u)u=\int \left(|(-\Delta)^{\frac{\sigma}{2}}u|^{2}+|x|^{\sigma} u^{2}\right)-\int|x|^{\gamma}|u|^{p+1}\,{\rm d}\kern0.06em x. \]

We define the Nehari manifold by

\[ \mathcal{N}_{rd}:=\{u\in X;\ {u\neq 0},\ {N(u)=0 }\}. \]

Define

\[ d=\inf \{K(u),\ u=0,\ u\in \mathcal{N}_{rd}\} \]

the depth of the potential well.

Claim 1: $d>0.$

For $u\in \mathcal {N}_{rd}$, via a use of proposition 1.1 (2) we have

\begin{align*} K(u)=\left(\frac{1}{2}-\frac{1}{p+1}\right)\|u\|^{2}_X& =\left(\frac{1}{2}-\frac{1}{p+1}\right)\int |x|^{\gamma} u^{p+1}\\ & \lesssim \||x|^{\sigma/2}u\|^{A}_2\|(-\Delta)^\frac{\alpha}{2}u\|^{B}_2\\ & \lesssim \|u\|^{A+B}_X. \end{align*}

Since $A+B>2$, we obtain $\|u\|_X\geq c>0$ for some uniform constant $c>0$. It follows that for $u\in \mathcal {N}_{rd}$,

\[ K(u)\geq \left(\frac{1}{2}-\frac{1}{p+1}\right)c^{2}>0, \]

which implies that

\[ d\geq\left(\frac{1}{2}-\frac{1}{p+1}\right)c^{2}>0. \]

Claim 2: There exists $u\in \mathcal {N}_{rd},$ such that $d=K(u)$.

To prove this, we may take a minimizing sequence $(u_j)\subset \mathcal {N}_{rd}, K(u_j)\rightarrow d.$ By this we may assume $\|u_j\|_X \leq c$ for some uniform constant $c>0$. Then using lemma 1.4, there exists a subsequence, still denoted by $(u_j)$ with the weak limit $u\in X$ such that $u_j\rightarrow u\ a.e.$ and $u_j\rightarrow u$ in $L^{p+1}(|x|^{\gamma }\,{\rm d}\kern0.06em x).$

On one hand,

\[ \underline{\lim}_jK(u_j)=\underline{\lim}_j\left(\frac{1}{2}-\frac{1}{p+1}\right)\|u_j\|^{2}_X\geq K(u)=\left(\frac{1}{2}-\frac{1}{p+1}\right)\|u\|^{2}_X. \]

On the other hand,

\[ K(u_j)=\left(\frac{1}{2}-\frac{1}{p+1}\right)\int |x|^{\gamma} |u_j|^{p+1}\,{\rm d}\kern0.06em x\rightarrow \left(\frac{1}{2}-\frac{1}{p+1}\right)\int |x|^{\gamma} |u|^{p+1}\,{\rm d}\kern0.06em x. \]

According to $K(u_j)\rightarrow d,$ we get

\[ \int |x|^{\gamma}|u|^{p+1}=\left(\frac{1}{2}-\frac{1}{p+1}\right)^{{-}1}d>0. \]

Then $u\neq 0$ and $N(u)\leq 0.$

If $N(u)<0$ and

\[ N(tu)=\frac{t^{2}}{2}\|u\|^{2}_{D_{rd}^{\alpha}}-\frac{t^{p+1}}{p+1}\int |u|^{p+1}>0 \]

for $t>0$ small, then we have $N(t_cu)=0$ for some $t_c\in (0,1)$. Then $t_cu\in \mathcal {N}_{rd}$ and $K(t_cu)\geq d$. However, by direct computation, we have

\[ K(t_cu)=\left(\frac{1}{2}-\frac{1}{p+1}\right)t_c^{p+1}\int |x|^{\gamma} u^{p+1}<\left(\frac{1}{2}-\frac{1}{p+1}\right)\int |x|^{\gamma} u^{p+1}=K(u)=d. \]

It is absurd. Then $N(u)=0$ and $K(u)=d$, which implies that $u$ is a minimizer of $K$ on $\mathcal {N}_{rd}$. Then we have $J'(u)= 0$ in the sense that for any $\varphi \in C_0^{\infty } (R^{n})\bigcap D_{rd}^{\alpha },$

\[ J'(u)\varphi=\int \left((-\Delta)^{\frac{\sigma}{2}}u(-\Delta)^{\frac{\sigma}{2}}\varphi+u\varphi\right){\rm d}\kern0.06em x-\int|x|^{\gamma}|u|^{p-1}u\varphi\,{\rm d}\kern0.06em x=0. \]

This implies that $u$ is a nontrivial ground state as desired.

Using the argument we may also get similar conclusion for the case when $\alpha \in (0,1/2]$, $1< p<\frac {N}{N-\alpha }$, and $\frac {2\alpha }{(1-p)N+\alpha (p+1)}\gamma \leq \sigma$.

6. Proof of theorem 1.6

The argument in the proof of proposition 1.1 can be used to prove the following interesting interpolation result.

Lemma 6.1 Let $\alpha \in (1/2,N/2)$. There is a uniform constant $C$ such that for any $u\in H_{rd}^{\alpha }$,

\[ \int |x|^{\gamma} |u|^{p+1}\leq C \|u\|_2^{A_1}\|u\|_{\dot{H}^{\alpha}}^{B_1} \]

where $A_1=\theta _0(q+1)$, $B_1=(1-\theta _0)(q+1)+\frac {2\gamma }{N-2\alpha }$, $q+1=p+1-\frac {2\gamma }{N-2\alpha }$ and $\theta _0=\frac {N}{\alpha }\left (\frac 12-\frac 1{q+1}\right )$.

Proof. Let $u\in H_{rd}^{\alpha }$. Note that

\[ \int |x|^{\gamma} |u|^{p+1} = \int \left(|x|^{\frac{N}{2}-\alpha}|u|\right)^{\frac{2\gamma}{N-2\alpha}}|u|^{q+1}. \]

By lemma 2.1, we know that for $x\not =0$,

\[ \left(|x|^{\frac{N}{2}-\alpha}|u|\right)^{\frac{2\gamma}{N-2\alpha}}\leq C(N,\alpha)\|(-\Delta)^\frac\alpha 2u\|^{\frac{2\gamma}{N-2\alpha}} \]

Then we have

\[ \int |x|^{\gamma} |u_j|^{p+1}\leq C(N,\alpha)\|(-\Delta)^\frac\alpha 2u\|^{\frac{2\gamma}{N-2\alpha}}\int |u|^{q+1}. \]

Using the well-known fractional Gagliardo–Nirenberg inequality (proposition A.3 in [35]) that

\[ \|u\|_{q+1}^{q+1}\leq C \|u\|_2^{\theta_0 (q+1)}\|u\|_{\dot{H}^{\alpha}}^{(1-\theta_0)(q+1)} \]

for any $u\in H^{\alpha }$, where $\theta _0=\frac {N}{\alpha }\left (\frac 12-\frac 1{q+1}\right )$. Then we have

\[ \int |x|^{\gamma} |u|^{p+1}\leq C \|u\|_2^{\theta_0(q+1)}\|u\|_{\dot{H}^{\alpha}}^{(1-\theta_0)(q+1)+{\frac{2\gamma}{N-2\alpha}}}. \]

This completes the proof.

This result improves the power $\alpha \in (0,1)$ in theorem 2.1 (1) [33] to $\alpha \in \left (1,\frac {N}{2}\right )$.

The proof of theorem 1.6 is now given below.

Proof. To consider the existence of the ground state $\phi$, we take a minimizing sequence $(u_i)\in {M}, u_i\neq 0, S(u_i)\to m_0,$ and we may assume

\begin{align*} \left(\frac{1}{2}-\frac{1}{p+1}\right)\int{|x|^{\gamma}|u_i|^{p+1}}& =\left(\frac{1}{2}-\frac{1}{p+1}\right)\int|(-\Delta)^\frac{\alpha}{2}u_i|^{2}\,{\rm d}\kern0.06em x +\!\int||x|^\frac{\sigma}{2}u_i|^{2}\,{\rm d}\kern0.06em x+\!\int u_i^{2}\,{\rm d}\kern0.06em x\\ & \leq m_0+1. \end{align*}

This implies that $(u_i)\subset D_{rd}^{\alpha }$ is bounded. From the compact imbedding theorem, $D_{rd}^{\alpha }\hookrightarrow \Sigma$. We may choose a sequence $(u_i)$ such that $u_i\to u\in D_{rd}^{\alpha }$ weakly and almost everywhere, and

\[ \left(\frac{1}{2}-\frac{1}{p+1}\right)\int{|x|^{\gamma}|u_i|^{p+1}}\to \left(\frac{1}{2}-\frac{1}{p+1}\right)\int{|x|^{\gamma}|u|^{p+1}}. \]

Then,

\[ \left(\frac{1}{2}-\frac{1}{p+1}\right)\int|(-\Delta)^\frac{\alpha}{2}u|^{2}\,{\rm d}\kern0.06em x+\int||x|^\frac{\sigma}{2}u|^{2}\,{\rm d}\kern0.06em x+\int u^{2}\,{\rm d}\kern0.06em x\leq m_0, \]

\[ \left(\frac{1}{2}-\frac{1}{p+1}\right)\int{|x|^{\gamma}|u|^{p+1}}=m_0. \]

By this, we have $u\in D_{rd}^{\alpha }, u\neq 0$. In the following, we prove $u\in {M},$ and then $S(u)=m_0$, that is, $u$ is the standing wave by the convergence of $(u_i)$. First, we have

\begin{align*} \left(\frac{1}{2}-\frac{1}{p+1}\right)\int{|x|^{\gamma}|u|^{p+1}}& \geq\left(\frac{1}{2}-\frac{1}{p+1}\right)\int|(-\Delta)^\frac{\alpha}{2}u|^{2}\,{\rm d}\kern0.06em x\\ & \quad +\int||x|^\frac{\sigma}{2}u|^{2}\,{\rm d}\kern0.06em x+\int u^{2}\,{\rm d}\kern0.06em x, \end{align*}

that is, $Q(u)\leq 0.$ If $Q(u)<0,$ then for $\lambda \in (0,1),$

\begin{align*} Q(\lambda u)& =\frac{\lambda^{2}}{2}\left[\int|(-\Delta)^\frac{\alpha}{2}u|^{2}\,{\rm d}\kern0.06em x\!+\!\int||x|^\frac{\sigma}{2}u|^{2}\,{\rm d}\kern0.06em x\!+\!\int u^{2}\,{\rm d}\kern0.06em x\right]-\frac{\lambda^{p+1}}{p+1}\int{|x|^{\gamma}|u|^{p+1}}\,{\rm d}\kern0.06em x\\ & =\lambda^{2}\left[\frac{1}{2}\int|(-\Delta)^\frac{\alpha}{2}u|^{2}\,{\rm d}\kern0.06em x\!+\!\!\int||x|^\frac{\sigma}{2}u|^{2}\,{\rm d}\kern0.06em x\!+\!\!\int\! u^{2}\,{\rm d}\kern0.06em x\!-\frac{\lambda^{p-1}}{p+1}\int{|x|^{\gamma}|u|^{p+1}}\,{\rm d}\kern0.06em x\right]\!. \end{align*}

By this, for $\lambda >0$ small, $Q(\lambda u)>0$. Using the intermediate value theorem, we have $\lambda _0\in (0,1),\ Q(\lambda _0 u)=0,\ i.e.\ \lambda _0 u\in {M},$ it implies $S(\lambda _0 u)\geq m_0.$ Note that

\begin{align*} m_0\leq S(\lambda_0 u)& =\left(\frac{1}{2}-\frac{1}{p+1}\right)\int{|x|^{\gamma}|\lambda_0 u|^{p+1}}\,{\rm d}\kern0.06em x\\ & =\lambda^{p+1}_0\left(\frac{1}{2}-\frac{1}{p+1}\right)\int{|x|^{\gamma}|u|^{p+1}}\,{\rm d}\kern0.06em x\\ & =m_0\lambda_0^{p+1}< m_0, \end{align*}

a contradiction. Then $Q(u)=0,\ i.e.\ u\in {M}.$

Next, we prove the stability of standing wave and the idea is similar to [4] (see also [33]). Suppose there exists a sequence $(u^{0}_n)\in D_{rd}^{\alpha }$ such that for positive real numbers $(t_n)$ and $\varepsilon _0>0$, where for some $T^{*}\in (0, \infty ]$, $u_n\in C([0,T^{*}),D_{rd}^{\alpha })$ is the solution to (1.1) when taking $n\to \infty$,

(6.17)\begin{equation} ||u_n-e^{it_n}\phi||_{D_{rd}^{\alpha}}\to 0,\quad \inf_{\theta\in R}||u_n(t_n)-{\rm e}^{i\theta}\phi||_{D_{rd}^{\alpha}}>\varepsilon. \end{equation}

Denote $\phi _n:=u_n(t_n)$. Since $\phi$ is a ground state to (1.1), we have $S(\phi )=m_0, ||\phi ||=q>0$. Then

\[ S(u_n)=m_0\ and\ ||u_n||\to q. \]

By theorem 1.2, $\int |x|^{\gamma }|u_n-\phi |^{p+1}\,{\rm d}\kern0.06em x\leq ||u_n-\phi ||_{D_{rd}^{\alpha }}^{p+1}\to 0$, and using the mass conservation, $||u_n(t_n)||=||u_n^{0}||\to ||\phi ||.$ By (6.17) and proposition 2.2, we have

\[ ||\phi_n||\to q\quad \text{and}\quad S(\phi_n)\to m_0. \]

If $\phi _n\to \phi \in D_{rd}^{\alpha }$, then

\[ \varepsilon_0<\inf||\phi_n-{\rm e}^{i\theta}\phi||_{D_{rd}^{\alpha}}\leq||\phi_n-\phi||\to 0 \]

which is a contradiction. Then we need only to prove that $(\phi _n)$ is relatively compact in $D_{rd}^{\alpha }$ such that

\[ ||\phi_n||\to q\ and\ S(\phi_n)\to m_0. \]

The latter is

(6.18)\begin{equation} S(\phi_n)=\frac{1}{2}||\phi_n||_{D_{rd}^{\alpha}}^{2}+\frac{1}{2}||\phi_n||^{2}-\frac{1}{p+1}\int|x|^{\gamma}|\phi_n|^{p+1}\,{\rm d}\kern0.06em x\to m_0. \end{equation}

For large $n$, and $\varepsilon >0,$ using lemma 6.1 and proposition 2.2,

\begin{align*} m_0+\varepsilon \geq S(\phi_n)& \geq\frac{1}{2}||\phi_n||_{D_{rd}^{\alpha}}^{2}+\frac{1}{2}||\phi_n||^{2}_2-\frac{1}{p+1}\int|x|^{\gamma}|\phi_n|^{p+1}\,{\rm d}\kern0.06em x\\ & \geq \frac{c}{2}||\phi_n||_{\dot H^{\alpha}}^{2}-C||\phi_n||^{B_1}_{\dot H^{\alpha}}||\phi_n||^{A_1}\\ & \geq \frac{c}{2}||\phi_n||_{\dot H^{\alpha}}^{2}\left(1-\frac{C}{c}||\phi_n||^{B_1-2}_{\dot H^{\alpha}}||\phi_n||^{A_1}\right). \end{align*}

Since $B_1<2$, it follows that $\phi _n$ is bounded in $\dot H^{\alpha }$. This then implies that the term $\int |x|^{\gamma }|\phi _n|^{p+1}dx$ is bounded. Going back to (6.18), we then know that $\phi _n$ is bounded in $D_{rd}^{\alpha }$. This completes the proof.

We remark that in [27] the authors have proved existence results about ground states of related nonlinear problems with drifting term and the existence of related principal eigen-functions. We believe that related results there may be true for the fractional cases.