Foreword ¶ We are going to discuss an important technique of estimating asymptotics of integrals. It is very relevant to the kind of functions we deal in regular statistical theory as well. This method is called the Laplace Method. This read is quite nice, it involves some low cortisol exercises and theory, which includes some nice techniques and facts.
The Laplace Method for Integrals ¶ We want to approximate
F ( t ) = ∫ − ∞ ∞ φ ( x , t ) d x ( t → ∞ ) F(t) = \int_{-\infty}^{\infty}\varphi(x, t)dx \qquad(t \to \infty) F ( t ) = ∫ − ∞ ∞ φ ( x , t ) d x ( t → ∞ ) The idea is simple: If by chance the function has a sharp peak around some x 0 x_0 x 0 t t t
WLOG, say x 0 = 0 x_0 = 0 x 0 = 0
∫ − ∞ ∞ − ∫ − δ δ < < ∫ − δ δ \int_{-\infty}^{\infty} - \int_{-\delta}^{\delta} << \int_{-\delta}^{\delta} ∫ − ∞ ∞ − ∫ − δ δ << ∫ − δ δ It would be nicer if in ∣ x ∣ < δ |x| < \delta ∣ x ∣ < δ φ \varphi φ
Example
∫ − ∞ ∞ e − t x 2 log ( 1 + x + x 2 ) d x ≈ ∫ − 1 / 2 1 / 2 e − t x 2 ( x + x 2 2 − 2 3 x 3 ) d x \int_{-\infty}^{\infty}e^{-tx^2}\log(1 + x + x^2)dx \approx \int_{-1/2}^{1/2}e^{-tx^2}(x + \frac{x^2}{2} - \frac{2}{3}x^3)dx ∫ − ∞ ∞ e − t x 2 log ( 1 + x + x 2 ) d x ≈ ∫ − 1/2 1/2 e − t x 2 ( x + 2 x 2 − 3 2 x 3 ) d x Some Nice Facts ¶ The integrands of the type e − t x 2 x k e^{-tx^2}x^k e − t x 2 x k
For k odd, since the integrand is an odd function, the integral will of course be 0.
∫ − ∞ ∞ e − t x 2 x k d x = 0 \int_{-\infty}^{\infty}e^{-tx^2}x^kdx = 0 ∫ − ∞ ∞ e − t x 2 x k d x = 0 For k even, we do a simple substitution and then use the gamma integral (we thus require R e ( t ) > 0 Re(t) > 0 R e ( t ) > 0
∫ − ∞ ∞ e − t x 2 x k d x = 2 ∫ 0 ∞ e − t x 2 x k d x \int_{-\infty}^{\infty}e^{-tx^2}x^kdx = 2\int_{0}^{\infty}e^{-tx^2}x^kdx ∫ − ∞ ∞ e − t x 2 x k d x = 2 ∫ 0 ∞ e − t x 2 x k d x Set t x 2 = y tx^2 = y t x 2 = y 2 t x d x = d y 2txdx = dy 2 t x d x = d y
t − ( k + 1 ) / 2 ∫ 0 ∞ e − y y ( k + 1 ) / 2 d y = t − ( k + 1 ) / 2 Γ ( k + 1 2 ) t^{-(k + 1)/2}\int_0^{\infty}e^{-y}y^{(k+1)/2}dy = t^{-(k + 1)/2}\Gamma(\frac{k + 1}{2}) t − ( k + 1 ) /2 ∫ 0 ∞ e − y y ( k + 1 ) /2 d y = t − ( k + 1 ) /2 Γ ( 2 k + 1 ) We get these three estimates/equations as well.
∫ − ∞ ∞ ∣ e − t x 2 x k ∣ d x = O { ( Re t ) − 1 2 ( k + 1 ) } ( Re t > 0 ) . \int_{-\infty}^{\infty} |e^{-tx^2}x^k| dx = \mathrm{O}\{(\operatorname{Re} t)^{-\frac{1}{2}(k+1)}\} \qquad (\operatorname{Re} t > 0). ∫ − ∞ ∞ ∣ e − t x 2 x k ∣ d x = O {( Re t ) − 2 1 ( k + 1 ) } ( Re t > 0 ) . ∫ 0 ∞ e − t x x k d x = t − k − 1 k ! ( Re t > 0 ) , \int_{0}^{\infty} e^{-tx}x^k dx = t^{-k-1}k! \qquad (\operatorname{Re} t > 0), ∫ 0 ∞ e − t x x k d x = t − k − 1 k ! ( Re t > 0 ) , ∫ 0 ∞ ∣ e − t x x k ∣ d x = O { ( Re t ) − k − 1 } ( Re t > 0 ) , \int_{0}^{\infty} |e^{-tx}x^k| dx = \mathrm{O}\{(\operatorname{Re} t)^{-k-1}\} \qquad (\operatorname{Re} t > 0), ∫ 0 ∞ ∣ e − t x x k ∣ d x = O {( Re t ) − k − 1 } ( Re t > 0 ) , General Case ¶ Let us tackle the general case. Just observe that we are making exactly the same assumptions that regularity imposes on − K n ( ω ) -K_n(\omega) − K n ( ω )
We will consider the function
F ( t ) = ∫ − ∞ ∞ e t h ( x ) d x F(t) = \int_{-\infty}^{\infty}e^{th(x)}dx F ( t ) = ∫ − ∞ ∞ e t h ( x ) d x with the following assumptions on h ( x ) h(x) h ( x )
h ( x ) h(x) h ( x ) x = 0 x = 0 x = 0 h h h
There exist real numbers b , c b, c b , c
h ( x ) ≤ − b if ∣ x ∣ ≥ c h(x) \leq -b \quad \text{if } \quad |x| \geq c h ( x ) ≤ − b if ∣ x ∣ ≥ c This is somewhat more general. This happens whenever h ( x ) → − ∞ h(x) \to -\infty h ( x ) → − ∞ ∣ x ∣ → ∞ |x| \to \infty ∣ x ∣ → ∞
The integral should converge for sufficiently large values of t t t t = 1 t = 1 t = 1
Assume h ′ ( x ) h'(x) h ′ ( x ) h ′ ′ ( 0 ) h''(0) h ′′ ( 0 ) h ′ ( 0 ) = 0 h'(0) = 0 h ′ ( 0 ) = 0
It follows from our assumptions that given any positive δ \delta δ η ( δ \eta(\delta η ( δ h ( x ) ≤ − η ( δ ) h(x) \leq -\eta(\delta) h ( x ) ≤ − η ( δ ) ∣ x ∣ ≥ δ |x| \geq \delta ∣ x ∣ ≥ δ
Draw a graph and make an argument, this is pretty easy.
Now since t h ( x ) = ( t − 1 ) h ( x ) + h ( x ) th(x) = (t - 1)h(x) + h(x) t h ( x ) = ( t − 1 ) h ( x ) + h ( x )
∫ − ∞ − δ + ∫ δ ∞ < e − ( t − 1 ) η ( δ ) ∫ − ∞ ∞ e h ( x ) d x ( t > 1 ) . \int_{-\infty}^{-\delta} + \int_{\delta}^{\infty} < e^{-(t-1)\eta(\delta)} \int_{-\infty}^{\infty} e^{h(x)} dx \qquad (t > 1). ∫ − ∞ − δ + ∫ δ ∞ < e − ( t − 1 ) η ( δ ) ∫ − ∞ ∞ e h ( x ) d x ( t > 1 ) . Why did we do this? We cannot directly bound t h ( x ) th(x) t h ( x ) ∞ \infty ∞ t → ∞ t \to \infty t → ∞
Thus, outside a δ \delta δ
We do a taylor approximation, as illustrated in the first example. We justify the use formally.
∣ h ( x ) − x 2 h ′ ′ ( 0 ) 2 ∣ = O ( x 2 ) |h(x) - \frac{x^2h''(0)}{2}| = O(x^2) ∣ h ( x ) − 2 x 2 h ′′ ( 0 ) ∣ = O ( x 2 ) Now, this is not a one liner since the third derivative may not exist. So you can show this by considering g ( x ) = h ( x ) − x 2 h ′ ′ ( 0 ) 2 g(x) = h(x) - \dfrac{x^2h''(0)}{2} g ( x ) = h ( x ) − 2 x 2 h ′′ ( 0 )
Choosing a small enough δ \delta δ h ( x ) h(x) h ( x )
∫ − δ δ e 1 2 t x 2 ( h ′ ′ ( 0 ) − 2 ε ) d x < ∫ − δ δ e t h ( x ) d x < ∫ − δ δ e 1 2 t x 2 ( h ′ ′ ( 0 ) + 2 ε ) d x . \int_{-\delta}^{\delta} e^{\frac{1}{2}tx^2(h''(0)-2\varepsilon)} dx < \int_{-\delta}^{\delta} e^{th(x)} dx < \int_{-\delta}^{\delta} e^{\frac{1}{2}tx^2(h''(0)+2\varepsilon)} dx. ∫ − δ δ e 2 1 t x 2 ( h ′′ ( 0 ) − 2 ε ) d x < ∫ − δ δ e t h ( x ) d x < ∫ − δ δ e 2 1 t x 2 ( h ′′ ( 0 ) + 2 ε ) d x . Now the upper bound and lower bound can both be calculated, we already have show how.
∫ − ∞ ∞ e t h ( x ) d x ∼ ( 2 π ) 1 2 ( − t h ′ ′ ( 0 ) ) − 1 2 ( t → ∞ ) . \int_{-\infty}^{\infty} e^{th(x)} dx \sim (2\pi)^{\frac{1}{2}}(-th''(0))^{-\frac{1}{2}} \qquad (t \to \infty). ∫ − ∞ ∞ e t h ( x ) d x ∼ ( 2 π ) 2 1 ( − t h ′′ ( 0 ) ) − 2 1 ( t → ∞ ) . Asymptotic Expansions ¶ For simplicity, we will assume that in some interval ∣ x ∣ ≤ δ |x| \leq \delta ∣ x ∣ ≤ δ
h ( x ) = a 2 x 2 + a 3 x 3 + . . . h(x) = a_2x^2 + a_3x^3 + ... h ( x ) = a 2 x 2 + a 3 x 3 + ... and that a 2 < 0 a_2 < 0 a 2 < 0
We will now discuss the more general case
F ( t ) = ∫ − ∞ ∞ g ( x ) e t h ( x ) d x F(t) = \int_{-\infty}^{\infty}g(x)e^{th(x)}dx F ( t ) = ∫ − ∞ ∞ g ( x ) e t h ( x ) d x where g ( x ) g(x) g ( x ) ∣ x ∣ ≤ δ |x| \leq \delta ∣ x ∣ ≤ δ
g ( x ) = b 0 + b 1 x + b 2 x 2 + . . . g(x) = b_0 + b_1x + b_2x^2 + ... g ( x ) = b 0 + b 1 x + b 2 x 2 + ... We need some rough estimate saying that the contributions of the intervals +-( δ , ∞ ) (\delta, \infty) ( δ , ∞ ) M M M
∫ − ∞ − δ g ( x ) e t h ( x ) d x = O ( t − m ) ∫ δ ∞ g ( x ) e t h ( x ) d x = O ( t − m ) \int_{-\infty}^{-\delta}g(x)e^{th(x)}dx = O(t^{-m}) \quad \int_{\delta}^{\infty}g(x)e^{th(x)}dx = O(t^{-m}) ∫ − ∞ − δ g ( x ) e t h ( x ) d x = O ( t − m ) ∫ δ ∞ g ( x ) e t h ( x ) d x = O ( t − m ) if t → ∞ t \to \infty t → ∞ h ( x ) = − O ( x 2 ) h(x) = -O(x^2) h ( x ) = − O ( x 2 )
We consider exp ( t a 2 x 2 ) \exp(ta_2x^2) exp ( t a 2 x 2 )
g ( x ) exp { t x 3 ( a 3 + a 4 x + a 5 x 2 + . . . ) } g(x)\exp\{tx^3(a_3 + a_4x + a_5x^2 + ...)\} g ( x ) exp { t x 3 ( a 3 + a 4 x + a 5 x 2 + ... )} We expand this as a double power series
P ( t x 3 , x ) = ∑ m = 0 ∞ ∑ n = 0 ∞ c m n ( t x 3 ) m x n P(tx^3, x) = \sum_{m = 0}^{\infty}\sum_{n = 0}^{\infty}c_{mn}(tx^3)^mx^n P ( t x 3 , x ) = m = 0 ∑ ∞ n = 0 ∑ ∞ c mn ( t x 3 ) m x n We want to approximate P P P t x 3 tx^3 t x 3 ∣ x ∣ ≤ t − 1 / 3 |x| \leq t^{-1/3} ∣ x ∣ ≤ t − 1/3 t − 1 / 3 = τ t^{-1/3} = \tau t − 1/3 = τ 0 ≤ τ ≤ δ 0 \leq \tau \leq \delta 0 ≤ τ ≤ δ ( τ , δ ) (\tau, \delta) ( τ , δ )