I wanted to do a quick post to verify an identity I encountered while reading about SteinGANs. For this post, we’ll consider probability densities in the exponential family, which have the following form,
\begin{align}
p\left(x\vert\theta\right) &= \frac{1}{Z} \exp\left(f\left(x;\theta\right)\right) \\

Z &= \int_{x} p\left(x\vert\theta\right) dx.
\end{align}
Then the claim is that the gradient of the log-likelihood function be be expressed as follows,
\begin{align}
\frac{\nabla_{\theta} \log L}{n} = \frac{1}{n}\sum_{i=1}^{n} \nabla_{\theta} f\left(x_i;\theta\right) - \mathbb{E}_{x\sim p}\left[\nabla_{\theta} f\left(x;\theta\right)\right].
\end{align}

To see this, we can proceed as follows,
\begin{align}
L\left(\theta\vert X\right) &= \prod_{i=1}^{n} \frac{1}{Z} \exp\left(f\left(x_i;\theta\right)\right) \\

\log L\left(\theta\vert X\right) &= \sum_{i=1}^{n} f\left(x_i;\theta\right) - n\log\int_{x} \exp\left(f\left(x;\theta\right)\right) dx \\

\nabla_{\theta} \log L &= \sum_{i=1}^{n} \nabla_{\theta} f\left(x_i;\theta\right) - \frac{n}{Z} \int_{x} \nabla_{\theta} f\left(x;\theta\right) \exp\left(f\left(x;\theta\right)\right) dx \\

\frac{\nabla_{\theta} \log L}{n} &= \frac{1}{n}\sum_{i=1}^{n} \nabla_{\theta} f\left(x_i;\theta\right) - \mathbb{E}_{x\sim p}\left[\nabla_{\theta} f\left(x;\theta\right)\right],
\end{align}
which is what we wanted to show.