The Bayesian approach to estimation is appealing for several reasons: it comes with theoretical guarantees in various statistical settings, performs well empirically, and offers several practical advantages such as flexibility, simplicity, automatic uncertainty quantification, and the availability of efficient implementation tools (e.g., MCMC algorithms, variational Bayes). Many questions remain to be explored, and in particular the issue of theoretical guarantees is still an active area of research, to which this work contributes.
We focus on semiparametric inference, where the goal is to estimate a low-dimensional aspect of an infinite-dimensional parameter, for example, the squared l2-norm of a density or the average causal treatment effect on the treated. In this setting, the Bayesian approach involves specifying a prior on the entire parameter, forming the posterior distribution, and then conducting inference based on the marginal posterior of the quantity of interest.
There is a rich body of literature suggesting that Bayesian methods can provide optimal procedures in this setting. However, a central message from this literature is that the default use of nonparametric priors designed for inference on the entire unknown parameter may lead to suboptimal results for semiparametric inference. To achieve good performance, Bayesian methods must be adapted. Two main strategies have been explored so far: (i) tuning the hyperparameters of standard nonparametric priors, and (ii) correcting the posterior distribution arising from such priors in order to improve performance for semiparametric inference. While both approaches yield improvements, they also have limitations.
In this work, we investigate an alternative approach in the specific context of estimating the squared l2-norm of the signal in the Gaussian white noise model. The idea is based on the reference prior approach, developed in parametric models, and consists in defining the prior using spherical coordinates, with a flat prior on the squared l2–norm and an isotropic prior on the set of angles. We demonstrate that, when appropriately tuned, this prior results in a posterior on the squared l2–norm that achieves several desirable properties such as consistency, minimax convergence rates over Sobolev balls, and a Bernstein–von Mises theorem for highly Sobolev-regular parameters. As a result, our method improves upon other Bayesian approaches proposed in the literature. We derive a Gibbs sampler to (approximately) sample from the posterior and illustrate our theoretical results through a simulation study. Furthermore, we present some preliminary findings on whether the method can be made adaptive to the parameter’s regularity class via an empirical Bayes approach. This is joint work with Ismaël Castillo.