论文检索

最小范数插值估计与正则化经验风险最小化的稳健性AI 解读

This article develops a general theory for minimum norm interpolating estimators and regularized empirical risk minimizers (RERM) in linear models in the presence of additive, potentially adversarial, errors. In particular, no conditions on the errors are imposed. A quantitative bound for the prediction error is given, relating it to the Rademacher complexity of the covariates, the norm of the minimum norm interpolator of the errors and the size of the subdifferential around…

高阶全变差正则化最小二乘的预测界AI 解读

We establish adaptive results for trend filtering: least squares estimation with a penalty on the total variation of (k−1)th order differences. Our approach is based on combining a general oracle inequality for the ℓ1-penalized least squares estimator with “interpolating vectors” to upper bound the “effective sparsity.” This allows one to show that the ℓ1-penalty on the kth order differences leads to an estimator that can adapt to the number of jumps in the (k−1)th order…

稳健低秩矩阵估计AI 解读

Many results have been proved for various nuclear norm penalized estimators of the uniform sampling matrix completion problem. However, most of these estimators are not robust: in most of the cases the quadratic loss function and its modifications are used. We consider robust nuclear norm penalized estimators using two well-known robust loss functions: the absolute value loss and the Huber loss. Under several conditions on the sparsity of the problem (i.e., the rank of the…

高维模型的半参数效率界AI 解读

Asymptotic lower bounds for estimation play a fundamental role in assessing the quality of statistical procedures. In this paper, we propose a framework for obtaining semiparametric efficiency bounds for sparse high-dimensional models, where the dimension of the parameter is larger than the sample size. We adopt a semiparametric point of view: we concentrate on one-dimensional functions of a high-dimensional parameter. We follow two different approaches to reach the lower…

高维模型中渐近最优的置信区域与检验AI 解读

We propose a general method for constructing confidence intervals and statistical tests for single or low-dimensional components of a large parameter vector in a high-dimensional model. It can be easily adjusted for multiplicity taking dependence among tests into account. For linear models, our method is essentially the same as in Zhang and Zhang [J. R. Stat. Soc. Ser. B Stat. Methodol. 76 (2014) 217–242]: we analyze its asymptotic properties and establish its asymptotic…

稀疏回归中的置信集AI 解读

The problem of constructing confidence sets in the high-dimensional linear model with $n$ response variables and $p$ parameters, possibly $p\ge n$, is considered. Full honest adaptive inference is possible if the rate of sparse estimation does not exceed $n^{-1/4}$, otherwise sparse adaptive confidence sets exist only over strict subsets of the parameter spaces for which sparse estimators exist. Necessary and sufficient conditions for the existence of confidence sets that…

稀疏有向无环图的 ℓ₀ 惩罚最大似然估计AI 解读

We consider the problem of regularized maximum likelihood estimation for the structure and parameters of a high-dimensional, sparse directed acyclic graphical (DAG) model with Gaussian distribution, or equivalently, of a Gaussian structural equation model. We show that the $\ell_{0}$-penalized maximum likelihood estimator of a DAG has about the same number of edges as the minimal-edge I-MAP (a DAG with minimal number of edges representing the distribution), and that it…

组稀疏性下的预言不等式与最优统计推断AI 解读

We consider the problem of estimating a sparse linear regression vector β* under a Gaussian noise model, for the purpose of both prediction and model selection. We assume that prior knowledge is available on the sparsity pattern, namely the set of variables is partitioned into prescribed groups, only few of which are relevant in the estimation process. This group sparsity assumption suggests us to consider the Group Lasso method as a means to estimate β*. We establish oracle…

高维加性建模AI 解读

We propose a new sparsity-smoothness penalty for high-dimensional generalized additive models. The combination of sparsity and smoothness is crucial for mathematical theory as well as performance for finite-sample data. We present a computationally efficient algorithm, with provable numerical convergence properties, for optimizing the penalized likelihood. Furthermore, we provide oracle results which yield asymptotic optimality of our estimator for high dimensional but…

Logistic 回归中的组 LassoAI 解读

Summary The group lasso is an extension of the lasso to do variable selection on (predefined) groups of variables in linear regression models. The estimates have the attractive property of being invariant under groupwise orthogonal reparameterizations. We extend the group lasso to logistic regression models and present an efficient algorithm, that is especially suitable for high dimensional problems, which can also be applied to generalized linear models to solve the…

讨论：风险最小化中的局部 Rademacher 复杂度与 Oracle 不等式AI 解读

This paper unifies and extends important theoretical results on empirical risk minimization and model selection.It makes extensive and efficient use of new probability inequalities for the amount of concentration of the (possibly symmetrized) empirical process around its mean.The results are very subtle and very pleasing indeed, as they show that oracle inequalities exist for very general problems.There are in my view two aspects which need special attention.First, the paper…

加性与乘性风险率函数的核估计AI 解读

We propose new procedures for estimating the component functions in both additive and multiplicative nonparametric marker-dependent hazard models. We work with a full counting process framework that allows for left truncation and right censoring and time-varying covariates. Our procedures are based on kernel hazard estimation as developed by Nielsen and Linton and on the idea of marginal integration. We provide a central limit theorem for the marginal integration estimator.…

相互依赖偏好的计量分析AI 解读

The theoretical model of Gaertner (1974) and Pollak (1976) for the interdependence of preferences in the Linear Expenditure System is estimated for a cross-section of households. The interdependence of consumption of different households has implications for the stochastic structure of the model and for the identifiability of its parameters. Both aspects are dealt with. The empirical results indicate a significant role played by the interdependence of preferences. One of its…

部分线性模型中的惩罚准似然估计AI 解读

Consider a partial linear model, where the expectation of a random variable Y depends on covariates $(x, z)$ through $F(\theta_0 x + m_0(z))$, with $\theta_0$ an unknown parameter, and $m_0$ an unknown function. We apply the theory of empirical processes to derive the asymptotic properties of the penalized quasi-likelihood estimator.

局部自适应回归样条AI 解读

Least squares penalized regression estimates with total variation penalties are considered. It is shown that these estimators are least squares splines with locally data adaptive placed knot points. The definition of these variable knot splines as minimizers of global functionals can be used to study their asymptotic properties. In particular, these results imply that the estimates adapt well to spatially inhomogeneous smoothness. We show rates of convergence in bounded…

非参数回归最小二乘估计量的一致性AI 解读

We shall study the general regression model $Y = g_0 (X) + \varepsilon$, where X and $varepsilon$ are independent. The available information about $g_0$ can be expressed by $g_0 \epsilon \mathscr{G}$ for some class $\mathscr{G}$. As an estimator of $g_0$ we choose the least squares estimator. We shall give necessary and sufficient conditions for consistency of this estimator in terms of (basically) geometric properties of $\mathscr{G}$. Our main tool will be the theory of…

鞅的指数不等式及其在计数过程最大似然估计中的应用AI 解读

We obtain an exponential probability inequality for martingales and a uniform probability inequality for the process $\int g dN$, where $N$ is a counting process and where $g$ varies within a class of predictable functions $\mathscr{G}$. For the latter, we use techniques from empirical process theory. The uniform inequality is shown to hold under certain entropy conditions on $\mathscr{G}$. As an application, we consider rates of convergence for (nonparametric) maximum…

若干非参数最大似然估计量的 Hellinger 一致性AI 解读

Consider a class $\mathscr{P}={P_\theta:\theta\in\Theta}$ of probability measures on a measurable space $(\mathscr{X},\mathscr{A})$, dominated by a $\sigma$ -finite measure $\mu$. Let $f_\theta=dP_\theta/d_\mu$, $\theta\ in\Theta$, and let $\theta_n$ be a maximum likelihood estimator based on n independent observations from $P_{\theta_0}$, $\theta_0\in\Theta$. We use results from empirical process theory to obtain convergence for the Hellinger distance $h(f_{\hat{\theta}_n}, f_{\theta_0})$…

回归函数的估计AI 解读

In this paper, an entropy approach is proposed to establish rates of convergence for estimators of a regression function. General regression problems are considered, with linear regression, splines and isotonic regression as special cases. The estimation methods studied are least squares, least absolute deviations and penalized least squares. Common features of these methods and various regression problems are highlighted.

最小二乘估计的新方法及其应用AI 解读

The regression model $\mathbf{y} = g(\mathbf{x}) + \mathbf{\varepsilon}$ and least-squares estimation are studied in a general context. By making use of empirical process theory, it is shown that entropy conditions on the class $\mathscr{G}$ of possible regression functions imply $L^2$-consistency of the least-squares estimator $\hat{\mathbf{g}}_n$ of $g$. This result is applied in parametric and nonparametric regression.