论文检索

未知连接函数的一般非参数回归模型的速率最优估计AI 解读

This paper discusses a nonparametric regression model that naturally generalizes neural network models. The model is based on a finite number of one-dimensional transformations and can be estimated with a one-dimensional rate of convergence. The model contains the generalized additive model with unknown link function as a special case. For this case, it is shown that the additive components and link function can be estimated with the optimal rate by a smoothing spline that…

讨论：Lasso、L2Boosting 与 Dantzig 三种相关方法AI 解读

We would like to congratulate the authors for their thought-provoking and interesting paper.The Dantzig paper is on the timely topic of high-dimensional data modeling that has been the center of much research lately and where many exciting results have been obtained.It also falls in the very hot area at the interface of statistics and optimization: ℓ 1 -constrained minimization in linear models for computationally efficient model selection, or sparse model estimation (Chen,…

自旋玻璃中的估计：第一步AI 解读

The Sherrington–Kirkpatrick model of spin glasses, the Hopfield model of neural networks and the Ising spin glass are all models of binary data belonging to the one-parameter exponential family with quadratic sufficient statistic. Under bare minimal conditions, we establish the $\sqrt{N}$-consistency of the maximum pseudolikelihood estimate of the natural parameter in this family, even at critical temperatures. Since very little is known about the low and critical…

长记忆模型中遗漏非线性的检验AI 解读

This article constructs tests for the presence of nonlinearity of unknown form in addition to a fractionally integrated, long-memory component in a time series process. The tests are based on artificial neural network approximations and do not restrict the parametric form of the nonlinearity. Some theoretical results for the new tests are obtained, and detailed simulation evidence on the power of the tests is presented. The new methodology is then applied to a wide variety…

摘要暂缺

讨论：风险最小化中的局部 Rademacher 复杂度与预言不等式AI 解读

In this magnificent paper, Professor Koltchinskii offers general and powerful performance bounds for empirical risk minimization, a fundamental principle of statistical learning theory.Since the elegant pioneering work of Vapnik and Chervonenkis in the early 1970s, various such bounds have been known that relate the performance of empirical risk minimizers to combinatorial and geometrical features of the class over which the minimization is performed.This area of research…

最优子集选择、高维统计学习的持续性与 l1 约束优化AI 解读

Let (Y, X1, …, Xm) be a random vector. It is desired to predict Y based on (X1, …, Xm). Examples of prediction methods are regression, classification using logistic regression or separating hyperplanes, and so on. We consider the problem of best subset selection, and study it in the context m=nα, α>1, where n is the number of observations. We investigate procedures that are based on empirical risk minimization. It is shown, that in common cases, we should aim to find the…

多元非参数回归中的分量选择与平滑AI 解读

We propose a new method for model selection and model fitting in multivariate nonparametric regression models, in the framework of smoothing spline ANOVA. The “COSSO” is a method of regularization with the penalty functional being the sum of component norms, instead of the squared norm employed in the traditional smoothing spline method. The COSSO provides a unified framework for several recent proposals for model selection in linear models and smoothing spline ANOVA models.…

凸性、分类与风险界AI 解读

Many of the classification algorithms developed in the machine learning literature, including the support vector machine and boosting, can be viewed as minimum contrast methods that minimize a convex surrogate of the 0–1 loss function. The convexity makes these algorithms computationally efficient. The use of a surrogate, however, has statistical consequences that must be balanced against the computational virtues of convexity. To study these issues, we provide a general…

调查抽样中的非参数模型校准估计AI 解读

Calibration is commonly used in survey sampling to include auxiliary information at the estimation stage of a population parameter. Calibrating the observation weights on population means (totals) of a set of auxiliary variables implies building weights that when applied to the auxiliaries give exactly their population mean (total). Implicitly, calibration techniques rely on a linear relation between the survey variable and the auxiliary variables. However, when auxiliary…

Boosting is one of the most significant advances in machine learning for classification and regression. In its original and computationally flexible version, boosting seeks to minimize empirically a loss function in a greedy fashion. The resulting estimator takes an additive function form and is built iteratively by applying a base estimator (or learner) to updated samples depending on the previous iterations. An unusual regularization technique, early stopping, is employed…

局部—全局神经网络AI 解读

We propose the local-global neural networks model within the context of time series models. This formulation encompasses some already existing nonlinear models and also admits the mixture of experts approach. We emphasize the linear expert case and extensively discuss the theoretical aspects of the model: stationarity conditions, existence, consistency and asymptotic normality of the parameter estimates, and model identifiability. The proposed model consists of a mixture of…

周期高斯再生核正则化方法的统计性质AI 解读

The method of regularization with the Gaussian reproducing kernel is popular in the machine learning literature and successful in many practical applications. In this paper we consider the periodic version of the Gaussian kernel regularization. We show in the white noise model setting, that in function spaces of very smooth functions, such as the infinite-order Sobolev space and the space of analytic functions, the method under consideration is asymptotically minimax; in…

利用广义谱与非线性时间序列模型推断汇率可预测性AI 解读

It is often documented, based on autocorrelation, variance ratio, and power spectrum, that exchange rates approximately follow a martingale process. Because these data check serial uncorrelatedness rather than martingale difference, they may deliver misleading conclusions in favor of the martingale hypothesis when the test statistics are insignificant. In this paper, we explore whether there exists a gap between serial uncorrelatedness and martingale difference for exchange…

摘要暂缺

提升核密度估计：一种偏差缩减技术？AI 解读

This paper proposes an algorithm for boosting kernel density estimates. We show that boosting is closely linked to a previously proposed method of bias reduction and indicate how it should enjoy similar properties. Numerical examples and simulations are used to illustrate the findings, and we also suggest further areas of research.

关于提升方法论文的讨论及作者回应AI 解读

Discussions of: "Process consistency for AdaBoost" [Ann. Statist. 32 (2004), no. 1, 13-29] by W. Jiang; "On the Bayes-risk consistency of regularized boosting methods" [ibid., 30-55] by G. Lugosi and N. Vayatis; and "Statistical behavior and consistency of classification methods based on convex risk minimization" [ibid., 56-85] by T. Zhang. Includes rejoinders by the authors.

正则化提升方法的贝叶斯风险一致性AI 解读

The probability of error of classification methods based on convex combinations of simple base classifiers by "boosting" algorithms is investigated. The main result of the paper is that certain regularized boosting algorithms provide Bayes-risk consistent classifiers under the sole assumption that the Bayes classifier may be approximated by a convex combination of the base classifiers. Nonasymptotic distribution-free bounds are also developed which offer interesting new…

三篇关于提升法的论文：导言AI 解读

The notion of boosting originated in the Machine Learning literature in the 1980's [VALIANT, L.G. (1984). A theory of the learnable. In Proc. 16th Annual ACM Symposium on Theory of Computing 436-445. ACM Press, New York]. The goal of boosting is to improve the generalization performance of weak (or base) learning algorithms by combining them in a certain way. The first algorithm of this type was discovered by Schapire [SCHAPIRE, R.E. (1990). The strength of weak…

Boosting 集成的总体理论AI 解读

Tree ensembles are looked at in distribution space, that is, the limit case of "infinite" sample size. It is shown that the simplest kind of trees is complete in D-dimensional $L_2(P)$ space if the number of terminal nodes T is greater than D. For such trees we show that the AdaBoost algorithm gives an ensemble converging to the Bayes risk.

基于凸风险最小化的分类方法：统计性质与一致性AI 解读

We study how closely the optimal Bayes error rate can be approximately reached using a classification algorithm that computes a classifier by minimizing a convex upper bound of the classification error function. The measurement of closeness is characterized by the loss function used in the estimation. We show that such a classification scheme can be generally regarded as a (nonmaximum-likelihood) conditional in-class probability estimate, and we use this analysis to compare…

利用广义谱与非线性时间序列模型推断汇率可预测性AI 解读

It is often documented, based on autocorrelation, variance ratio, and power spectrum, that exchange rates approximately follow a martingale process. Because these data check serial uncorrelatedness rather than martingale difference, they may deliver misleading conclusions in favor of the martingale hypothesis when the test statistics are insignificant. In this paper, we explore whether there exists a gap between serial uncorrelatedness and martingale difference for exchange…

循环神经网络的可识别性AI 解读

We examine the identifiability of a nonlinear state space system under general assumptions. The discrete time evolution of the state is generated by a recurrent Elman network. For a large set of Elman networks we determine the class of observationally equivalent minimal systems, i.e., minimal systems that exhibit the same input-output behavior.The authors are grateful for discussions with M. Deistler and D. Bauer. A.A. Al-Falou was funded by the ERNSI network within the…

ψ学习研究AI 解读

The concept of large margins have been recognized as an important principle in analyzing learning methodologies, including boosting, neural networks, and support vector machines (SVMs). However, this concept alone is not adequate for learning in nonseparable cases. We propose a learning methodology, called ψ-learning, that is derived from a direct consideration of generalization errors. We provide a theory for ψ-learning and show that it essentially attains the optimal rates…