论文检索

贪婪函数逼近：梯度提升机AI 解读

Function estimation/approximation is viewed from the perspective of numerical optimization in function space, rather than parameter space. A connection is made between stagewise additive expansions and steepest-descent minimization. A general gradient descent “boosting” paradigm is developed for additive expansions based on any fitting criterion.Specific algorithms are presented for least-squares, least absolute deviation, and Huber-M loss functions for regression, and…

不规则采样曲线的函数型线性判别分析AI 解读

Summary We introduce a technique for extending the classical method of linear discriminant analysis (LDA) to data sets where the predictor variables are curves or functions. This procedure, which we call functional linear discriminant analysis (FLDA), is particularly useful when only fragments of the curves are observed. All the techniques associated with LDA can be extended for use with FLDA. In particular FLDA can be used to produce classifications on new (test) curves,…

图像形变的惩罚似然方法AI 解读

Summary A warping is a function that deforms images by mapping between image domains. The choice of function is formulated statistically as maximum penalized likelihood, where the likelihood measures the similarity between images after warping and the penalty is a measure of distortion of a warping. The paper addresses two issues simultaneously, of how to choose the warping function and how to assess the alignment. A new, Fourier–von Mises image model is identified, with…

小波逼近的正则化AI 解读

In this paper, we introduce nonlinear regularized wavelet estimators for estimating nonparametric regression functions when sampling points are not uniformly spaced. The approach can apply readily to many other statistical contexts. Various new penalty functions are proposed. The hard-thresholding and soft-thresholding estimators of Donoho and Johnstone are specific members of nonlinear regularized wavelet estimators. They correspond to the lower and upper envelopes of a…

随机神经网络及其在非线性时间序列中的应用AI 解读

We consider a variant of the conventional neural network model, called the stochastic neural network, that can be used to approximate complex nonlinear stochastic systems. We show how the expectation-maximization algorithm can be used to develop efficient estimation schemes that have much lower computational complexity than those for conventional neural networks. This enables us to carry out model selection procedures, such as the Bayesian information criterion, to choose…

基于软阈值的高维广义线性模型推断AI 解读

Summary We further develop and analyse penalized likelihood estimators for generalized linear models with a large number of coefficients. The methodology proposed leads to an adaptive selection of model terms without substantial variance inflation. Our proposal extends the soft thresholding strategy of Donoho and Johnstone and the lasso of Tibshirani to generalized linear models and multiple predictor variables. In addition, we develop an estimator for the covariance matrix…

利用金融与经济变量预测股票收益的非线性可预测性AI 解读

In a recent article by Qi, neural networks trained by Bayesian regularization were used to predict excess returns on the S&P 500. The article concluded that the switching portfolio based on the recursive neural-network forecasts generates higher accumulated wealth with lower risks than that based on linear regression. Unfortunately, attempts to replicate the results were unsuccessful. Replicated results using the same software, approach and data detailed by Qi indicate that,…

通过模型选择实现自回归或 β 混合回归中的自适应估计AI 解读

We study the problem of estimatingsome unknown regression function in a $\beta$-mixing dependent framework. To this end, we consider some collection of models which are finite dimensional spaces. A penalized least-squares estimator (PLSE) is built on a data driven selected model among this collection. We state non asymptotic risk bounds for this PLSE and give several examples where the procedure can be applied (autoregression, regression with arithmetically $\beta$-mixing…

实参数进化蒙特卡洛及其在贝叶斯混合模型中的应用AI 解读

We propose an evolutionary Monte Carlo algorithm to sample from a target distribution with real-valued parameters. The attractive features of the algorithm include the ability to learn from the samples obtained in previous steps and the ability to improve the mixing of a system by sampling along a temperature ladder. The effectiveness of the algorithm is examined through three multimodal examples and Bayesian neural networks. The numerical results confirm that the real-coded…

蒙特卡洛计算中动态加权的理论AI 解读

This article provides a first theoretical analysis of a new Monte Carlo approach, the dynamic weighting algorithm, proposed recently by Wong and Liang. In dynamic weighting Monte Carlo, one augments the original state space of interest by a weighting factor, which allows the resulting Markov chain to move more freely and to escape from local modes. It uses a new invariance principle to guide the construction of transition rules. We analyze the behavior of the weights…

曲线的贝叶斯小波回归及其光谱校准应用AI 解读

Motivated by calibration problems in near-infrared (NIR) spectroscopy, we consider the linear regression setting in which the many predictor variables arise from sampling an essentially continuous curve at equally spaced points and there may be multiple predictands. We tackle this regression problem by calculating the wavelet transforms of the discretized curves, then applying a Bayesian variable selection method using mixture priors to the multivariate regression of…

通过混合实现自适应回归AI 解读

Adaptation over different procedures is of practical importance. Different procedures perform well under different conditions. In many practical situations, it is rather hard to assess which conditions are (approximately) satisfied so as to identify the best procedure for the data at hand. Thus automatic adaptation over various scenarios is desirable. A practically feasible method, named adaptive regression by mixing (ARM), is proposed to convexly combine general candidate…

自回归时间序列建模的非渐近界AI 解读

The subject of this paper is autoregressive (AR) modeling of a stationary, Gaussian discrete time process, based on a finite sequence of observations. The process is assumed to admit an AR($\infty$) representation with exponentially decaying coefficients. We adopt the nonparametric minimax framework and study how well the process can be approximated by a finite­order AR model. A lower bound on the accuracy of AR approximations is derived, and a nonasymptotic upper bound on…

摘要暂缺

校准与经验贝叶斯变量选择AI 解读

For the problem of variable selection for the normal linear model, selection criteria such as aic, Cp , bic and ric have fixed dimensionality penalties. Such criteria are shown to correspond to selection of maximum posterior models under implicit hyperparameter choices for a particular hierarchical Bayes formulation. Based on this calibration, we propose empirical Bayes selection criteria that use hyperparameter estimates instead of fixed choices. For obtaining these…

自适应优化与 D 最优实验设计AI 解读

We consider the situation where one has to maximize a function $\eta(\theta, \mathbf{x})$ with respect to $\mathbf{x} \epsilon \mathbb{R}^q$, when $\theta$ is unknown and estimated by least squares through observations $y_k = \mathbf{f}^{\top(\mathbf{x}_k)\theta + \varepsilon_k$, with $\varepsilon_k$ some random error. Classical applications are regulation and extremum control problems. The approach we adopt corresponds to maximizing the sum of the current estimated objective and a penalization for poor estimation: $…

回归模型选择的渐近极小极大遗憾规则与维数惩罚AI 解读

This paper addresses the topic of model selection in regression.We emphasize the case of two models, testing which model provides a better prediction based on $n$ observations. Within a family of selection rules, based on maximizing a penalized log-likelihood under a normal model, we search for asymptotically minimax rules over a class $\mathscr{G}$ of possible joint distributions of the explanatory and response variables. For the class $\mathscr{G}$ of multivariate normal…

大规模伯努利数据的平滑样条 ANOVA 模型与随机化 GACVAI 解读

We propose the randomized Generalized Approximate Cross Validation (ranGACV) method for choosing multiple smoothing parameters in penalized likelihood estimates for Bernoulli data. The method is intended for application with penalized likelihood smoothing spline ANOVA models. In addition we propose a class of approximate numerical methods for solving the penalized likelihood variational problem which, in conjunction with the ranGACV method allows the application of smoothing…

变量选择问题AI 解读

The problem of variable selection is one of the most pervasive model selection problems in statistical applications. Often referred to as the problem of subset selection, it arises when one wants to model the relationship between a variable of interest and a subset of potential explanatory variables or predictors, but there is uncertainty about which subset to use. This vignette reviews some of the key developments that have led to the wide variety of approaches for this…

非线性回归与神经网络训练的局部贪心逼近AI 解读

A criterion for local estimation and approximation in nonlinear regres- sion and neural network training is introduced and motivated. $N$th-order greedy approximation for the regression (or target) function based on the criterion is shown to converge at rate $O(1/N^{1/2})$ in the nonsampling case.

Lasso 型估计量的渐近性质AI 解读

We consider the asymptotic behavior ofregression estimators that minimize the residual sum of squares plus a penalty proportional to $\sum|\beta_j|^{\gamma}$. for some $\gamma > 0$. These estimators include the Lasso as a special case when $\gamma = 1$. Under appropriate conditions, we show that the limiting distributions can have positive probability mass at 0 when the true value of the parameter is 0.We also consider asymptotics for “nearly singular” designs.

通过模型选择自适应估计二次泛函AI 解读

We consider the problem of estimating $\|s\|^2$ when $s$ belongs to some separable Hilbert space and one observes the Gaussian process $Y(t) = \langles, t\rangle + \sigmaL(t)$, for all $t \epsilon \mathbb{H}$,where $L$ is some Gaussian isonormal process. This framework allows us in particular to consider the classical “Gaussian sequence model” for which $\mathbb{H} = l_2(\mathbb{N}*)$ and $L(t) = \sum_{\lambda\geq1}t_{\lambda}\varepsilon_{\lambda}$, where $(\varepsilon_{\lambda})_{\lambda\geq1}$…

偏最小二乘回归的特殊收缩性质AI 解读

Summary Partial least squares regression has been widely adopted within some areas as a useful alternative to ordinary least squares regression in the manner of other shrinkage methods such as principal components regression and ridge regression. In this paper we examine the nature of this shrinkage and demonstrate that partial least squares regression exhibits some undesirable properties.

包含交互项与平滑效应的贝叶斯回归建模AI 解读

There have been many recent suggestions as to how to build and estimate flexible Bayesian regression models, using constructs such as trees, neural networks, and Gaussian processes. Although there is much to commend these methods, their implementation and interpretation can be daunting for practitioners. This article presents a spline-based methodology for flexible Bayesian regression that is quite simple in terms of computation and interpretation. Smooth bivariate…