论文检索

数据挖掘 LIFT 指标的渐近分布与置信区间AI 解读

A LIFT measure, such as the response rate, lift, or the percentage of captured response, is a fundamental measure of effectiveness for a scoring rule obtained from data mining, which is estimated from a set of validation data. In this article, we study how to construct confidence intervals of the LIFT measures. We point out the subtlety of this task and explain how simple binomial confidence intervals can have incorrect coverage probabilities, due to omitting variation from…

非加性经验风险下 Gibbs 后验的一般不等式AI 解读

The Gibbs posterior is a useful tool for risk minimization, which adopts a Bayesian framework and can incorporate convenient computational algorithms such as Markov chain Monte Carlo. We derive risk bounds for the Gibbs posterior using some general nonasymptotic inequalities, which can be used to derive nearly optimal convergence rates and select models to optimally balance the approximation errors and the stochastic errors. These inequalities are formulated in a very…

非参数条件矩限制模型的后验一致性AI 解读

This paper addresses the estimation of the nonparametric conditional moment restricted model that involves an infinite-dimensional parameter g0. We estimate it in a quasi-Bayesian way, based on the limited information likelihood, and investigate the impact of three types of priors on the posterior consistency: (i) truncated prior (priors supported on a bounded set), (ii) thin-tail prior (a prior that has very thin tail outside a growing bounded set) and (iii) normal prior…

带变量选择的时间序列二元选择风险最小化AI 解读

This paper considers the problem of predicting binary choices by selecting from a possibly large set of candidate explanatory variables, which can include both exogenous variables and lagged dependent variables. We consider risk minimization with the risk function being the predictive classification error. We study the convergence rates of empirical risk minimization in both the frequentist and Bayesian approaches. The Bayesian treatment uses a Gibbs posterior constructed…

矩不等式模型的贝叶斯分析AI 解读

This paper presents a study of the large-sample behavior of the posterior distribution of a structural parameter which is partially identified by moment inequalities. The posterior density is derived based on the limited information likelihood. The posterior distribution converges to zero exponentially fast on any δ-contraction outside the identified region. Inside, it is bounded below by a positive constant if the identified region is assumed to have a nonempty interior.…

高维分类与数据挖掘中的 Gibbs 后验变量选择AI 解读

In the popular approach of “Bayesian variable selection” (BVS), one uses prior and posterior distributions to select a subset of candidate variables to enter the model. A completely new direction will be considered here to study BVS with a Gibbs posterior originating in statistical mechanics. The Gibbs posterior is constructed from a risk function of practical interest (such as the classification error) and aims at minimizing a risk function without modeling the data…

Bayesian variable selection has gained much empirical success recently in a variety of applications when the number K of explanatory variables $(x_1,\ldots,x_K)$ is possibly much larger than the sample size $n$. For generalized linear models, if most of the $x_j$’s have very small effects on the response $y$, we show that it is possible to use Bayesian variable selection to reduce overfitting caused by the curse of dimensionality $K\gg n$. In this approach a suitable prior…

关于提升方法论文的讨论及作者回应AI 解读

Discussions of: "Process consistency for AdaBoost" [Ann. Statist. 32 (2004), no. 1, 13-29] by W. Jiang; "On the Bayes-risk consistency of regularized boosting methods" [ibid., 30-55] by G. Lugosi and N. Vayatis; and "Statistical behavior and consistency of classification methods based on convex risk minimization" [ibid., 56-85] by T. Zhang. Includes rejoinders by the authors.

AdaBoost 的过程相合性AI 解读

Recent experiments and theoretical studies show that AdaBoost can overfit in the limit of large time. If running the algorithm forever is suboptimal, a natural question is how low can the prediction error be during the process of AdaBoost? We show under general regularity conditions that during the process of AdaBoost a consistent prediction is generated, which has the prediction error approximating the optimal Bayes error as the sample size increases. This result suggests…

弱基假设及其对提升回归与分类的含义AI 解读

When studying the training error and the prediction error for boosting, it is often assumed that the hypotheses returned by the base learner are weakly accurate, or are able to beat a random guesser by a certain amount of difference. It has been an open question how much this difference can be, whether it will eventually disappear in the boosting process or be bounded by a positive amount. This question is crucial for the behavior of both the training error and the…

样条非参数回归中的模型选择AI 解读

Summary A Bayesian approach is presented for model selection in nonparametric regression with Gaussian errors and in binary nonparametric regression. A smoothness prior is assumed for each component of the model and the posterior probabilities of the candidate models are approximated using the Bayesian information criterion. We study the model selection method by simulation and show that it has excellent frequentist properties and gives improved estimates of the regression…

非参数回归问题中的参数化与推断AI 解读

Summary We consider local likelihood or local estimating equations, in which a multivariate function Θ(·) is estimated but a derived function λ(·) of Θ(·) is of interest. In many applications, when most naturally formulated the derived function is a non-linear function of Θ(·). In trying to understand whether the derived non-linear function is constant or linear, a problem arises with this approach: when the function is actually constant or linear, the expectation of the…

指数族回归的层次专家混合模型：逼近与最大似然估计AI 解读

We consider hierarchical mixtures-of-experts (HME) models where exponential family regression models with generalized linear mean functions of the form $\psi(\alpha + \mathbf{x}^T \mathbf{\beta})$ are mixed. Here $\psi(\cdot)$ is the inverse link function. Suppose the true response $y$ follows an exponential family regression model with mean function belonging to a class of smooth functions of the form $\psi(h(\mathbf{x}))$ where $h(\cdot)\in W_{2; K_0}^{\infty}$ (a Sobolev…

含随机效应与测量误差的重复事件半参数回归AI 解读

Statistical methodology is presented for the regression analysis of multiple events in the presence of random effects and measurement error. Omitted covariates are modeled as random effects. Our approach to parameter estimation and significance testing is to start with a naive model of semiparametric Poisson process regression, and then to adjust for random effects and any possible covariate measurement error. We illustrate the techniques with data from a randomized clinical…

含随机效应与测量误差的重复事件半参数回归模型AI 解读

Statistical methodology is presented for the regression analysis of multiple events in the presence of random effects and measurement error. Omitted covariates are modeled as random effects. Our approach to parameter estimation and significance testing is to start with a naive model of semiparametric Poisson process regression, and then to adjust for random effects and any possible covariate measurement error. We illustrate the techniques with data from a randomized clinical…