By making use of martingale representations, we derive the asymptotic normality of particle filters in hidden Markov models and a relatively simple formula for their asymptotic variances. Although repeated resamplings result in complicated dependence among the sample paths, the asymptotic variance formula and martingale representations lead to consistent estimates of the standard errors of the particle filter estimates of the hidden states.
Tze Leung Lai · Shulamith T. Gross · David Bo Shen·刊期 2011-10-01 · 卷 39（5）·The Annals of Statistics已编入正刊
概率预测的评价AI 解读
Probability forecasts of events are routinely used in climate predictions, in forecasting default probabilities on bank loans or in estimating the probability of a patient’s positive response to treatment. Scoring rules have long been used to assess the efficacy of the forecast probabilities after observing the occurrence, or nonoccurrence, of the predicted events. We develop herein a statistical theory for scoring rules and propose an alternative approach to the evaluation…
Summary We develop a new approach to using estimating equations to estimate marginal regression models for longitudinal data with time-dependent covariates. Our approach classifies time-dependent covariates into three types—types I, II and III. The type of covariate determines what estimating equations can be used involving the covariate. We use the generalized method of moments to make optimal use of the estimating equations that are made available by the covariates.…
Tze Leung Lai · Wenzhi Li·刊期 2006-09-01 · 卷 93（3）·Biometrika已编入正刊
随机组大小的组序贯试验置信区间及生存分析应用AI 解读
A new ordering scheme for defining quantiles of the multivariate distribution of a stopping time and a stopped stochastic process is introduced. This ordering scheme is used in conjunction with resampling methods to construct confidence intervals for a population mean following a group sequential test with random group sizes, and for the regression parameter of a proportional hazards model following a time-sequential clinical trial with censored survival data. It is shown…
Stochastic approximation, introduced by Robbins and Monro in 1951, has become an important and vibrant subject in optimization, control and signal processing. This paper reviews Robbins' contributions to stochastic approximation and gives an overview of several related developments.
We consider a variant of the conventional neural network model, called the stochastic neural network, that can be used to approximate complex nonlinear stochastic systems. We show how the expectation-maximization algorithm can be used to develop efficient estimation schemes that have much lower computational complexity than those for conventional neural networks. This enables us to carry out model selection procedures, such as the Bayesian information criterion, to choose…
Asymptotic approximations for the error probabilities of sequential tests of composite hypotheses in multiparameter exponential families are developed herein for a general class of test statistics, including generalized likelihood ratio statistics and other functions of the sufficient statistics. These results not only generalize previous approximations for Type I error probabilities of sequential generalized likelihood ratio tests, but also pro- vide a unified treatment of…
In many prospective and retrospective studies, survival data are subject to left truncation in addition to the usual right censoring. For left-truncated data without covariates, only the conditional distribution of the survival time Y given Y ≥ τ can be estimated nonparametrically, where τ is the lower boundary of the support of the left-truncation variable T. If the data are also right censored, then the conditional distribution can be consistently estimated only at points…
SUMMARY After a brief survey of a large variety of sequential detection procedures that are widely scattered in statistical references on quality control and engineering references on fault detection and signal processing, we study some open problems concerning these procedures and introduce a unified theory of sequential changepoint detection. This theory leads to a class of sequential detection rules which are not too demanding in computational and memory requirements for…
Stochastic regression models of the form $y_i = f_i(\theta) + \varepsilon_i$, where the random disturbances $\varepsilon_i$ form a martingale difference sequence with respect to an increasing sequence of $\sigma$-fields $\{\mathcal{G}_i\}$ and $f_i$ is a random $\mathcal{G}_{i - 1}$-measurable function of an unknown parameter $\theta$, cover a broad range of nonlinear (and linear) time series and stochastic process models. Herein strong consistency and asymptotic normality…
A general missing information principle is proposed for constructing $M$-estimators of regression parameters in the presence of left truncation and right censoring on the observed responses. By making use of martingale central limit theorems and empirical process theory, the asymptotic normality of $M$-estimators is established under certain assumptions. Asymptotically efficient $M$-estimators are also developed by using data-dependent score functions. In addition,…
A general weak convergence theory is developed for time-sequential censored rank statistics in the two-sample problem of comparing time to failure between two treatment groups, such as in the case of a clinical trial in which patients enter serially and, after being randomly allocated to one of two treatments, are followed until they fail or withdraw from the study or until the study is terminated. Applications of the theory to time-sequential tests based on these censored…
Buckley and James proposed an extension of the classical least squares estimator to the censored regression model. It has been found in some empirical and Monte Carlo studies that their approach provides satisfactory results and seems to be superior to other extensions of the least squares estimator in the literature. To develop a complete asymptotic theory for this approach, we introduce herein a slight modification of the Buckley-James estimator to get around the…
A class of rank estimators is introduced for regression analysis in the presence of both left-truncation and right-censoring on the response variable. By making use of martingale theory and a tightness lemma for stochastic integrals of multiparameter empirical processes, the asymptotic normality of the estimators is established under certain assumptions. Adaptive choice of the score functions to give asymptotically efficient rank estimators is also discussed.
A minor modification of the product-limit estimator is proposed for estimating a distribution function (not necessarily continuous) when the data are subject to either truncation or censoring, or to both, by independent but not necessarily identically distributed truncation-censoring variables. Making use of martingale integral representations and empirical process theory, uniform strong consistency of the estimator is established and weak convergence results are proved for…
A class of simple adaptive allocation rules is proposed for the problem (often called the "multi-armed bandit problem") of sampling $x_1, \cdots x_N$ sequentially from $k$ populations with densities belonging to an exponential family, in order to maximize the expected value of the sum $S_N = x_1 + \cdots + x_N$. These allocation rules are based on certain upper confidence bounds, which are developed from boundary crossing theory, for the $k$ population parameters. The rules…
Strong consistency and asymptotic normality of least squares estimates in stochastic regression models are established under certain weak assumptions on the stochastic regressors and errors. We discuss applications of these results to interval estimation of the regression parameters and to recursive on-line identification and control schemes for linear dynamic systems.
It is well known that Wald's SPRT for testing simple hypotheses based on i.i.d. observations minimizes the expected sample size both under the null and under the alternative hypotheses among all tests with the same or smaller error probabilities and with finite expected sample sizes under the two hypotheses. In this paper it is shown that this optimum property can be extended, at least asymptotically as the error probabilities tend to 0, to invariant SPRTs like the…
While usual sequential analysis deals with i.i.d. observations, this paper studies sequential tests for the dependent case of sampling without replacement from a finite population. A general weak convergence theorem is obtained and it is applied to the asymptotic analysis of the tests. Motivated by such applications as election predictions and acceptance sampling, the case of hypergeometric populations is studied in detail and a simple test with a triangular continuation…
In this paper Pitman's method of constructing and comparing tests based on statistics which are asymptotically normal under the null hypothesis and the local alternatives is extended to sequential tests of statistical hypotheses. The asymptotic normality assumption in Pitman's theory is replaced in its sequential analogue by the weak convergence of normalized processes formed from these statistics under the null hypothesis and the local alternatives. Uniform invariance…
This paper studies the properties of open-ended power-one tests of $H_0: \theta \leqq \theta_0$ versus $H_1: \theta > \theta_0$ or of $H: \theta = \theta_0$ versus $K: \theta \neq \theta_0$ based on sample sums stopped at moving boundaries. The behavior of the expected sample size is analyzed and certain asymptotic results as $\theta \rightarrow \theta_0$ are obtained in the case of a location parameter and also in the case of an exponential family of distributions.
In this paper, we shall represent a generalized Chernoff-Savage statistic as the sum of i.i.d. random variables plus a remainder term and analyze the order of magnitude of the remainder term. While Chernoff and Savage have proved that the remainder term, when suitably normalized, converges to O in probability, we obtain a stronger form of convergence in this paper. Our result gives an invariance principle and a law of the iterated logarithm for generalized Chernoff-Savage…
It is well known that Wald's SPRT terminates with probability one and in fact the stopping time is exponentially bounded for every distribution $P$ except in the trivial case where the $\log$ likelihood ratio vanishes with probability one. The results in the literature for invariant SPRT's, however, have been considerably less complete. In all the parametric problems studied, moment conditions of certain random variables have been assumed to prove termination, and the…
In a continuous production process, samples of fixed size are taken at regular intervals of time and a statistic $X_n$ is computed from the $n$th sample, $n = 1,2, \cdots$. In this paper, we study process inspection schemes which stop the production and take corrective action with $N = \text{first} n \geqq 1$ such that $\sum^n_{i=1} c_{n-i} X_i \geqq h$, where $h$ is a preassigned constant and $c_0 \geqq c_1 \geqq \cdots \geqq c_{k-1} > 0 = c_k = c_{k+1} = \cdots$ is a…
Stochastic regression models of the form yi=fi(θ)+εi, where the random disturbances εi form a martingale difference sequence with respect to an increasing sequence of σ-fields {Gi} and fi is a random Gi−1-measurable function of an unknown parameter θ, cover a broad range of nonlinear (and linear) time series and stochastic process models. Herein strong consistency and asymptotic normality…
A Missing Information Principle and M-Estimators in Regression Analysis with Censored and Truncated Data
A general missing information principle is proposed for constructing M-estimators of regression parameters in the presence of left truncation and right censoring on the observed responses. By making use of martingale central limit theorems and empirical process theory, the asymptotic normality of M-estimators is established under certain assumptions. Asymptotically efficient M-estimators are also developed by using data-dependent score functions. In addition,…
A class of simple adaptive allocation rules is proposed for the problem (often called the "multi-armed bandit problem") of sampling x1,⋯xN sequentially from k populations with densities belonging to an exponential family, in order to maximize the expected value of the sum SN=x1+⋯+xN. These allocation rules are based on certain upper confidence bounds, which are developed from boundary crossing theory, for the k population parameters. The rules…
This paper studies the properties of open-ended power-one tests of H0:θ≦θ0 versus H1:θ>θ0 or of H:θ=θ0 versus K:θ=θ0 based on sample sums stopped at moving boundaries. The behavior of the expected sample size is analyzed and certain asymptotic results as θ→θ0 are obtained in the case of a location parameter and also in the case of an exponential family of distributions.
It is well known that Wald's SPRT terminates with probability one and in fact the stopping time is exponentially bounded for every distribution P except in the trivial case where the log likelihood ratio vanishes with probability one. The results in the literature for invariant SPRT's, however, have been considerably less complete. In all the parametric problems studied, moment conditions of certain random variables have been assumed to prove termination, and the…
In a continuous production process, samples of fixed size are taken at regular intervals of time and a statistic Xn is computed from the nth sample, n=1,2,⋯. In this paper, we study process inspection schemes which stop the production and take corrective action with N=firstn≧1 such that ∑i=1ncn−iXi≧h, where h is a preassigned constant and c0≧c1≧⋯≧ck−1>0=ck=ck+1=⋯ is a…