The observation model $y_i = \beta(i/n) + \varepsilon_i, 1 \leq i \leq n$, is considered, where the $\varepsilon$'s are i.i.d. with mean zero and variance $\sigma^2$ and $\beta$ is an unknown smooth function. A Gaussian prior distribution is specified by assuming $\beta$ is the solution of a high order stochastic differential equation. The estimation error $\delta = \beta - \hat{\beta}$ is analyzed, where $\hat{\beta}$ is the posterior expectation of $\beta$. Asymptotic…
For experiments where the strength of association between a response variable $Y$ and a covariate $X$ is different over different regions of values for the covariate $X$, we propose local nonparametric dependence functions which measure the strength of association between $Y$ and $X$ as a function of $X = x$. Our dependence functions are extensions of Galton's idea of strength of co-relation from the bivariate normal case to the nonparametric case. In particular, a…
This paper studies the shapes of low dimensional projections from high dimensional data. After standardization, let $\mathbf{x}$ be a $p$-dimensional random variable with mean zero and identity covariance. For a projection $\beta'\mathbf{x}, \|\beta\| = 1$, find another direction $b$ so that the regression curve of $b'\mathbf{x}$ against $\beta'\mathbf{x}$ is as nonlinear as possible. We show that when the dimension of $\mathbf{x}$ is large, for most directions $\beta$ even…
The spectral distribution function of a stationary stochastic process standardized by dividing by the variance of the process is a linear function of the autocorrelations. The integral of the sample standardized spectral density (periodogram) is a similar linear function of the autocorrelations. As the sample size increases, the difference of these two functions multiplied by the square root of the sample size converges weakly to a Gaussian stochastic process with a…
This paper is mainly devoted to the following statistical problem: in the case of random variables of any finite dimension and both simple or parametric hypotheses, how to construct convenient "empirical" processes which could provide the basis for goodness of fit tests-more or less in the same way as the uniform empirical process does in the case of simple hypothesis and scalar random variables. The solution of this problem is connected here with the theory of…
Suppose that $X_1,\ldots,X_n$ are i.i.d. $\sim F$, and we wish to test the null hypothesis that $F$ is a member of the parametric family $\mathscr{F} = \{F_\theta(x); \theta \in \Theta\}$ where $\Theta \subset \mathbb{R}^q$. The classical Pearson-Fisher chi-square test involves partitioning the real axis into $k$ cells $I_1,\ldots, I_k$ and forming the chi-square statistic $X^2 = \sum^k_{i = 1}(O_i - nF_{\hat{\theta}}(I_i))^2/nF_{\hat{\theta}}(I_i)$, where $O_i$ is the…
In a recent paper Pukelsheim and Studden determined the $E$-optimal design for the polynomial regression model on the interval $\lbrack - 1, 1\rbrack$ where the variances of different observations are assumed to be constant. In this note we show that these results can be generalized for polynomial regression models with non constant variances proportional to specific functions.
We consider a model robust version of the $c$-optimality criterion minimizing a weighted product with factors corresponding to the variances of the least squares estimates for linear combinations of the parameters in different models. A generalization of Elfving's theorem is proved for the optimal designs with respect to this criterion by an application of an equivalence theorem for mixtures of optimality criteria. As a special case an Elfving theorem for the $D$-optimal…
This paper obtains some extensions of Gilliland and Hannan's results on equivariance and the compound decision problem. Consider a compound decision problem with restricted component risk and component distributions in a norm compact set of mutually absolutely continuous probability measures. Then the method of proof of a theorem of Gilliland and Hannan translates the results of Mashayekhi on symmetrization of product measures into uniform convergence to zero of the excess…
This paper investigates the density estimation problem in the $L^\infty$ norm for dependent data. It is shown that the iid optimal minimax rates are also optimal for smooth classes of stationary sequences satisfying certain $\beta$-mixing (or absolutely regular) conditions. Moreover, for given $\beta$-mixing coefficients, bounds on uniform convergence rates of kernel estimators are computed in terms of the mixing coefficients. The rates and the bounds obtained are not only…
Optimal local smoothing of a curve estimator requires knowledge of various derivatives of the curve in the neighbourhood of the point at which estimation is being conducted. One empirical approach to selecting the amount of smoothing is to employ pilot estimators to approximate those derivatives, and substitute the approximate values into an analytical formula for the desired local bandwidth. In the present paper we study how bandwidth choice for the pilot estimators affects…
There has been an increasing interest in modelling regression with heavy-tailed conditional error distributions, mostly in the parametric setting. Nonparametric regression procedures have been studied almost exclusively for the cases where the conditional variance of the regressed variable is finite in the region of interest. We initiate a study of the infinite variance case. Some results in strong uniform consistency of the nearest neighbor estimator with rates are proven.…
Pollak and Siegmund compared the Shiryayev-Roberts procedure with the CUSUM procedure for detecting a change in the drift of a Brownian motion based on the conditional average delay time. In this paper, the exponentially weighted moving average (EWMA) procedure proposed by Roberts is compared with the Shiryayev-Roberts and CUSUM procedures. The comparison is based on the stationary average delay time as advocated by Shiryayev. The optimal design for the EWMA procedure and…
Consider the problem of estimating $\mu$, based on the observation of $Y_0, Y_1, \ldots, Y_n$, where it is assumed only that $Y_0, Y_1, \ldots, Y_\kappa \operatorname{iid} N(\mu, \sigma^2)$ for some unknown $\kappa$. Unlike the traditional change-point problem, the focus here is not on estimating $\kappa$, which is now a nuisance parameter. When it is known that $\kappa = k$, the sample mean $\bar{Y}_k = \sum^k_0Y_i/(k + 1)$, provides, in addition to wonderful efficiency…
This paper concerns self-consistent estimators for survival functions based on doubly censored data. We establish strong uniform consistency, asymptotic normality and asymptotic efficiency of the estimators under mild conditions on the distributions of the censoring variables.
In this paper, a method for finding global minimax lower bounds is introduced. The idea is to adjust automatically the direction of a local one-dimensional subproblem at each location to the nearly hardest one, and to use locally the difficulty of the one-dimensional subproblem. This method has the advantages of being easily implemented and understood. The lower bound is then applied to nonparametric deconvolution to obtain the optimal rates of convergence for estimating a…
Consider an unknown regression function $f$ of the response $Y$ on a $d$-dimensional measurement variable $X$. It is assumed that $f$ belongs to a class of functions having a smoothness measure $p$. Let $T$ denote a known linear operator of order $q$ which maps $f$ to another function $T(f)$ in a space $G$. Let $\hat{T}_n$ denote an estimator of $T(f)$ based on a random sample of size $n$ from the distribution of $(X, Y)$, and let $\|\hat{T}_n - T(f)\|_G$ be a norm of $\hat{T}_n - T(f)$…
Renormalization arguments are used to derive optimal rates of convergence, under integrated squared error loss, for parameter spaces having a certain rectangular structure.
We compute the asymptotic distribution of the maximum likelihood ratio test when we want to check whether the parameters of normal observations have changed at an unknown point. The proof is based on the limit distribution of the largest deviation between a $d$-dimensional Ornstein-Uhlenbeck process and the origin.
It is shown that serial correlation can produce striking effects in distributions of change-point statistics. Failure to account for these effects is shown to invalidate change-point tests, either through increases in the type 1 error rates if low frequency spectral mass predominates in the spectrum of the noise process, or through diminution of the power of the tests when high frequency mass predominates. These effects are characterized by the expression ${2\pi f(0)/\int^\pi_-\pi f(\lambda)d\lambda}$…
In an AR(p) model, R-estimation of a subset of parameters is considered when the complementary subset is possibly redundant. Along with the rank test of the full hypothesis and the subhypothesis of the parameters, both preliminary test and shrinkage R-estimators are considered. In the light of asymptotic distributional risk, the relative asymptotic risk-efficiency results are given. Though, the shrinkage R-estimator may dominate their classical versions, they do not in…
Let $S_1$, $S_2,\cdots$ denote a driftless random walk with values in an inner product space $\mathscr{W}$; let $Z_1$, $Z_2,\cdots$ denote a perturbed random walk of the form $Z_n=n+\langle c,S_n \rangle+\xi_n$, $n = 1, 2,\cdots$, where $\xi_1,\xi_2,\cdots$ are slowly changing, $\langle\centerdot,\centerdot\rangle$ denotes the inner product, and $c\in\mathscr{W}$; and let $t=t_a=inf{n\geq1:Z_n>a}$ for $0\leq a<\infty$. Conditions are developed under which the first four…
We consider the problem of sequentially estimating one parameter in a class of two-parameter exponential family of distributions. We assume a weighted squared error loss with a fixed cost of estimation error. The stopping rule, based on the maximum likelihood estimate of the nuisance parameter, is shown to be independent of the terminal estimate. The asymptotic normality of the stopping variable is established and approximations to its mean and to the regret associated with…
Model robustness in optimal regression design is studied by introducing a family of nonparametric models, which are defined as neighborhoods of classical parametric models in terms of the uniform norm. Optimal designs are sought under a minimax criterion for estimating linear functionals on such models that may be put as integrals using measures of finite support. A set of conditions equivalent to design optimality is derived using a Lagrangian principle applicable when the…
The observation model yi=β(i/n)+εi,1≤i≤n, is considered, where the ε's are i.i.d. with mean zero and variance σ2 and β is an unknown smooth function. A Gaussian prior distribution is specified by assuming β is the solution of a high order stochastic differential equation. The estimation error δ=β−β^ is analyzed, where β^ is the posterior expectation of β. Asymptotic…
For experiments where the strength of association between a response variable Y and a covariate X is different over different regions of values for the covariate X, we propose local nonparametric dependence functions which measure the strength of association between Y and X as a function of X=x. Our dependence functions are extensions of Galton's idea of strength of co-relation from the bivariate normal case to the nonparametric case. In particular, a…
This paper studies the shapes of low dimensional projections from high dimensional data. After standardization, let x be a p-dimensional random variable with mean zero and identity covariance. For a projection β′x,∥β∥=1, find another direction b so that the regression curve of b′x against β′x is as nonlinear as possible. We show that when the dimension of x is large, for most directions β even…
Suppose that X1,…,Xn are i.i.d. ∼F, and we wish to test the null hypothesis that F is a member of the parametric family F={Fθ(x);θ∈Θ} where Θ⊂Rq. The classical Pearson-Fisher chi-square test involves partitioning the real axis into k cells I1,…,Ik and forming the chi-square statistic X2=∑i=1k(Oi−nFθ^(Ii))2/nFθ^(Ii), where Oi is the…
A Note on E-Optimal Designs for Weighted Polynomial Regression
In a recent paper Pukelsheim and Studden determined the E-optimal design for the polynomial regression model on the interval [−1,1] where the variances of different observations are assumed to be constant. In this note we show that these results can be generalized for polynomial regression models with non constant variances proportional to specific functions.
Elfving's Theorem for D-Optimality
We consider a model robust version of the c-optimality criterion minimizing a weighted product with factors corresponding to the variances of the least squares estimates for linear combinations of the parameters in different models. A generalization of Elfving's theorem is proved for the optimal designs with respect to this criterion by an application of an equivalence theorem for mixtures of optimality criteria. As a special case an Elfving theorem for the D-optimal…
Density Estimation in the L∞ Norm for Dependent Data with Applications to the Gibbs Sampler
This paper investigates the density estimation problem in the L∞ norm for dependent data. It is shown that the iid optimal minimax rates are also optimal for smooth classes of stationary sequences satisfying certain β-mixing (or absolutely regular) conditions. Moreover, for given β-mixing coefficients, bounds on uniform convergence rates of kernel estimators are computed in terms of the mixing coefficients. The rates and the bounds obtained are not only…
Consider the problem of estimating μ, based on the observation of Y0,Y1,…,Yn, where it is assumed only that Y0,Y1,…,YκiidN(μ,σ2) for some unknown κ. Unlike the traditional change-point problem, the focus here is not on estimating κ, which is now a nuisance parameter. When it is known that κ=k, the sample mean Yˉk=∑0kYi/(k+1), provides, in addition to wonderful efficiency…
Consider an unknown regression function f of the response Y on a d-dimensional measurement variable X. It is assumed that f belongs to a class of functions having a smoothness measure p. Let T denote a known linear operator of order q which maps f to another function T(f) in a space G. Let T^n denote an estimator of T(f) based on a random sample of size n from the distribution of (X,Y), and let ∥T^n−T(f)∥G be a norm of T^n−T(f)…
We compute the asymptotic distribution of the maximum likelihood ratio test when we want to check whether the parameters of normal observations have changed at an unknown point. The proof is based on the limit distribution of the largest deviation between a d-dimensional Ornstein-Uhlenbeck process and the origin.
It is shown that serial correlation can produce striking effects in distributions of change-point statistics. Failure to account for these effects is shown to invalidate change-point tests, either through increases in the type 1 error rates if low frequency spectral mass predominates in the spectrum of the noise process, or through diminution of the power of the tests when high frequency mass predominates. These effects are characterized by the expression 2πf(0)/∫−ππf(λ)dλ…
Let S1, S2,⋯ denote a driftless random walk with values in an inner product space W; let Z1, Z2,⋯ denote a perturbed random walk of the form Zn=n+⟨c,Sn⟩+ξn, n=1,2,⋯, where ξ1,ξ2,⋯ are slowly changing, ⟨⋅,⋅⟩ denotes the inner product, and c∈W; and let t=ta=infn≥1:Zn>a for 0≤a<∞. Conditions are developed under which the first four…