Consider a partial linear model, where the expectation of a random variable Y depends on covariates $(x, z)$ through $F(\theta_0 x + m_0(z))$, with $\theta_0$ an unknown parameter, and $m_0$ an unknown function. We apply the theory of empirical processes to derive the asymptotic properties of the penalized quasi-likelihood estimator.
Least squares penalized regression estimates with total variation penalties are considered. It is shown that these estimators are least squares splines with locally data adaptive placed knot points. The definition of these variable knot splines as minimizers of global functionals can be used to study their asymptotic properties. In particular, these results imply that the estimates adapt well to spatially inhomogeneous smoothness. We show rates of convergence in bounded…
We give a stochastic expansion for the empirical distribution function $\hat{F}_n$ of residuals in a p-dimensional linear model. This expansion holds for p increasing with n. It shows that, for high-dimensional linear models, $\hat{F}_n$ strongly depends on the chosen estimator $\hat{\theta}$ of the parameter $\theta$ of the linear model. In particular, if one uses an ML-estimator $\hat{\theta}_{ML}$ which is ML motivated by a wrongly specified error distribution function G,…
Recent work of several authors has focussed on first-order properties (e.g., consistency) of general bootstrap algorithms, where the numbers of times that data values are resampled form an exchangeable sequence. In the present paper we develop second-order properties of such algorithms, in a very general setting. Performance is discussed in the context of distribution estimation, and formulae for higher-order moments and cumulants are developed. Arguing thus, necessary and…
In this paper two bootstrap procedures are considered for the estimation of the distribution of linear contrasts and of F-test statistics in high dimensional linear models. An asymptotic approach will be chosen where the dimension p of the model may increase for sample size $n\rightarrow\infty$. The range of validity will be compared for the normal approximation and for the bootstrap procedures. Furthermore, it will be argued that the rates of convergence are different for…
We propose a new nonparametric regression estimate. In contrast to the traditional approach of considering regression functions whose $m$th derivatives lie in a ball in the $L_\infty$ or $L_2$ norm, we consider the class of functions whose $(m - 1)$st derivative consists of at most $k$ monotone pieces. For many applications this class seems more natural than the classical ones. The least squares estimator of this class is studied. It is shown that the speed of convergence is…
The problem of estimating a smooth monotone regression function $m$ will be studied. We will consider the estimator $m_{SI}$ consisting of a smoothing step (application of a kernel estimator based on a kernel $K$) and of a isotonisation step (application of the pool adjacent violator algorithm). The estimator $m_{SI}$ will be compared with the estimator $m_{IS}$ where these two steps are interchanged. A higher order stochastic expansion of these estimators will be given…
A stochastic expansion for $M$-estimates in linear models with many parameters is derived under the weak condition $\kappa n^{1/3}(\log n)^{2/3} \rightarrow 0$, where $n$ is the sample size and $\kappa$ the maximal diagonal element of the hat matrix. The expansion is used to study the asymptotic distribution of linear contrasts and the consistency of the bootstrap. In particular, it turns out that bootstrap works in cases where the usual asymptotic approach fails.
Let $\mathscr{E}^n$ be a statistical experiment based on $n$ i.i.d. observations. We compare $\mathscr{E}^n$ with $\mathscr{E}^{n+r_n}$. The gain of information due to the $r_n$ additional observations is measured by the deficiency distance $\Delta (\mathscr{E}^n, \mathscr{E}^{n+r_n})$, i.e., the maximum diminution of the risk functions. We show that under general dimensionality conditions $\Delta(\mathscr{E}^n, \mathscr{E}^{n+r_n})$ is of order $r_n/n$. Further the behavior…
Consider a partial linear model, where the expectation of a random variable Y depends on covariates (x,z) through F(θ0x+m0(z)), with θ0 an unknown parameter, and m0 an unknown function. We apply the theory of empirical processes to derive the asymptotic properties of the penalized quasi-likelihood estimator.
We give a stochastic expansion for the empirical distribution function F^n of residuals in a p-dimensional linear model. This expansion holds for p increasing with n. It shows that, for high-dimensional linear models, F^n strongly depends on the chosen estimator θ^ of the parameter θ of the linear model. In particular, if one uses an ML-estimator θ^ML which is ML motivated by a wrongly specified error distribution function G,…
In this paper two bootstrap procedures are considered for the estimation of the distribution of linear contrasts and of F-test statistics in high dimensional linear models. An asymptotic approach will be chosen where the dimension p of the model may increase for sample size n→∞. The range of validity will be compared for the normal approximation and for the bootstrap procedures. Furthermore, it will be argued that the rates of convergence are different for…
We propose a new nonparametric regression estimate. In contrast to the traditional approach of considering regression functions whose mth derivatives lie in a ball in the L∞ or L2 norm, we consider the class of functions whose (m−1)st derivative consists of at most k monotone pieces. For many applications this class seems more natural than the classical ones. The least squares estimator of this class is studied. It is shown that the speed of convergence is…
The problem of estimating a smooth monotone regression function m will be studied. We will consider the estimator mSI consisting of a smoothing step (application of a kernel estimator based on a kernel K) and of a isotonisation step (application of the pool adjacent violator algorithm). The estimator mSI will be compared with the estimator mIS where these two steps are interchanged. A higher order stochastic expansion of these estimators will be given…
A stochastic expansion for M-estimates in linear models with many parameters is derived under the weak condition κn1/3(logn)2/3→0, where n is the sample size and κ the maximal diagonal element of the hat matrix. The expansion is used to study the asymptotic distribution of linear contrasts and the consistency of the bootstrap. In particular, it turns out that bootstrap works in cases where the usual asymptotic approach fails.
Let En be a statistical experiment based on n i.i.d. observations. We compare En with En+rn. The gain of information due to the rn additional observations is measured by the deficiency distance Δ(En,En+rn), i.e., the maximum diminution of the risk functions. We show that under general dimensionality conditions Δ(En,En+rn) is of order rn/n. Further the behavior…