The flexibility of the restricted least squares (RLS) method described by Buse and Lim (1977) for cubic spline regression models is used to consider other polynomial splines. Some of the conclusions of the analysis of a sample data set in the Buse and Lim article are criticized for failing to consider less restrictive models. The data are reanalyzed, the end result being a simpler quadratic spline regression model.
Let $X_1, X_2, \cdots, X_n$ be i.i.d. random variables with common density function $f$. A method of density estimation based on "delta sequences" is studied and mean square rates established. This method generalizes certain others including kernel estimators, orthogonal series estimators, Fourier transform estimators, and the histogram. Rates are obtained for densities in Sobolev spaces and for densities satisfying Lipschitz conditions. The former generalizes some results…
The objective in nonparametric regression is to infer a function $m(x)$ on the basis of a finite collection of noisy pairs $\{(X_i, m(X_i) + N_i)\}^n_{i=1}$, where the noise components $N_i$ satisfy certain lenient assumptions and the domain points $X_i$ are selected at random. It is known a priori only that $m$ is a member of a nonparametric class of functions (that is, a class of functions like $C\lbrack 0, 1\rbrack$ which, under customary topologies, does not admit a…
Orthogonal series estimators of univariate densities are proposed that are derived from and motivated by kernel estimators optimal in Whittle's (1958) sense. A preliminary fit of the data from within a one or two parameter class of densities plays the role of a prior mean density. The ratio of the true density and the prior mean density is assumed to have a series expansion in terms of functions orthogonal with respect to the prior mean density. Coefficients of terms of the…
Summary Spline and generalized spline smoothing is shown to be equivalent to Bayesian estimation with a partially improper prior. This result supports the idea that spline smoothing is a natural solution to the regression problem when one is given a set of regression functions but one also wants to hedge against the possibility that the true model is not exactly in the span of the given regression functions. A natural measure of the deviation of the true model from the span…
The estimation of a density and its derivatives by the kernel method is considered. Uniform consistency properties over the whole real line are studied. For suitable kernels and uniformly continuous densities it is shown that the conditions $h \rightarrow 0$ and $(nh)^{-1} \log n \rightarrow 0$ are sufficient for strong uniform consistency of the density estimate, where $n$ is the sample size and $h$ is the "window width." Under certain conditions on the kernel, conditions…
Orthogonal series estimators of univariate densities are proposed that are derived from and motivated by kernel estimators optimal in Whittle's (1958) sense. A preliminary fit of the data from within a one or two parameter class of densities plays the role of a prior mean density. The ratio of the true density and the prior mean density is assumed to have a series expansion in terms of functions orthogonal with respect to the prior mean density. Coefficients of terms of the…
D. M. Titterington·刊期 1977-12 · 卷 64(3)·Biometrika已编入正刊
不完整多元二元数据的核方法分析AI 解读
The procedure, due to Aitchison & Aitken (1976), of analyzing multivariate binary data by kernel methods is extended to deal with missing data problems. The performance of the techniques is discussed for univariate, bivariate and for specific multivariate data. Comparisons are drawn with other approaches.
Summary This paper presents a new family of non-parametric estimators of a smooth regression function, which are shown to have theoretical advantages in small samples over some alternative estimators. The new estimator has practical computational advantages over spline functions, while its smoothing properties are very close to the optimal smoothing properties of cubic splines. A simulation experiment demonstrates the remarkable success of the cross-validation technique as a…
Let $T \subseteqq I$ be sets of real numbers. Let $\{Y(t): t \in I\}$ be a real time series whose mean is an unknown element of a known class of functions on $I$ and whose covariance kernel $k$ is assumed known. For each fixed $s \in I, Y(s)$ is predicted by a minimum mean square error unbiased linear predictor $\hat{Y}(s)$ based on $\{Y(t): t \in T\}$. If $\hat{y}(s)$ is the evaluation of $\hat{Y}(s)$ given a set of observations $\{Y(t) = g(t): t \in T\}$, then the function…
Estimation of the value of a density function at a point of continuity using a kernel-type estimator is discussed and improvements of the technique are presented. The generalized jackknife method is employed to reduce the asymptotic and small sample mean square error and bias of these estimators. The procedure presented has the flexibility to afford the user a choice between bias reduction, variance reduction, or both.
The rate at which the mean integrated square error decreases as sample size increases is evaluated for general $L^1$ kernel estimates and for the Fourier integral estimate for a probability density. The rates are compared to that of the minimum M.I.S.E.; the Fourier integral estimate is found to be asymptotically optimal.
Based on a random sample from a univariate distribution with density $f$, this note exhibits a class of kernel estimators of the $p$th order derivative $f^{(p)}$ of $f, p \geqq 0$ fixed. These estimators improve some known estimators of $f^{(p)}$ by weakening the conditions, sharpening the rates of convergence, or both for the properties of strong consistency, asymptotic unbiasedness and mean square consistency, each uniform on the real line.
In establishing weak convergence of von Mises' differentiable statistical functions to a normal distribution usually square integrability conditions with respect to the underlying kernel function are assumed. It is shown that these conditions can be weakened by assuming integrability of the von Mises' functional itself. In addition it is pointed out that in nontrivial cases the conditions of square integrability of the kernel do not hold whereas weak convergence of the von…
The standard cubic spline regression method is shown to be a special case of the restricted least-squares estimator. The equivalence of the two procedures under a common set of restrictions is proved. The greater flexibility of the restricted least-squares estimator in terms of the number of restrictions and tests of hypotheses that can be utilized is illustrated by an application to a set of data that has been previously analyzed by the spline method.
Let $X_1, X_2,\cdots$ be $R^p$-valued random variables having unknown density function $f$. If $K$ is a density on the unit sphere in $R^p, \{k(n)\}$ a sequence of positive integers such that $k(n) \rightarrow \infty$ and $k(n) = o(n)$, and $R(k, z)$ is the distance from a point $z$ to the $k(n)$th nearest of $X_1,\cdots, X_n$, then $f_n(z) = (nR(k, z)^p)^{-1} \sum K((z - X_i)/R(k, z))$ is a nearest neighbor estimator of $f(z).$ When $K$ is the uniform kernel, $f_n$ is an…
D. M. TITTERINGTON·刊期 1977 · 卷 64(3)·Biometrika已编入正刊
利用核方法分析不完整多元二元数据AI 解读
The procedure, due to Aitchison & Aitken (1976), of analyzing multivariate binary data by kernel methods is extended to deal with missing data problems. The performance of the techniques is discussed for univariate, bivariate and for specific multivariate data. Comparisons are drawn with other approaches.
J. Aitchison · C. G. G. Aitken·刊期 1976-12 · 卷 63(3)·Biometrika已编入正刊
核方法在多元二元数据判别中的应用AI 解读
An extension of the kernel method of density estimation from continuous to multivariate binary spaces is described. Its simple nonparametric nature together with its consistency properties make it an attractive tool in discrimination problems, with some advantages over already proposed parametric counterparts. The method is illustrated by an application to a particular medical diagnostic problem. Simple extensions of the method to categorical data and to data of mixed binary…
This paper develops some efficient algorithms for linear spline and piecewise multiple linear regression. A plotting procedure that shows the existence and location of changes in linear regression models is developed as an adjunct to one of the algorithms. The algorithms are compared with other presently available algorithms both in terms of efficiency and in terms of performance on sets of artificial data. An example shows how the algorithms, implemented in portable FORTRAN…
Summary The histospline considered here is a quadratic spline density estimate analogous to the Boneva–Kendall–Stefanov (BKS) histospline, with their equally spaced knots replaced by knots at every knth-order statistic. This estimate can be expected to be relatively more “flexible” where the density is greatest. We obtain “mean square error at a point” convergence rates. The rates obtained are uniform over classes of densities which have first (alternatively second or third)…
J. AITCHISON · C. G. G. AITKEN·刊期 1976 · 卷 63(3)·Biometrika已编入正刊
基于核方法的多元二元数据判别AI 解读
An extension of the kernel method of density estimation from continuous to multivariate binary spaces is described. Its simple nonparametric nature together with its consistency properties make it an attractive tool in discrimination problems, with some advantages over already proposed parametric counterparts. The method is illustrated by an application to a particular medical diagnostic problem. Simple extensions of the method to categorical data and to data of mixed binary…
G. F. de Montricher · R. A. Tapia · J. R. Thompson·刊期 1975-11-01 · 卷 3(6)·The Annals of Statistics已编入正刊
惩罚函数法下概率密度的非参数最大似然估计AI 解读
Tne maximum likelihood estimate of a probability density function based on a random sample does not exist in the nonparametric case. For this reason and others based on heuristic Bayesian considerations Good and Gaskins suggested adding a penalty term to the likelihood. They proposed two penalty terms; however they did not establish existence or uniqueness of their maximum penalized likelihood estimates. Good and Gaskins also suggested an alternate approach for calculating…
The rate at which the mean square error decreases as sample size increases is evaluated for general $L^1$ kernel estimates and for the Fourier integral estimate for a probability density function. The estimates are then compared on the basis of these rates.
Let X1,X2,⋯,Xn be i.i.d. random variables with common density function f. A method of density estimation based on "delta sequences" is studied and mean square rates established. This method generalizes certain others including kernel estimators, orthogonal series estimators, Fourier transform estimators, and the histogram. Rates are obtained for densities in Sobolev spaces and for densities satisfying Lipschitz conditions. The former generalizes some results…
The objective in nonparametric regression is to infer a function m(x) on the basis of a finite collection of noisy pairs {(Xi,m(Xi)+Ni)}i=1n, where the noise components Ni satisfy certain lenient assumptions and the domain points Xi are selected at random. It is known a priori only that m is a member of a nonparametric class of functions (that is, a class of functions like C[0,1] which, under customary topologies, does not admit a…
The estimation of a density and its derivatives by the kernel method is considered. Uniform consistency properties over the whole real line are studied. For suitable kernels and uniformly continuous densities it is shown that the conditions h→0 and (nh)−1logn→0 are sufficient for strong uniform consistency of the density estimate, where n is the sample size and h is the "window width." Under certain conditions on the kernel, conditions…
Let T⫅I be sets of real numbers. Let {Y(t):t∈I} be a real time series whose mean is an unknown element of a known class of functions on I and whose covariance kernel k is assumed known. For each fixed s∈I,Y(s) is predicted by a minimum mean square error unbiased linear predictor Y^(s) based on {Y(t):t∈T}. If y^(s) is the evaluation of Y^(s) given a set of observations {Y(t)=g(t):t∈T}, then the function…
The rate at which the mean integrated square error decreases as sample size increases is evaluated for general L1 kernel estimates and for the Fourier integral estimate for a probability density. The rates are compared to that of the minimum M.I.S.E.; the Fourier integral estimate is found to be asymptotically optimal.
Based on a random sample from a univariate distribution with density f, this note exhibits a class of kernel estimators of the pth order derivative f(p) of f,p≧0 fixed. These estimators improve some known estimators of f(p) by weakening the conditions, sharpening the rates of convergence, or both for the properties of strong consistency, asymptotic unbiasedness and mean square consistency, each uniform on the real line.
Let X1,X2,⋯ be Rp-valued random variables having unknown density function f. If K is a density on the unit sphere in Rp,{k(n)} a sequence of positive integers such that k(n)→∞ and k(n)=o(n), and R(k,z) is the distance from a point z to the k(n)th nearest of X1,⋯,Xn, then fn(z)=(nR(k,z)p)−1∑K((z−Xi)/R(k,z)) is a nearest neighbor estimator of f(z). When K is the uniform kernel, fn is an…
The rate at which the mean square error decreases as sample size increases is evaluated for general L1 kernel estimates and for the Fourier integral estimate for a probability density function. The estimates are then compared on the basis of these rates.