论文检索

Beta 树：具有置信陈述的多元直方图AI 解读

Multivariate histograms are difficult to construct due to the curse of dimensionality. Motivated by k-d trees in computer science, we show how to construct an efficient data-adaptive partition of Euclidean space that possesses the following two properties: With high confidence the distribution from which the data are generated is close to uniform on each rectangle of the partition; and despite the data-dependent construction we can give guaranteed finite sample simultaneous…

扫描统计量的校准：有限样本表现与渐近理论AI 解读

Abstract We consider the problem of detecting an elevated mean on an interval with unknown location and length in the univariate Gaussian sequence model. Recent results have shown that using scale-dependent critical values for the scan statistic allows to attain asymptotically optimal detection simultaneously for all signal lengths, thereby improving on the traditional scan, but this procedure has been criticised for losing too much power for short signals. We explain this…

具有块结构的大规模统计推断AI 解读

The detection of weak and rare effects in large amounts of data arises in a number of modern data analysis problems. Known results show that in this situation the potential of statistical inference is severely limited by the large-scale multiple testing that is inherent in these problems. Here, we show that fundamentally more powerful statistical inference is possible when there is some structure in the signal that can be exploited, for example, if the signal is clustered in…

本质直方图AI 解读

Summary The histogram is widely used as a simple, exploratory way of displaying data, but it is usually not clear how to choose the number and size of the bins. We construct a confidence set of distribution functions that optimally deal with the two main tasks of the histogram: estimating probabilities and detecting features such as increases and modes in the distribution. We define the essential histogram as the histogram in the confidence set with the fewest bins. Thus the…

多样本对齐稀疏信号的最优检测AI 解读

We describe, in the detection of multi-sample aligned sparse signals, the critical boundary separating detectable from nondetectable signals, and construct tests that achieve optimal detectability: penalized versions of the Berk–Jones and the higher-criticism test statistics evaluated over pooled scans, and an average likelihood ratio over the critical boundary. We show in our results an inter-play between the scale of the sequence length to signal length ratio, and the…

利用扫描统计量最优且快速地检测空间聚集AI 解读

We consider the detection of multivariate spatial clusters in the Bernoulli model with N locations, where the design distribution has weakly dependent marginals. The locations are scanned with a rectangular window with sides parallel to the axes and with varying sizes and aspect ratios. Multivariate scan statistics pose a statistical problem due to the multiple testing over many scan windows, as well as a computational problem because statistics have to be evaluated on many…

利用多尺度最大似然检测混合分布AI 解读

A test of homogeneity tries to decide whether observations come from a single distribution or from a mixture of several distributions. A powerful theory has been developed for the case where the component distributions are members of an exponential family. When no parametric assumptions are appropriate, the standard approach is to test for bimodality, which is known not to be very sensitive for detecting heterogeneity. To develop a more sensitive procedure, this article…

半参数模型的多尺度最大似然分析及应用AI 解读

A special semiparametric model for a univariate density is introduced that allows analyzing a number of problems via appropriate transformations. Two problems treated in some detail are testing for the presence of a mixture and detecting a wear-out trend in a failure rate. The analysis of the semiparametric model leads to an approach that advances the maximum likelihood theory of the Grenander estimator to a multiscale analysis. The construction of the corresponding test…

利用 Gap 统计量估计数据集中的聚类数AI 解读

Summary We propose a method (the ‘gap statistic’) for estimating the number of clusters (groups) in a set of data. The technique uses the output of any clustering algorithm (e.g. K-means or hierarchical), comparing the change in within-cluster dispersion with that expected under an appropriate reference null distribution. Some theory is developed for the proposal and a simulation study shows that the gap statistic usually outperforms other methods that have been proposed in…

粒度平滑AI 解读

A new method for smoothing a multivariate data set is introduced that is based on a simple geometric operation. This method is applied to the problem of estimating level sets of a density and minimum volume sets with given probability content. Building on existing techniques, the resulting estimator combines excellent theoretical and computational properties: It converges with the minimax rates (up to log factors) in most cases where these rates are known and, at the same…

在对偶空间中用蒙特卡洛抽样近似经验半空间距离AI 解读

The Kolmogorov-Smirnov distance is an important tool for constructing confidence sets and tests in univariate problems. In multivariate settings, an analogous role is played by the halfspace distance, which has the merit of being invariant under linear transformations. However, the evaluation of the halfspace distance between two samples is a computationally very intensive combinatorial problem even in moderate dimensions, which severely restricts the use of the halfspace…