Proposal: DMS 95- 05151 PI(s): Iain Johnstone - 1981, David Donoho - 1984 Institution: Stanford Title: Adaptive Estimation, New Tools, New Settings Abstract: The research develops fundamental tools in theoretical statistics to aid understanding of such adaptive procedures, and shows how to tune them so they are noise-cognizant and stable. Underlying the approach are a) the idea of oracles, which know perfectly well how to adapt representations ideally, b) the idea that the goal of adaptation in the presence of noisy data is to quantify how closely realizable procedures (which do not have privileged information about the object) can mimic an oracle, and c) the design of procedures coming as close as possible to the oracle. The project also develops methods for comparing different adaptation schemes by comparing oracles of different kinds, for example time-frequency oracles and time-scale oracles. This is an outgrowth of our earlier results on wavelets, where this approach was used to show that wavelets have a property of being nearly-ideally spatially adaptive. In addition a computational environment is being developed for implementing and systematically testing such approaches. As a further outgrowth of the proposers' earlier work on wavelets, the project studies a number of improvements and extensions of wavelet shrinkage, for example in the directions of classification, confidence bands, correlated data and selection of orthogonal bases. This research seeks to develop statistical theory and computational tools in the general area of adaptive methods of representing and analyzing signals, images and other objects. On the one hand, this project is prompted by the extremely high interest in adaptation on the part of people working in fields of signal processing, image processing, time-frequency analysis, speech processing analysis. New signal representations appear in these fields almost daily, along with new principles fo r selecting representations. Such representations are useful for data compression, which is not the main interest here; but they are also useful for noise removal and signal interpretation, which are important areas for statisticians to consider. On the other hand, this project is prompted by the goal of developing statistical theory which can give clear understanding of such adaptive schemes. Some of the most highly adaptive schemes being suggested in signal processing pose a real challenge to traditional statistical thinking. For example, some adaptation schemes search (either explicitly or implicitly) through thousands or millions of representations of a signal in order to arrive at their final result. A statistician, when thinking about applying such methods to a noisy signal, is by training forced to ask to what extent the result merely reflects the effects of snooping through noise, detecting pseudo-structure which actually due to noise, and due to the vigorous search. The research seeks to develop fundamental tools in theoretical statistics to aid understanding of such adaptive procedures, and shows how to tune them so they are noise-cognizant and stable. This project may have two spin-offs. First, some of the results may be stimulating and/or useful to the community of ``inventors of adaptive procedures'' in signal, image, speech, and time/frequency, and related communities. Second, the theoretical work may stimulate statisticians to take more interest in making further contributions in such directions.
