In this paper, we consider a nonparametric regression model with replicated observations based on the φ-mixing and the ρ-mixing error’s structures respectively, for exhibiting dependence among the units. The wavelet procedures are developed to estimate the regression function. Under suitable conditions, we obtain expansions for the bias and the variance of wavelet estimator, prove the moment consistency, the strong consistency, the strong convergence rate of it, and establish the asymptotic normality of wavelet estimator.
Keywords:nonparametric regression model; wavelet; weak dependence; φ-mixing; ρ-mixing
Consider the following nonparametric regression model:
from a discrete set of observations of the process at the points , is a zero mean stochastic process, defined on a probability space , and is an unknown function defined on a closed interval .
It is well known that regression model has a wide range of applications in filtering and prediction in communications and control systems, pattern recognition, classification and econometrics, and is an important tool of data analysis. Of much interest about the problem have been the weighted function estimates of g; see, for example, Priestley and Chao , Gasser and Müller [2,3], Prakasa Rao , Clark  and the references therein for the independent case; Roussas , Fan , Roussas and Tran , Liang and Jing , Yang and Li , Yang  for the various dependent cases.
In this article, we discuss a nonparametric estimation problem in the model (1.1) with repeated measurements. We assume that a random sample of m experimental units is available and the observed data for the jth unit are the values, ( ), of a response variable corresponding to the values, ( ), of a controlled variable. Let obey the model (1.1), i.e.,
where the ’s are fixed with , and the ’s are zero mean random errors. The model can be applied to many fields. For instance, in hydrology many phenomena may be represented by a sequence of continuous responses , where represents the time elapsed from the beginning of a certain year, j indicates the corresponding year, is the measurement of the deviation from the annual mean ; in some biological phenomena as the growth of individual (or populations) will be the growth points of the j-individual, the measurement of the deviation from the mean growth of the response of the j-individual, and the points where measurements are taken . It is clear that the observations ( ) made on the same experimental unit will in general be correlated. Hart and Wehrly  studied the asymptotic square mean error of a kernel estimator in the model with zero mean errors ’s be correlated, that is, for and zero elsewhere where is a correlation function. The uncorrelated assumption between units is unrealistic. In practice, sometimes the observed responses in different units are also correlated, more precisely a sequence of responses curve has an intrinsic dependence structure, such as mixing conditions. Under a weak error’s structure among units, Fraiman and Iribarren  proposed nonparametric estimates of in model (1.2) based on locally weighted averages, and gave their local and global asymptotic behaviors.
In the paper, we develop wavelet methods to estimate a regression function in the model (1.2) with the φ-mixing and ρ-mixing error’s structures respectively, that is, is a φ-mixing or ρ-mixing process. One motivation for using wavelet for the model (1.2) is that most of the nonparametric analyses of regression models are based on an important assumption that the regression function is smooth; but in reality, the assumption may not be satisfied. The major advantage of the wavelet is that the hypotheses of degrees of smoothness of the underlying function are less restrictive. Up to now, there have been no results on wavelet estimation for model (1.2). Another motivation for considering the model (1.2) with weakly dependent processes is that our interest is to avoid as far as possible any assumptions on the error’s structure within for each j ( ), and in the meantime to exhibit weakly dependence among the units.
For a systematic discussion of wavelets and their applications in statistics, see the recent monographs by Härdle et al. and Vidakovic . Due to their ability to adapt to local features of unknown curves, many authors have applied wavelet procedures to estimate the general nonparametric model. See recent works, for example, Antoniadis et al. and Xue  on independent errors; Johnstone and Silverman  for correlated noise; Liang et al. on martingale difference errors; Li and Xiao  for long memory data; Li et al. on associated samples; Xue , Sun and Chai , Li and Guo  and Liang  on mixing error assumptions.
For dealing with weakly dependent data, bootstrap and blockwise are well known. They are useful techniques of resampling, which can preserve the dependent properties of the data by appropriately choosing blocks of data. They have been sufficiently investigated by many papers, for example, Bühlman and Künsch , Yuichi , Lahiri , Lin and Zhang [29,30] and Lin et al.. For the nonparametric regression model without repeated observations under weakly dependent processes, Lin and Zhang  respectively adopted bootstrap wavelet and blockwise bootstrap wavelet to generate an independent blockwise sample from the original dependent data, defined the wavelet estimators of , and then took advantage of the independence of the blockwise sample to prove some asymptotic properties of the wavelet estimators. In addition, the weakly dependent conditions and in Lin and Zhang  are weaker than the conditions for the consistency and asymptotic normality of ρ-mixing and φ-mixing dependent sequences. In our paper, we consider the nonparametric regression model with repeated observations under the specific ρ-mixing and φ-mixing dependent processes. We do not use bootstrap and/or blockwise to define wavelet estimator of , but construct it by the following simple formula (2.2), and can show directly some asymptotic behaviors of the wavelet estimator by applying the fundamental properties of ρ-mixing and φ-mixing sequences and some proof techniques.
Recall the definitions of the sequences of the φ-mixing and ρ-mixing random variables. Let be a sequence of random variables defined on a probability space , be σ-algebra generated by , and denote to be the set of all -measurable random variables with second moments.
A sequence of random variables is said to be φ-mixing if
A sequence of random variables is said to be ρ-mixing if maximal correlation coefficient
The concept of a mixing sequence is central in many areas of economics, finance and other sciences. A mixing time series can be viewed as a sequence of random variables for which the past and distant future are asymptotically independent. A number of limit theorems for φ-mixing and ρ-mixing random variables have been studied by many authors. For example, see Shao , Peligrad , Utev , Kiesel , Chen et al. and Zhou  for φ-mixing; Peligrad , Peligrad and Shao [39,40], Shao  and Bradley  for ρ-mixing. Some limit theories can be found in the monograph of Lin and Lu .
The article is structured as follows. In Section 2, we introduce the wavelet estimation procedures and establish main results. The proofs of the main results are provided in Section 3.
2 Estimators and main results
Defining , from (1.2), we have
where . Expressing the model in this way is useful since the problem of estimating may now be regarded as that of fitting a curve through the sample means . The wavelet technique is applied to estimate the regression function in model (1.2). The detailed procedure is summarized below.
For convenience, we introduce some symbols and definitions along the line of Antoniadis et al.. Suppose that is a given scaling function in the Schwartz space with order l. A multiresolution analysis of consists of an increasing sequence of closed subspaces , , where is the set of square integrable functions over the real line. The associated integral kernel of is given by
where Z denotes the set of integers, is an integer depending only on n. Let be a partition of the interval ℐ with for . From (2.1), we now construct the wavelet estimator of g,
In the sequel, let denote generic finite positive constants, whose values are unimportant and may change from line to line. Set and suppose that for and .
Before formulating the main results, we first give some assumptions, which are quite mild and can be easily satisfied.
(i) is a sequence of φ-mixing (ρ-mixing);
(ii) is a sequence of identically distributed φ-mixing (ρ-mixing);
(iii) is a sequence of strictly stationary φ-mixing (ρ-mixing).
(ii) ( );
(iii) for some ;
(iv) ( ) for some .
A4. , , where is the Sobolev space of order υ, i.e., if , then , where denotes the Fourier transform of h.
A5. satisfies the Lipschitz condition of order .
A6. is q regular with , satisfies the Lipschitz condition with order 1 and has a compact support, and satisfies as , where is the Fourier transform of .
A8. , as , and , as , such that , and , where υ and γ are defined in (A4) and (A5), respectively.
Remark 2.1 We refer to the monograph of Doukhan  for properties of φ-mixing and ρ-mixing, and more mixing conditions.
Remark 2.2 It is well known that (A4)-(A7) are the mild regularity conditions for wavelet smoothing; see Antoniadis et al., Chai and Xu , Xue , Sun and Chai , Zhou and You  and Li and Guo .
Remark 2.3 (A8) is satisfied easily. For example, for any , and for and .
Our results are listed as follows.
Theorem 2.1Assume that (A1)(i) and (A2)(i), and (A4)-(A7) are satisfied. Then
(a) , where is defined in Lemma 3.2;
(b) φ-mixing: ;
Theorem 2.2Under (A1)(i), (A2)(i)(ii), and (A4)-(A7), we have
Theorem 2.3Assume that [(A1)(ii) and (A2)(i)(ii)] or [(A1)(i) and (A2)(iii)], and (A4)-(A7) are satisfied. Then
Theorem 2.4Assume that (A1)(ii), (A2)(i)(ii), and (A4)-(A7) are satisfied. If , , and , where and , then
Theorem 2.5Assume that (A1)(iii), (A2)(i)(iv), (A3), and (A4)-(A8) are satisfied. For a fixedxand each , there exists verifying , where as . Then
where denotes convergence in distribution.
3 Proofs of the main results
In order to prove the main results, we first present several lemmas.
Lemma 3.1Suppose that (A6) holds. We have
(c) uniformly in , as .
Lemma 3.2Suppose that (A6)-(A7) hold, and satisfies (A4)-(A5). Then
It follows easily from Theorem 3.2 of Antoniadis et al. .
Lemma 3.3 (a) Let be aφ-mixing sequence, , with and . Then
(b) Let be aρ-mixing sequence, , with and . Then
Lemmas 3.3(a) and (b) respectively come from Lemmas 10.1.d and 10.1.c of Lin and Lu.
Lemma 3.4 (a) Let be aφ-mixing sequence.
(i) If , then
(ii) Suppose that there exists an array of positive numbers such that for every , . Then, for any , there exists a positive constant such that
(b) Let be aρ-mixing sequence with . Then, for any , there exists a positive constant such that
Lemma 3.5Let be aφ-mixing (ρ-mixing) sequence of identically distributed random variables with
for some . Then
Lemma 3.6Let be a strictly stationaryφ-mixing (ρ-mixing) sequence of random variables with , and . If ( ), then
We are now in a position to give the proofs of the main results.
Proof of Theorem 2.1 From (1.2) and (2.2), we have
By Lemma 3.2, (a) holds.
Denote . By and Lemma 3.1(b),
For φ-mixing, by Lemmas 3.3(a) and (3.2), we have
Therefore, (b) holds for φ-mixing. Similar to the arguments, we obtain (b) for ρ-mixing by Lemma 3.3(b). □
Proof of Theorem 2.2 We know that . It follows easily from Theorem 2.1. □
Proof of Theorem 2.3 From (1.2) and (2.2), we have
where is defined in the proof of Theorem 2.1. Note that
as by Lemma 3.2. It remains to show that
(1) φ-mixing. Here, we consider under two different assumptions: [(A1)(ii) and (A2)(i)(ii)] and [(A1)(i) and (A2)(iii)], respectively.
If the assumptions are [(A1)(ii) and (A2)(i)(ii)], denote and for , , . Note that . We have
For , by Lemma 3.4(a), we have
Therefore, it follows from the Borel-Cantelli lemma that
Note that ( ). By Lemma 3.5, we have a.s. Therefore,
By (3.8) and Lemma 3.1(ii), one gets
Further, we have
From (3.6), (3.7), (3.9) and (3.10), we obtain (3.5).
If the assumptions are [(A1)(i) and (A2)(iii)], note that as , hence . Further, for any and . Take , next take small enough such that . By Lemma 3.4(a), we have
Therefore, from the Borel-Cantelli lemma, we obtain (3.5).
(2) ρ-mixing. We also consider under two different assumptions: [(A1)(ii) and (A2)(i)(ii)] and [(A1)(i) and (A2)(iii)], respectively.
Note that as , hence for . Further, for any and .
If the assumptions are [(A1)(ii) and (A2)(i)(ii)], from (3.6)-(3.10), it is known that we only need to prove (3.7) for obtaining (3.5). Taking , we have and . Next, take small enough such that and . By Lemma 3.4(b), we have
Therefore, (3.7) holds.
If the assumptions are [(A1)(i) and (A2)(iii)], take , next take small enough such that . By Lemma 3.4(b), we have
Thus, we obtain (3.5).
So, we complete the proof of Theorem 2.3. □
Proof of Theorem 2.4 Here, we use some symbols of the proof of Theorem 2.3. From (3.3), we have
By Lemma 3.2, for , one gets
Note that satisfies the Lipschitz condition of order 1 on x. We have
Note that , for , by Lemma 3.5, we have
By (3.14), one gets
Thus, we obtain
By Lemma 3.1(ii), (3.14), and , we have
To complete the proof of the theorem, it is suffices to show that
by (3.11)-(3.13) and (3.15)-(3.16).
Here, we show (3.17) under φ-mixing and ρ-mixing, respectively.
(1) φ-mixing. Taking , we have , , and . By Lemma 3.4(a), we have
Thus, (3.17) holds by the Borel-Cantelli lemma.
(2) ρ-mixing. Similar to the arguments in the proof of Theorem 2.3, for any and . Taking , we have and . Next, take small enough such that . By Lemma 3.4(b), we have
Therefore, we also obtain (3.17).
So, we complete the proof of Theorem 2.4. □
Proof of Theorem 2.5 Denote . For each , we have
Since we have
by Lemma 3.2 and (A8), it suffices to show that
Let , and . Under the assumption of φ-mixing, denote ; if we consider ρ-mixing, then is . By Lemma 3.3, we have
which can be made arbitrarily small if we first choose ς such that is small, and for this ς we choose m large enough so that makes arbitrarily small. Thus, we obtain (3.18).
It remains to establish (3.19). Denote and . By Lemma 3.4, and (iv) implies (ii), for φ-mixing and ρ-mixing, we have
For any , from (3.20) and the dominated convergence theorem,
Hence, converges absolutely for φ-mixing and ρ-mixing, that is,
We easily obtain
since by Lemma 3.6, by (3.21), and by Lemma 3.1.
Thus, we complete the proof of Theorem 2.5. □
4 Conclusion and discussion
The paper studies a nonparametric regression model with replicated observations under weakly dependent processes by wavelet procedures. For exhibiting dependence among the units, we assume that is a φ-mixing or ρ-mixing process, and avoid as far as possible any assumptions on the error’s structure among for each j ( ). Under suitable conditions, we obtain expansions for the bias and the variance of wavelet estimator, prove the moment consistency, the strong consistency, the strong convergence rate of it, and establish the asymptotic normality of wavelet estimator. For the general nonparametric model, consistency results of linear wavelet estimator can be derived from general results on regression estimators in the case of dependent errors. But our results cannot be derived directly from general results on regression estimators because the nonparametric regression model with repeated measurements we considered has complex dependent error’s structure. Bootstrap and blockwise are useful techniques of resampling, which can preserve the dependent properties of the data by appropriately choosing blocks of data. They have been sufficiently investigated for weakly dependent data by many papers, for example, Bühlman and Künsch , Yuichi , Lahiri , Lin and Zhang [29,30] and Lin et al.. In the future, we may try bootstrap and blockwise methods into our model. Since linear wavelet is not adaptive, nonlinear wavelet and design-adapted wavelet have also received considerable attention recently; see, for example, Li , Liang and Uña-Álvarez  and Chesneau  for (conditional) density estimator; Li and Xiao  and Uña-Álvarez et al. for nonparametric models; and Delouille and Sachs  for nonlinear autoregressive models, and so on. At present, our paper mainly concentrates on linear wavelet estimator in nonparametric regression model with repeated measurements under weakly dependent processes. Although it is easy to construct nonlinear wavelet estimator in our model, it is very difficult to establish asymptotic theory of nonlinear wavelet estimator and to prove it since the structure of our model is complex. It will be a challenging work. Interesting work for further research includes nonlinear wavelet and design-adapted wavelet estimations for our model.
The authors declare that they have no competing interests.
The three authors contributed equally to this work. All authors read and approved the final manuscript.
The authors thank the associate editor and two anonymous referees, whose valuable comments greatly improved the paper. This work is partially supported by Key Natural Science Foundation of Higher Education Institutions of Anhui Province (KJ2012A270), NSFC (11171065, 11061002), Youth Foundation for Humanities and Social Sciences Project from Ministry of Education of China (11YJC790311), NSFJS (BK2011058), National Natural Science Foundation of Guangxi (2011GXNSFA018126) and Postdoctoral Research Program of Jiangsu Province of China (1202013C).
Roussas, GG: Consistent regression with fixed design points under dependence condition. Stat. Probab. Lett.. 8, 41–50 (1989). Publisher Full Text
Fan, Y: Consistent nonparametric multiple regression for dependent heterogeneous processes: the fixed design case. J. Multivar. Anal.. 33, 72–88 (1990). Publisher Full Text
Roussas, GG, Tran, LT: Fixed design regression for time series: asymptotic normality. J. Multivar. Anal.. 40, 262–291 (1992). Publisher Full Text
Liang, HY, Jing, BY: Asymptotic properties for estimates of nonparametric regression models based on negatively associated sequences. J. Multivar. Anal.. 95, 227–245 (2005). Publisher Full Text
Yang, SC: Maximal moment inequality for partial sums of strong mixing sequences and application. Acta Math. Sin. Engl. Ser.. 23, 1013–1024 (2007). Publisher Full Text
Fraiman, R, Iribarren, GP: Nonparametric regression estimation in models with weak error’s structure. J. Multivar. Anal.. 37, 180–196 (1991). Publisher Full Text
Hart, JD, Wehrly, TE: Kernel regression estimation using repeated measurements data. J. Am. Stat. Assoc.. 81, 1080–1088 (1986). Publisher Full Text
Antoniadis, A, Gregoire, G, Mckeague, IW: Wavelet methods for curve estimation. J. Am. Stat. Assoc.. 89, 1340–1352 (1994). Publisher Full Text
Johnstone, IM, Silverman, BW: Wavelet threshold estimators for data with correlated noise. J. R. Stat. Soc. B. 59, 319–351 (1997). Publisher Full Text
Liang, HY, Zhang, DX, Lu, BX: Wavelet estimation in nonparametric model under martingale difference errors. Appl. Math. J. Chin. Univ. Ser. B. 19, 302–310 (2004). Publisher Full Text
Li, LY, Xiao, YM: Wavelet-based estimators of mean regression function with long memory data. Appl. Math. Mech.. 27, 901–910 (2006). Publisher Full Text
Li, YM, Yang, SC, Zhou, Y: Consistency and uniformly asymptotic normality of wavelet estimator in regression model with associated samples. Stat. Probab. Lett.. 78, 2947–2956 (2008). Publisher Full Text
Li, YM, Guo, JH: Asymptotic normality of wavelet estimator for strong mixing errors. J. Korean Stat. Soc.. 38, 383–390 (2009). Publisher Full Text
Liang, HY: Asymptotic normality of wavelet estimator in heteroscedastic model with α-mixing errors. J. Syst. Sci. Complex.. 24, 725–737 (2011). Publisher Full Text
Yuichi, K: Empirical likelihood methods with weakly dependent processes. Ann. Stat.. 25, 2084–2102 (1997). Publisher Full Text
Lahiri, SH: Theoretical comparison of block bootstrap methods. Ann. Stat.. 27, 386–404 (1999). Publisher Full Text
Lin, L, Zhang, RC: Blockwise empirical Euclidean likelihood for weakly dependent processes. Stat. Probab. Lett.. 53, 143–152 (2001). Publisher Full Text
Peligrad, M: The r-quick version of the strong law for stationary φ-mixing sequences. Proceedings of the International Conference on Almost Everywhere Convergence in Probability and Statistics, pp. 335–348. Academic Press, New York (1989)
Kiesel, R: Summability and strong laws for φ-mixing random variables. J. Theor. Probab.. 11, 209–224 (1998). Publisher Full Text
Chen, PY, Hu, TC, Volodin, A: Limiting behaviour of moving average processes under φ-mixing assumption. Stat. Probab. Lett.. 79, 105–111 (2009). Publisher Full Text
Zhou, XC: Complete moment convergence of moving average processes under φ-mixing assumptions. Stat. Probab. Lett.. 80, 285–292 (2010). Publisher Full Text
Peligrad, M: On the central limit theorem for ρ-mixing sequences of random variables. Ann. Probab.. 15, 1387–1394 (1987). Publisher Full Text
Peligrad, M, Shao, QM: Estimation of variance for ρ-mixing sequences. J. Multivar. Anal.. 52, 140–157 (1995). Publisher Full Text
Shao, QM: Maximal inequalities for partial sums of ρ-mixing sequences. Ann. Probab.. 23, 948–965 (1995). Publisher Full Text
Bradley, R: A stationary rho-mixing Markov chain which is not “interlaced” rho-mixing. J. Theor. Probab.. 14, 717–727 (2001). Publisher Full Text
Zhou, X, You, JH: Wavelet estimation in varying-coefficient partially linear regression models. Stat. Probab. Lett.. 68, 91–104 (2004). Publisher Full Text
Ibragimov, IA: Some limit theorems for stationary processes. Theory Probab. Appl.. 7, 349–382 (1962). Publisher Full Text
Li, LY: Non-linear wavelet-based density estimators under random censorship. J. Stat. Plan. Inference. 117, 35–58 (2003). Publisher Full Text
Liang, HY, Uña-Álvarez, J: Wavelet estimation of conditional density with truncated, censored and dependent data. J. Multivar. Anal.. 102, 448–467 (2011). Publisher Full Text
Chesneau, C: On the adaptive wavelet deconvolution of a density for strong mixing sequences. J. Korean Stat. Soc.. 41, 423–436 (2012). Publisher Full Text
Uña-Álvarez, J, Liang, HY, Rodríguez-Casal, A: Nonlinear wavelet estimator of the regression function under left-truncated dependent data. J. Nonparametr. Stat.. 22, 319–344 (2010). Publisher Full Text
Delouille, V, Sachs, RV: Estimation of nonlinear autoregressive models using design-adapted wavelets. Ann. Inst. Stat. Math.. 57, 235–253 (2005). Publisher Full Text