Research

# The consistency for estimator of nonparametric regression model based on NOD errors

Wenzhi Yang, Xuejun Wang, Xinghui Wang and Shuhe Hu*

Author Affiliations

School of Mathematical Science, Anhui University, Hefei 230039, P. R. China

For all author emails, please log on.

Journal of Inequalities and Applications 2012, 2012:140 doi:10.1186/1029-242X-2012-140

 Received: 24 November 2011 Accepted: 15 June 2012 Published: 15 June 2012

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

### Abstract

By using some inequalities for NOD random variables, we give its application to investigate the nonparametric regression model based on these errors. Some consistency results for the estimator of g(x) are presented, including the mean convergence, uniform convergence, almost sure convergence and convergence rate. We generalize some related results and as an example of designed assumptions for weight functions, we give the nearest neighbor weights.

AMS Mathematical Subject Classification 2000: 62G05; 62G08.

##### Keywords:
NOD sequence; almost sure convergence; convergence rate; nearest neighbor weights

### 1 Introduction

Consider a fixed design regression model

(1.1)

where xni are design points on a set A in for some q ≥ 1, g(·) is an unknown function on A and εni are random errors. Assume that for each n, {εni, 1 ≤ i n} has the same distribution as {εi, 1 ≤ i n}. As an estimator of g(·), the following weighted regression estimator is given:

where Wni(x) = Wni(x,xn1,...,xnn) are weighted functions.

The above estimator was first proposed by Georgiev [1] and subsequently has been studied by many authors. In the independent case, consistency and asymptotic normality have been investigated by Georgiev and Greblicki [2], Georgiev [3], Müller [4], and the references therein. Fan [5] extended the work of Georgiev [3] and Müller [4] in the estimation of the regression model to the case of Lq-mixingale for some 1 ≤ q ≤ 2. Roussas [6] discussed strong consistency and quadratic mean consistency of gn(x), and Roussas et al. [7] established asymptotic normality of gn(x) assuming that the errors are from a strictly stationary stochastic process and satisfying the strong mixing condition. Tran et al. [8] obtained the asymptotic normality of gn(x) assuming that the errors form a linear time series, more precisely, a weakly stationary linear process based on a martingale difference sequence. Hu et al. [9] generalized the main results of Tran et al. [8]. Liang and Jing [10] established the consistency, uniform consistency, and asymptotic normality of gn(x) under negatively associated (NA) samples. Meanwhile, for the semiparametric regression model, Ren and Chen [11] obtained the strong consistency for the least squares estimator of β and the nonparametric estimator of g(t) based on NA samples, Hu [12] obtained the consistency and complete consistency for these estimations based on the linear time series, Baek and Liang [13] established some asymptotic results for these estimations under NA samples, Liang et al. [14] also established some asymptotic results for a linear process based NA samples, etc. For more details of semiparametric regression model, one can refer to Hardle et al. [15] and the references therein.

In this article, we investigate the nonparametric regression model based on negatively orthant dependent (NOD) random variables, which is weaker than NA random variables. Some related definitions are given as follows:

Definition 1.1 Two random variables X and Y are said to be NQD if for x, y R,

A sequence of random variables {Xn, n ≥ 1} is said to be pairwise NQD if for all i, j N, I j, and Xi and Xj are NQD.

The concept of NQD was intruduced by Lehmann [16] and he pointed out some useful properties of NQD, for example, let X and Y be NQD, then

(i) EXY EXEY,

(ii) P(X > x, Y > y) ≤ P(X > x)P(Y > y), for ∀ x, y R,

(iii) if f, g are both nondecreasing (or nonincreasing) functions, then f(X) and g(Y) are NQD.

Definition 1.2 A finite collection of random variables X1, X2,..., Xn is said to be NA if for every pair of disjoint subsets A1, A2 of {1, 2,..., n},

whenever f and g are coordinatewise nondecreasing such that this covariance exists.

An infinite sequence {Xn}n≥1 is NA if every finite subcollection is NA.

Definition 1.3 A finite collection of random variables X1, X2,...,Xn is said to be negatively upper orthant dependent (NUOD) if for all real numbers x1,x2,...,xn,

and negatively lower orthant dependent (NLOD) if for all real numbers x1,x2,...,xn,

A finite collection of random variables X1, X2,...,Xn is said to be NOD if they are both NUOD and NLOD.

An infinite sequence {Xn}n≥1 is said to be NOD (NUOD or NLOD) if every finite subcollection is NOD (NUOD or NLOD).

The concepts of NA and NOD sequences were introduced by Joag-Dev and Proschan [17]. They pointed out that NA random variables are NOD random variables, but neither NUOD nor NLOD implies NA. Various results and examples of NOD random variables can be found in Joag-Dev and Proschan [17], Bozorgnia et al. [18], Asadian et al. [19], Wang et al. [20], Wu [21,22], Wang et al. [23,24], Li et al. [25] and Sung [26], etc. Obviously, by the definitions of NOD and pairwise NQD, NOD random variables are pairwise NQD random variables. For more results and examples of pairwise NQD random variables, one can refer to Lehmann [16], Matula [27], Wu [28], Gan and Chen [29], Li and Yang [30], etc. But unlike NOD random variables, pairwise NQD random variables have not some nice inequalities such as Bernstein-type inequality as we know.

Inspired by Liang and Jing [10] and other articles referred above, we investigate the nonparametric regression model based on NOD random errors. By using the moment inequality, Bernstein-type inequality and truncating method for NOD random variables, we obtain some consistency results for estimator of g(x) such as the mean convergence, uniform convergence, almost sure convergence and convergence rate. We generalize some results of Liang and Jing [10] for NA random variables to the case of NOD random variables. Meanwhile, as an example of designed assumptions for weight functions, we give the nearest neighbor weights.

For any function g(x), we use c(g) to denote all continuity points of function g on the set A in for some q ≥ 1. Let c, c1, c2, C, C1, C2,... denote the positive constants whose values may vary at each occurrence. ⌈x⌉ denotes the largest integer not exceeding x, I(B) is the indicator function of set B, x+ = xI(x ≥ 0), x- = -xI(x < 0) and ∥x∥ denotes Euclidean norm of x. In this article, main results are presented in Section 2, some lemmas and the proofs of main results are presented in Sections 3 and 4, respectively.

### 2 The main results

Under the nonparametric regression model of (1.1), for any fixed point x A, some assumptions on weighted function Wni(x) = Wni(x, xn1,..., xnn) are given as follows:

(H1) as n → ∞;

(H2) for all n;

(H3) as n → ∞;

(H4) as n → ∞ for all a > 0.

Theorem 2.1 Let {εn, n ≥ 1} be a mean zero NOD sequence. Assume that the conditions (H1)-(H4) hold true. If , then for x c(g) and some p ∈ (0, 2],

(2.1)

If for some p > 2, then (2.1) also holds true.

In order to obtain uniform convergence for the estimator of g(x), for any fixed point x on a compact set A in for some q ≥ 1, some uniform version of assumptions on Wni(x) = Wni(x,xn1,...,xnn) are replaced by that as follows:

Theorem 2.2 Let {εn, n > 1} be a mean zero NOD sequence. Assume that the conditions hold true and g is continuous on the compact set A. If , then for some p ∈ (0, 2],

(2.2)

If for some p > 2, then (2.2) also holds true.

Next, we will study the almost sure convergence and convergence rate for the estimator of g(x). Similarly, for any fixed point x on the compact set A in for some q ≥ 1, some assumptions on the Wni(x) = Wni(x, xn1,...,xnn) are shown as follows:

(H5) ;

(H6) for all n ≥ 1 and ;

(H7) for some a > 0.

Theorem 2.3 Let {εn, n ≥ 1} be a mean zero NOD sequence such that . Suppose that the conditions (H5)-(H7) hold true and g(x) satisfies a local Lipschitz condition around the point x. Then for x A,

(2.3)

Theorem 2.4 Let {εn, n ≥ 1} be a mean zero NOD sequence such that . Suppose that the conditions (H5)-(H7) hold true and g(x) satisfies a local Lipschitz condition around the point x. Then for x A,

(2.4)

Remark 2.1 The similar assumptions on weighted functions can be found in Ren and Chen [11], Hu et al. [31] and Liang and Jing [10], etc. Under the NA sequence and other assumptions, for some p > 1, Liang and Jing [10] obtained the result E|gn(x)-g(x)|p → 0 as n → ∞ (see Liang and Jing [10, Theorem 2.1]). In our Theorem 2.1, we give the result E|gn(x)-g(x)|p → 0 as n → ∞ for some p > 0. Liang and Jing [10] also studied the strong consistency of the estimator for g(x). In our Theorems 2.3 and 2.4, the strong consistency and convergence rate of the estimator for g(x) are presented. Since NA sequence is a NOD sequence, we generalize some results of Liang and Jing [10] to the case of NOD sequence.

Example 2.1 Here, we give an example that the designed assumptions (H5)-(H7) are satisfied for the nearest neighbor weights. Without loss of generality, let A = [0,1] and . For x A, we rewrite |xn1 - x|, |xn2 - x|,..., |xnn - x| as follows

(2.5)

if |xni - x| = |xnj - x|, |xni - x| is in frond of |xnj - x| for i < j. Let kn = ⌈n5/8⌉ and define the nearest neighbor weight functions as following

(2.6)

Consequently, for every x ∈ [0,1], we can find by definition of Ri (x) and choice of xni that

(2.7)

(2.8)

(2.9)

If g is continuous on [0,1], then by (2.6)-(2.9), it can find that the assumptions of (H1)-(H7) and are satisfied.

### 3 Some lemmas

Lemma 3.1 (cf. Bozorgnia et al. [18]). Let random variables X1,X2,...,Xn be NOD, f1,f2,...,fn be all nondecreasing (or nonincreasing) functions, then random variables f1(X1), f2(X2),..., fn(Xn) are NOD.

Lemma 3.2 (cf. Asadian et al. [19]). Let {Xn, n ≥ 1} be a NOD sequence such that EXn = 0 and E|Xn|p < ∞ for all n ≥ 1 and p ≥ 2. Then

where cp depends only on p.

Lemma 3.3 (cf. Wang et al. [20]). Let {Xn}n≥1 be a sequence of NOD random variables such that EXn = 0 and |Xn| ≤ b for each n ≥ 1, where b is a positive constant.

Denote for each n ≥ 1. Then for every ε > 0,

### 4 Proofs of the main results

Proof of Theorem 2.1: By Cr inequality, it has

(4.1)

For x c(g) and a > 0,

So, similar to the proof of (2.10) in Hu et al. [31], by conditions (H1), (H2) and (H4), it is easy to have that

(4.2)

On the other hand, by Lemma 3.1, for the fixed x, we can see that and are also NOD sequences. Combining with , without loss of generality, we assume Wni(x) ≥ 0 in the proof. If 0 < p ≤ 2, by Jensen's inequality, Lemma 3.2, (H3) and , we have

(4.3)

following from that {εni, 1 ≤ i n} has the same distribution as {εi, 1 ≤ i n} for each n. Otherwise, for p > 2, by Lemma 3.2, and (H3) again, we obtain

(4.4)

since for any positive number sequence {ai,1 ≤ i n} and 1 ≤ α β. Therefore, by (4.1)-(4.4), the desired result (2.1) has been proved completely.

Proof of Theorem 2.2: Since g is continuous in the compact set A, g is uniformly continuous in the compact set A. Consequently, similar to the proof of Theorem 2.1, we can get that

Therefore,

which implies the desired result (2.2).

Proof of Theorem 2.3: Combining the proof of (4.2) with the assumptions of (H5)-(H7) and g(x) satisfying a local Lipschitz condition around the point x, we can get that

(4.5)

Therefore, for x A, to prove (2.3), we only have to show that

(4.6)

Without loss of generality, we assume that Wni(x) ≥ 0 in the proof. Let

Since for each n, it is easy to see that

(4.7)

Obviously, for the fixed x and n, is a NOD sequence with mean zero. Meanwhile, by the condition (H6), it has

Since {εni, 1 ≤ i n} has the same distribution as {εi, 1 ≤ i n} for each n, we obtain by applying Lemma 3.3 that for every ϵ > 0,

which implies

(4.8)

following from Borel-Cantelli lemma.

Next, we turn to estimate Tn2 and Tn3. It can be checked by that

(4.9)

which implies

Consequently, by Kronecker's lemma, we have that

Thus, by the condition (H6), it is easy to see that

(4.10)

Obviously, by and (H6) again,

(4.11)

Combining (4.10) with (4.11), it follows

(4.12)

Likewise, by , we will found that

which implies

Then, by Kronecker's lemma

Consequently, by (H6), it has that

On the other hand, by (H6) and again,

Finally,

(4.13)

Therefore, by (4.7), (4.8), (4.12) and (4.13), (4.6) is completely proved. The desired result (2.3) follows from (4.5) and (4.6) immediately.

Proof of Theorem 2.4: By the estimation of (4.5), to prove (2.4), we only need to prove that |gn(x) - Egn(x)| = O(n-1/4), a.s. It is also to assume that Wni(x) ≥ 0 in the proof. Similar to the proof of Theorem 2.3, we will use the same notation and Tnq for q = 1, 2, 3, where i1/2 is replaced by i1/4. Obviously implies , by (H6), it has

Since {εni, 1 ≤ i n} has the same distribution as {εi, 1 ≤ i n} for each n, we obtain by applying Lemma 3.3 that for every ϵ > 0

which implies by Borel-Cantelli lemma that

(4.14)

Meanwhile, it can be checked by that

which implies

Then, we have by Kronecker's lemma that

Consequently, by (H.6), it follows

(4.15)

and

(4.16)

On the other hand, it can be checked that

which implies

So, by Kronecker's lemma,

Consequently, by (H.6), we have

(4.17)

and

(4.18)

Finally, similar to the proof of (2.3), by (4.14)-(4.18), it easily has that |gn(x) -Egn(x)| = O(n-1/4), a.s..

### Competing interests

The authors declare that they have no competing interests.

### Authors' contributions

All authors read and approved the final manuscript.

### Acknowledgements

The authors are grateful to Associate Editor Prof. Andrei Volodin and two anonymous referees for their careful reading and insightful comments. This work was supported by the National Natural Science Foundation of China (11171001, 11126176), HSSPF of the Ministry of Education of China (10YJA910005), Natural Science Foundation of Anhui Province (1208085QA03) and Provincial Natural Science Research Project of Anhui Colleges (KJ2010A005).

### References

1. Georgiev, AA: Local properties of function fitting estimates with applications to system identification. Mathematical Statistics and Applications, Proceedings of the 4th Pannonian Symposium on Mathematical Statistics, Bad Tatzmannsdorf, pp. 141–151. Bad Tatzmannsdorf, Austria. Reidel, Dordrecht (1985) Austria, 4-10, Sept 1983

2. Georgiev, AA, Greblicki, W: Nonparametric function recovering from noisy observations. J Stat Plan Infer. 13, 1–14 (1986)

3. Georgiev, AA: Consistent nonparametric multiple regression: the fixed design case. J Multivar Anal. 25(1), 100–110 (1988). Publisher Full Text

4. Müller, HG: Weak and universal consistency of moving weighted averages. Period Math Hungar. 18(3), 241–250 (1987). Publisher Full Text

5. Fan, Y: Consistent nonparametric multiple regression for dependent heterogeneous processes: the fixed design case. J Multivar Anal. 33(1), 72–88 (1990). Publisher Full Text

6. Roussas, GG: Consistent regression estimation with fixed design points under dependence conditions. Stat Probab Lett. 8(1), 41–50 (1989). Publisher Full Text

7. Roussas, GG, Tran, LT, Ioannides, DA: Fixed design regression for time series: Asymptotic normality. J Multivar Anal. 40(2), 262–291 (1992). Publisher Full Text

8. Tran, L, Roussas, G, Yakowitz, S, Van, BT: Fixed-design regression for linear time series. Ann Stat. 24(3), 975–991 (1996)

9. Hu, SH, Zhu, CH, Chen, YB: Fixed-design regression for linear time series. Acta Math Sci Ser B Engl Ed. 22(1), 9–18 (2002)

10. Liang, HY, Jing, BY: Asymptotic properties for estimates of nonparametric regression models based on negatively associated sequences. J Multivar Anal. 95(2), 227–245 (2005). Publisher Full Text

11. Ren, Z, Chen, MH: Strong consistency of a class of estimators in partial linear model for negative associated samples. Chinese J Appl Prob Stat. 18(1), 60–66 (2002)

12. Hu, SH: Fixed-design semiparametric regression for linear time series. Acta Math Sci Ser B Engl Ed. 26(1), 74–82 (2006)

13. Baek, J-II, Liang, HY: Asymptotics of estimators in semi-parametric model under NA samples. J Stat Plan Infer. 136(10), 3362–3382 (2006). Publisher Full Text

14. Liang, HY, Mammitzsch, V, Steineback, J: On a semiparametric regression model whose errors form a linear process with negatively associated innovations. Statistics. 40(3), 207–226 (2006). Publisher Full Text

15. Härdle, W, Liang, H, Gao, JT: Partially Linear Models, Springer Series in Economics and Statistics. Physica-Verlag, New York (2000)

16. Lehmann, EL: Some concepts of dependence. Ann Math Stat. 37(5), 1137–1153 (1966). Publisher Full Text

17. Joag-Dev, K, Proschan, F: Negative association of random variables with applications. Ann Stat. 11(1), 286–295 (1983). Publisher Full Text

18. Bozorgnia, A, Patterson, RF, Taylor, RL: Limit Theorems for Dependent Random Variables. World Congress Nonlinear Analysts'92, pp. 1639–1650. de Gruyter, Berlin (1996) (Tampa, FL, 1992)

19. Asadian, N, Fakoor, V, Bozorgnia, A: Rosenthal's type inequalities for negatively orthant dependent random variables. J Iranian Stat Soc. 5(1-2), 69–75 (2006)

20. Wang, XJ, Hu, SH, Yang, WZ, Ling, NX: Exponential inequalities and inverse moment for NOD sequence. Stat Probab Lett. 80(5-6), 452–461 (2010). Publisher Full Text

21. Wu, QY: Complete convergence for negatively dependent sequences of random variables. J Inequal Appl. 2010, 10 (Article ID 507293) (2010)

22. Wu, QY: A strong limit theorem for weighted sums of sequences of negatively dependent random variables. J Inequal Appl. 2010, 8 (Article ID 383805) (2010)

23. Wang, XJ, Hu, SH, Shen, AT, Yang, WZ: An exponential inequality for a NOD sequence and a strong law of large numbers. Appl Math Lett. 24(2), 219–223 (2011). Publisher Full Text

24. Wang, XJ, Hu, SH, Volodin, AI: Strong limit theorems for weighted sums of NOD sequence and exponential inequalities. Bull Korean Math Soc. 48(5), 923–938 (2011). Publisher Full Text

25. Li, XQ, Yang, WZ, Hu, SH, Wang, XJ: The Bahadur representation for sample quantile under NOD sequence. J Nonparametr Stat. 23(1), 59–65 (2011). Publisher Full Text

26. Sung, SH: On the exponential inequalities for negatively dependent random variables. J Math Anal Appl. 381(2), 538–545 (2011). Publisher Full Text

27. Matula, P: A note on the almost sure convergence of sums of negatively dependent random variables. Stat Probab Lett. 15(3), 209–213 (1992). Publisher Full Text

28. Wu, QY: Probability Limit Theory of Mixing Sequences. Science Press, Beijing (2006)

29. Gan, SX, Chen, PY: Some limit theorems for sequences of pairwise NQD random variables. Acta Math Sci Ser B Engl. 28(2), 269–281 (2008)

30. Li, R, Yang, WG: Strong convergence of pairwise NQD random sequences. J Math Anal Appl. 344(2), 741–747 (2008). Publisher Full Text

31. Hu, SH, Pan, GM, Gao, QB: Estimate problem of regression models with linear process errors. Appl Math A J Chinese Univ. 18A(1), 81–90 (2003)