The consistency for estimator of nonparametric regression model based on NOD errors

By using some inequalities for NOD random variables, we give its application to investigate the nonparametric regression model based on these errors. Some consistency results for the estimator of g(x) are presented, including the mean convergence, uniform convergence, almost sure convergence and convergence rate. We generalize some related results and as an example of designed assumptions for weight functions, we give the nearest neighbor weights.

AMS Mathematical Subject Classification 2000: 62G05; 62G08.

Consider a fixed design regression model

where x_ni are design points on a set A in for some q ≥ 1, g(·) is an unknown function on A and ε_ni are random errors. Assume that for each n, {ε_ni, 1 ≤ i ≤ n} has the same distribution as {ε_i, 1 ≤ i ≤ n}. As an estimator of g(·), the following weighted regression estimator is given:

where W_ni(x) = W_ni(x,x_n1,...,x_nn) are weighted functions.

The above estimator was first proposed by Georgiev [1] and subsequently has been studied by many authors. In the independent case, consistency and asymptotic normality have been investigated by Georgiev and Greblicki [2], Georgiev [3], Müller [4], and the references therein. Fan [5] extended the work of Georgiev [3] and Müller [4] in the estimation of the regression model to the case of L^q-mixingale for some 1 ≤ q ≤ 2. Roussas [6] discussed strong consistency and quadratic mean consistency of g_n(x), and Roussas et al. [7] established asymptotic normality of g_n(x) assuming that the errors are from a strictly stationary stochastic process and satisfying the strong mixing condition. Tran et al. [8] obtained the asymptotic normality of g_n(x) assuming that the errors form a linear time series, more precisely, a weakly stationary linear process based on a martingale difference sequence. Hu et al. [9] generalized the main results of Tran et al. [8]. Liang and Jing [10] established the consistency, uniform consistency, and asymptotic normality of g_n(x) under negatively associated (NA) samples. Meanwhile, for the semiparametric regression model, Ren and Chen [11] obtained the strong consistency for the least squares estimator of β and the nonparametric estimator of g(t) based on NA samples, Hu [12] obtained the consistency and complete consistency for these estimations based on the linear time series, Baek and Liang [13] established some asymptotic results for these estimations under NA samples, Liang et al. [14] also established some asymptotic results for a linear process based NA samples, etc. For more details of semiparametric regression model, one can refer to Hardle et al. [15] and the references therein.

In this article, we investigate the nonparametric regression model based on negatively orthant dependent (NOD) random variables, which is weaker than NA random variables. Some related definitions are given as follows:

Definition 1.1 Two random variables X and Y are said to be NQD if for ∀ x, y ∈ R,

A sequence of random variables {X_n, n ≥ 1} is said to be pairwise NQD if for all i, j ∈ N, I ≠ j, and X_i and X_j are NQD.

The concept of NQD was intruduced by Lehmann [16] and he pointed out some useful properties of NQD, for example, let X and Y be NQD, then

(i) EXY ≤ EXEY,

(ii) P(X > x, Y > y) ≤ P(X > x)P(Y > y), for ∀ x, y ∈ R,

(iii) if f, g are both nondecreasing (or nonincreasing) functions, then f(X) and g(Y) are NQD.

Definition 1.2 A finite collection of random variables X₁, X₂,..., X_n is said to be NA if for every pair of disjoint subsets A₁, A₂ of {1, 2,..., n},

whenever f and g are coordinatewise nondecreasing such that this covariance exists.

An infinite sequence {X_n}_n≥1 is NA if every finite subcollection is NA.

Definition 1.3 A finite collection of random variables X₁, X₂,...,X_n is said to be negatively upper orthant dependent (NUOD) if for all real numbers x₁,x₂,...,x_n,

and negatively lower orthant dependent (NLOD) if for all real numbers x₁,x₂,...,x_n,

A finite collection of random variables X₁, X₂,...,X_n is said to be NOD if they are both NUOD and NLOD.

An infinite sequence {X_n}_n≥1 is said to be NOD (NUOD or NLOD) if every finite subcollection is NOD (NUOD or NLOD).

The concepts of NA and NOD sequences were introduced by Joag-Dev and Proschan [17]. They pointed out that NA random variables are NOD random variables, but neither NUOD nor NLOD implies NA. Various results and examples of NOD random variables can be found in Joag-Dev and Proschan [17], Bozorgnia et al. [18], Asadian et al. [19], Wang et al. [20], Wu [21,22], Wang et al. [23,24], Li et al. [25] and Sung [26], etc. Obviously, by the definitions of NOD and pairwise NQD, NOD random variables are pairwise NQD random variables. For more results and examples of pairwise NQD random variables, one can refer to Lehmann [16], Matula [27], Wu [28], Gan and Chen [29], Li and Yang [30], etc. But unlike NOD random variables, pairwise NQD random variables have not some nice inequalities such as Bernstein-type inequality as we know.

Inspired by Liang and Jing [10] and other articles referred above, we investigate the nonparametric regression model based on NOD random errors. By using the moment inequality, Bernstein-type inequality and truncating method for NOD random variables, we obtain some consistency results for estimator of g(x) such as the mean convergence, uniform convergence, almost sure convergence and convergence rate. We generalize some results of Liang and Jing [10] for NA random variables to the case of NOD random variables. Meanwhile, as an example of designed assumptions for weight functions, we give the nearest neighbor weights.

For any function g(x), we use c(g) to denote all continuity points of function g on the set A in for some q ≥ 1. Let c, c₁, c₂, C, C₁, C₂,... denote the positive constants whose values may vary at each occurrence. ⌈x⌉ denotes the largest integer not exceeding x, I(B) is the indicator function of set B, x⁺ = xI(x ≥ 0), x^- = -xI(x < 0) and ∥x∥ denotes Euclidean norm of x. In this article, main results are presented in Section 2, some lemmas and the proofs of main results are presented in Sections 3 and 4, respectively.

Under the nonparametric regression model of (1.1), for any fixed point x ∈ A, some assumptions on weighted function W_ni(x) = W_ni(x, x_n1,..., x_nn) are given as follows:

(H₁) as n → ∞;

(H₂) for all n;

(H₃) as n → ∞;

(H₄) as n → ∞ for all a > 0.

Theorem 2.1 Let {ε_n, n ≥ 1} be a mean zero NOD sequence. Assume that the conditions (H₁)-(H₄) hold true. If , then for x ∈ c(g) and some p ∈ (0, 2],

If for some p > 2, then (2.1) also holds true.

In order to obtain uniform convergence for the estimator of g(x), for any fixed point x on a compact set A in for some q ≥ 1, some uniform version of assumptions on W_ni(x) = W_ni(x,x_n1,...,x_nn) are replaced by that as follows:

Theorem 2.2 Let {ε_n, n > 1} be a mean zero NOD sequence. Assume that the conditions hold true and g is continuous on the compact set A. If , then for some p ∈ (0, 2],

If for some p > 2, then (2.2) also holds true.

Next, we will study the almost sure convergence and convergence rate for the estimator of g(x). Similarly, for any fixed point x on the compact set A in for some q ≥ 1, some assumptions on the W_ni(x) = W_ni(x, x_n1,...,x_nn) are shown as follows:

(H₅) ;

(H₆) for all n ≥ 1 and ;

(H₇) for some a > 0.

Theorem 2.3 Let {ε_n, n ≥ 1} be a mean zero NOD sequence such that . Suppose that the conditions (H₅)-(H₇) hold true and g(x) satisfies a local Lipschitz condition around the point x. Then for x ∈ A,

Theorem 2.4 Let {ε_n, n ≥ 1} be a mean zero NOD sequence such that . Suppose that the conditions (H₅)-(H₇) hold true and g(x) satisfies a local Lipschitz condition around the point x. Then for x ∈ A,

Remark 2.1 The similar assumptions on weighted functions can be found in Ren and Chen [11], Hu et al. [31] and Liang and Jing [10], etc. Under the NA sequence and other assumptions, for some p > 1, Liang and Jing [10] obtained the result E|g_n(x)-g(x)|^p → 0 as n → ∞ (see Liang and Jing [10, Theorem 2.1]). In our Theorem 2.1, we give the result E|g_n(x)-g(x)|^p → 0 as n → ∞ for some p > 0. Liang and Jing [10] also studied the strong consistency of the estimator for g(x). In our Theorems 2.3 and 2.4, the strong consistency and convergence rate of the estimator for g(x) are presented. Since NA sequence is a NOD sequence, we generalize some results of Liang and Jing [10] to the case of NOD sequence.

Example 2.1 Here, we give an example that the designed assumptions (H₅)-(H₇) are satisfied for the nearest neighbor weights. Without loss of generality, let A = [0,1] and . For x ∈ A, we rewrite |x_n1 - x|, |x_n2 - x|,..., |x_nn - x| as follows

if |x_ni - x| = |x_nj - x|, |x_ni - x| is in frond of |x_nj - x| for i < j. Let k_n = ⌈n^5/8⌉ and define the nearest neighbor weight functions as following

Consequently, for every x ∈ [0,1], we can find by definition of R_i (x) and choice of x_ni that

If g is continuous on [0,1], then by (2.6)-(2.9), it can find that the assumptions of (H₁)-(H₇) and are satisfied.

Lemma 3.1 (cf. Bozorgnia et al. [18]). Let random variables X₁,X₂,...,X_n be NOD, f₁,f₂,...,f_n be all nondecreasing (or nonincreasing) functions, then random variables f₁(X₁), f₂(X₂),..., f_n(X_n) are NOD.

Lemma 3.2 (cf. Asadian et al. [19]). Let {X_n, n ≥ 1} be a NOD sequence such that EX_n = 0 and E|X_n|^p < ∞ for all n ≥ 1 and p ≥ 2. Then

where c_p depends only on p.

Lemma 3.3 (cf. Wang et al. [20]). Let {X_n}_n≥1 be a sequence of NOD random variables such that EX_n = 0 and |X_n| ≤ b for each n ≥ 1, where b is a positive constant.

Denote for each n ≥ 1. Then for every ε > 0,

Proof of Theorem 2.1: By C_r inequality, it has

For x ∈ c(g) and a > 0,

So, similar to the proof of (2.10) in Hu et al. [31], by conditions (H₁), (H₂) and (H₄), it is easy to have that

On the other hand, by Lemma 3.1, for the fixed x, we can see that and are also NOD sequences. Combining with , without loss of generality, we assume W_ni(x) ≥ 0 in the proof. If 0 < p ≤ 2, by Jensen's inequality, Lemma 3.2, (H₃) and , we have

following from that {ε_ni, 1 ≤ i ≤ n} has the same distribution as {ε_i, 1 ≤ i ≤ n} for each n. Otherwise, for p > 2, by Lemma 3.2, and (H₃) again, we obtain

since for any positive number sequence {a_i,1 ≤ i ≤ n} and 1 ≤ α ≤ β. Therefore, by (4.1)-(4.4), the desired result (2.1) has been proved completely.

Proof of Theorem 2.2: Since g is continuous in the compact set A, g is uniformly continuous in the compact set A. Consequently, similar to the proof of Theorem 2.1, we can get that

Therefore,

which implies the desired result (2.2).

Proof of Theorem 2.3: Combining the proof of (4.2) with the assumptions of (H₅)-(H₇) and g(x) satisfying a local Lipschitz condition around the point x, we can get that

Therefore, for x ∈ A, to prove (2.3), we only have to show that

Without loss of generality, we assume that W_ni(x) ≥ 0 in the proof. Let

Since for each n, it is easy to see that

Obviously, for the fixed x and n, is a NOD sequence with mean zero. Meanwhile, by the condition (H₆), it has

Since {ε_ni, 1 ≤ i ≤ n} has the same distribution as {ε_i, 1 ≤ i ≤ n} for each n, we obtain by applying Lemma 3.3 that for every ϵ > 0,

which implies

following from Borel-Cantelli lemma.

Next, we turn to estimate T_n2 and T_n3. It can be checked by that

which implies

Consequently, by Kronecker's lemma, we have that

Thus, by the condition (H₆), it is easy to see that

Obviously, by and (H₆) again,

Combining (4.10) with (4.11), it follows

Likewise, by , we will found that

which implies

Then, by Kronecker's lemma

Consequently, by (H₆), it has that

On the other hand, by (H₆) and again,

Finally,

Therefore, by (4.7), (4.8), (4.12) and (4.13), (4.6) is completely proved. The desired result (2.3) follows from (4.5) and (4.6) immediately.

Proof of Theorem 2.4: By the estimation of (4.5), to prove (2.4), we only need to prove that |g_n(x) - Eg_n(x)| = O(n^-1/4), a.s. It is also to assume that W_ni(x) ≥ 0 in the proof. Similar to the proof of Theorem 2.3, we will use the same notation and T_nq for q = 1, 2, 3, where i^1/2 is replaced by i^1/4. Obviously implies , by (H₆), it has

Since {ε_ni, 1 ≤ i ≤ n} has the same distribution as {ε_i, 1 ≤ i ≤ n} for each n, we obtain by applying Lemma 3.3 that for every ϵ > 0

which implies by Borel-Cantelli lemma that

Meanwhile, it can be checked by that

which implies

Then, we have by Kronecker's lemma that

Consequently, by (H.6), it follows

and

On the other hand, it can be checked that

which implies

So, by Kronecker's lemma,

Consequently, by (H.6), we have

and

Finally, similar to the proof of (2.3), by (4.14)-(4.18), it easily has that |g_n(x) -Eg_n(x)| = O(n^-1/4), a.s..

The authors declare that they have no competing interests.

All authors read and approved the final manuscript.

The authors are grateful to Associate Editor Prof. Andrei Volodin and two anonymous referees for their careful reading and insightful comments. This work was supported by the National Natural Science Foundation of China (11171001, 11126176), HSSPF of the Ministry of Education of China (10YJA910005), Natural Science Foundation of Anhui Province (1208085QA03) and Provincial Natural Science Research Project of Anhui Colleges (KJ2010A005).

1. Georgiev, AA: Local properties of function fitting estimates with applications to system identification. Mathematical Statistics and Applications, Proceedings of the 4th Pannonian Symposium on Mathematical Statistics, Bad Tatzmannsdorf, pp. 141–151. Bad Tatzmannsdorf, Austria. Reidel, Dordrecht (1985) Austria, 4-10, Sept 1983

2. Georgiev, AA, Greblicki, W: Nonparametric function recovering from noisy observations. J Stat Plan Infer. 13, 1–14 (1986)

3. Georgiev, AA: Consistent nonparametric multiple regression: the fixed design case. J Multivar Anal. 25(1), 100–110 (1988). Publisher Full Text

4. Müller, HG: Weak and universal consistency of moving weighted averages. Period Math Hungar. 18(3), 241–250 (1987). Publisher Full Text

5. Fan, Y: Consistent nonparametric multiple regression for dependent heterogeneous processes: the fixed design case. J Multivar Anal. 33(1), 72–88 (1990). Publisher Full Text

6. Roussas, GG: Consistent regression estimation with fixed design points under dependence conditions. Stat Probab Lett. 8(1), 41–50 (1989). Publisher Full Text

7. Roussas, GG, Tran, LT, Ioannides, DA: Fixed design regression for time series: Asymptotic normality. J Multivar Anal. 40(2), 262–291 (1992). Publisher Full Text

8. Tran, L, Roussas, G, Yakowitz, S, Van, BT: Fixed-design regression for linear time series. Ann Stat. 24(3), 975–991 (1996)

9. Hu, SH, Zhu, CH, Chen, YB: Fixed-design regression for linear time series. Acta Math Sci Ser B Engl Ed. 22(1), 9–18 (2002)

10. Liang, HY, Jing, BY: Asymptotic properties for estimates of nonparametric regression models based on negatively associated sequences. J Multivar Anal. 95(2), 227–245 (2005). Publisher Full Text

11. Ren, Z, Chen, MH: Strong consistency of a class of estimators in partial linear model for negative associated samples. Chinese J Appl Prob Stat. 18(1), 60–66 (2002)

12. Hu, SH: Fixed-design semiparametric regression for linear time series. Acta Math Sci Ser B Engl Ed. 26(1), 74–82 (2006)

13. Baek, J-II, Liang, HY: Asymptotics of estimators in semi-parametric model under NA samples. J Stat Plan Infer. 136(10), 3362–3382 (2006). Publisher Full Text

14. Liang, HY, Mammitzsch, V, Steineback, J: On a semiparametric regression model whose errors form a linear process with negatively associated innovations. Statistics. 40(3), 207–226 (2006). Publisher Full Text

15. Härdle, W, Liang, H, Gao, JT: Partially Linear Models, Springer Series in Economics and Statistics. Physica-Verlag, New York (2000)

16. Lehmann, EL: Some concepts of dependence. Ann Math Stat. 37(5), 1137–1153 (1966). Publisher Full Text

17. Joag-Dev, K, Proschan, F: Negative association of random variables with applications. Ann Stat. 11(1), 286–295 (1983). Publisher Full Text

18. Bozorgnia, A, Patterson, RF, Taylor, RL: Limit Theorems for Dependent Random Variables. World Congress Nonlinear Analysts'92, pp. 1639–1650. de Gruyter, Berlin (1996) (Tampa, FL, 1992)

19. Asadian, N, Fakoor, V, Bozorgnia, A: Rosenthal's type inequalities for negatively orthant dependent random variables. J Iranian Stat Soc. 5(1-2), 69–75 (2006)

20. Wang, XJ, Hu, SH, Yang, WZ, Ling, NX: Exponential inequalities and inverse moment for NOD sequence. Stat Probab Lett. 80(5-6), 452–461 (2010). Publisher Full Text

21. Wu, QY: Complete convergence for negatively dependent sequences of random variables. J Inequal Appl. 2010, 10 (Article ID 507293) (2010)

22. Wu, QY: A strong limit theorem for weighted sums of sequences of negatively dependent random variables. J Inequal Appl. 2010, 8 (Article ID 383805) (2010)

23. Wang, XJ, Hu, SH, Shen, AT, Yang, WZ: An exponential inequality for a NOD sequence and a strong law of large numbers. Appl Math Lett. 24(2), 219–223 (2011). Publisher Full Text

24. Wang, XJ, Hu, SH, Volodin, AI: Strong limit theorems for weighted sums of NOD sequence and exponential inequalities. Bull Korean Math Soc. 48(5), 923–938 (2011). Publisher Full Text

25. Li, XQ, Yang, WZ, Hu, SH, Wang, XJ: The Bahadur representation for sample quantile under NOD sequence. J Nonparametr Stat. 23(1), 59–65 (2011). Publisher Full Text

26. Sung, SH: On the exponential inequalities for negatively dependent random variables. J Math Anal Appl. 381(2), 538–545 (2011). Publisher Full Text

27. Matula, P: A note on the almost sure convergence of sums of negatively dependent random variables. Stat Probab Lett. 15(3), 209–213 (1992). Publisher Full Text

28. Wu, QY: Probability Limit Theory of Mixing Sequences. Science Press, Beijing (2006)

29. Gan, SX, Chen, PY: Some limit theorems for sequences of pairwise NQD random variables. Acta Math Sci Ser B Engl. 28(2), 269–281 (2008)

30. Li, R, Yang, WG: Strong convergence of pairwise NQD random sequences. J Math Anal Appl. 344(2), 741–747 (2008). Publisher Full Text

31. Hu, SH, Pan, GM, Gao, QB: Estimate problem of regression models with linear process errors. Appl Math A J Chinese Univ. 18A(1), 81–90 (2003)