An iterative procedure for general probability measures to obtain \(I\)-projections onto intersections of convex sets. (English) Zbl 1117.62003

Bhattacharya, Bhaskar

Ann. Stat. 34, No. 2, 878-902 (2006).

Summary: The iterative proportional fitting procedure (IPFP) was introduced formally by W. E. Deming and F. F. Stephan [Ann. Math. Stat., Ann. Arbor 11, 427–444 (1940; JFM 66.0652.02)]. For bivariate densities, this procedure has been investigated by S. Kullback [Ann. Math. Stat. 39, 1236–1243 (1968; Zbl 0165.20303)] and L. Rüschendorf [Ann. Stat. 23, No. 4, 1160–1174 (1995; Zbl 0851.62038)]. It is well known that the IPFP is a sequence of successive \(I\)-projections onto sets of probability measures with fixed marginals. However, when finding the \(I\)-projection onto the intersection of arbitrary closed, convex sets (e.g., marginal stochastic orders), a sequence of successive \(I\)-projections onto these sets may not lead to the actual solution. Addressing this situation, we present a new iterative \(I\)-projection algorithm. Under reasonable assumptions and using tools from Fenchel duality, convergence of this algorithm to the true solution is shown. The cases of infinite dimensional IPFP and marginal stochastic orders are worked out in this context.

MSC:

62A01	Foundations and philosophical topics in statistics
60B10	Convergence of probability measures
65K10	Numerical optimization and variational techniques
60E15	Inequalities; stochastic orderings
62B10	Statistical aspects of information-theoretic topics
62B99	Sufficiency and information
62H99	Multivariate analysis

Keywords:

algorithm; convergence; convex sets; Fenchel duality; functions; I-projection; inequality constraints; stochastic order

Citations:

JFM 66.0652.02; Zbl 0165.20303; Zbl 0851.62038

References:

[1]	Barbu, V. and Precupanu, T. (1986). Convexity and Optimization in Banach Spaces , 2nd ed. Reidel, Dordrecht. · Zbl 0594.49001
[2]	Bhattacharya, B. and Dykstra, R. (1995). A general duality approach to I-projections. J. Statist. Plann. Inference 47 203–216. · Zbl 0844.49024 · doi:10.1016/0378-3758(94)00142-I
[3]	Bhattacharya, B. and Dykstra, R. (1997). A Fenchel duality aspect of iterative I-projection procedures. Ann. Inst. Statist. Math. 49 435–446. · Zbl 0935.62010 · doi:10.1023/A:1003110627413
[4]	Billingsley, P. (1968). Convergence of Probability Measures . Wiley, New York. · Zbl 0172.21201
[5]	Cover, T. M. and Thomas, J. (1991). Elements of Information Theory . Wiley, New York. · Zbl 0762.94001
[6]	Csiszár, I. (1975). I-divergence geometry of probability distributions and minimization problems. Ann. Probab. 3 146–158. · Zbl 0318.60013 · doi:10.1214/aop/1176996454
[7]	Csiszár, I. (1989). A geometric interpretation of Darroch and Ratcliff’s generalized iterative scaling. Ann. Statist. 17 1409–1413. · Zbl 0681.62010 · doi:10.1214/aos/1176347279
[8]	Darroch, J. N. and Ratcliff, D. (1972). Generalized iterative scaling for loglinear models. Ann. Math. Statist. 43 1470–1480. · Zbl 0251.62020 · doi:10.1214/aoms/1177692379
[9]	Deming, W. E. and Stephan, F. F. (1940). On a least squares adjustment of a sampled frequency table when the expected marginal totals are known. Ann. Math. Statist. 11 427–444. · Zbl 0024.05502 · doi:10.1214/aoms/1177731829
[10]	Dykstra, R. (1985). An iterative procedure for obtaining I-projections onto the intersection of convex sets. Ann. Probab. 13 975–984. · Zbl 0571.60006 · doi:10.1214/aop/1176992918
[11]	Dykstra, R. and Lemke, J. H. (1988). Duality of I-projections and maximum likelihood estimation for log-linear models under cone constraints. J. Amer. Statist. Assoc. 83 546–554. JSTOR: · Zbl 0702.62047 · doi:10.2307/2288876
[12]	Good, I. J. (1963). Maximum entropy for hypothesis formulation, especially for multidimensional contingency tables. Ann. Math. Statist. 34 911–934. · Zbl 0143.40705 · doi:10.1214/aoms/1177704014
[13]	Haberman, S. J. (1974). The Analysis of Frequency Data . Univ. Chicago Press. · Zbl 0325.62017
[14]	Haberman, S. J. (1984). Adjustment by minimum discriminant information. Ann. Statist. 12 971–988. · Zbl 0583.62020 · doi:10.1214/aos/1176346715
[15]	Ireland, C. T. and Kullback, S. (1968). Contingency tables with given marginals. Biometrika 55 179–188. JSTOR: · Zbl 0155.26701 · doi:10.1093/biomet/55.1.179
[16]	Jaynes, E. T. (1957). Information theory and statistical mechanics. Phys. Rev. ( 2 ) 106 620–630. · Zbl 0084.43701 · doi:10.1103/PhysRev.106.620
[17]	Kagan, A. M., Linnik, Y. V. and Rao, C. R. (1973). Characterization Problems in Mathematical Statistics . Wiley, New York. · Zbl 0271.62002
[18]	Kruithof, R. (1937). Telefoonverkeersrekening. De Ingenieur 52 E15–E25.
[19]	Kullback, S. (1959). Information Theory and Statistics . Wiley, New York. · Zbl 0088.10406
[20]	Kullback, S. (1968). Probability densities with given marginals. Ann. Math. Statist. 39 1236–1243. · Zbl 0165.20303 · doi:10.1214/aoms/1177698249
[21]	Liese, F. (1977). On the existence of \(f\)-projections. In Topics in Information Theory (I. Csiszár and P. Elias, eds.) 431–446. North-Holland, Amsterdam. · Zbl 0361.94034
[22]	Luenberger, D. G. (1969). Optimization by Vector Space Methods . Wiley, New York. · Zbl 0176.12701
[23]	Rao, C. R. (1973). Linear Statistical Inference and Its Applications , 2nd ed. Wiley, New York. · Zbl 0256.62002
[24]	Robertson, T., Wright, F. T. and Dykstra, R. L. (1988). Order Restricted Statistical Inference. Wiley, New York. · Zbl 0645.62028
[25]	Rüschendorf, L. (1995). Convergence of the iterative proportional fitting procedure. Ann. Statist. 23 1160–1174. · Zbl 0851.62038 · doi:10.1214/aos/1176324703
[26]	Rüschendorf, L. and Thomsen, W. (1993). Note on the Schrodinger equation and I-projections. Statist. Probab. Lett. 17 369–375. · Zbl 0780.60036 · doi:10.1016/0167-7152(93)90257-J
[27]	Sanov, I. N. (1957). On the probability of large deviations of random magnitudes. Mat. Sb. N.S. 42 11–44.
[28]	Winkler, W. (1990). On Dykstra’s iterative fitting procedure. Ann. Probab. 18 1410–1415. · Zbl 0708.90092 · doi:10.1214/aop/1176990752