×

A mathematical framework to optimize ATR systems with non-declarations and sensor fusion. (English) Zbl 1139.90438

Summary: Combat identification is one example where incorrect automatic target recognition (ATR) output labels may have substantial decision costs. For example, the incorrect labeling of hostile targets vs. friendly non-targets may have high costs; yet, these costs are difficult to quantify. One way to increase decision confidence is through fusion of data from multiple sources or from multiple looks through time. Numerous methods have been published to determine a Bayes’ optimal fusion decision if decision costs are well known. This research presents a novel mathematical programming ATR evaluation framework. A new objective function inclusive of time is introduced to optimize and compare ATR systems. Constraints are developed to enforce both decision maker preferences and traditional engineering measures of performance. This research merges rejection and receiver operating characteristic (ROC) analysis by incorporating rejection and ROC thresholds as decision variables. The rejection thresholds specify non-declaration regions, while the ROC thresholds explore viable true positive and false positive tradeoffs for output target labels. This methodology yields an optimal ATR system subject to decision maker constraints without using explicit costs for each type of output decision. A sample application is included for the fusion of two channels of collected polarized radar data for 10 different ground targets. A Boolean logic and probabilistic neural network fusion method are optimized and compared. Sensitivity analysis of significant performance parameters then reveals preferred regions for each of the fusion algorithms.

MSC:

90C99 Mathematical programming
92B20 Neural networks for/in biological studies, artificial life and related topics

Software:

HandTill2001
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] Laine, T. I.; Bauer, K. W., A mathematical framework for the optimization of rejection and ROC thresholds in the fusion of correlated sensor data, Proceedings of SPIE, 5434, 37-48 (2004)
[2] Laine TI. Optimization of automatic target recognition with a reject option using fusion and correlated sensor data, PhD dissertation, Air Force Institute of Technology, Wright-Patterson AFB, OH, 2005.; Laine TI. Optimization of automatic target recognition with a reject option using fusion and correlated sensor data, PhD dissertation, Air Force Institute of Technology, Wright-Patterson AFB, OH, 2005.
[3] Department of the Air Force (DAF). Air Warfare. AFDD2-1, Washington, DC: HQ USAF; 2000.; Department of the Air Force (DAF). Air Warfare. AFDD2-1, Washington, DC: HQ USAF; 2000.
[4] Department of the Air Force (DAF). USAF intelligence targeting guide. AFP 14-210. Washington, DC: HQ USAF; 1998.; Department of the Air Force (DAF). USAF intelligence targeting guide. AFP 14-210. Washington, DC: HQ USAF; 1998.
[5] Alsing SG. The evaluation of competing classifiers. PhD dissertation, Air Force Institute of Technology, Wright-Patterson AFB, OH, 2000.; Alsing SG. The evaluation of competing classifiers. PhD dissertation, Air Force Institute of Technology, Wright-Patterson AFB, OH, 2000.
[6] Albrecht TW. Combat identification with sequential observations, rejection option, and out-of-library targets. PhD dissertation, Air Force Institute of Technology, Wright-Patterson AFB, OH, 2005.; Albrecht TW. Combat identification with sequential observations, rejection option, and out-of-library targets. PhD dissertation, Air Force Institute of Technology, Wright-Patterson AFB, OH, 2005.
[7] Department of the Air Force (DAF). Intelligence, surveillance, and reconnaissance operations. AFDD2-5.2. Washington; DC: HQ USAF; 1999.; Department of the Air Force (DAF). Intelligence, surveillance, and reconnaissance operations. AFDD2-5.2. Washington; DC: HQ USAF; 1999.
[8] Hall DL, Llinas J, editors. Handbook of multisensor data fusion. New York: CRC Press; 2001.; Hall DL, Llinas J, editors. Handbook of multisensor data fusion. New York: CRC Press; 2001.
[9] Swets, J. A., Signal detection and recognition by human observers (1964), Wiley: Wiley New York
[10] Swets, J. A.; Dawes, R. M.; Monahan, J., Psychological science can improve diagnostic decisions, Psychological Science in the Public Interest, 1, 1-26 (2000)
[11] Chow, C. K., On optimum rejection error and reject tradeoff, IEEE Transactions on Information Theory, IT-16, 41-46 (1970) · Zbl 0185.47804
[12] Devijver, P. A.; Kittler, J., Pattern recognition—a statistical approach (1982), Prentice-Hall: Prentice-Hall London · Zbl 0476.68057
[13] Fumera, G.; Roli, F.; Giorgio, G., Reject option with multiple thresholds, Pattern Recognition, 33, 2099-2101 (2000)
[14] Haspert KJ. Optimum ID sensor fusion for multiple target types. Institute for Defense Analysis (IDA) Document D-2451, Virginia, March 2000.; Haspert KJ. Optimum ID sensor fusion for multiple target types. Institute for Defense Analysis (IDA) Document D-2451, Virginia, March 2000.
[15] Ross, T. D.; Westerkamp, L. A.; Zelnio, E. G.; Burns, T. J., Extensibility and other model-based ATR evaluation concepts, Proceedings of SPIE, 3070, 213-222 (1997)
[16] Alsing, S. G.; Blasch, E. P.; Bauer, K. W., Three-dimensional receiver operating characteristic trajectory concepts for the evaluation of target recognition algorithms faced with the unknown target detection problem, Proceedings of SPIE, 3718, 449-458 (1999)
[17] Wald, A., Sequential analysis (1947), Wiley: Wiley New York · Zbl 0041.26303
[18] Peterson, W. W.; Birdsall, T. G.; Fox, W. C., The theory of signal detectability, Transactions of the IRE Professional Group on Information Theory, PGIT-4, 171-212 (1954)
[19] Mossman, D., Three-way ROCs, Medical decision making, 19, 78-89 (1999)
[20] Hand, D. J.; Till, R. J., A simple generalisation of the area under the ROC curve for multiple class classification problems, Machine Learning, 45, 171-186 (2001) · Zbl 1007.68180
[21] Varner T. ACE TestJig evaluation metrics. Jacobs Sverdrup Working Papers, AFRL COMPASE Center, Wright-Patterson AFB, OH, 2002.; Varner T. ACE TestJig evaluation metrics. Jacobs Sverdrup Working Papers, AFRL COMPASE Center, Wright-Patterson AFB, OH, 2002.
[22] Ross, T. D.; Westerkamp, L. A.; Dilsavor, R. L.; Mossing, J. C., Performance measures for summarizing confusion matrices—the AFRL COMPASE approach, Proceedings of SPIE, 4727, 310-321 (2002)
[23] Laine, T. I.; Bauer, K. W., Input feature selection for automatic target recognition of temporal data, Military Operations Research Journal, 10, 2, 51-65 (2005)
[24] Sadowski C. Measuring Combat Identification, presentation, 69th MORS symposium, Annapolis MD, June 2001.; Sadowski C. Measuring Combat Identification, presentation, 69th MORS symposium, Annapolis MD, June 2001.
[25] Audet, C.; Dennis, J. E., Pattern search algorithms for mixed variable programming, SIAM Journal of Optimization,, 11, 3, 573-594 (2000) · Zbl 1035.90048
[26] Abramson MA. Pattern search algorithms for mixed variable general constrained optimization problems. PhD thesis, Department of Computational and Applied Mathematics, Rice University, Houston TX, August 2002.; Abramson MA. Pattern search algorithms for mixed variable general constrained optimization problems. PhD thesis, Department of Computational and Applied Mathematics, Rice University, Houston TX, August 2002.
[27] Sriver TA. Pattern search ranking and selection algorithms for mixed-variable optimization of stochastic systems. Dissertation, Air Force Institute of Technology, Wright-Patterson AFB OH, 2004.; Sriver TA. Pattern search ranking and selection algorithms for mixed-variable optimization of stochastic systems. Dissertation, Air Force Institute of Technology, Wright-Patterson AFB OH, 2004.
[28] Klein, L. A., Sensor and data fusion: a tool for information assessment and decision making (2004), SPIE Press: SPIE Press Bellingham WA
[29] Kuncheva, L. I., Combining pattern classifiers, methods and algorithms (2004), Wiley: Wiley New Jersey · Zbl 1066.68114
[30] Leap, N. J.; Clemans, P. P.; Bauer, K. W.; Oxley, M. E., An investigation on the effects of correlation and autocorrelation on classifier fusion and optimal classifier ensembles, (Intelligent Engineering Systems through ANNs, vol. 14 (2004), ASME Press: ASME Press New York), 627-632 · Zbl 1214.62099
[31] Mindrup FM. An investigation of the effects of correlation and autocorrelation on classifier fusion with non-declarations. Thesis AFIT/GOR/ENS/05-13, Air Force Institute of Technology, Wright-Patterson AFB OH, 2005.; Mindrup FM. An investigation of the effects of correlation and autocorrelation on classifier fusion with non-declarations. Thesis AFIT/GOR/ENS/05-13, Air Force Institute of Technology, Wright-Patterson AFB OH, 2005.
[32] Çetin M. Feature-enhanced synthetic aperture radar imaging. Dissertation, Boston University, Boston MA, 2001.; Çetin M. Feature-enhanced synthetic aperture radar imaging. Dissertation, Boston University, Boston MA, 2001.
[33] Çetin, M.; Karl, W. C.; Castañon, D. A., Feature enhancement and ATR performance using nonquadratic optimization-based SAR imaging, IEEE Transactions on Aerospace and Electronic Systems, 39, 4, 1375-1395 (2003)
[34] Williams, R.; Westerkamp, J.; Gross, D.; Palomino, A., 1D high range resolution radar, IEEE AES Systems Magazine, 10, 37-43 (2000)
[35] Williams, R.; Westerkamp, J.; Gross, D.; Palomino, A.; Kaufman, T.; Fister, T., Analysis of 1-D HRR moving target ATR, Proceedings of SPIE, 3721, 413-424 (1999)
[36] Mitchell RA, Westerkamp JJ. A statistical feature based classifier for robust high range resolution radar target identification. Technial report, AFRL/SN, WPAFB OH, 1998.; Mitchell RA, Westerkamp JJ. A statistical feature based classifier for robust high range resolution radar target identification. Technial report, AFRL/SN, WPAFB OH, 1998.
[37] Varshney, P. K., Distributed detection and data fusion (1997), Springer: Springer New York
[38] Robinson GS, Aboutalib AO. Trade-off analysis of multisensor fusion levels. Proceedings of the second national symposium on sensors and sensor fusion GACIAC PR-89-01, Chicago, IL, 1990, pp. 21-34.; Robinson GS, Aboutalib AO. Trade-off analysis of multisensor fusion levels. Proceedings of the second national symposium on sensors and sensor fusion GACIAC PR-89-01, Chicago, IL, 1990, pp. 21-34.
[39] Hall, D. L.; Llinas, J., An introduction to multisensor data fusion, Proceedings of the IEEE, 85, 1, 6-23 (1997)
[40] Specht, D. F., Probabilistic neural networks, Neural Networks, 3, 109-118 (1990)
[41] Wasserman, P.; Nostrand, V., Advanced methods in neural computing (1993), Wiley: Wiley New York
[42] Brown, G. G., Top ten secrets to success with optimization, Phalanx, 37, 4, 12-13 (2004), 25-26
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.