×

Campus grids meet applications: Modeling, metascheduling and integration. (English) Zbl 1096.92054

Summary: Air Quality Forecasting (AQF) is a new discipline that attempts to reliably predict atmospheric pollution. An AQF application has complex workflows and in order to produce timely and reliable forecast results, each execution requires access to diverse and distributed computational and storage resources. Deploying AQF on Grids is one option to satisfy such needs, but requires the related Grid middleware to support automated workflow scheduling and execution on Grid resources.
We analyze the challenges in deploying an AQF application in a campus Grid environment and present our current efforts to develop a general solution for Grid-enabling scientific workflow applications in the GRACCE project. In GRACCE, an application’s workflow is described using GAMDL, a powerful dataflow language for describing application logic. The GRACCE metascheduling architecture provides the functionalities required for co-allocating Grid resources for workflow tasks, scheduling the workflows and monitoring their execution. By providing an integrated framework for modeling and metascheduling scientific workflow applications on Grid resources, we make it easy to build a customized environment with end-to-end support for application Grid deployment, from the management of an application and its dataset, to the automatic execution and analysis of its results.

MSC:

92D40 Ecology
93A30 Mathematical modelling of systems (MSC2010)
92-04 Software, source code, etc. for problems pertaining to biology
93C95 Application models in control theory
PDF BibTeX XML Cite
Full Text: DOI

References:

[1] Bose, A., Wickman, B., Wood, C.: MARS: A metascheduler for distributed resources in campus Grids, Proceedings of Fifth IEEE/ACM International Workshop on Grid Computing, 2004
[2] Chapman, B.M., Raghunath, P., Sundaram, B., Yan, Y.: Air quality prediction in a production quality Grid environment. In: Dongarra, J., et al. (eds.) Engineering the Grid: Status and Perspective. (Spring 2005)
[3] Chapman, B.M., Donepudi, H., He, J., Li, Y., Raghunath, P., Sundaram, B., Yan, Y.: Grid Environment with Web-Based Portal Access for Air Quality Modeling, Parallel and Distributed Scientific and Engineering Computing, Practice and Experience, 2003
[4] Chapman, B.M., Li, Y., B., Sundaram, He, J.: Computational Environment for Air Quality Modeling in Texas, Use of High Performance Computing in Meteorology. World Scientific (2003)
[5] Churches, D., Gombas, G., Harrison, A., Maassen, J., Robinson, C., Shields, M., Taylor, I., Wang, I.: Programming Scientific and Distributed Workflow with Triana Services, Special Issue of Concurrency and Computation: Practice and Experience, 2005
[6] Byun, D.W., Pleim, J., Tang, R., Bourgeois, A.: Meteorology-Chemistry Interface Processor (MCIP) for Models-3 Community Multiscale Air Quality (CMAQ) Modeling System. Environmental Protection Agency, Office of Research and Development, Washington, District of Columbia (1999)
[7] Byun, D.W., Schere, K.: EPA’s third generation air quality modeling system: Description of the models-3 Community Multiscale Air Quality (CMAQ) Model. Journal of Mech. Review (2004)
[8] Deelman, E., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Patil, S., Su, M., Vahi, K., Livny, M.: Pegasus: Mapping Scientific Workflows onto the Grid, Across Grids Conference 2004, Nicosia, Cyprus
[9] Grell, G., Dudhia, J., Stauffer, D.: A Description of the Fifth-Generation Penn State/NCAR Mesoscale Model (MM5), NCAR Tech Notes, TN-398+STR
[10] Foster, I., Kesselman, C.: Globus: A metacomputing infrastructure toolkit. I. J. Supercomput. Appl. (Summer 1997)
[11] Foster, I., Kesselman, C., Lee, C., Lindell, R., Nahrstedt, K., Roy, A.: A distributed resource management architecture that supports advance reservations and co-allocation. Intl Workshop on Quality of Service, 1999
[12] Foster, I., Kesselman, C., Tsudik, G., Tuecke, S.: A security architecture for computational Grids, ACM Conference on Computers and Security, 83-91, 1998
[13] Foster, I., Kesselman, C., Tuecke, S.: The anatomy of the Grid: Enabling scalable virtual organizations. Int. J. High Perform. Comput. Appl. 15(3), 200-222 (2001)
[14] Foster, I., Vockler, J., Wilde, M., Zhao, Y.: Chimera: A virtual data system for representing, querying, and automating data derivation, Proceedings of the 14th International Conference on Scientific and Statistical Database Management, pp. 37-46, 2002
[15] Cao, J., Jarvis, S.A., Saini, S., Nudd, G.R.: GridFlow: Workflow management for Grid computing, Proceedings of 3rd International Symposium on Cluster Computing and the Grid, at Tokyo, Japan, May 12-15, 2003
[16] Yu, J., Buyya, R.: A taxonomy of workflow management systems for Grid computing, Technical Report, GRIDS-TR-2005-1, University of Melbourne, Australia, March, 2005
[17] Czajkowski, K., Foster, I., Kesselman, C.: Resource co-allocation in computational Grids, Proceedings of the Eighth IEEE International Symposium on High Performance Distributed Computing (HPDC-8), pp. 219-228, 1999
[18] Czajkowski, K., Foster, I., Kesselman, C., Sander, V., Tuecke, S.: SNAP: A Protocol for negotiating service level agreements and coordinating resource management in distributed systems. Lect. Notes Comput. Sci. 2537, 153-183 (2002) · Zbl 1024.68682
[19] Czajkowski, K., Foster, I., Karonis, N., Kesselman, C., Martin, S., Smith, W., Tuecke, S.: A resource management architecture for metacomputing systems, Workshop on Job Scheduling Strategies for Parallel Processing, pp. 62-82, 1998
[20] Wieczorek, M., Prodan, R., Fahringer, T.: Scheduling of scientific workflows in the ASKALON Grid environment. ACM SIGMOD Record Journal (2005)
[21] Buyya, R., Abramson, D., Giddy, J.: Nimrod/G: An architecture for a resource management and scheduling system in a global computational Grid, The 4th International Conference on High Performance Computing in Asia-Pacific Region (HPC Asia 2000), May 2000 · Zbl 1042.68514
[22] Fahringer, T., Qin, J., Hainzer, S.: Specification of Grid workflow applications with AGWL: An abstract Grid workflow language, Proceedings of Cluster Computing and Grid, 2005
[23] Oinn, T., et al.: Taverna: Lessons in creating a workflow environment for the life sciences, Concurrency and Computation: Practice and Experience Grid Workflow Special Issue, 09, 2002
[24] Dabberdt, W.F., Carroll, M.A., Baumgardner, D., Carmichael, G., Cohen, R.: Meteorological research needs for improved air quality forecasting, The 11th Prospectus Development Team of the U.S. Weather Research Program, 2004
[25] Smith, W., Foster, I., Taylor, V.: Scheduling with advanced reservations, Proceedings of the 14th International Parallel and Distributed Processing Symposium (IPDPS’00), 2000
[26] Adelman, Z., Houyoux, M.: Processing the National Emissions Inventory 96 (NEI96) version 3.11 with SMOKE, The Emission Inventory Conference: One Atmosphere, One Inventory, Many Challenges, 1–3 May, Denver, Colorado Environmental Protection Agency, 2001
[27] BPEL4WS: Business Process Execution Language for Web Services v1.0, http://www.106.ibm.com/developerworks/webservices/library/wsbpel
[28] Community Scheduler Framework: http://www.platform.com/products/Globus
[29] DAGMan (Directed Acyclic Graph Manager): http://www.cs.wisc.edu/condor/dagman
[30] GRACCE: Grid Application Coordination, Collaboration and Execution: http://www.cs.uh.edu/\(\sim\)yanyh/gracce
[31] High Performance Computing Center, University of Houston: http://www.hpcc.uh.edu/
[32] Java CoG Kit Karajan/Gridant Workflow Guide: http://www.cogkit.org/release/4_0_a1/manual/workflow.pdf
[33] Job Submission Description Language, GGF: https://forge.Gridforum.org/projects/jsdl-wg
[34] Load Sharing Facility, Resource Management and Job Scheduling System: http://www.platform.com/products/HPC/
[35] Maui Moab Grid Scheduler (Silver): http://www.clusterresources.com/products/mgs
[36] Sun Grid Engine, Sun Microsystems: http://gridengine.sunsource.net/
[37] The Globus Resource Specification Language RSL v1.0: http://www-fp.globus.org/gram/rsl_spec1.html
[38] The University of Houston’s Sun Microsystems Center of Excellence in Geosciences: http://www.suncoe.uh.edu
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.