Workflow management system
Pegasus is an open-source workflow management system.[1][2][3] It provides the necessary abstractions for scientists to create scientific workflows[4] and allows for transparent execution of these workflows on a range of computing platforms including high performance computing clusters, clouds, and national cyberinfrastructure.[5][6] In Pegasus, workflows are described abstractly as directed acyclic graphs (DAGs) using a provided API for Jupyter Notebooks, Python, R, or Java.[7] During execution, Pegasus translates the constructed abstract workflow into an executable workflow[8][9] which is executed and managed by HTCondor.[10][11]
Pegasus is being used in a number of different disciplines including astronomy, gravitational-wave physics, bioinformatics, earthquake engineering, and helioseismology.[12] Notably, the LIGO Scientific Collaboration has used it to directly detect a gravitational wave for the first time.[8][13][14]
- ^ E. Deelman, K. Vahi, G. Juve, M. Rynge, S. Callaghan, P. J. Maechling, R. Mayani, W. Chen, R. Ferreira da Silva, M. Livny, and K. Wenger, "Future Generation Computer Systems", Elsevier; 46, pp. 17-35 (2015)
- ^ E.A. Huerta, R. Haas, E. Fajardo, D.S. Katz, S. Anderson, P. Couvares ,J. Willis, T. Bouvet, J. Enos, W.T.C. Kramer, H.W. Leong, and D. Wheeler, "BOSS-LDG: A Novel Computational Framework That Brings Together Blue Waters, Open Science Grid, Shifter and the LIGO Data Grid to Accelerate Gravitational Wave Discovery", 2017 IEEE 13th International Conference on e-Science (e-Science); pp. 335-344 (2017)
- ^ B. Riedel, B. Bauermeister, L. Bryant, J. Conrad, P. de Perio, R. W. Gardner ,L. Grandi, F. Lombardi, A. Rizzo, G. Sartorelli, M. Selvi, E. Shockley, J. Stephen, S. Thapa, and C. Tunnell "Distributed Data and Job Management for the XENON1T Experiment", PEARC '18: Proceedings of the Practice and Experience on Advanced Research Computing;9, pp. 1-8 (2018)
- ^ G. Amalarethinam, T. Lucia, A. Beena, “Scheduling Framework for Regular Scientific Workflows in Cloud”, International Journal of Applied Engineering Research; 10, no. 82 (2015)
- ^ Cite error: The named reference
framework was invoked but never defined (see the help page).
- ^ The Scientific Workflow Integrity with Pegasus (SWIP), by
Center for Applied Cybersecurity Research; published 16 September 2016; retrieved 1 May 2020
- ^ D. Weitzel, B. Bockelman, D. Brown, P. Couvares, F. Würthwein, and E.F. Hernandez, “Data Access for LIGO on the OSG”, Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact - PEARC17; 24, no. 1-6 (2017)
- ^ a b "Testing LIGO's Sensitivity". Research.gov. September 1, 2007. Retrieved April 30, 2020.
- ^ Duncan Brown and Ewa Deelman, "Looking for gravitational waves: A computing perspective", at Science Node; published June 8, 2011; retrieved April 30, 2020
- ^ $1M NSF award goes to IU-led data integrity project, by Indiana University; published 16 September 2016; retrieved 1 May 2020
- ^ Brian Mattmiller, "High Throughput Computing helps LIGO confirm Einstein's last unproven theory", at Morgridge Institute for Research; published March 7, 2016; retrieved May 1, 2020
- ^ Sanden Totten, "Caltech Wasn't the Only SoCal School Helping Discover Gravitational Waves", at KPCC; published 11 February 2016; retrieved May 1, 2020
- ^ D.A. Brown, P.R. Brady, A. Dietz, J. Cao, B. Johnson, J. McNabb, “A Case Study on the Use of Workflow Technologies for Scientific Analysis: Gravitational Wave Data Analysis. In: I.J Taylor, E. Deelman, D.B. Gannon, M. Shields (eds) Workflows for e-Science”, Springer, London; 13, pp. 39-59 (2007)
- ^ D. Davis, T. Massinger, A. Lundgren, J.C. Driggers, A.L. Urban, and L. Nuttall, “Improving the sensitivity of Advanced LIGO using noise subtraction”, Classical and Quantum Gravity; 36, no. 5 (2019)