AstroPortal
The astronomy community has an abundance of imaging datasets at its disposal
which are essentially the “crown jewels” for the astronomy community; however
the terabytes of data makes the traditional analysis of these datasets a very
difficult process. Large astronomy datasets are generally terabytes in size and
contain hundreds of millions of objects separated into millions of files. We
propose to use grid computing as the main mechanism to enable the dynamic
analysis of large astronomy datasets on the TeraGrid spanning many physical
resources. The key question we address is: “How can the analysis of large
astronomy datasets be made a reality for the astronomy community using Grid
resources?” Our answer is the “AstroPortal”, a science gateway to grid resources
that is specifically tailored for the astronomy community. We have implemented
our prototype as a web service using the Globus Toolkit 4 (GT4) and it is
deployed on the TeraGrid. The astronomy dataset we are using is the Sloan
Digital Sky Survey (SDSS), DR4, which is comprised of about 300 million objects
dispersed over 1.3 million files adding up to 3 terabytes of compressed data.
The analysis currently supported by the AstroPortal prototype is “stacking”, the
summation of multiple observations of the same part of the sky; “stacking” will
both help identify variable sources and to detect faint objects. The AstroPortal
will give the astronomy community a new tool to advance their research and to
open new doors to opportunities never before possible on such a large scale.
Collaborators:
-
Ioan Raicu,
Computer Science Dept. The University of Chicago
- Ian Foster,
Math and Computer Science Div. Argonne National Laboratory & Computer
Science Dept. The University of Chicago
- Alex Szalay,
Dept. of Physics and Astronomy The Johns Hopkins University
- Gabriela Turcu,
Computer Science Dept. The University of Chicago
Documents:
-
Ioan Raicu, Ian Foster. “Harnessing
Grid Resources to Enable the Dynamic Analysis of Large Astronomy Datasets:
Year 1 Status and Year 2 Proposal”, NASA GSRP Year 1 Progress Report
and Year 2 Proposal, Ames Research Center, NASA, February 2007 --
Award funded 10/1/07 - 9/30/08.
-
I. Raicu, I. Foster.
“Harnessing
Grid Resources to Enable the Dynamic Analysis of Large Astronomy Datasets”,
NASA GSRP Proposal, Ames Research Center, NASA, February 2006 --
Award funded 10/1/06 - 9/30/07.
-
Ioan Raicu, Ian Foster, Alex Szalay. “Harnessing
Grid Resources to Enable the Dynamic Analysis of Large Astronomy Datasets”,
poster presentation, IEEE/ACM SuperComputing 2006.
-
Ioan Raicu, Ian Foster, Alex Szalay, Gabriela Turcu.
“AstroPortal: A Science Gateway for Large-scale Astronomy Data Analysis”,
TeraGrid Conference 2006, June 2006.
-
Alex Szalay, Julian Bunn, Jim Gray, Ian Foster, Ioan Raicu.
“The Importance of Data Locality in Distributed Computing Applications”,
NSF Workflow Workshop 2006.
-
Ioan Raicu, Ian Foster. “SkyServer
Web Service”, Technical Report, University of Chicago, 2006.
-
I. Raicu,
, I. Foster. “Characterizing
the SDSS DR4 Dataset and the SkyServer Workloads,” Technical Report,
University of Chicago, 2006
-
Ioan Raicu, Ian Foster, Alex Szalay. "AstroPortal",
Handout, University of Chicago, 2006.
-
Ioan Raicu, Ian Foster, Alex Szalay. "Harnessing
Grid Resources to Enable the Dynamic Analysis of Large Astronomy Datasets",
Technical Report, University of Chicago, 2006.
-
I. Raicu,
, I. Foster. “Characterizing
Storage Resources Performance in Accessing the SDSS Dataset,”
Technical Report, University of Chicago, 2005
Presentations:
-
AstroPortal: A Science Gateway for Large-scale Astronomy Data Analysis, IEEE/ACM SuperComputing 2006,
November 2006.
-
Storage and Compute Resource Management via DYRE, 3DcacheGrid, and
CompuStore, University of Chicago, Department of Computer Science, Distributed
Systems Lab Seminar,
November 2006.
-
Harnessing Grid Resources to Enable the Dynamic Analysis of Large Astronomy
Datasets, DSLW 2006, June 2006
-
AstroPortal: A Science Gateway for Large-scale Astronomy Data Analysis, TeraGrid 2006, June 2006
-
Harnessing Grid Resources to Enable the Dynamic Analysis of Large Astronomy
Datasets, University of Chicago, Department of Computer Science, Graduate
Seminar,
February 2006.
-
AstroPortal: A Science Portal to Grid Resources, University of Chicago, Department of Computer Science, Distributed
Systems Lab Seminar,
January 2006.
Links:

Last modified:
January 11, 2008