The Digital Sky Project
Author
George Kremenek
Project Name
National Virtual Observatory (NVO), 2 micron all sky survey (2MASS)
Area
Astronomy, computational astronomy
Problem
Significant computing capabilities are required to support computational astronomy, including large scans over data collections, data mining and cataloguing. This project combines grid data-intensive computing and technologies; in addition to remote data access and analysis.
Players Involved
SDSC, Caltech, NVO, IPAC Roy Williams, Caltech
Storage Resource Broker (SRB) and Metadata Catalog (MCAT):
George Kremenek, SDSC
Time-line
Completed
Data Sizes
2MASS has around 5,000,000 data files in a 10 TB collection, DPOSS has about 3 TB.
SRB Solution
SRB was utilized to provide distributed data access; data and metadata management. The standard open source FITS library was modified to provide a transparent SRB data access for NVO application. Several mosaic and data cataloguing software packages were written by astronomers to support this effort.
Comments
The data ingestion and sorting was finished very successfully. JPL, Caltech and IPAC created mosaics on demand and cataloguing applications in later phases.