The Digital Sky Project
National Virtual Observatory (NVO), 2 micron all sky survey (2MASS)
Astronomy, computational astronomy
Significant computing capabilities are required to support computational astronomy, including large scans over data collections, data mining and cataloguing. This project combines grid data-intensive computing and technologies; in addition to remote data access and analysis.
SDSC, Caltech, NVO, IPAC Roy Williams, Caltech
Storage Resource Broker (SRB) and Metadata Catalog (MCAT):
George Kremenek, SDSC
2MASS has around 5,000,000 data files in a 10 TB collection, DPOSS has about 3 TB.
SRB was utilized to provide distributed data access; data and metadata management. The standard open source FITS library was modified to provide a transparent SRB data access for NVO application. Several mosaic and data cataloguing software packages were written by astronomers to support this effort.
The data ingestion and sorting was finished very successfully. JPL, Caltech and IPAC created mosaics on demand and cataloguing applications in later phases.