SRB Case Studies | National Virtual Observatory (NVO)

The Digital Sky Project

Author

George Kremenek

Project Name
National Virtual Observatory (NVO), 2 micron all sky survey (2MASS)

Area
Astronomy, computational astronomy

Problem
Significant computing capabilities are required to support computational astronomy, including large scans over data collections, data mining and cataloguing. This project combines grid data-intensive computing and technologies; in addition to remote data access and analysis.

Players Involved
SDSC, Caltech, NVO, IPAC Roy Williams, Caltech
Storage Resource Broker (SRB) and Metadata Catalog (MCAT):
George Kremenek, SDSC

Time-line
Completed

Data Sizes
2MASS has around 5,000,000 data files in a 10 TB collection, DPOSS has about 3 TB.

SRB Solution
SRB was utilized to provide distributed data access; data and metadata management. The standard open source FITS library was modified to provide a transparent SRB data access for NVO application. Several mosaic and data cataloguing software packages were written by astronomers to support this effort.

Comments
The data ingestion and sorting was finished very successfully. JPL, Caltech and IPAC created mosaics on demand and cataloguing applications in later phases.