CosmoSim Database for cosmological simulations released
Website of the CosmoSim database.
Credit: AIPThe CosmoSim database (www.cosmosim.org) has now been released after an intensive testing period. This service to the scientific community is the successor of the MultiDark database (www.multidark.org) and it is used as a platform for publishing and sharing data products from cosmological simulations.
It provides access to currently six cosmological simulations - including a highresolution resimulation of selected regions with hydrodynamics and star formation - which originate from different international projects in collaboration with the Leibniz Institute for Astrophysics Potsdam (AIP).
Outputs of cosmological simulations are typically stored at supercomputing centres with restricted access and encompass terabytes of data - too much to be downloaded by everyone. By providing the data via CosmoSim, scientists from all over the world are able to access the data, filter or combine the results directly on the server and use them for their own research.
The available data products include catalogues of dark matter halos, their inner properties, merging histories, information about the cosmic web and for selected snapshots even the raw particle distributions allowing for much deeper studies of dark matter halos and their environment. All the simulations and database tables are made available through a modern web interface. Additional features include an extensive documentation, along with some background on the database structure and selected images and movies for outreach purposes.
The increase in resolution for cosmological simulations has led to larger data volumes, with individual tables reaching sizes in the terabyte range. By exploring a new database technology, it is now possible to store and analyse snapshots from simulations with nearly 60 billion particles directly in the CosmoSim database.
The new database technology, the Spider engine for MariaDB/MySQL, allows to spread the data over many server nodes, and one head node, resulting in a distribution of the computational task over many server nodes. Several additional software components were developed by AIP’s E-Science team to enable transparent handling of parallel queries. Further, a job queue was implemented as a direct plugin for MariaDB/MySQL, so that even long running queries are permitted without stalling the servers.
CosmoSim utilizes the modern web framework Daiquiri, which was developed by the E-Science team during the recent years. It provides direct data access via an SQL query form, a database browser and SQL validation before sending the query to the server. An interface to Virtual Observatory tools like TopCat allows further quick investigations and processing of the retrieved results.
CosmoSim is based fully on open source technology. The modules developed by the E-Science team are available on GitHub at https://github.com/adrpar and https://github.com/jochenklar.
Demo movie:
http://www.cosmosim.org/cms/documentation/demos-and-tutorials/first-steps-with-cosmosim
The CosmoSim-Team:
- Kristin Riebe, kriebe@aip.de (data management and support)
- Jochen Klar, jklar@aip.de (web interface - backend and frontend)
- Harry Enke, henke@aip.de (management)
- Stefan Gottloeber, sgottloeber@aip.de (simulation data)
- Adrian Partl, apartl@aip.de (backend and database development)
Website of the CosmoSim database.
Credit: AIPThe CosmoSim database (www.cosmosim.org) has now been released after an intensive testing period. This service to the scientific community is the successor of the MultiDark database (www.multidark.org) and it is used as a platform for publishing and sharing data products from cosmological simulations.
It provides access to currently six cosmological simulations - including a highresolution resimulation of selected regions with hydrodynamics and star formation - which originate from different international projects in collaboration with the Leibniz Institute for Astrophysics Potsdam (AIP).
Outputs of cosmological simulations are typically stored at supercomputing centres with restricted access and encompass terabytes of data - too much to be downloaded by everyone. By providing the data via CosmoSim, scientists from all over the world are able to access the data, filter or combine the results directly on the server and use them for their own research.
The available data products include catalogues of dark matter halos, their inner properties, merging histories, information about the cosmic web and for selected snapshots even the raw particle distributions allowing for much deeper studies of dark matter halos and their environment. All the simulations and database tables are made available through a modern web interface. Additional features include an extensive documentation, along with some background on the database structure and selected images and movies for outreach purposes.
The increase in resolution for cosmological simulations has led to larger data volumes, with individual tables reaching sizes in the terabyte range. By exploring a new database technology, it is now possible to store and analyse snapshots from simulations with nearly 60 billion particles directly in the CosmoSim database.
The new database technology, the Spider engine for MariaDB/MySQL, allows to spread the data over many server nodes, and one head node, resulting in a distribution of the computational task over many server nodes. Several additional software components were developed by AIP’s E-Science team to enable transparent handling of parallel queries. Further, a job queue was implemented as a direct plugin for MariaDB/MySQL, so that even long running queries are permitted without stalling the servers.
CosmoSim utilizes the modern web framework Daiquiri, which was developed by the E-Science team during the recent years. It provides direct data access via an SQL query form, a database browser and SQL validation before sending the query to the server. An interface to Virtual Observatory tools like TopCat allows further quick investigations and processing of the retrieved results.
CosmoSim is based fully on open source technology. The modules developed by the E-Science team are available on GitHub at https://github.com/adrpar and https://github.com/jochenklar.
Demo movie:
http://www.cosmosim.org/cms/documentation/demos-and-tutorials/first-steps-with-cosmosim
The CosmoSim-Team:
- Kristin Riebe, kriebe@aip.de (data management and support)
- Jochen Klar, jklar@aip.de (web interface - backend and frontend)
- Harry Enke, henke@aip.de (management)
- Stefan Gottloeber, sgottloeber@aip.de (simulation data)
- Adrian Partl, apartl@aip.de (backend and database development)
Images
Website of the CosmoSim database.
