Deployment
Creating a configuration file
HEPnOS relies on the Bedrock Mochi microservice for bootstrapping and configuration.
The first step before deploying HEPnOS is to create a configuration file. This configuration file should be in JSON format and at least contain the following.
{
"ssg" : [
{
"name" : "hepnos",
"bootstrap" : "mpi",
"group_file" : "hepnos.ssg",
"swim" : { "disabled" : true }
}
],
"libraries" : {
"yokan" : "libyokan-bedrock-module.so"
},
"providers" : [
{
"name" : "hepnos",
"type" : "yokan",
"config" : {
"databases" : [
{
"name" : "hepnos-datasets",
"type" : "map",
"config": {}
},
{
"name" : "hepnos-runs",
"type" : "map",
"config": {}
},
{
"name" : "hepnos-subruns",
"type" : "map",
"config": {}
},
{
"name" : "hepnos-events",
"type" : "map",
"config": {}
},
{
"name" : "hepnos-products",
"type" : "map",
"config": {}
}
]
}
}
]
}
This example configuration file only provides the bare minimum to get started. The “ssg” section sets up the group management component. The only important field here is the name of the group file, which we will use later.
The “providers” section should contain at least one Yokan provider with a number of databases. These databases must have a name that starts with “hepnos-datasets”, “hepnos-runs”, “hepnos-subruns”, “hepnos-events”, or “hepnos-products”. At least one database for each type of data should be provided, but you are free to use more than one database for some types of data, as long as their name starts with the above prefixes. For example, you can have two databases to store events, named “hepnos-events-1” and “hepnos-events-2”.
Configuring with the HEPnOS Wizard
An easy way of creating a HEPnOS configuration for Bedrock is to use the HEPnOS Wizard, which can be installed as follows.
$ spack install py-hepnos-wizard
Once installed and loaded, you can use it as follows.
$ hepnos-gen-config --address na+sm --output=myconfig.json
The only required parameter is the address, which should be a valid protocol accepted by the underlying Mercury library (e.g. na+sm, ofi+tcp, and so on).
Passing –help to hepnos-gen-config will provide information on all the available arguments and their meaning.
Deploying HEPnOS on a single node
To deploy HEPnOS on a single node, simply ssh into the node and type the following.
bedrock na+sm -c config.json
Change na+sm into the protocol that you want to use for communication. This tells Bedrock to start and initialize itself with the provided configuration. The command will block. To run it as a daemon, put it in the background, use nohup, or another other mechanism available on your platform.
Deploying HEPnOS on multiple nodes
The bedrock program can just as simply be deployed on multiple nodes, using your favorite MPI laucher (mpirun, aprun, etc.), for instance:
mpirun -np 4 -f hostfile bedrock na+sm -c config.json
Getting connection information
Once deployed, run the following command to obtain connection information readable by the client.
hepnos-list-databases na+sm -s ssg-file > connection.json
Where ssg-file is the name of the SSG file as specified in your HEPnOS configuration file.
This command will query the service and print a JSON representation of the information required for a client to connect to HEPnOS (addresses, database ids, etc.). Hence we pipe its output to a connection.json file that the clients will use later.
Important
On some platforms, you will need to launch this command as an MPI application running on a single process/node (typically if your login node does not connect to the compute nodes via the same type of network as across compute nodes).