NorduGrid Toolkit Installation - HOWTO

NorduGrid Support < nordugrid-support@nbi.dk>

2002-10-19 09:32


This document provides information on how to install and configure the NorduGrid Toolkit.

1. Introduction

2. Quick start

Log in as root on your Red Hat 7.2 system. Make sure that you have enough space on the /opt partition. Download and install the latest version of the following RPMs:

found under: ftp://ftp.nordugrid.org/pub/nordugrid/software/.

You can use wget to retrieve the files from the command line and then use rpm -Uvh *.rpm to install the RPMs. Alernatively you can use rpm -Uvh ftp://ftp.nordugrid.org/pub... to use RPMs download feature.

Add a dedicated queue to your PBS system for Grid users. Call it grid-queue. Add a generic account (eg. griduser) on your system.

3. Overview

4. Getting, installling and configuring the software

4.1 Globus

Configuring Globus

Here is a small example of a Globus MDS configuration of a NorduGrid GRIS with a registration to a country level GIIS.


$ cat /etc/globus.conf
[common]
globus_flavor_name=gcc32dbgpthr

[mds/gris/provider/nordugrid]

[mds/gris/registration/MyCountry]
reghn=giis.my.country
regperiod=30      # register every 30 seconds
$

4.2 NorduGrid

Preparations

Before installing any Grid related software some preparations must be done. First is the system going to be a compute cluster or a storage element. If is a storage element (SE) the only thing needed is a UNIX machine with storage capacity. The SE setup is described later.

LRMS

For a compute cluster we need to be some kind of local resource management system. At the moment PBS (either PBSPro or OpenPBS ) with or without the Maui scheduler is preferred.

Grid accounts

On the cluster you need to create those unix accounts which will be used by the Grid. If you only allow the local users who already have accounts you don't need to do anything. If you allow GRID users the simplest scenario is to create a single account (eg. 'griduser'), or you can have separate accounts for the different grid user groups. It is recommended to put all the grid accounts into the same unix group.

Directories

Create the directories which will be used by the Grid services, some of them have to be available on the nodes (below indica ted as 'NFS'), example locations are indicated too. It is recommended to put these directory onto separate disks.

PBS configuration configuration

Follow the document ... We recommend not to use routing queues.

a

4, Download & install the Globus packages from the nordugrid website (we recommend to use the nordugrid-distributed Globus since it contains some critical fixes,... and .... However you can install the nordugrid software on top of your existing Globus installation too. In the later case you need to get the nordugrid source and recompile the toolkit against your Globus)

Get the following RPMS and install them in this order: gpt, globus, globus-config

9, Copy the globus.conf_template to /etc/globus.conf, modify it to fit your system. With globus.conf you basically set up your core MDS services (configure the openldap servers and the registration processes for the NorduGrid MDS tree)

10, if You want to run a GIIS then you may edit the so called policy files which control which GRISes can register to your GIIS .

10, Copy the nordugrid.conf_template to /etc/nordugrid.conf, modify it to fit your system.

11, Start the grid-services on your system: /etc/init.d/globus-mds start (this starts the MDS, and the registrations to GIISes) /etc/init.d/nordugrid start (this starts the gridftp-server & the grid-manager daemon)

12, Test your system (we need to work on it, put together a couple of basic test utility which must come with the server.rpm!)

Download

For a minimal server setup the several RPMs are needed. The main components are Globus, NorduGrid and Certificate Authorities. The RPMs can be fetched through the Downloads section from the homepage or directly from the ftp server.

Globus

Globus installation consists of only 3 RPMs. Installing these RPMs will results in a full Globus installation. We have opted the solution to place all the basic binaries and libraries in 1 RPM. The gpt RPM contains the Grid Packaging Toolkit, and globus-config contains the alternate Globus configuration used in NorduGrid and European DataGrid.

NorduGrid

The core NorduGrid installation consists of 2 RPMs:

So for a server- or client-only installation, only one NorduGrid RPM is needed. However we do recommend that you install the nordugrid-client on the server for testing purposes.

Certificate Authorities / authentication policy

In this section there are several RPMs. They come in two types. For the NorduGrid Certificate Authority (CA) they are:

The first type determine the authentication policy. That is, it allows to authenticate certificates issued by the NorduGrid Certificate Authority. This does not mean that people using NorduGrid will be accepted to access your site! It only means that you can establish users and servers which uses certificates issued by the respective Certificate Authorites. Install all the ca_* RPMs are thus relatively harmless and generally a good idea. Note that a cron job installed as part of the nordugrid-server RPM (/etc/cron.d/grid-update-crls.cron) will automatically download the respective Certificate Relocation Lists (CRLs) from the CA's. The CRLs contain a list of revocate (invalid) certificates issued by a CA.

The Globus installations comes with a tool called grid-cert-request that can generate a certificate request. So if your users or you need to request user or service certificates from say NorduGrid you need to install the ca_NorduGrid-local RPM. You can install multiple ca_*-local RPMs if you need be able to generate requests to different CA's. The script: /etc/grid-security/ca-set-default is used to set the default request CA using the hash of the CA as argument. For the NorduGrid CA this is done by:


/etc/grid-security/ca-set-default 1f0e8352

Note that the last installed ca_*-local RPM will determine the default request CA.

Using grid-cert-request you can use the -ca option to select the request CA between those available on the system.

For your NorduGrid server installation you need at least a host certificate, but we also recomment an ldap certificate. Run


grid-cert-request  -help 

To get information about how to get the various types of certificate requests.

Once a request has been made instructions on how to get the request signed by the CA will be printed on the screen.

Once the request has been send and a signed certificate returned by the CA they certificate and key needs to be placed in the proper place with the correct permissions and ownership. The ownership is for the service certificates is that of the user running the daemons (normally root) and user certificates should be owned by the user. The certificate is public and can be readable by anyone while the key must only be readable by the owner. Note that execute permissions seems to be disallowed as well.

For the host certificate and key:


-rw-r--r-- /etc/grid-security/hostcert.pem
-r-------- /etc/grid-security/hostkey.pem 

For the ldap certificate and key:


-rw-r--r-- /etc/grid-security/ldap/ldapkey.pem
-r-------- /etc/grid-security/ldap/ldapcert.pem

User certificates are normally located in:


-rw-r--r-- $HOME/.globus/usercert.pem
-r-------- $HOME/.globus/userkey.pem

Authorization policy

This deals we which users, services or groups (Virtual Organisations (VO's)) of such entities are allowed to access your site. In the current scheme this authorization is done by mapping allowed Grid users to local UNIX users. This is done in the /etc/grid-security/grid-mapfile. The basic format of this file is:


...
"Distinguised name of grid user" local_unix_username
...

This file is normally more or less identical to that of other sites collaborating on the same projects. However manually maintaing this kind of distributed "password" like file can get quite tedious. There is therefore an automatic mechanism for doing this:

First the "/etc/grid-security/local-grid-mapfile" (which has the same format as the grid-mapfile) contains a list of users allowed by the local site only. Furthermore /etc/grid-security/nordugridmap.conf needs to be modified to list which VO user databases the site want to subscribe to. For further info on authorization read the VO-document. The grid-mapfile is in this case updated automatically several times a day using cron.

Configuring

As with the Globus configuration the NorduGrid configuration is quite centralized so most of the configuration is handled through the single /etc/nordugrid.conf file. The file is use the same basic format as globus.conf. The available main sections are:


[common]

[cluster]

[grid-manager]

[queue]

[grid-ftp]



$ cat /etc/nordugrid.conf
[common]
globustime_command="/opt/nordugrid/bin/globus-generalized-time"
hostname_command="/bin/hostname"
pbs_bin_path="/usr/bin"
gridmap="/etc/grid-security/grid-mapfile"
gridarea="/shared/grid"
controldir="/var/spool/nordugrid/jobstatus"
runtimedir="/shared/runtime"

[cluster]
cluster_alias="My Cluster"
lrmstype="OpenPBS"
lrmsversion="2.3.16"
lrmsconfig="single job per processors"
homogeneity="True"
gm_port=2811
gm_mount_point=/jobs
nodecpu="AMD Athlon(tm) Processor 700 MHz"
nodememory="128"
totalcpu="45"
cpudistribution="1cpu:10,2cpu:8"
clustersupport="grid-support@my.email.address"
middleware="nordugrid-0.3.4"
middleware="globus-2.0"
cachetime=30

[queue/short]
queue_name="gridshort"
scheduling_policy="strict FIFO"

[queue/long]
queue_name="gridlong"
scheduling_policy="strict FIFO"
$

Enabling the software

4.3 PBS configuration

Appendix

5. Configuration files

5.1 globus.conf

5.2 nordugrid.conf reference