The latest major production release of ARC is the "v0.8.x". The release cycle contains alpha, beta and rc which are available from the nordugrid testing repositories. After the rc, a final release is made available from the nordugrid stable repository.
Source and binary packages for Debian/Ubuntu and RPM-based systems are also be provided.
The candidate release bundles are available from the test area of http://download.nordugrid.org:
RPMs and debs for ARC 0.8.x are available at:
The "0.8.x" is the latest major production release of ARC after the 0.6.x releases which started in May 2007. In addition to the numerous improvements, the "0.8.x" represents technology change in two areas (see draft release notes for details):
The installation instructions were tested on two specific platforms: Ubuntu and CentOS. We believe the instructions are general enough to be followed on other platforms as well. The main installation steps are the usual ones:
APT sources (the dot at the end of the lines below is part of the configuration):
sudo apt-key add DEB-GPG-KEY-nordugrid.asc
You can then install the arc packages using the way you prefer: apt-get, synaptic, or ...Yum repositories:
[nordugrid] name=NorduGrid - $basearch - Stable baseurl=ftp://ftp.nordugrid.org/repos/redhat/el4/$releasever/$basearch/stable enabled=1 gpgcheck=1 gpgkey=http://download.nordugrid.org/RPM-GPG-KEY-nordugrid
[nordugrid] name=NorduGrid - $basearch - Stable baseurl=ftp://ftp.nordugrid.org/repos/redhat/el5/$releasever/$basearch/stable enabled=1 gpgcheck=1 gpgkey=http://download.nordugrid.org/RPM-GPG-KEY-nordugrid
[nordugrid] name=NorduGrid - $basearch - Stable baseurl=ftp://ftp.nordugrid.org/repos/fedora/$releasever/$basearch/stable enabled=1 gpgcheck=1 gpgkey=http://download.nordugrid.org/RPM-GPG-KEY-nordugrid
After creating a file in /etc/yum.repos.d containing the repository definition you can install the packages the way you prefer: yum, package-kit, ...
In order to build and deploy the ARC v0.8.x you will need to satisfy some build and runtime dependencies. Most of the build dependencies come from the compiler-toolchain and common standard libraries. In addition to that, you also need some Grid-specific libraries during both compilation and runtime.
ARC external dependencies are grouped into two categories:
These packages are needed:
Build for ARC-corePlease note that depending on operating system distribution in order to build A-REX you may need to install development versions of mentioned packages.
Runtime for ARC coreUbuntu and Debian, install with:
aptitude install build-essential automake1.9 libtool gettext python-dev libxml2-dev libldap2-dev \ libexpat1-dev doxygen libmysqlclient15-dev alien flex bison uuid-dev swig \ gsoap wget libssl-dev pkg-config libglib2.0-dev libglibmm-2.4-dev libxml2-dev e2fsprogs libxml-dom-perl \ libcrypt-ssleay-perl libnet-ldap-perl slapd
CentOS/Fedora/RedHat, install with:
yum install make automake autoconf libtool gettext python-devel libxml2-devel openldap-devel \ expat-devel doxygen mysql-devel \ flex bison e2fsprogs-devel swig gsoap wget openssl-devel pkgconfig \ glibmm24-devel libxml2-devel openldap
For generic Linux distribution and for non-Linux OSes packages may be found at their respective websites:
The Grid-specific software is identical for all OS distributions and it includes:
Build and Runtime for ARC-corePlease note that depending on operating system distribution in order to build A-REX you may need to install development versions of mentioned packages.
All of these can be found in the NorduGrid repositories. Alternatively, you can build them yourself.
ARC v0.8.x can be built on top of any decent Globus distribution, nevertheless we recommend to use the NorduGrid provided Globus packages. NorduGrid offers a patched and selected set of Globus packages.
The migration to the new native LDAP-based information system resulted in a much shorter Globus dependency list for the ARC 0.8.x. There is no need to install Globus-LDAP and Globus-MDS packages any longer. The actual list of packages is given below.
The following Globus packages (and their prerequisities) needs to be installed in order to build and deploy ARC v0.8.x including the A-REX component as well. We note that the A-REX subpackage has a smaller list of Globus dependencies:
If you choose to build own Globus Toolkit you can find it and GPT at http://www.globus.org/toolkit/ . You will have to build the modules specified above. ARC requires threaded flavor of Globus modules (for what flavor means read Globus documentation).
An improved BDII package, the "bdii-ng" is provided by Nordugrid and available from the nordugrid download area and via the repositories. The bdii-ng package has been tested with different OpenLDAP versions ranging from 2.2.13 to 2.4.15 on both 32 and 64bit. Other versions may work as well.
NOTE: If you use a different bdii than bdii-ng, for example the gLite supplied bdii v4. Then you will need to specify where the bdii script and the bdii-update script are. This is done in arc.conf by setting bdii_cmd and bdii_update_cmd. If you do not know whether you need to do this, it will be indicated in the log files when you try to start the information system.
For example:
Instructions how to migrate from ARC 0.6 LDAP to this new information system are also available in this document.
Install BDII
From here on, bdii_etc will on bdii packages refer to /opt/bdii/etc and on bdii-ng packages refter to /etc/bdii4
After installation you may, for optimization reasons adjust BDII (ldap) logging by selecting one of the two options below depending on what version of BDB you have:
An untuned example of a DB_CONFIG file can be found here. Such file after your modifications should go into bdii_etc.
NOTE: If you try to set for example DB_TXN_NOT_DURABLE on a BDB version that does not support it, the nordugrid infosys will not work.
Additional info about the original glite BDII can be found at the BDII Homepage.
All of the required Grid-specific software can be found in the NorduGrid repositories. Below we give repository info.
The easiest way to install the Grid-specific software is by correctly configuring the NorduGrid repositories and using yum utility with syntax like
yum install *globus*
for e.g. installing all the Globus packages.
Note: You can also use RPMs directly. They can be downloaded from ftp://ftp.nordugrid.org.
For Debian/Ubuntu there is no official repository on the NorduGrid servers. For distributions Squeeze and later the Globus packages are in the regular Debian repository. For Ubuntu the packages are in since Karmic.
For older Ubuntu/Debian distributions there is however a repository held by Mattias Ellert with these packages:
Debian (set etch or lenny as appropriate):
Ubuntu (change dapper to gutsy, hardy, intrepid or jaunty as appropriate):
Some environment variables used frequently while building external software from source:
Couple of guides describing building external software are available at NorduGrid Wiki, some of these are fairly Ubuntu-centric. Should you have any problems, please contact (daniel@ndgf.org).
Before trying to build and install the nordugrid-arc subpackage please make sure you have all the required dependencies installed.
We use rpath for hardcoding the location of libraries into ARC - beware that this implies that one cannot relocate the external software that ARC depends on (the adavantage is that we avoid problems with setting $LD_LIBRARY_PATH correctly). The use of $LD_LIBRARY_PATH here is only needed for building ARC (the build process invokes gsoap2cpp).
Note that on Ubuntu 6.06 or up to and including hardy on 64 bit (because gsoap is not compiled with -fPIC) you need to add -rpath=$GSOAP_LOCATION/lib to the CFLAGS line and also do export LD_LIBRARY_PATH=$GSOAP_LOCATION/lib.
export CFLAGS="-Wl,-rpath=$GLOBUS_LOCATION/lib,-rpath=$LFC_LOCATION/lib" export CXXFLAGS=$CFLAGS export C_INCLUDE_PATH= export CPLUS_INCLUDE_PATH=
Because of potential conflicts which may occur between core ARC and A-REX, make sure ARC_LOCATION and ARC_CONFIG environment variables are NOT set at global system level.
If you have gotten globus from some other place than nordugrid repos, you may have to set the following environment variables (note that GPT_LOCATION is usually same as GLOBUS_LOCATION):
export GLOBUS_LOCATION=path_to_globus_installation_directory export GPT_LOCATION=path_to_gpt_installation_directory export PATH=$PATH:$GLOBUS_LOCATION/sbin:$GLOBUS_LOCATION/bin:$GPT_LOCATION/sbin:$GPT_LOCATION/bin
Alternatively, instead of the last variable, one may specify paths needed by Globus/GPT utilities directly:
export GPT_FLAVOR_CONFIGURATION=$GPT_LOCATION/sbin/gpt-flavor-configuration export GPT_QUERY=$GPT_LOCATION/sbin/gpt-query export GLOBUS_MAKEFILE_HEADER=$GLOBUS_LOCATION/bin/globus-makefile-header
Depending on how Globus libraries were installed, following variable may also need to be set
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$GLOBUS_LOCATION/lib
If you have some external packages installed in an unusual location where pkg-config can't find them, add them to PKG_CONFIG_PATH variable. For example, if you have gSOAP installed in /opt/gsoap:
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/opt/gsoap/lib/pkgconfig
Location of some packages may be specified as configuration parameters. In the previous case that would be -with-gsoap-location=/opt/gsoap .
It was reported that on Ubuntu 6.06 and some 64 bit system one has to add --with-gsoap-location=$GSOAP_LOCATION --with-voms-location=$VOMS_LOCATION --with-lfc-location=$LFC_LOCATION to the configure line.
You then install ARC by going to the source directory and typing
./configure make make install
You may add some options to ./configure command as e.g. pathes described above. It is advisable to run ./configure --help first and check if there is something you would like to adjust. Inspect output produced by these commands and report any error found to developers using Bugzilla interface at http://bugzilla.nordugrid.org/ .
If everything finishes without errors, you will have ARC middleware installed. If you have not specified --prefix option, ARC will be in the default location - /opt/nordugrid. Installation procedure also sets up configuraation files in /etc location.
If you just want to install the binaries (for example to test it out as an ordinary user), then run:
make install-exec cd arclib/arc make install-arcincludeHEADERS
If You also want to have arclib-python bindings, then run these commands too
cd arclib make install-pythonPYTHON make install-pythonLTLIBRARIES
Hint: You may run make distclean && ./bootstrap before ./configure to make all generated code re-created. That may make it more suitable for your distribution.
One of the new components which is shipped with 0.8.x release is the A-REX, next generation of ARC job management service. Below we describe how to build and install A-REX on a generic Linux distribution. Note, the A-REX subpackage can be built and deployed either on top of a newly built ARC v0.8rcx (see above) or even on top of an existing ARC v0.6x deployment.
We recommend to unset the ARC_LOCATION variable before building the A-REX subpackage.
Before trying to build and install the nordugrid-arc-arex subpackage please make sure you have all the required dependencies installed.
Because of potential conflicts which may occur between core ARC and A-REX when using ARC_LOCATION environment variable it is advised to make sure it is not set as global variable.
Installation from source consists of the usual steps similar to those described in the core ARC section above:
./configure make make install
By default if --prefix option is not specified, A-REX installs into /usr/local location.
On some systems you may need to use gmake instead of make.
Hint: You may run make distclean && ./autogen.sh before ./configure to make all generated code re-created. That may make it more suitable for your distribution.
When migrating to A-REX from Grid Manager you will find these instructions helpful.
This section presents simple steps to evaluate successfulness of installation and setup of ARC 0.8.x by performing basic functionality tests.
Following steps are intended to check whether ARC works. We assume that you are running supported local resource management system and you have properly set up arc.conf file with all the certificates paths and other necessary information is present. If not, please start from the beginning of this document.
First start gridftpd server:
/etc/init.d/gridftpd start
and the the information system:
/etc/init.d/grid-infosys start
You can use either GridManager or
A-REX. Check Grid manager migration
notes for more information. Start one of them:
/usr/local/etc/init.d/a-rex start
for A-REX, or
/etc/init.d/grid-manager start
for Grid Manager.
If the services started without problems we can continue with further tests.
gridftpd
To check gridftpd you can use the ngls command line utility:
ngls gsiftp://<cluster_name>/jobs
should return list of actual jobs on the cluster.
Information system
You can use ldapsearch command line utility to check whether the information system actually works:
ldapsearch -h <server_name> -p 2135 -x -b 'mds-vo-name=local,o=grid'
A-REX and Grid Manager
We can use ps command line utility to check whether the service started properly. Type for instance:
ps aux | grep arex
or
ps aux | grep grid-manager
Nordugrid ARC comes with a simple test suite available via the command line tool ngtest. The ngtest command can test various things related to a client or server setup. To learn about all its options type man ngtest.
Test jobs
The test jobs available through ngtest are:
Examples
ngtest -job 1 -c <clustername>
will submit a test job number 1 to the specified cluster.
ngtest -certificate
will print certificate information about the user's certificate and installed CA-certificates.
During the deployment of 0.8.x you may came over some problems. Some of them may be already known and in such case should be collected in NorduGrid Wiki. If you can not find the problem there please let us know through our bug tracking system.
Previous versions of ARC (v0.6x and earlier) used an information system based upon a very old version of a patched LDAP server which was distributed as part of the Globus packages. The local information system used the "GRIS" backend of Globus-LDAP while the index services relied on the "GIIS" backend of Globus-LDAP. Both of these backends were developed by Globus.
With the 0.8.x release the LDAP-based information system of ARC got reimplemented in order to use a native LDAP server. For the local information server a solution based upon the BDII LDAP backend was developed while as an information index server a new shell-based LDAP backend was written.
As an extra feature, the new local information server can publish its content in GLUE 1.2 schema as well (in addition to the NorduGrid schema).
The migration to native LDAP server based solution affects only internal components of ARC, no changes in the interfaces were introduced.
The local information server of the ARC 0.8.x release have moved to using glite BDIIv4 and native system OpenLDAP instead of Globus MDS GRIS and the OpenLDAP provided by Globus. This affects you in the way that you need to install the gLite BDII package together with a distribution provided native LDAP server.
The gLite BDIIv4 consists of a set of Perl scripts that take care of populating the LDAP database with the information that will be available in the local LDAP information server. This is done by giving it a set of providers that will mine data from the system. The BDIIv4 runs two LDAP servers consecutively and while one LDAP server is populated, the other one is serving requests. Then when the new data has been entered, BDII uses its port-forward service to let the LDAP server with the new data service requests while the old one is repopulated with data.
To deploy the 0.8.x ARC release you must fetch and install a BDIIv4 package. NorduGrid provides its own version of the BDIIv4 package, the bdii-ng. The nordgurid version comes with several improvements over the original gLite BDII We recommend to use the nordugrid version of BDII. Source and binary packages of the nordugrid BDII are available from the usual nordugrid download area and via the nordugrid repositories (the nordugrid BDII package in the download area: http://download.nordugrid.org/software/bdii/)
With the 0.8.x release there is no need for Globus MDS and Globus-LDAP packages any longer!
However, there are two minor bits from Globus MDS that have been kept and integrated into the ARC codebase (no need to install any Globus packages for them):
In order to continue widespread support for different platforms, some non-portable LDAP optimizations has been turned off. If you find performance issues (very unlikely), you might want to try turning on some optimizations:
This requires your OpenLDAP to be newer than 2.3.x (the version in RHEL5 is known to not work with this setting.)
If you want to optimize OpenLDAP more, you can do that by updating the BDII.conf, if you do however, you need to know what BDB backend you use since the information system will not start if you enable the wrong thing.
Since we now use the native system LDAP, we need to be wary of SELinux on for example CentOS and Apparmor on for example Ubuntu. The first thing that you need to do if you have any problems with LDAP server not starting is to check your system logs in /var/log if this is your problem. You need to allow the OpenLDAP server to access /opt/bdii, /var/run/nordugrid, /var/run/bdii, /var/tmp/bdii and it needs to have access to ports 2135, 2136 and 2137.
The easiest solution for Apparmor is to remove /etc/apparmor.d/usr.sbin.slapd
The existing arc.conf file (with respect to information system components) is fully compatible with the new system. An arc.conf from a 0.6.x installation should work out-of-the-box in a 0.8.x deployment as well.
However, a couple of new configuration parameters have been introduced as part of the LDAP migration. These parameters configure the BDII LDAP backend. All of these parameters come with decent defaults, normally there is no need to change them.
The new local infoserver comes with a new feature: it can publish its content through GLUE 1.2 schema as well in addition to the NorduGrid schema. This feature is optional and can be turned on by setting the following variables in arc.conf:
If you do not set them then you will get the default behavior which is identical to 0.6, i.e. the same as nordugrid=enable and glue=disable. If GLUE is turned on, it will be added under "mds-vo-name=glue12,o=grid"
These variables are necessary if you enable glue output. They should be under the [infosys/glue12] section(adjust to your geographical location):
The BDII LDAP backend can be fully configured from arc.conf; this however should not be necessary. More information can be found in the arc.conf documentation (http://svn.nordugrid.org/trac/nordugrid/browser/arc0/branches/v_0_8/doc/arc.conf.template).
There is no change in the way how information index services and registrations are configured.
Important: The new native-LDAP based infosystem is a complete replacement of the old/previous system. We don't recommend the two systems being deployed simultaneously on the same box.
In order to migrate from a Globus-MDS based 0.6 ARC system to a native LDAP-based 0.8.x ARC system the following steps are necessary:
The above instructions apply to both ordinary resources (e.g. clusters) and to Information Indexing services.
ARC 0.8.x includes the A-REX component, the next generation ARC job management service (see technical documentation). This component is an optional replacement of the previous Grid Manager component. It is possible to have both the Grid Manager and the A-REX deployed in the same box and by a simple service start/stop change between the two at any time. The A-REX is configured to share the same gridftp-based job submission interface the Grid Manager is using as well.
In order to be able to turn on the A-REX component, we assume that you have already successfully built and installed all the A-REX prerequisities and the full ARC 0.8.x bundle including the A-REX itself.
When replacing the Grid Manager with A-REX no additional configuration is necessary. A-REX understands the exact same configuration as the Grid Manager, it takes all the configuration parameters from the same arc.conf file used also by the Grid Manager.
If you install any of the Grid Manager or A-REX in an unusual location, make sure pluginpath="/opt/nordugrid/lib" in arc.conf points to the actual location of plugins. When unsure, simply remove this line - a correct path will be picked up automatically.
Any computing element utilising the pre-configured A-REX as distributed with the v0.8.x release is fully compatible with any other ARC service and client. This is because the WS-inteface of A-REX is turned off and A-REX is configured to use the gridftp-based job submission interface of the Grid Manager.
The A-REX service, as it is distributed with the v0.8, offers an out-of-the-box full replacement for the Grid Manager component. You can decide which one to run and it is possible to swap between them any time. It is important that you run only one of them at the same time: either A-REX or the Grid Manager.
To turn on A-REX first make sure that the Grid Manager is not running:
Stop the Grid Manager:
Start A-REX
Stop A-REX (e.g. if needed for maintenance)
A-REX is a very powerful component, a next generation job execution service. For example, it provides a standard-compliant Web Service interface to handle job submission/management. The WS interface of A-REX is however disabled in the v0.8.x distribution. If you are interested to experiment with A-REX advanced features, a configuration file (arex_service-secure.xml.example located in $ARC_LOCATION/share/doc/arc) is provided which should be loaded into A-REX through the arched daemon (see the technical documentation for details):