NORDIC TESTBED FOR WIDE AREA COMPUTING AND DATA HANDLING 
2/24/03 
EXTENDED RESOURCE SPECIFICATION LANGUAGE 
Abstract 
The paper represents an overview and a reference manual for the NorduGrid Extended RSL, developed on the basis of the Globus RSL. 
CONTENTS 
1.Introduction.......................................................................................................................................4

2.General remarks ................................................................................................................................4

3.Supported attributes..........................................................................................................................5
3.1.User-side attributes...................................................................................................................5
executable..................................................................................................................................5
arguments..................................................................................................................................5
inputFiles..................................................................................................................................6
executables................................................................................................................................6
outputFiles.................................................................................................................................6
jobName....................................................................................................................................7
stdin...........................................................................................................................................7
stdout.........................................................................................................................................7
stderr.........................................................................................................................................8
stdlog.........................................................................................................................................8
join............................................................................................................................................8
notify.........................................................................................................................................8
cluster........................................................................................................................................9
queue.........................................................................................................................................9
startTime...................................................................................................................................9
CpuTime...................................................................................................................................9
lifeTime.....................................................................................................................................9
memory...................................................................................................................................10
disk..........................................................................................................................................10
runTimeEnvironment .............................................................................................................10
middleware..............................................................................................................................11
replicaCollection.....................................................................................................................11
rerun........................................................................................................................................11
architecture.............................................................................................................................11
dryRun.....................................................................................................................................12
rsl_substitution........................................................................................................................12
environment............................................................................................................................12
ftpThreads...............................................................................................................................12
jobtype.....................................................................................................................................13
count........................................................................................................................................13
parallelenvironment.................................................................................................................13

3.2.GM internal attributes............................................................................................................14
sstdin.......................................................................................................................................14
action.......................................................................................................................................14
savestate..................................................................................................................................14
lrmstype...................................................................................................................................14
hostName.................................................................................................................................14
jobid........................................................................................................................................15

4.Unsupported Globus attributes........................................................................................................15

5.References.......................................................................................................................................16

Appendix A. Example of a user-side RSL script................................................................................17

Appendix B. Example of a GM input RSL (job submission).............................................................18

1. INTRODUCTION 
Execution of jobs at batch clusters is the major part of the data analysis, production and other computational tasks and applications in many research areas. In the Grid infrastructure[4], such clusters are not necessarily local to a job submission site, but can be widely distributed. This implies extra requirements for the proper description of job options. The Globus project[2] developed a Grid middleware toolkit, which makes use of the Resource Specification Language (RSL)[3] to parse job options and definitions to resource management systems. 
The NorduGrid project[1] developed an architectural solution[5] for a Grid testbed, suitable for the High Energy Physics tasks. This solution requires certain extensions to the RSL. This concerns not only introduction of new attributes, but also differentiation between two levels of the job options specifications: 
&#8226;&#9;User-side RSL, i.e., the set of attributes specified by a user in a job-specific file. This file is interpreted by the User Interface (UI)[6], and after the necessary modifications is parsed to the Grid-manager (GM)[7] 
&#8226;&#9;GM-side RSL, i.e., the set of attributes pre-processed by the UI, and ready to be interpreted by the GM 
In what follows, the description of the NorduGrid-extended RSL is given, using the following notations: 
&#60;xxxx&#62; parameter to be substituted with a corresponding string or a number
[&#60;xxxx&#62;] optional parameter
xxx|yyy|zzz list of possible values of a parameter
-&#34;-&#34;same as above&#34;

2. GENERAL REMARKS 
For a complete description of Globus RSL, see reference [3]. xRSL uses the same syntax conventions, although changes the meaning and interpretation of some attributes. 
xRSL attributes are normally parsed via a so-called xRSL-file (suggested extension .xrsl). Such a file contains a plain list of attribute strings and boolean operands &#8220;&#38;&#8221; (for AND) and &#8220;|&#8221; (for OR). Typically, an xRSL job description starts with an ampersand, to indicate implicit conjunction of all the attributes (see Appendices). Whenever a disjunct-request of two or more attributes is needed, the following construction can be used: (|(attribute=value1)(attribute=value2)...). 
In expressions, operands &#8220;=&#8221;, &#8220;!=&#8221;, &#8220;&#62;&#8221;, &#8220;&#60;&#8221;, &#8220;&#62;=&#8221; and &#8220;&#60;=&#8221; are allowed. 
Commented lines should start with &#8220;(*&#8221; and be closed with &#8220;*)&#8221; (see Appendices). 
Multiple job description in one file is realized via a standard Globus RSL multi-request operand &#8220;+&#8221;, which should precede additional job desrcription, e.g.: (&#38;(...))+(&#38;(...)) 
URL strings mentioned in this document are ordinary URL for a protocol (GsiFTP, HTTP,HTTPs or FTP), which can be extended in certain special cases: 
&#8226;&#9;Replica Catalog URL: URL of type rc://[location[-location[...]]][@rc_host/rc_distinguished_name]/lfn 
where 
&#8226;&#9;location has a form host[:port] and specifies a physical file destination. If none or more than one location is specified, a choice is made by the Grid Manager. 
&#8226; rc_host has a form of host[:port] and describes a ReplicaCatalog server. 
&#8226;&#9;rc_distinguished_name has an LDAP DN form and is used to specify the logical collection of the Replica Catalog. 
&#8226;&#9;lfn is a &#8220;logical&#8221; file name, containing a physical path at the destination, with respect to the mount point. 
&#8226; Extended URL: an URL containing additional options, added after host[:port] parts and 
separated by a semicolumn &#8220;;&#8221;. In case of the 'Replica Catalog URL', options are located after each location.The options are: 
&#8226; thread=&#60;integer&#62; 
&#8226; cache=&#60;yes|no&#62; Examples of URLs are: ftp://grid.domain.org/dir/script.sh gsiftp://grid.domain.org:2811;thread=10/dir/input 12378.dat ldap://grid.domain.org:389/lc=collection1,rc=Nordugrid,dc=nordugrid,dc=org rc://@grid.domain.org/lc=collection1,rc=Nordugrid,dc=nordugrid,dc=org/zebra/f1.zebra 
3. SUPPORTED ATTRIBUTES 
Most of the attributes introduced by Globus are supported, some with modifications as indicated. Several new attributes are introduced, of which some are to be specified in the user's script, and others are internal for the GM (are specified by the UI) 
3.1. USER-SIDE ATTRIBUTES 
The following attributes can be specified in a user's RSL script. Some are to be modified by the UI before being passed to the GM. 
The executable to be submitted to LRMS 
string -file name (including path), local to the computing element (CE). 
If an executable has to be transferred from the submission node, it has to be specified in the inputFiles list, otherwise it will be added to that list by the UI. 
If the file name starts with leading slash (&#8220;/&#8221;), it is considered to be the full path to the executable at a CE; otherwise the location of the file is relative to the session directory (where job input and files are stored). If the file name starts with an environment variable (&#8220;$...&#8221;), the value of this variable is resolved at the remote computing element. 
For internal GM use, the UI creates a dummy executable attribute with, e.g., &#34;/bin/echo&#34; as an argument (unless the specified executable itself starts with &#8220;/&#8221;). The actual executable is transferred by the UI to lists in the arguments, inputFiles and executables internal GM attributes. 
List of the arguments for the executable string -an argument 
executable -the executable to be run by LRMS, taken by the UI from the user-specified executable attribute 
List of files to be copied to the computing element before the execution 
filename&#9;-file name, local to the computing element and always relative to the session directory. 
location -location of the file (gsiftp, https, ftp, http URLs, or a path, local to the submission node). If void (&#8220;&#8221;), the input file is taken from the submssion directory 
URL -URL of the file (gsiftp, https, ftp, or http protocol) 
size -file size in bytes 
checksum -file checksum (as returned by cksum) 
If the list does not contain the standard input file (as specified by stdin) and/or the executable file (as specified by executable if given name is relative to the session directory), the UI appends these files to the list. If the &#60;location&#62; is a URL, it is passed by the UI to the GM without changes; if it is a local path (or void, &#8220;&#8221;), the UI converts it to &#60;size&#62; and &#60;checksum&#62; and uploads those files. 
List of files from the inputFiles set, which will be given executable permissions 
string -file name, local to the computing element and relative to the session directory. 
If the executable file (as specified in executable and relative to the session directory) is not listed, it will be added to the list by the UI. 
string -file name, local to the computing element (CE) 
URL -URL of the remote file (gsiftp, https, ftp, http, or a Replica Catalog (RC) pseudo-URL); if void (&#8220;&#8221;), the file is kept for manual retrieval. 
RC pseudo-URL has the form: 
rc://[location[|location[...]]][@ldap_host[:ldap_port]/replica_collection_DN]/logical_file_name Here ldap_host:ldap_port is an internet address of the LDAP server serving RC; location is logical location as defined in the RC (usually this is a host name of a Storage Element). Location should be already defined in the RC. If few locations are specified, only those found in the RC will be used. GM stores output files in one location only. If the first one in the list fails, it tries next. If no locations are specified, all found in the RC are used. Attribute logical_file_name is the name given to the file in the Replica Catalog. If ldap_host[:ldap_port]/replica_collection_DN is not specified, the one given in the replicaCollection attribute is used. 
If the list does not contain the standard output and/or standard error files (as specified by stdout/stderr), the UI appends these file names to the list. If the &#60;URL&#62; is not specified (void, &#8220;&#8221;), files are not considered uploadable by the GM and should be downloaded by the UI. 
User-specified job name 
string -job name 
This name is meant for convenience of the user. It can be used to select the job while using the UI. It is also available through the Information System. 
The standard input file. 
string -file name, local to the computing element. 
The standard input file should be listed in the inputFiles attribute; otherwise it will be forced to that list by the UI 
The standard output file. 
string -file name, local to the computing element and relative to the session directory. 
The standard output file should be listed in the outputFiles attribute; otherwise it will be forced to that list by the UI. If the standard output is not defined, UI assigns a name. 
The standard error file. 
string -file name, local to the computing element and relative to the session directory. 
The standard error file should be listed as an outputFiles attribute; otherwise it will be forced to that list by the UI. If the standard error is not defined, UI assigns a name. 
The job log file, containing all the job-related messages from the GM. 
string -file name, local to the computing element and relative to the session directory. 
The job log file should be listed as an outputFiles attribute; otherwise it will be forced to that list by the UI. 
If &#34;yes&#34;, joins stderr and stdout files into the stdout one 
Request e-mail notifications on job status change 
string - string of the format: &#34;[b][q][f][e][c] user1@domain1 [user2@domain2] ...&#34; b begin (PREPARING) q queued (INLRMS) f finalizing (FINISHING) e end (FINISHED) c cancellation (CANCELLED) 
No more than 3 e-mail addresses per status change accepted 
The name of the execution cluster 
string - known cluster name, or a substring of it 
Disjunct-requests of the kind (|(cluster=clus1)(cluster=clus2)) are supported. It is possible to exclude a cluster by requesting (cluster != clus3). 
The name of the remote batch queue string - known queue name 
Time to start job processing time -time string, YYYY-MM-DD hh:mm:ss tttt -time string, YYYYMMDDhhmmss[Z] (converted by the UI from time) 
Maximal CPU time required for the job time -time, minutes for PBS 
Maximal time to keep job files (the session directory) on the gatekeeper 
tttt -time, seconds (converted by the UI from time) May be overwritten by the local settings 
Memory required for the job integer - size (Mbytes) 
Disk space required for the job integer -disk space ,Mbytes Required runtime environment 
string -environment name. The site to submit the job to will be chosen by the UI among those advertising specified runtime environments. Before starting the job, the GM will set up environment according to those requested. 
To request several environments, repeat the attribute string: (runTimeEnvironment=ENV1)(runTimeEnvironment=ENV2) etc. Before being submitted to the GM, a grouping is done by the UI, e.g., (runTimeEnvironment=ENV1 ENV2). 
To make a disjunct-request, use a boolean expression: (|(runTimeEnvironment=env1)(runTimeEnvironment=env2)) 
Environment string interpretation is case-insensitive. Substring specification is possible: in such a case, a partial match will be substituted with a full advertised in the MDS value and only then submitted to the GM. 
Required middleware 
string &#8211; Grid middleware. The site to submit the job to will be chosen by the UI among those advertising specified middleware. 
Usage is similar to that of the runTimeEnvironment. 
Location of a logical collection in the Replica Catalog URL -LDAP directory specified as URL (ldap://host[:port]/dn) 
Number of reruns (if a system failure occurs) integer - an integer number 
If not specified, the default is 0. Default maximal allowed value is 2. 
Request a specific architecture string -architecture (as produced by uname -a) 
If &#34;yes&#34;, do dry-run: RSL is parsed, but no job submission to LRMS made 
Substitutes &#60;string2&#62; with &#60;string1&#62; for internal RSL use string1 -new internal RSL variable string2 -existing combination of variables/paths 
Only one pair per substitution is allowed 
Defines execution shell environment variables VAR - new variable name string -existing combination of variables/paths 
Defines how many parallel threads will be used by the GM during gsiftp transfer of the files integer &#8211; a number from 1 to 10 
If not specified, parallel transfers are not used. 
Specifies a type of job (default is &#8220;single&#8221;) 
Specifies amount of jobs to be submitted for jobtype &#8220;multiple&#8221; integer &#8211; a number (default is 1) 
Specifies a required parallel runtime environment 
string &#8211; a string describing the type of parallel environment 
Syntax and usage of this attribute is completely analogous to that of the runTimeEnvironment,apart of its interpretation by the GM, which depends on the requested environment. 

3.2. GM INTERNAL ATTRIBUTES 
The following attributes are constructed by the UI and are parsed to the GM, although users may specify them manually (not advised!). 
Internal attribute for the standard input. 
Action to be taken by the gatekeeper: submit the job, cancel job execution, or clear the results of the job (also cancels the job). 
If &#34;yes&#34;, input RSL is stored in a temporary file at the gatekeeper. Must be always set as &#34;yes&#34; at the moment. 
LRMS type, indicating which submission script is to be invoked string -LRMS type, at the moment only &#34;pbs&#34; is supported 
string - host name 
Unique job identification string, needed for cancellation and clean-up. string -global job ID It can also be provided during submission of the job and should be unique to a computing element. 
4. UNSUPPORTED GLOBUS ATTRIBUTES 
The following Globus attributes are not supported (a warning is issued by the UI, and the corresponding attribute is ignored): 
&#8226; (resourceManagerContact=&#60;string&#62;) 
&#8226; (directory=&#60;string&#62;) 
&#8226; (maxWallTime=&#60;time&#62;) 
&#8226; (maxTime=&#60;time&#62;) 
&#8226; (gramMyJob=independent|collective) 
&#8226; (project=&#60;string&#62;) 
&#8226; (hostCount=&#60;number&#62;) 
&#8226; (label=&#60;string&#62;) 
&#8226; (subjobCommsType=blocking-join|independent) 
&#8226; (subjobStartType=strict-barrier|loose-barrier|no-barrier) and other attributes specific to different jobmanagers. 
5. REFERENCES

1. I. Foster and C.Kesselman: The Grid: Blueprint for a New Computing Infrastructure (1998)

2. Globus project, http://www.globus.org

3. RSL 1.0, http://www.globus.org/gram/rsl_spec1.html

4. NorduGrid project, http://www.nordugrid.org

5. A. Wnnen et al.: An Overview of an Architecture Proposal for a High Energy Physics Grid,
PARA 2002, LNCS 2367, pp. 76, J. Fagerholm et al. (Eds.) Springer-Verlag Berlin Heidelberg
(2002)

6. NorduGrid User Interface description,
http://www.nordugrid.org/documents/NorduGrid-UI.pdf

7. NorduGrid Grid-Manager description, http://www.nordugrid.org/documents/GM.pdf

APPENDIX A. EXAMPLE OF A USER-SIDE RSL SCRIPT 
Strings enclosed in (*...*) are comments 
&#38; (* test run: if &#34;yes&#34;, only submits RSL without actual job start *) (dryRun=no) 
(* some local variables defined for further convenience *) (rsl_substitution=(TOPDIR /home/oxana)) (rsl_substitution=(NGTEST $(TOPDIR)/examples/ngtest)) (rsl_substitution=(BIGFILE /scratch/oxana/100mb.tmp)) 
(* some environment variables, to be used by the job *) (environment=(ATLAS /opt/atlas) (CERN /cern)) (* the main executable file to be staged in and submitted to the PBS *) (executable=checkall.sh) (* the arguments for the executable above *) (arguments=pal) 
(* files to be staged in before the execution *) (inputFiles = (be_kaons &#34;&#34;) (file1 gsiftp://grid.uio.no$(TOPDIR)/remfile.txt) 
(bigfile.dat $(BIGFILE) ) ) (* files to be given executable permissions after staging in *) (executables=be_kaons) (* files to be staged out after the execution *) 
(outputFiles= (file1 gsiftp://grid.tsl.uu.se/tmp/file1.tmp) (100mb.tmp rc://grid.fi.uib.no/bigfile) (be_kaons.hbook gsiftp://grid.quark.lu.se$(NGTEST)/kaons.hbook) 
) (* location of the Replica collection *) (replicaCollection=&#34;ldap://grid.uio.no:389/lc=TestCollection,rc=NorduGrid,dc=nordugrid,dc=org&#34;) (* user-specified job name *) (jobName=NGtest) (* standard input file *) (stdin=&#34;myinput.dat&#34;) (* standard output file *) (stdout=&#34;myoutput.dat&#34;) (* standard error file *) (stderr=&#34;myerror.dat&#34;) (* flag whether to merge stdout and stderr *) (join=&#34;no&#34;) (* request e-mail notification on status change *) (notify=&#34;bqfe oxana.smirnova@quark.lu.se oxana.smirnova@cern.ch&#34;) (* specific queue to submit the job *) (queue=pc) (* maximal CPU time required for the job, minutes for PBS*) (maxCpuTime=60) (* maximal time for the session directory to exist on the remote node, days *) (lifeTime=7) (* maximal memory required for the job, Mbytes *) (maxMemory=200) (* minimal memory required for the job, Mbytes *) (minMemory=50) (* wall time to start job processing *) (startTime=&#34;2002-04-28 17:15:00&#34;) (* maximal disk space required for the job, Gbytes *) (maxDisk=0.5) (* required architecture of the execution node *) (architecture=i686) 
(* required run-time environment *) (runTimeEnvironment=&#34;Atlas-1.1&#34;) (* number of re-runs, in case of a system failure *) (rerun=2) 
APPENDIX B. EXAMPLE OF A GM INPUT RSL (JOB SUBMISSION) 
Strings enclosed in (*...*) are comments 
&#38; (* saves RSL in a temporary file if &#34;yes&#34;*) (savestate=yes) (* job submission to be performed if action is &#34;request&#34; *) (action=request) (* LRMS type, so far only pbs *) (lrmstype=pbs) (* submission host name *) (hostName=&#34;grid.quark.lu.se&#34;) (* test run: if &#34;yes&#34;, only submits RSL without actual job start *) (dryRun=no) 
(* some local variables defined for further convenience *) (rsl_substitution=(TOPDIR /home/oxana)) (rsl_substitution=(NGTEST $(TOPDIR)/examples/ngtest)) (rsl_substitution=(BIGFILE /scratch/oxana/100mb.tmp)) 
(* some environment variables, to be used by the job *) (environment=(ATLAS /opt/atlas) (CERN /cern)) (* a dummy executable *) (executable=/bin/echo) (* the main executable itself, and its arguments *) (arguments=checkall.sh pal) (* files to be staged in before the execution *) 
(inputFiles = (checkall.sh 210.279320915) (myinput.dat 14.3980640923) (be_kaons 880788.2035414420) (file1 gsiftp://grid.uio.no$(TOPDIR)/remfile.txt) (bigfile.dat 104857600.4087847787) 
) (* files to be given executable permissions after staging in *) (executables=checkall.sh be_kaons) (* files to be staged out after the execution *) 
(outputFiles=(file1 gsiftp://grid.tsl.uu.se/tmp/file1.tmp) (100mb.tmp rc://grid.fi.uib.no/bigfile) (be_kaons.hbook gsiftp://grid.quark.lu.se$(NGTEST)/kaons.hbook) (myoutput.dat gsiftp://grid.quark.lu.se/home/oxana/examples/ngtest/myoutput.dat) (myerror.dat gsiftp://grid.quark.lu.se/home/oxana/examples/ngtest/myerror.dat) 
) (* location of the Replica collection *) (replicaCollection=&#34;ldap://grid.uio.no:389/lc=TestCollection,rc=NorduGrid,dc=nordugrid,dc=org&#34;) (* user-specified job name *) (jobName=NGtest) (* standard input file *) (stdin=&#34;myinput.dat&#34;) (* standard output file *) (stdout=&#34;myoutput.dat&#34;) (* standard error file *) (stderr=&#34;myerror.dat&#34;) (* flag whether to merge stdout and stderr *) (join=&#34;no&#34;) (* request e-mail notification on status change *) 
 (notify=&#34;bqfe oxana.smirnova@quark.lu.se oxana.smirnova@cern.ch&#34;) (* specific queue to submit the job *) 
(queue=pc) (* maximal CPU time required for the job, minutes for PBS*) 
(maxCpuTime=60) (* maximal time for the session directory to exist on the remote node, seconds *) 
(lifeTime=604800) (* maximal memory required for the job, Mbytes *) 
(maxMemory=200) (* minimal memory required for the job, Mbytes *) 
(minMemory=50) (* wall time to start job processing *) 
(startTime=20020128171500) (* maximal disk space required for the job, Gbytes *) 
(maxDisk=0.5) (* required architecture of the execution node *) 
(architecture=i686) (* required run-time environment *) 
(runTimeEnvironment=&#34;Atlas-1.1&#34;) (* number of re-runs, in case of a system failure *) 
(rerun=2) (* job ID -needed for cancellation and cleanup *) 
(*(jobid=157111017133827)*) 

INDEX 
action 14 architecture 11 arguments 5 cluster 9 count 13 CpuTime 9 disk 10 dryRun 12 environment 12 executable 5 executables 6 ftpThreads 12 hostName 14 inputFiles 6 jobid 15 jobName 7 jobtype 13 join 8 lifeTime 9 lrmstype 14 memory 10 middleware 11 notify 8 outputFiles 6 parallelenvironment 13 queue 9 replicaCollection 11 rerun 11 rsl_substitution 12 runTimeEnvironment 10 savestate 14 sstdin 14 startTime 9 stderr 8 stdin 7 stdlog 8 stdout 7 


