Resource Management and Accounting Notebook - page 12 of 150

EditDeleteAnnotateNotarize
First PagePrevious PageNext PageLast PageTable of ContentsSearch

Date and Author(s)

Consolidated Job Attributes

Consolidated Job Attributes

I went through the notebooks (and one document not in a notebook) and collected all the XML that applies to “jobs.” I then organized them into a “life cycle,” or the flow of a job from the request for a (price?) quote, through execution, and through the debit on the job’s account. The result is immediately below. At the bottom of the document, I’ve collated all the XML attributes into (1) identifications, (2) parameters to an individual node, (3) user-specified parameters to the system software, and (4) quantities calculated by the system.

This exercise gives an overview of all the things everybody has been thinking about. An obvious step would be to develop a uniform naming scheme and a class-subclass structure for all the attributes.

Life Cycle of a Job

Scott Jackson – Resource Management Group

1. Request Price Quote: [AccountName, UserName, JobId, Processors, Wallclock, Machine, Nodes, Memory, Disk, Class, QOS, NodeType]

2. Create Reservation: [AccountName, UserName, Amount, Wallclock, Processors, JobId, Machine, Nodes, Memory, Disk, Class, QOS, NodeType, QuoteId, TracebackInfo]

Dave Jackson – Resource Management Group

2. Create reservation object:

   
<CMD OBJECT="reservation" ACTION="create"  NAME="[ANY]"> 
     <ARGS><ATTR NAME="acl"  VALUE="QOS:3"/> 
           <ATTR NAME="acl"  VALUE="CLASS:batch"/> 
           <ATTR NAME="duration"  VALUE="3600"/> 
           <ATTR NAME="processors"  VALUE="8"/> 
           <ATTR NAME="starttime" VALUE="924853324"/> 
     </ARGS>  
</CMD>  

3. Create job object:

 
<CMD OBJECT="job"  ACTION="create" NAME="[ANY]"> 
    <ARGS><ATTR  NAME="duration" VALUE="3600"/> 
          <ATTR  NAME="taskcount" VALUE="4"/> 
    </ARGS> 
</CMD> 

Rusty Lusk and Narayan Desai – Process Management Group

3. Create Process XML Schema:

   
<?xml version='1.0' ?>    
  <xs:schema xmlns='http://www.w3.org/2001/XMLSchema'    
             targetNamespace='urn:scidac-org:sss:pm'    
             version='0.1'>    
    <xs:element name='process-manager-equests'>    
      <xs:attribute name='version' type=xs:string use='required'/>    
      <xs:element name='kill-processes'>    
          <xs:attribute name='jobid' use='required'/>    
      </xs:element>    
      <xs:element name='create-processes'>    
          <xs:attribute name='id' type=xs:id use='required'/>    
          <!-- This is the unique id for requests within PMRequests -->    
          <xs:attribute name='totalprocs' type=xs:integer use='required'/>    
          <xs:attribute name='search-path' type=xs:string use='required'/>    
          <xs:attribute name='userid' type=xs:string use='required'/>    
          <xs:attribute name='cwd' type=xs:string use='required'/>    
          <xs:attribute name='host' type=xs:string use='optional'/>    
          <xs:attribute name='jobid' type=xs:string use='optional'/>    
          <!-- This is so that the scheduler can request that a job    
               have a particular id -->    
          <xs:attribute name='exec' type=xs:string use='required'/>    
          <xs:simpleType name='stdio' base=xs:string use='optional'>    
            <xs:enumeration value='separate'/>    
            <xs:enumeration value='merged'/>    
            <xs:enumeration value='merged-labelled'/>    
            <xs:enumeration value='discard'/>    
          </xs:simpleType>    
          <!-- This is to handle stdio. separate creates one file per    
               process for stdio. merged merges all of the outputs.    
               merged-labelled labels merged output by hostname.    
               discard throws it away -->    
          <xs:sequence>    
            <xs:element name='exec' minOccurs='0' maxOccurs='unbounded'>    
              <xs:attribute name='name' type=xs:string use='required'/>    
              <xs:attribute name='range' type=xs:string use='required'/>    
              <xs:attribute name='search-path' type=xs:string use='required'/>    
            </xs:element>    
            <xs:element name='env'>    
              <xs:element name='var'>    
                <xs:attribute name='name' type=xs:string use='required'/>    
                <xs:attribute name='value' type=xs:string use='required'/>    
              </xs:element>    
              <xs:attribute name='range' type=xs:string use='required'/>    
            </xs:element>    
            <xs:element name='args'>    
              <xs:attribute name='range' type=xs:string use='required'/>    
              <xs:element name='arg'>    
                <xs:attribute name='value' type=xs:string use='required'/>    
                <xs:attribute name='idx' type=xs:integer use='required'/>    
              </xs:element>    
            </xs:element>    
            <xs:element name='host'>    
              <xs:attribute name='range' type=xs:string use='required'/>    
              <xs:attribute name='hostname' type=xs:string use='required'/>    
            </xs:element>    
            <xs:element name='cwd'>    
              <xs:attribute name='range' type=xs:string use='required'/>    
              <xs:attribute name='path' type=xs:string use='required'/>    
            </xs:element>    
          </xs:sequence>    
      </xs:element>    
    </xs:element>    
  </xs:schema>   
   

Erik DeBenedictis – Process Management Group

3-4. Meta Create Process:

 
<create-process  exec=sample load="512..1023"  host0="A0,P0"..host1023="A1023, P1023">   

Dave Jackson – Resource Management Group

4. Create node object:

 
<CMD  OBJECT="node" ACTION="create"  NAME="sn001"> 
    <ARGS><ATTR NAME="processors"  VALUE="8"/> 
          <ATTR NAME="opsys"  VALUE="AIX432"/> 
    </ARGS> 
</CMD> 

Scott Jackson – Resource Management Group

5. Withdraw Funds from an Account: [AccountName, UserName, Amount, Wallclock, Processors, JobId, QuoteId, Machine, Nodes, QOS, Memory, Class, Disk, NodeType, Tracebackinfo]

Collated XML Attributes

Identifications

JobId (SJackson) – integer job number
QuoteID (SJackson) – integer job number
id (Lusk and Desai) – integer job number
jobid (Lusk and Desai) – integer job number

AccountName (SJackson) – user’s (financial) account
UserName (SJackson) – user’s (Unix) name
userid (Lusk and Desai) – user’s (Unix) name

Parameters to an Individual Node

   
exec (Lusk and Desai) – parameter
stdio (Lusk and Desai) – parameter
env (Lusk and Desai) – parameter
var (Lusk and Desai) – parameter
args (Lusk and Desai) – parameter
cwd (Lusk and Desai) – parameter
TracebackInfo (SJackson) – (I don’t know what this is)
search-path (Lusk and Desai) – parameter

User-Specified Parameters to the System Software

Processors (SJackson) – number of PEs
processors (DJackson) – number of PEs
taskcount value=4 (DJackson) – number of PEs
Nodes (SJackson) – number of PEs
totalprocs (Lusk and Desai) – number of Pes

Wallclock (SJackson) – elapsed time
duration (DJackson) – elapsed time
starttime value=924853324 (DJackson) – time

Machine (SJackson) – name of a parallel computer
host (Lusk and Desai) – network address

NodeType (SJackson) – parameter (CPU type, CPU speed, memory size, etc.)
opsys value=AIX432 (DJackson) – parameter (operating system)

Class (SJackson) – execution priority??
Acl value=”Class:batch” (DJackson) – execution priority

QOS (SJackson) – execution priority??
Acl value=”QOS:3” (DJackson) – execution priority

Memory (SJackson) – Max megabytes per PE
Disk (SJackson) – Max gigabytes per job

load (DeBenedictis) – subset of processors to load

Quantities Calculated by the System

Amount (SJackson) – dollars
HostN value=”address,port” (DeBenedictis) – I/O mapping table


Addendum:   Al Geist   Date: Wed Oct 10 17:11:44 2001 (GMT)
I did a little reformating of the page to insert "enote:showtags" tags around the XML snipets so they display correctly on a browser. I also backeted them with "pre" tags to preserve line breaks and indents.