|Date and Author(s)|
Scalable Systems Software - Process Management Working Group Teleconference 9/20/01 10:00 AM (PDT) ------------------------------------ Attendees --------- Paul Hargrove LBNL Jason Duell LBNL Eric Roman LBNL Rusty Lusk ANL Narayan Desai ANL Al Geist ORNL Stephen Scott ORNL Scott Jackson PNNL Brett Bode, Mike Welcome, and Katie Vargo were unable to attend. Agenda ------ * Reading material for XML * XML Schema for process startup request See pages 9 and 10 of PMWG e-notebook * System monitoring * Set time/date for next meeting * System wide issues (common socket behavior and interoperability) Action Items ------------ * Read description of "global concerns" on XML delimiting (posted on build/configure website) * Post links to XML reading material on project notebook. [All] * Look at how other systems (XMLRPC, SOAP, Slide) handle passing parsing multiple XML request through a single socket [All] * Place 1 sentence reviews of known XML reading material on PMWG/XML page (page 7 of PMWG notebook). [Rusty] * Type up a more detailed, thoroughly commented description of process manager requests. [Narayan] * Discuss conflicts with OSCAR system model [Stephen] XML Reading Material -------------------- Rusty's group is learning XML in detail. They believe that DTD's are now obsolete, and that schema's are more popular. They are reading _Essential XML_. Rusty's group is trying to find a validating XML parser that can validate against a schema. Most current validating parsers validate against a DTD, not a schema. Paul asked whether any of the group had looked at SOAP or at XMLRPC. None of the group has had a chance to look at these protocols in detail. Narayan Desai believes that SOAP and XMLRPC standards may not be useful to us at this point, and that the standards may not be frozen. Al Geist suggested looking at Slide. XML Schema for Process Startup Request -------------------------------------- Paul asked whether the standard input, standard output, or standard error of a job could be specified in the process startup request. Paul asked whether anyone could be notified when a job completes. Rusty had not looked at this issue. Rusty said that one missing attribute from the process startup request is the current working directory to use when the process starts. Rusty is looking for a way to say: "Process 8 writes it output to the file stdout.8" Paul suggested a mechanism similar to printf. Rusty agreed to look at this in the next version of the request. Rusty stated that he would like to learn whether XML has a way to turn string arguments into array arguments. Narayan suggested the group try to take advantage of "well-formed" values for schemas. He suggested using well-formed schemas to perform input validation for every software component. If it parses, then it's valid. Elements of a request that were not meaningful at a particular site would not even parse. Al Geist asked whether site administrators should have to configure the request format for every part of the system software. Narayan says that this could be automated as part of the system build process. There was a brief discussion about host ranges. How complicated should host ranges be? How do you specify ranges? Jason asked whether there was a way to specify environment variables globally ( across a range of nodes). Rusty said that there were no complicated rules to handle range overlap, so that the request processing could be kept very concise. Stephen suggested adding a field to specify a particular cluster within the process startup request. Rusty stated that there would be only one process manager per cluster. Stephen contested that a process manager should be able to forward process startup requests to another process manager. Paul stated that a request to start a process should only be sent to the process manager responsible that will start the process. If the user does not know in advance where a process should be started, the decision should be left to the scheduler. Stephen agreed that this decision would be more appropriate for the scheduler. Al Geist asked whether there was a distinction between a required and an unrequired field of an XML request. Narayan stated that by default, all fields were optional; "required" is necessary to make a tag required. "optional" may be used to make an element optional, and "prohibited" may be used to forbid a particular element. Paul asked how to refer to the process started by a process startup request? Rusty has not thought about this. Al stated that this was part of a more general problem. Every Scalable Systems component will expect some response. What should this response be? Paul suggested a tuple containing an opaque response and a location identifier (opaque, locationId). The location identifier would specify who to contact to operate on the job. There was some discussion. This format might only be appropriate for instantiation or query requests. It would not be appropriate for a "stop job" request to return a tuple of this form, but only a status code specifying whether the request was successful. Al proposed that schema's be developed for both responses, and for requests. We want some consistency in the response schemas. Two responses that would be appropriate for all components are an "I don't understand your request" response, and a "No, I won't do that, but the following component can" response. Monitoring ---------- Paul opened the monitoring discussion by asking which requests or objects do we need to monitor? Paul also asked about how monitoring works through this system in general. Is there a way to "stream" XML information? Rusty stated that XML has a way to stream XML information through a special model. For requests, he suggested only using a document model. Scott suggested using a subscription model for monitoring. The subscription model would bring the system closer to an event model. Paul summarized the discussion by identifying 3 event models. First, a streaming model for continuous or interactive monitoring. Second, a request/ response model. Third, a subscription model for event processing. Next meeting ------------ Rusty is unable to attend the October 4th meeting due to a schedule conflict. The next meeting is scheduled for September 27th. System Wide Issues ------------------ Narayan opened this discussion by asking whether multiple requests could be sent down a socket to another component. Can sockets be held open? This raised the issue of how to delimit XML requests. Jason suggested encapsulating requests inside another format, possibly HTTP. If the length of a request is sent before the request, there is no need to delimit the end with a special marker. How are different types of request sent through a stream of requests? Is a higher level protocol or convention to encapsulate requests in a stream needed? There may be issues with lingering sockets. The group agreed that we should start simple, by assuming a single request per socket. All the working groups will use this convention. The group discussed issues that arise when sockets close unexpectedly. They agreed that the parser needs to be smart about what happens in this case.
Addendum: Al Geist Date: Wed Sep 26 21:22:08 2001 (GMT)
There are some comments on using HTTP format between components in the main notebook. (Addendums of "General Ideas" page 8)