Scalable Systems Software Project Notebook - page 61 of 88

First PagePrevious PageNext PageLast PageTable of ContentsSearch

Date and Author(s)

Meeting Notes from Aug. 26-27 Argonne meeting

Hi Folks,

Here are my notes from our Scalable Systems meeting at Argonne National Lab in Chicago. At this meeting we had four big results:


Al Geist ORNL
Paul Hargrove LBNL
Ron Brightwell SNL
William McLendon SNL
Ron Oldfield SNL
Craig Steffen NCSA
Narayan Desai ANL
Stephen Scott ORNL
Thomas Naughton ORNL
Scott Jackson PNNL
Brett Bode Ames Lab
Rick Bradshaw ANL
Rusty Lusk ANL
Pete Beckman ANL
Rob Ross ANL
Susan Coghlan ANL

Agenda and ppt slides

Thursday August 26                      
 8:00  Continental Breakfast                                                   
 8:30  Al Geist - Project Status and Fred's comments                      
 9:30  Scott Jackson - Resource Managment Report       
       Not shown but slides made available for Gold Allocation and Accounting Package       
10:30  Paul Hargrove - Process                  
Management and Monitoring Report                                                       
11:30  Narayan Desai - Node Build,                 
Configure Report                                                           
12:30  Lunch (caffeteria)                                                  
 1:30  Rusty Lusk -  Comparison of RS and SSSRMAP followed by       
      Introduction of Less Restrictive Syntax              
 3:00  Break       
       FastOS presentations were each followed by discussions of how       
       Scalable Systems Software could help and/or play a part.       
 3:30  Pete Beckman - FastOS Project led by ANL                                                                                       
 4:00  Ron Brightwell - FastOS Project led by SNL        
 4:30  Steven Scott - FastOS Project led by ORNL                                                                  
 5:00  Paul Hargrove - K42 FastOS Project proposed by LBL              
 5:30  Adjourn for dinner                  
 7:30  Return to hack on software release             

Friday August 27 8:00 Continental Breakfast 8:30 Rusty Lusk - New SDK for SSS components 9:00 Will McClendon - Validation and Testing 10:00 Thomas Naughton - SSS-OSCAR V1.0 Release Plans 10:30 Break 11:00 Al Geist - Discussion of Response to Fred next meeting date: Hacking mtg at ORNL next regular meeting: Jan 25-26, 2005 (Check with Fred) Location: A Gathersburg Hotel so Fred can come 12:00 Meeting Ends

Meeting Goals

Meeting notes

Al's Talk. - (slides above)                                       
His talk was divided into three sections:                                         
Review of results of last meeting,                                        
Fred's items, and                                         
expectations for this meeting.                           
(also includes these notes on end)                         
Scott's Talk - Resource Management Working Group Status                
Updated and implemented SSSRMAP v3 spec       
Second alpha release including Maui, Bamboo, Warehouse, Gold       
Added interactive FAQOMATIC       
Completed merger of Maui 3.2 and Maui SSS – to use SSS interfaces       
   (commercial versions of these will also as a matter of course)       
QM – interactive job support finished and tested.      
Packaging updated to separate out components required on the execution nodes.       
Accounting and allocation- complete rewrite in PERL.        
Significantly improved accounting design and account report       
Completed allocation, reservation, quotation and charge rates GOLD GUI       
Metascheduler (grid scheduler-Silver) migrated interface to use SSS       
Future work – Beta release of all components including Silver       
                     FT supporting 25% cluster loss       
                     Continued OS support for Linux, AIX, Tru-64.        
                     Future OS-X, Unicos, HPUX       
Who using these alpha components? Few just looking at it        
Production deployment of GOLD on 11.8TF PNNL cluster in November 2004       
Also think - MauiHPC center, ANL, and DOD centers likely to deploy it.      
Paul – Process Management Working Group status      
Checkpoint status – full save and restore of: registers, memory, signals, PID,      
Files (open but unmodified, open and appended, pipes between processes),       
and communication (via LAM/MPI over TCP)       
Handles in flight data (drains), linear scaling and migration.      
In future OpenMPI? Paul will check      
Discussion of handling files      
Will always be a Linux-only solution (across all of them)        
Presently x86 only – Alpha and PPC as possible future work-not high priority      
Future work – more on files (mutable files, directories), process groups      
Checkpoint Manager works with Bamboo and MPDPM      
Process Manger – continued daily use on Chiba      
New option to signal entire unix process group      
Misc hardening of MPD system. Prompted by Intel use (cook & associates)      
Future- Intel donated a IA64 test cluster could be used to test SSS      
Warehouse- major bug fix, works with RM components      
Ssslib version with RMAP delayed due to harddrive crash.          
Narayan – Build configure working group status      
Infrastructure improvements – ssslib wire protocol user support additions      
   SSS SDK development      
Components Improvements – efficiency of service directory and event mgr      
   Node state manager – simplified implementation by using other SSS components particularly from PM       
More discussion of SC04 demos – GUIs and handling failure.      
Rusty – Syntax Discussion      
We agreed on XML as basis of communication mechanism. Many benefits.      
Allowed multiple wire protocol and service directory to keep track      
We have created a couple XML styles.     
Rusty thinks having two or more is fine.     
He is not suggesting we have only one, although others in group have said this.     
Steps: match a set of objects,       
           apply function with args to set of objects, and       
           construct return message.          
RS  syntax is  predicate       
Command is the function to apply (args go here)      
Predicate is a field value match to select set of objects      
Return message includes info on all fields in predicate      
Goes through a few examples in RS and explains them      
In SSSRMAP       
Go through same examples in RMAP      
Matching is in the “where” clause      
Args are in the “Option” object      
Return message indicated by “Get” object      
Looking at Both      
Completeness - Probably equal  – both lack general negation      
Validation – RS is somewhat better here      
Extensibility – SSSRMAP is somewhat better      
Readability – SSSRMAP is somewhat better here      
Conciseness – RS is better      
Atomicity- equal      
(see slides for "Less Restrictive Syntax" proposal)      
The Less Restrictive Syntax      
Keep high-level spec of commands like RS for validation      
Move attributes in RS to subobjects as in SSSRMAP      
Explicitly specify fields to return      
Show and discuss examples in new syntax style      
   List of objects to match      
     matching criteria      
Still has the same implicit AND and OR that was in RS.      
Argonne is starting to transition to the Less Restriction Syntax.      
Pete – FastOS at ANL and U Oregon (Budget starts Oct. at 2/3 request)      
Future systems (smart memory, message processor, stream processor)      
   Functional decomposition and Hierarchical organization      
Example BG/L uses 4 OS SuSE 8, SuSE 9, embedded Linux, microkernel      
   Get a BG/L in December      
For Petascale how many OS will be required      
What are their performance characteristics and requirements      
Can they be dynamic      
What is the cost of each component. What if a part is left out      
Are collective – coupled OSes needed      
Can we build experimental framework for FT      
Four focus areas-       
Flexible OS suites, Scalable system calls, FT, performance tools      
Interact with SSS      
Dynamic node builds and kernel loads      
Tao will be added to kernel and middleware could compliment SSS      
Faulty Towers provide info to SSS layers via component interface      
OS is Linux 2.6 kernel       
   Embedded Linux       
Ron – SNL, UNM, CalTech also got 2/3 budget      
Need OS functionality out on network interface – distributing bits of OS      
   OS bypass, offload, splintering      
Light Weight Kernel influences – hard to make changes      
Programming Models – problems with mixing PIM, MPI, OpenMP      
Usage Models – Apps number and time change over time      
External services – parallel file system, chkpt, dynamic libraries      
   Build a collection of micro services.       
   Small components with well defined interfaces      
   Combine services specifically specifically for an app and system      
Tools for combining Micro services      
Building custom OS on the fly      
Stephen – ORNL, UNM, NCSU, OSU, LousianaTech  got 1/2 budget      
RAS for scientific and engineering apps      
Paul – getting K42 to work on clusters      

DAY 2 NOTES Rusty – SDK for SSS components Lots of components in the future, some we have never imagined Crucial to make component development easy Ssslib and event manager, service directory for an foundation Also need to encapsulate functionality of an abstract component Have been trying this with Python Classes Useful for BG/L and FastOS experiments Low Levels of SDK multiple wire protocols, EM, SD, and communication for any language Upper Level of SDK Server and Event receiver classes provide all the services that are independent of component – registers, logging, errors, XML validating, … Shows the “stack” Goes through echo example – makes SSS coding pretty easy! Goes through job submitter example (several slides) Conclusion makes writing SSS components easy currently for Python components but other languages possible Scott says this should be easy to implement RMAP syntax in this SDK Will – Validation and Testing status Mainly working on APItest current release v0.2.0 Available for download on SNL ftp site (see slides) Easier to define a new test type (already does shell, script, and SSS) There is some caution with SUID Packages required Python 2.3, ElementTree (, Twisted, and ssslib SSS- Service Directory test – need to extend to all SSS components Discussion about details how to use for SSS tests What about a user manual? Future work Future work develop more tests for SSS components test developer GUI additional native tests types – http, TCP, XMLRPC user guide ability to SU jobs to different users Discussion Thomas – SSS-OSCAR Current status v0.2a8 prerelease for v1.0 at SC04 Two more items are in CVS (at this meeting) need testing Starting work on v0.3 w/ new GOLD pkg OSCAR support for BCWG schema Future work Integrate Gold integrate APItest in OSCAR – authors create their own test cases Improve documentation for v1.0 Start weekly builds for testing Release schedule Nov 8 SC04 release v1.0 Oct 4 code freeze Sept weekly builds – available first day of week by noon for developer to test their component Test resources – ORNL “Test1” cluster Al Geist – Group discussion of Response to Fred Priorities> v1.0 SSS-OSCAR release formal collaboration with FastOS projects SSS on NLCF machines rather than production clusters run scale tests on big clusters (short windows) steps towards long-term support priority to vote on written component interfaces Discussion that a hacking meeting may be needed before SC04 to get ready for SSS-OSCAR V1.0 release. Meeting ended with folks feeling good about progress.