|Date and Author(s)|
Here is my plan for performing the scalability testing for QBank and Gold.
First, for both QBank and Gold, I will document the time that it takes to respond to 1000 requests each for withdrawals, reservations, quotations(QBank only), balance checks and default account lookups. I will do this for a local client request and remote client request -- which should yield information allowing me to estimate a maximum supportable job rate (measured say in jobs per minute). These performance and scalability tests will be carried out via client scripts.
In addition, I intend to carry out simulations via Maui to see how many jobs can be simultaneously serviced by the system by running an endless supply of short jobs on an unfillably large system for a predetermined time (say 20 minutes). These tests will include a reservation and withdrawal interaction with both QBank and Gold.
Finally, I would like to try packing a real system (Colony or Chiba City or an ORNL cluster) with real zero-second jobs as fast as they can be submitted and handled using two base combinations of RM systems,
1) Maui, PBS, and QBank (PBS tested with and without Gary's patches).
2) Maui, Gold, Brett's QM, NCSA's NM/CM, ANL's PM (Tests will also be performed toggling use of Event Manager and Service Directory, as well as replacement with PBS (and a subtest with Kevin's PBS front-end)). The metric will be to determine how many zero-runtime jobs can be serviced in a the various configurations.
Note that this is only an allocation manager centric view of the tests and that other tests should be performed (such as the RM system without allocation management etc.)