Study and Evaluation of 3 UDP protocols
3.3. Fast Object-Based data transfer System(FOBS)
FOBS is an application library implemented in C and uses many of the NETBLT ideas but at the user level rather than the kernel level. The primary paper on FOBS, An Evaluation of Object-Based Data Transfers on High Performance Networks , states that by 'object-based' they mean that none of the data transferred can be assumed to be correct until all of the data has been received. The code seems to indicate that "all of the data" means a "CHUNK_SIZE" which is currently preset at 100 MB. The application does not have access to any of the data until the "CHUNK" is successfully received. That assumes that both sender and receiver have pre-allocated buffers large enough to accommodate the complete size of a "CHUNK". Each CHUNK of data is transmitted in segments--10000(or fewer depending on the amount of data remaining) packets carrying 1466 bytes of data. Loss information is sent from receiver to sender after a segment of data has been sent. FOBS presets a tolerated loss rate(1%) and makes the assumption that packet losses are not necessarily the result of congestion.
1. use of TCP/UDP channel(s)
FOBS uses two TCP channels to transfer control information between sender and receiver. One channel for ENDOFSEGMENT/DONE/FEEDBACK/ACK packets and one for COMPLETEDPKT/WRITECOMPLETEDPKT packets so that the receiver can signal the sender when all the data in the current CHUNK has been successfully received and written to the file. Setsockopt() is used to set all the sender's socket buffers(SO_SNDBUF , SO_RCVBUF) to socketBufferSize(100000). SO_REUSEADDR and TCP_NODELAY are set for all TCP sockets for both sender and receiver.
One UDP channel is used to transfer data from sender to receiver-- the UDP socket SO_SNDBUF was also set to socketBufferSize(100000) and SO_REUSEADDR is set for both sender and receiver. The UDP socket SO_RCVBUF did not appear to be set anywhere???
A timed select() is used to detect incoming packets on TCP and UDP sockets for both sender and receiver.
TCP control packets from sender to receiver: #define DONEPKT 1 - sent after all original data or all retransmitted data in a CHUNK has been sent #define ENDOFSEGMENTPKT 2 - sent at the end of each segment TCP control packets from receiver to sender: #define COMPLETEDPKT 1 - sent after all packets in a CHUNK have been received #define FEEDBACKPKT 2 - sent in response to the ENDOFSEGMENTPKT or DONEPKT-- when all packets in a segment have not been received #define NACKPKT 3 - sent in response to a DONEPKT #define ACKPKT 4 - sent in response to the ENDOFSEGMENTPKT or DONEPKT-- when all packets in a segment have been received #define STOPPKT 5 - not used #define WRITECOMPLETEDPKT 6 - sent after a COMPLETEDPKT indicating that all data in the CHUNK has now been written to the file
MaxRate[NS_GREEN]=nicRate; MinRate[NS_GREEN]=0.85 * nicRate; MidRate[NS_GREEN]=(MaxRate[NS_GREEN]+MinRate[NS_GREEN])/2; increasingStep[NS_GREEN]=(MaxRate[NS_GREEN]-MinRate[NS_GREEN])/5; decreasingStep[NS_GREEN]=(MaxRate[NS_GREEN]-MinRate[NS_GREEN])/5; MaxRate[NS_YELLOW]=MinRate[NS_GREEN]-1; MinRate[NS_YELLOW]=0.4 * nicRate; MidRate[NS_YELLOW]=(MaxRate[NS_YELLOW]+MinRate[NS_YELLOW])/2; increasingStep[NS_YELLOW]=(MaxRate[NS_YELLOW]-MinRate[NS_YELLOW])/10; decreasingStep[NS_YELLOW]=(MaxRate[NS_YELLOW]-MinRate[NS_YELLOW])/7; MaxRate[NS_RED] = MinRate[NS_YELLOW] - 1 ; MinRate[NS_RED] = 0.02 * nicRate; if(!MinRate[NS_RED]) MinRate[NS_RED] = 1; MidRate[NS_RED] = (MaxRate[NS_RED]) / 2; increasingStep[NS_RED] = (MaxRate[NS_RED] - MinRate[NS_RED]) / 15; decreasingStep[NS_RED] = (MaxRate[NS_RED] - MinRate[NS_RED]) / 5;The currentLossPercent, calculated from the data sent by the receiver in response to an ENDOFSEGMENTPKT, is input to the following formula to determine the current available bandwidth.
availableBW = (currentRate - (((double)(currentLossPercent - acceptableLossRate)/100)* nicRate));The availableBW is then compared to the previously calculated table to determine the current state of the network and, correspondingly, whether the newSendingRate should be increased or decreased from the currentSendingRate. The sender checks the actual sending rate against the desired sending rate after a burst of packets has been sent. Right now, burst size is set at 25. If the sending rate is greater than the currentSendingRate, a gettimeofday() calculation is done until time to send the next burst.
5. unique features
Data that has been sent n times has preference over data that has been sent n+1 times. Similar to a version of NETBLT, FOBS is lock-step--after every CHUNK_SIZE of data has been sent, the sender waits until the receiver has written all the data to the file before beginning to send the next CHUNK.
***object-based data transfer
According to the authors of FOBS,
"the fundamental characteristic of an object-based data transfer that is leveraged by FOBS is the assumption that the user-level data buffer spans the entire object to be transferred.... this characteristic allows FOBS to push to the limit the basic concept of the "Large Window" extensions developed for TCP."
There is a java drag-n-drop interface that has been written for FOBS however I had trouble getting it to always complete a file transfer. The command line version seemed to be more reliable.
The sending rates are calculated before the transfer begins and adjusted using the pre-calculated rates pulled from an array as the flow continues. Also the rate control mechanism checks the actual sending rate after a specified number of packets(25) have been sent--not after every packet.