The Fortran compiler on the ORNL IBM SP is IBM's XL Fortran. It supports the Fortran 95 standard, POSIX Threads, and OpenMP.
Although there is only one actual Fortran compiler, you can invoke the compiler using an array of different commands. Each command has different default options. Here is a description of some of these commands.
| xlf or f77 | Assumes Fortran-77-style ``fixed format'' source code. Does understand Fortran-95 syntax. Links with Fortran libraries that are not thread safe. By default, all local variables are static, as if they were declared with ``SAVE''. Use ``-qnosave'' to change this behavior. |
| xlf_r | Like ``xlf'', but links with thread-safe Fortran libraries and automatically links with IBM's Fortran POSIX-Threads library. |
| xlf90 | Assumes ``free format'' source code. Links with Fortran libraries that are not thread safe. By default, all local variables are automatic, like in C and C++. |
| xlf90_r | Like ``xlf90'', but links with thread-safe Fortran libraries and automatically links with IBM's Fortran POSIX-Threads library. |
| mpxlf | Like ``xlf'', but automatically links with MPI and LAPI libraries. |
| mpxlf_r | Like ``xlf_r'', but automatically links with MPI and LAPI libraries. |
| mpxlf90 | Like ``xlf90'', but automatically links with MPI and LAPI libraries. |
| mpxlf90_r | Like ``xlf90_r'', but automatically links with MPI and LAPI libraries. |
Various options control the way memory is used by Fortran programs. These options can be critical for large-memory applications and applications ported from other systems. Here is a description of some options for memory management.
| -q64 | Creates an executable with a 64-bit address space. All object files making up the executable must be compiled with ``-q64''. The Parallel Environment is currently only available in 32-bit form, so MPI executables cannot be compiled with ``-q64''. |
| -bmaxdata | By default, 32-bit executables only access one segment, or 256 MB,
of memory. By linking with ``-bmaxdata'', you can increase
this range up to eight segments, or 2 GB. Specifying
``0x80000000'' allows the full 2 GB range.
|
| -qrealsize=8 | The default size of variables declared ``REAL'' is 4 bytes. On some other architectures, like Cray Research systems, the default size of ``REAL'' is 8 bytes. You can use the ``-qrealsize'' option to change the XL-Fortran default from 4 to 8, which can simplify porting from Cray-like systems. It is important to note that this option also changes the size of ``DOUBLE PRECISION'' from 8 bytes to 16 bytes. |
| -qautodbl=dbl4 | Some codes rely on the ability to upgrade the size of ``REAL'' without changing the size of ``DOUBLE PRECISION'', making them both 8 bytes. This option does this, unlike ``-qrealsize=8''. |
| -qnosave | By default, the ``xlf'' command creates all variables as if they were declared ``SAVE''. This is useful for FORTRAN-77 codes. This default is inappropriate, however, for fixed-format codes that use some modern features, like recursion or thread parallelism. Use the ``-qnosave'' option to override the default. The ``xlf90'' commands have an implicit ``-qnosave''. |
The following options provide a high level of optimization that is also safe.
For potentially higher performance, you may want to experiment with yet higher levels of optimization. The following options provide ``high-order transformations'', which help optimize loops. These transformations are particularly important for optimizing Fortran-90 array statements.
The General Parallel File System (GPFS) is a large high-performance temporary-storage area with shared access from all SP nodes. It is a common location for building executables. For more information on GPFS, see our online documentation, available at the following URL.
http://www.csm.ornl.gov/ccs/filesystems.html#gpfs
Unfortunately, compiling programs that use Fortran modules is complicated by the fact that GPFS files cannot be memory mapped. The XL Fortran compiler memory maps the ``.mod'' files used to implement and check module interfaces. Therefore, these files must be stored elsewhere.
The XL Fotran compiler has options to facilitate maintaining ``.mod'' files elsewhere. Use the ``-qmoddir'' to write all ``.mod'' files to a particular directory, and use ``-I'' to include that directory in the search path. The following example maintains ``.mod'' files in a ``mod'' subdirectory of the user's home directory (in DFS).
Using these options, you can compile in GPFS while maintaining ``.mod'' files in DFS. Typically, the ``.mod'' files do not represent a significant amount of disk space.
The XL Fortran compiler supports both explicit and automatic shared-memory parallelization. For explicit parallelization, the compiler supports OpenMP.
Use the ``-qsmp'' option to turn on shared-memory parallelization. The compiler will automatically parallelize ``DO'' loops and array statements when it can prove that such parallelization is safe. You must use the thread-safe compiler commands (``..._r'') to use ``-qsmp''. The ``-qreport'' option causes the compiler to produce a loop-transformation report that includes information about automatic parallelization.
The automatic parallelization performed by the compiler is of limited utility, however. Performance may increase little or may even decrease. Another option is explicit parallelization using OpenMP directives. By default, the ``-qsmp'' option translates OpenMP directives and performs automatic parallelization. To turn off automatic parallelization, use the ``-qsmp=noauto'' option. For OpenMP programs compiled using ``xlf_r'', you probably want to add the ``-qnosave'' option for independent parallel calls to the same procedure.
In addition to OpenMP, XL Fortran directly supports POSIX Threads (Pthreads) through a Fortran API created by IBM. For more information on using Pthreads with XL Fortran, see the IBM Redbook POWER3 Introduction and Tuning Guide. The ``..._r'' compiler commands automatically link with the Pthreads library.
By default, the XL Fortran compilers link dynamically with external libraries. Each library is loaded at runtime from a file that is shared among different executables. This saves storage space and memory, and it allows libraries to be upgraded without having to relink executables.
Dynamic linking poses a problem, however, for executables that must stay exactly the same between runs. Changes to the shared libraries can change the way an executable behaves. To solve this problem, the XL Fortran compiler allows you to statically link executables, so all the executable code is stored within the executable itself.
Statically linking sequential or shared-memory executables is straightforward.
Single-thread programs:
Multiple-thread programs:
Statically linking MPI programs to use the US protocol over the SP interconnect is messier. You have to manually issue the options that would otherwise be encapsulated in the ``mpxlf'' commands.
First compile all the source files and point to the MPI include directory.Then link the object files into an executable using a long list of options.
Single-threaded MPI processes:
Multiple-threaded MPI processes:
The following compiler options are useful for debugging executables.
| -g | Includes debugger information in the object files. Allows a debugger to associate machine code with source code. Works with all levels of optimization! Note that the connection between source code and highly optimized machine code may not be accurate. |
| -C | Compiles for run-time array-bounds checking. |
| -qextchk | Checks for mismatched procedure interfaces and common blocks. This option cannot be used with MPI for MPI relies on weak type checking and mismatched procedure interfaces. |
| -qflttrap | Compiles the program to detect floating-point exceptions at
run time. The following form of this option causes the program to abort
on floating-point overflow or division by zero.
|
The following options are useful for creating performance profiles of executables.
| -p | Compiles the executable to produce limited performance-profile information. When run, the executable writes performance data to the file ``mon.out''. Use ``prof'' to analyze these data. |
| -pg | Compiles the executable to produce extensive performance-profile information. When run, the executable writes performance data to the file ``gmon.out''. Use ``gprof'' or the gui-based tool ``xprofiler'' to analyze these data. |
For more information on XL-Fortran compiler options, see ``man xlf''. The CCS has hard copies of the XL Fortran User's Guide and Language Reference. Unfortunately, these manuals are not available online. E-mail ``consult@ccs.ornl.gov'' to arrange access to these manuals.
For more information on XL-Fortran performance optimization and parallelization, see the following IBM Redbooks, available online.
POWER3 Introduction and Tuning Guide
http://www.redbooks.ibm.com/abstracts/sg245155.html
Scientific Applications in RS/6000 SP Environments
http://www.redbooks.ibm.com/abstracts/sg245611.html