How do I build applications


Common Compiler Options

The following are common options for various vendors C/C++ and Fortran compilers. In particular they show how to:

  1. Generate optimized code
  2. Enable OpenMP
  3. Enable traceback features

For production builds you may want to remove the debug "-g" and traceback options. For development you may want to set optimization to -O0.

Portland Group Compilers

Intel Compilers

Intel Compilers Notes:

The Intel C compiler can be invoked using the command. icpc. The icpc command uses the same compiler options as the icc command. Invoking the compiler using icpc compiles .c and .i files as C++. Invoking the compiler using icc compiles .c and .i files as C. Using icpc always links in C++ libraries. Using icc only links in C++ libraries if C++ source is provided on the command line.

For ifort, filenames with the suffix .f90 are interpreted as free-form Fortran 95/90 source files. Filenames with the suffix .f, .for, or .ftn are interpreted as fixed-form Fortran source files.

IBM Compilers

IBM Compilers for Blue Gene Q (Mc2)

IBM Compiler notes:

The IBM C compilers can be invoked using any of the commands: xlc, xlc++, xlC, cc, c89, c99, xlc_r, xlc++_r, xlC_r, cc_r, c89_r, c99_r. All invocations with a suffix of _r allow for thread-safe compilation. See the man page below for additional information on the differences in the invocations.

The IBM fortran compilers can be invoked using any of the commands: xlf, xlf_r, f77, fort77, xlf90, xlf90_r, f90, xlf95, xlf95_r, f95, xlf2003, xlf2003_r, f2003, xlf2008, xlf2008_r. All invocations with a suffix of _r allow for thread-safe compilation. See the man page below for additional information on the differences in the invocations.

Portland Group Compiler Documentation
(Served from Portland Group Site)

Portland Group Compiler manpages


Intel Compiler Documentation

Intel C compiler
(Only available on Campus)

Intel Fortran compiler
(Only available on Campus)

Samples
(Only available on Campus)

AuN: There are many sample programs in the directory: /opt/intel/2016/samples/en_US
On Mio, see: /opt/intel/2016/parallel_studio_xe_2016.0.047/samples_2016/en

MKL (Math Kernel Library)
(Only available on Campus)

For additional information on the Intel Compilers see:
https://software.intel.com/en-us/intel-software-technical-documentation

Intel Compiler manpages

IBM Compiler Documentation

There are different compilers for the Mc2 compute nodes and the front end node because they have different (but similar) processors. If you build an application with the IBM compilers it will most likely not work on the front end node. If you build an application with the default gcc or gfortran compilers it will most likely not work on the compute nodes. See below for some examples.

Blue Gene Q manpages

Power manpages and reference docs

We have a complete example of build and run both the IBM and gnu compilers on Mc2. To see this example do a wget on Mc2 followed by a tar command. The look at the README file.

mkdir test
cd test
wget http://geco.mines.edu/prototype/How_do_you_build_applications/bgq.tgz
tar -xzf bgq.tgz
cat README

Mc2 gcc and gfortran versions - more information

There are different compilers for the Mc2 compute nodes and the front end node because they have different (but similar) processors. Error messages of the form:


2016-11-08 09:30:30.074 (FATAL) [0xfff81c88f40]
3574:ibm.runjob.client.Job: could not start job: job failed to start
2016-11-08 09:30:30.074 (FATAL) [0xfff81c88f40] 3574:ibm.runjob.client.Job: 
Load failed on R00-ID-J02: Application executable ELF header contains 
invalid value, errno 8 Exec format error

are caused by trying to run an application on the compute nodes that was built with the head node version of the compiler. The opposite can also occur.

Consider the simple Fortran program. It will sum values of sin and print it. The output should be close to 0.

      program sinsum
       integer, parameter:: b8 = selected_real_kind(14)
       real(b8), parameter :: pi = 3.141592653589793239_b8
       real(b8) :: da,a,s,tot
       da=2.0_b8*pi/100.0_b8
       tot=0.0
       do i=0,100
          a=da*i
          s=sin(a)
          tot=tot+s
       end do
       write(*,*)tot
      end program
Or consider the similar C program.
[joeminer@mc2 fort]$ cat dosin.c
#include <stdlib.h>
#include <stdio.h>
#include <math.h>
int main() {

#define pi 3.141592653589793239
  double  da,a,s,tot;
  int i;
  da=2.0*pi/100.0;
  tot=0.0;
  for (i=0;i<=100;i++) {
    a=da*i;
    s=sin(a);
    tot=tot+s;
  }
  printf("%lg\n",tot);
  return(0);
}

We will build these programs with the default gcc (/usr/bin/gcc) and gfortran (/usr/bin/gfortran) compilers and see that they do not work properly on the compute nodes. We will run interactively on a compute node using the "srun" command.

   
[joeminer@mc2 fort]$ which gcc
/usr/bin/gcc

[joeminer@mc2 fort]$ gcc dosin.c -lm -o dosin_c.head

[joeminer@mc2 fort]$ which gfortran
/usr/bin/gfortran

[joeminer@mc2 fort]$ gfortran dosin.f90 -o dosin_f.head

[joeminer@mc2 fort]$ ls -l *head
-rwxrwxr-x 1 joeminer joeminer 7864 Nov  8 09:29 dosin_c.head
-rwxrwxr-x 1 joeminer joeminer 9577 Nov  8 09:30 dosin_f.head


[joeminer@mc2 fort]$ srun -n 1 dosin_c.head
2016-11-08 09:30:30.074 (FATAL) [0xfff81c88f40]
3574:ibm.runjob.client.Job: could not start job: job failed to start
2016-11-08 09:30:30.074 (FATAL) [0xfff81c88f40] 3574:ibm.runjob.client.Job: 
Load failed on R00-ID-J02: Application executable ELF header contains 
invalid value, errno 8 Exec format error

[joeminer@mc2 fort]$ srun -n 1 dosin_f.head
2016-11-08 09:30:36.314 (FATAL) [0xfff8a308f40] 
3586:ibm.runjob.client.Job: could not start job: job failed to start
2016-11-08 09:30:36.314 (FATAL) [0xfff8a308f40] 3586:ibm.runjob.client.Job: 
Load failed on R00-ID-J02: Application executable ELF header contains 
invalid value, errno 8 Exec format error

Next we load the module that points to the compute node versions of the compilers. We will rebuild our applications and then run them on a compute node.


[joeminer@mc2 fort]$ module load PrgEnv/gcc/gcc-4.7.2.bgq

[joeminer@mc2 fort]$ which gcc
/bgsys/drivers/ppcfloor/gnu-linux-4.7.2/powerpc64-bgq-linux/bin/gcc


[joeminer@mc2 fort]$ gcc dosin.c -lm -o dosin_c.comp


[joeminer@mc2 fort]$ which gfortran
/bgsys/drivers/ppcfloor/gnu-linux-4.7.2/powerpc64-bgq-linux/bin/gfortran



[joeminer@mc2 fort]$ gfortran dosin.f90 -o dosin_f.comp

[joeminer@mc2 fort]$ ls -l *comp
-rwxrwxr-x 1 joeminer joeminer 4183006 Nov  8 09:33 dosin_c.comp
-rwxrwxr-x 1 joeminer joeminer 4996830 Nov  8 09:31 dosin_f.comp


[joeminer@mc2 fort]$ srun -n 1 dosin_c.comp
7.34622e-15

[joeminer@mc2 fort]$ srun -n 1 dosin_f.comp
   7.3462205710450375E-015