Is there anyone to try using Linux to run the model?

[Guest]

Is there anyone to try using Linux to run the model? I have problems compiling the model. I have tried f77, pgf90, pgf77, g77.

[dupontf]

Just in case somebody hasn't replied yet (I hope not!), we did compiled and run ROMS using g77 only on a linux box.

Fred
dupontf@mar.dfo-mpo.gc.ca

[arango]

ROMS version 2.0 is written in F90. So you need to compile the model with a F90 compiler. Several users have used the pgf90 compiler succesfully.

Good luck, H
arango@imcs.rutgers.edu

[nperlin]

Is there anybody working with such a combination of using the recent version of ROMS2.0 running under Linux and MPI? We make an attempt to run the model on a PC cluster, and use RedHat Linux with MPICH, where mpif90 is configured to utilize the pgf90 compiler; Netcd also has been compiled using Portland Group Fortran (pgf77).

The general test case using ocean_upw.in template has successfully completed the run on a single node. The attempt to request two nodes (with the proper tiling set up in the ocean_upw.in, 02x01) results in model blowup at the 88-91 time step. The standard printout shows the kinetic energy growing to high positive, then high negative values (next time step), followed by "-nan" and blowup.

I will appreciate any suggestions or hints how to solve the problem, or what kind of configuration should I try to use!!

Thanks in advance,
Natalie.

[enrique]

I've been running a comibnations of ROMS 2.0/ RedHat Linux/PGI compilers/MPICH with no problems. Some issues can arise with local installations of any of the above packages (and also netCDF). Try plotting some of the output and see whether things are going bad near one of the processor boundaries.

Enrique

[arango]

My experience with Linux computers is that you need to compile the NetCDF library with the same compiler as that for the model. I don't use Rehat Linux but Mandrake Linux in my desktop. I have the PGI compiler (pgf90 version 5.1) and Intel compilers (ifc version 7.1 and ifort version 8.0) installed in my computer. I had to compile the NetCDF library with each one of these compilers in order to get the model to work correctly.

However, I think that your problem is with the MPICH communications since the upwelling example does not IO during initialization. Another candidate is the compiler flags.

Then, I have been able to run the model successfully with these compilers. However if I compile with ifc (version 7.1) and optimize with the -O3 flag, I get the incorrect results and the model will be eventually blows-up. This is definetely a compiler bug. This bug was fixed in ifort (version 8.0). I recommend to compile with the following flags:

• ifc: -tpp7 -xW -ip -O2
  ifort: -tpp7 -xW -ip -O3
  pgf90: -O4 -fastsse -Mvect

Now, compiling the latest version (3.5.1) of the NetCDF library is a little bit tricky. In order to install the NetCDF library correctly with the Intel compilers, please try the following:

• (1) Go to the top-level source directory.
  
  (2) Perform steps 3 and 4 described at:
  
  http://my.unidata.ucar.edu/content/soft ... ngProblems
  
  (3) Set the relevant environment variables. Execute the commands
  
  export CC=gcc
  export CPPFLAGS="-DNDEBUG -DpgiFortran"
  export CFLAGS=-O
  export FC=ifort
  export F90=ifort
  export FLIBS="-Vaxlib"
  export FFLAGS="-O -mp -dynamic-libcxa -cm -w95"
  
  or
  
  setenv CC gcc
  setenv CPPFLAGS "-DNDEBUG -DpgiFortran"
  setenv CFLAGS -O
  setenv FC ifort
  setenv F90 ifort
  setenv FLIBS "-Vaxlib"
  setenv FFLAGS "-O -mp -dynamic-libcxa -cm -w95"
  
  depending on your user-shell.
  
  (4) Perform steps 6 and 9 described at above link.

Good luck, Hernan
arango@imcs.rutgers.edu

[nperlin]

As I undestand, one of the important things is compiling the Netcdf with the same compiler as the model. Here we got some troubles with pgf90 version 5.1-3, it fails passing the Netcdf test during the installation: there are some differences between example_goo.cdl and example.cdl.

The guy from support@unidata.ucar.edu replied that there is a bug in pgf90 compiler version 5.1, that causes this failure [... the second call obtains an array with wrong dimensionality (it's 2-D instead of 3-D) and wrong starting point (it's at the beginning of the "pressure" array instead of halfway through it -- which is why the output values are wrong)].

His recommendation was either not to build Fortran 90 interface, or to use other Fortran 90 compilers, including the older versions of pgf90 (till Portland Group would fix the bug). Therefore, I had to compile Netcdf with Portland Group pgf77 compiler instead of pgf90. Could this be the cause for the model failure, either because the Netcdf is compiled with pgf77, or because the same bug in pgf90 could have affected the ROMS simulation when run in multi-node regime?

Has anybody else used this version of pgf90, v. 5.1-3, and haven't encountered any problems with it?

Thanks!
Natalie.

[arango]

There is not need to build the Fortran 90 interface of the NetCDF library with the PGI compiler. ROMS doesn't use that interface. I was able to build this library with version 5.1-3.

Use the following macros when compiling the NetCDF library with the PGI compiler:

setenv CC gcc
setenv CPPFLAGS "-DNDEBUG -DpgiFortran"
setenv CFLAGS -O
setenv FC pgf90
setenv F90 " "
setenv FLIBS " "
setenv FFLAGS "-O -w"

Notice that the macros F90 and FLIBS are empty. If the F90 macro is empty the Fortran 90 interface is not built. Notice, however, that the macro FC is pgf90 and not pgf77.

Hernan
arango@imcs.rutgers.edu