Lagrangian floats problem/question

[schen]

Hi all,
I am new to the lagrangian float option in ROMS. My goal is to release line sources of drifters at different depth (top 8 vertical levels) and repeat the drifter release every hour for 0.5 day. The way I do this is kind of dumb (suggestion?):

in floats.in, I specify

POS = G, C, T, N, Ft0, Fx0, Fy0, Fz0, Fdt, Fdx, Fdy, Fdz

1 0 1 35 8.0d0 96.0d0 41.0d0 40.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0d0 96.0d0 41.0d0 38.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0d0 96.0d0 41.0d0 36.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0d0 96.0d0 41.0d0 34.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0d0 96.0d0 41.0d0 32.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0d0 96.0d0 41.0d0 30.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0d0 96.0d0 41.0d0 28.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0d0 96.0d0 41.0d0 26.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0417d0 96.0d0 41.0d0 40.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0417d0 96.0d0 41.0d0 38.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0417d0 96.0d0 41.0d0 36.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0417d0 96.0d0 41.0d0 34.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0417d0 96.0d0 41.0d0 32.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0417d0 96.0d0 41.0d0 30.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0417d0 96.0d0 41.0d0 28.0d0 0.d0 0.d0 1.0d0 0.d0
1 0 1 35 8.0417d0 96.0d0 41.0d0 26.0d0 0.d0 0.d0 1.0d0 0.d0
............

My understanding is that:

The 1st line of the code will release 35 drifters at i-index = 96, j-index = 24~58 (centered at j=41), vertical level = 40, at the beginning of day 8.

Is my interpretation correct?

Then I do the same thing for Fz0 = 38,36,34,32,30,28,26 and repeat the whole thing for day 8.0417.

Problem:
When I used the above floats.in and use tile of 12 x 8 (on a linux cluster with ifort and openmpi), the model hanged without any apparent error. The screen output just stopped.

When I use tile of 6 x 8, the model ran fine.
When I use tile of 12 x 8 and release only 35 drifters (use only the 1st line of code in floats.in), the model ran fine.

So, I think I must have made mistakes in my floats.in (attached). Could somebody give me hints or suggestions on how to properly do the drifter release?

Your input is appreciated. Thanks
Shih-Nan

[kate]

POS = G, C, T, N, Ft0, Fx0, Fy0, Fz0, Fdt, Fdx, Fdy, Fdz

1 0 1 35 8.0d0 96.0d0 41.0d0 40.0d0 0.d0 0.d0 1.0d0 0.d0

My understanding is that:

The 1st line of the code will release 35 drifters at i-index = 96, j-index = 24~58 (centered at j=41), vertical level = 40, at the beginning of day 8.

You should check your output, but I would expect j-index going 41-75.

Problem:
When I used the above floats.in and use tile of 12 x 8 (on a linux cluster with ifort and openmpi), the model hanged without any apparent error. The screen output just stopped.

When I use tile of 6 x 8, the model ran fine.
When I use tile of 12 x 8 and release only 35 drifters (use only the 1st line of code in floats.in), the model ran fine.

This is the worst kind of problem, because getting a debugger to work on 96 cores might not be practical. I don't see any reason from this to suspect a bad floats.in. After all, it runs with 48 cores. How about trying 72 cores or other tilings of the 96? You might be best off just going with the largest number you can get to work and pretending none of this ever happened. The alternative is looking for ROMS bugs and/or openmpi bugs.

I have run with 9648 floats on a tiling of 8x24 on Linux-pgi. I lay them out like:

Code: Select all

1  1  1  12  0.0d0   178.00        69.50  50.0d0 30.4375d0 0.d0 0.d0 0.d0

i.e., once a month from each location.

[schen]

Hi Kate,
Thanks for your reply.
Yes, you are right. The j-index range is 41-75.

I tried your suggestion to release drifters every Fdt interval (= 0.0417 day).

tile of 6 x 8 works (as before)
tile of 6 x 12 works
But, tile of 12 x 8 still failed. The model ran for 2.5 day and hanged.

[wilkin]

Shih-Nan,

What are the dimensions of your grid Lm x Mm? I'm wondering if on so many processors you are ending up with very small tiles that are triggering the problem.

Also, when you say "the model ran for 2.5 day and hanged" does this behavior repeat whenever you run the exact same configuration, and does this behavior occur both with and without FLOATS active?

[miguel.solano]

Hello,
I am having issues trying to use the lagrangian floats implemented in ROMS.
I compiled ROMS using svn version 655 with #define FLAOTS. My input file
looks like this:

Code: Select all

     Lfloats == T

! Switch to control the printing of floats positions to standard output
! file, [1:Ngrids].
!
      Fprint == T

! Flag indicating re-start from previous solution. If FRREC = 0, a new
! NetCDF output file is created.

       FRREC == 0

! Number of floats to release in each nested grid.  These values are
! essential because the FLOATS structure in "mod_floats" is dynamically
! allocated using these values, [1:Ngrids].

     NFLOATS == 4

! Initial floats locations for all grids:
!
!   G      Nested grid number
!   C      Initial horizontal coordinate type (0: grid units, 1: spherical)
!   T      Float trajectory type (1: Lagrangian, 2: isobaric, 3: Geopotential)
!   N      Number floats to be released at (Fx0,Fy0,Fz0)
!   Ft0    Float release time (days) after model initialization
!   Fx0    Initial float X-location (grid units or longitude)
!   Fy0    Initial float Y-location (grid units or latitude)
!   Fz0    Initial float Z-location (grid units or depth)
!   Fdt    Float cluster release time interval (days)
!   Fdx    Float cluster X-distribution parameter
!   Fdy    Float cluster Y-distribution parameter
!   Fdz    Float cluster Z-distribution parameter

POS = G, C, T, N,    Ft0,      Fx0,      Fy0,    Fz0,   Fdt,    Fdx,    Fdy,   Fdz

      1  1  1  1  2.62819d0 -67.466d0 18.381d0  -5.d0   0.d0    0.d0   0.0d0  0.0d0
      1  1  1  1  2.69780d0 -67.432d0 18.229d0  -5.d0   0.d0    0.d0   0.0d0  0.0d0
      1  1  1  1  2.59275d0 -67.433d0 18.227d0  -5.d0   0.d0    0.d0   0.0d0  0.0d0
      1  1  1  1  2.57310d0 -67.432d0 18.228d0  -5.d0   0.d0    0.d0   0.0d0  0.0d0

It produces the the floats.nc output file but with Nan's on the lat and lon variables.

I then tried re-compiling the code using the latest version (i.e. revision 757) and now I get another error.

Code: Select all

Output/Input Files:

             Output Restart File:  ocean_rst.nc
        Prefix for History Files:  ocean_his1
            Output Averages File:  ocean_avg.nc
            Output Stations File:  ocean_sta1.nc
              Output Floats File:  ocean_flt1.nc
                 Input Grid File:  ../PRE1/ROMS_FILES/roms_grdPR.nc
    Input Nonlinear Initial File:  ../PRE1/ROMS_FILES/roms_iniPR.nc

 READ_PHYPAR - could not find input file:  ^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@

I'm confused since I did not change any of the ROMS file names in the ROMS input file.
I heard from a fellow ROMS user that he also had some problems using FLOATS under some code revisions. Has anyone had a similar problem?
Thanks in advance!
-MS