ROMS 3.7 nesting problems

[Aljaz]

Hi everyone,
I am Aljaz and this is my first post on this forum. The forum is great, however this time I couldn't find the answers related to my grid nesting issues.
As I wrote in the title, I am using ROMS 3.7 and at the moment I cannot upgrade to the latest ROMS version.
I would like to simulate the effects of hydrogenerators on the currents using three grids: one coarse and two telescopic refinements as in the case of the Jersey Lake ADE grids. From now on I will only talk about the Lake Jersey ADE case because the errors are the same as in my case.
I managed to successfully run the Lake Jersey AD, ABD, ABDC cases and unsuccessfully the ADE and ACDE case where the second refined grid is placed on top of the first.
The blowup of the ADE case produces the following message and other details are in the "log_file.log":

At line 3539 of file nesting.f90
Fortran runtime error: Index '-999' of dimension 1 of array 'refined%on_u' below lower bound of 1

Error termination. Backtrace:
#0 0x7f3a8403b2ed in ???
#1 0x7f3a8403bed5 in ???
#2 0x7f3a8403c2a7 in ???
#3 0x55ee52402aeb in put_refine2d
at /mnt/B6E4B942E4B905A1/ROMS_Tidal_Array_master/Projects/Nesting/Lake_Jersey/Build_romsG/nesting.f90:3539
#4 0x55ee52459d92 in put_refine
at /mnt/B6E4B942E4B905A1/ROMS_Tidal_Array_master/Projects/Nesting/Lake_Jersey/Build_romsG/nesting.f90:1458
#5 0x55ee524866e7 in __nesting_mod_MOD_nesting
at /mnt/B6E4B942E4B905A1/ROMS_Tidal_Array_master/Projects/Nesting/Lake_Jersey/Build_romsG/nesting.f90:190
#6 0x55ee52344811 in main3d_
at /mnt/B6E4B942E4B905A1/ROMS_Tidal_Array_master/Projects/Nesting/Lake_Jersey/Build_romsG/main3d.f90:296
#7 0x55ee522328f3 in __ocean_control_mod_MOD_roms_run
at /mnt/B6E4B942E4B905A1/ROMS_Tidal_Array_master/Projects/Nesting/Lake_Jersey/Build_romsG/ocean_control.f90:173
#8 0x55ee52231fa8 in ocean
at /mnt/B6E4B942E4B905A1/ROMS_Tidal_Array_master/Projects/Nesting/Lake_Jersey/Build_romsG/master.f90:108
#9 0x55ee52231ff2 in main
at /mnt/B6E4B942E4B905A1/ROMS_Tidal_Array_master/Projects/Nesting/Lake_Jersey/Build_romsG/master.f90:50
-------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

Process name: [[5112,1],0]
Exit code: 2
--------------------------------------------------------------------------

The second problem related to nesting is that I noticed that the temperature interpolation of the nested grid boundary cell in contact with the shore is not correct as can be seen in Figure 1. The bathymetry in this case is flat and the water temperature is uniform for the entire water column. The error is difficult to notice, but with a constant flow along the coast, it can produce a deceptive inflow of cold water, as can be seen in Figure 2.

I would really appreciate some help solving these issues.
Any ideas what it could do?
Thanks in advance!

[wilkin]

That looks a MPI problem rather than a ROMS problem. You're only using only tile, so try compiling and running in serial mode instead of parallel.

I think those temperature plots (from the initial conditions?) might be an artifact of shading interp.

[Aljaz]

Hi, thanks for the suggestion.
Tried running in serial mode and there was no difference in the results and it is strange that all other Lake Jersey nesting cases are working fine except those where the second refined grid is above the first.

The temperature graphs are from the first steps of the calculation and the initial conditions are analytical, where the temperature is uniform.

Please help me with some more tips.

Thank you!

Aljaz

[jcwarner]

i am not familiar with this test case but maybe you should re-create the connectivity file. that error should not occur if the grids and connections are set up correctly.
can you show a plot of the 3 grids, and the outline of grid 2 and grid 3? how much space is there between the perimeter of grid 3 to the boundary of grid 2, and between the perimeter of grid 2 and the edges of grid 1?
-j

[Aljaz]

Here I am talking about the ROMS Lake Jersey ADE test case. I didn't create the connectivity file, I just use the files as sent in the example:
https://www.myroms.org/wiki/Lake_Jersey ... xample_ADE

But also when I create my own connectivity file I get the same results.

[arango]

I don’t know what you are doing wrong. First, I highly recommend that you use the latest version of ROMS. Right now, we are in version 3.9. We have improved ROMS nesting. It is more efficient. The latest version of ROMS is consistent with all the files in the current test repository. That’s the reason that you should use the current ROMS trunk version. A lot of things have changed since then. You need to use the distributed nested connectivity NetCDF files in the test repository.

I just ran the test case ADE with 8 processors, and the solutions are correct, reproducing the figures available in WikiROMS. Also, I ran the case ACDE with 12 processors with similar results.

[Aljaz]

Thanks for the reply. Yes, I also run the ADE case on the newer version of ROMS and it works perfectly. Unfortunately, I cannot upgrade at the moment because I am working with hydro generators and the code is entangled in 12 scripts, I have already tried updating the code, but at the moment it is out of my league. I think I have a problem mixing different versions of ROMS.

Thanks

[theresiaao]

Sorry for interupting, may I ask, did you use forcing file for the input of this test case running? cause I can't find the forcing file in the repository and I have a problem with that while trying tu run the test case of Lake Jersey ADE, thank you

[Aljaz]

No, I used the analytical forcing.

Cheers

Aljaz