ROMS blowing up problem
ROMS blowing up problem
Hi,
I ran ROMS with a blowing-up problem.
Found Error: 01 Line: 298 Source: ROMS/Nonlinear/main3d.F
Found Error: 01 Line: 298 Source: ROMS/Drivers/nl_ocean.h
Blowing-up: Saving latest model state into RESTART file
WRT_RST - wrote re-start fields (Index=2,2) in record = 0000003
Does anyone know why this happen? I checked the main3d.F, it gives:
! Compute horizontal mass fluxes (Hz*u/n and Hz*v/m), density related
! quatities and report global diagnostics.
!-----------------------------------------------------------------------
!
DO ig=1,GridsInLayer(nl)
ng=GridNumber(ig,nl)
DO tile=first_tile(ng),last_tile(ng),+1
CALL set_massflux (ng, tile, iNLM)
# ifndef TS_FIXED
CALL rho_eos (ng, tile, iNLM)
# endif
CALL diag (ng, tile)
# ifdef TLM_CHECK
CALL nl_dotproduct (ng, tile, Lnew(ng))
# endif
END DO
!$OMP BARRIER
END DO
IF (FoundError(exit_flag, NoError, __LINE__, &
& __FILE__)) RETURN
I ran ROMS with a blowing-up problem.
Found Error: 01 Line: 298 Source: ROMS/Nonlinear/main3d.F
Found Error: 01 Line: 298 Source: ROMS/Drivers/nl_ocean.h
Blowing-up: Saving latest model state into RESTART file
WRT_RST - wrote re-start fields (Index=2,2) in record = 0000003
Does anyone know why this happen? I checked the main3d.F, it gives:
! Compute horizontal mass fluxes (Hz*u/n and Hz*v/m), density related
! quatities and report global diagnostics.
!-----------------------------------------------------------------------
!
DO ig=1,GridsInLayer(nl)
ng=GridNumber(ig,nl)
DO tile=first_tile(ng),last_tile(ng),+1
CALL set_massflux (ng, tile, iNLM)
# ifndef TS_FIXED
CALL rho_eos (ng, tile, iNLM)
# endif
CALL diag (ng, tile)
# ifdef TLM_CHECK
CALL nl_dotproduct (ng, tile, Lnew(ng))
# endif
END DO
!$OMP BARRIER
END DO
IF (FoundError(exit_flag, NoError, __LINE__, &
& __FILE__)) RETURN
Re: ROMS blowing up problem
It is from that call to diag, which checks the status of some model fields. If it finds things have gone out of bounds, it dies like that. You should look at that third record of the restart file to see what is going bad.
Re: ROMS blowing up problem
kate wrote:It is from that call to diag, which checks the status of some model fields. If it finds things have gone out of bounds, it dies like that. You should look at that third record of the restart file to see what is going bad.
Yes, I found that the velocity went bad and exceeded 20 m/s.
Thanks, kate!
Re: ROMS blowing up problem
Hi,
I have exactly the same problem : my model blows up after a few time steps and I found with the restart file than the velocity is around 20 m/s in some points. This problem appeared when I used a 1/8° resolution instead of a 1/4° resolution.
So my question is : where does it come from ? And what did you do to fix this problem ? I tried to reduce dt and it works but only for dt lower than 10 which is not convenient for a long simulation. So I decided to run with dt=10 for a few days and then use the restart file to launch another simulation with a longer time step but the model still blows up.
Thank you in advance for your help.
I have exactly the same problem : my model blows up after a few time steps and I found with the restart file than the velocity is around 20 m/s in some points. This problem appeared when I used a 1/8° resolution instead of a 1/4° resolution.
So my question is : where does it come from ? And what did you do to fix this problem ? I tried to reduce dt and it works but only for dt lower than 10 which is not convenient for a long simulation. So I decided to run with dt=10 for a few days and then use the restart file to launch another simulation with a longer time step but the model still blows up.
Thank you in advance for your help.
Re: ROMS blowing up problem
Have you looked to see where things go bad? It might help you figure out how to get a longer timestep.
Re: ROMS blowing up problem
Have you checked the value of your CFL criteria?
If your Courant Number is too high, that means your grid can be in adapted. If you increased your horizontal resolution, you have to decrease your time step (That's information that Courant Number can provide). Some clues can be find there : viewtopic.php?f=14&t=4560&p=17748&hilit ... ber#p17748
Do you know where you're velocity became too high? If it appends on the sea floor (as it did for me), that may your bathymetry which is no more adapted to your new resolution. Do you see angular topography as you check your grid file? Maybe you should smooth it more ? I think that rx0 criteria can inform you on that point (in the print of your model). I think that it depicts a relation between your horizontal, your vertical resolution and your bathymetry (is your bottom rough or steep ? ).
Mine looks that :
in the print you can read as your model is running (and that I put in a 'log_file').Minimum barotropic Courant Number = 1.17951147E-02
Maximum barotropic Courant Number = 4.07452685E-01
If your Courant Number is too high, that means your grid can be in adapted. If you increased your horizontal resolution, you have to decrease your time step (That's information that Courant Number can provide). Some clues can be find there : viewtopic.php?f=14&t=4560&p=17748&hilit ... ber#p17748
Do you know where you're velocity became too high? If it appends on the sea floor (as it did for me), that may your bathymetry which is no more adapted to your new resolution. Do you see angular topography as you check your grid file? Maybe you should smooth it more ? I think that rx0 criteria can inform you on that point (in the print of your model). I think that it depicts a relation between your horizontal, your vertical resolution and your bathymetry (is your bottom rough or steep ? ).
Mine looks that :
I've read that a rx0<0.1 was good enough. In my case, as my bathymetry was not enough accurated (lower resolution)I had to smooth it a lot... And it solves my blowing up problem !Maximum grid stiffness ratios: rx0 = 1.000000E-02 (Beckmann and Haidvogel)
rx1 = 2.168223E+00 (Haney)
Re: ROMS blowing up problem
First, thank you for your answers !
The velocity seem to be go bad on the boundaries (from the bottom to the surface), I suspect my tidal forcing to be at the root of the problem, but I don’t really know how.
For the CFL criteria, I have :
Minimum barotropic Courant Number = 7.50545369E-03
Maximum barotropic Courant Number = 1.50438817E-01
For the bathymetry, I tried two :
rx0 = 2.500000E-01 (Beckmann and Haidvogel)
rx1 = 1.023680E+01 (Haney)
And another one more smooth :
rx0 = 1.000000E-01 (Beckmann and Haidvogel)
rx1 = 4.292327E+00 (Haney)
The model blows up for both of them. But your bathymetry seems to be even smoother so maybe I should try to smooth my bathymetry even more to see what happens.
Thank you again for your help !
The velocity seem to be go bad on the boundaries (from the bottom to the surface), I suspect my tidal forcing to be at the root of the problem, but I don’t really know how.
For the CFL criteria, I have :
Minimum barotropic Courant Number = 7.50545369E-03
Maximum barotropic Courant Number = 1.50438817E-01
For the bathymetry, I tried two :
rx0 = 2.500000E-01 (Beckmann and Haidvogel)
rx1 = 1.023680E+01 (Haney)
And another one more smooth :
rx0 = 1.000000E-01 (Beckmann and Haidvogel)
rx1 = 4.292327E+00 (Haney)
The model blows up for both of them. But your bathymetry seems to be even smoother so maybe I should try to smooth my bathymetry even more to see what happens.
Thank you again for your help !
Re: ROMS blowing up problem
Actually, I have a similar problem : my velocities, temperature (my ocean will boil soon !!!) and salinity behave abnormal near the boundaries (although the boundaries seem OK !?!)
You wrote :
I did not yet activate the tide but only the open lateral boundaries, and I see some similar problems. I have temperature, salt and velocity oscillating from the boundaries to the inner domain. I've tried to add a sponge layer (with increasing diffusivity and viscosity). But that does not seem enough...
Do you use sponge layer?
You wrote :
Have you tried without your tide forcing (just to verify whether your problem really comes from that...)? What if you deactivated them?The velocity seem to be go bad on the boundaries (from the bottom to the surface), I suspect my tidal forcing to be at the root of the problem, but I don’t really know how.
I did not yet activate the tide but only the open lateral boundaries, and I see some similar problems. I have temperature, salt and velocity oscillating from the boundaries to the inner domain. I've tried to add a sponge layer (with increasing diffusivity and viscosity). But that does not seem enough...
Do you use sponge layer?
Re: ROMS blowing up problem
Yes, I set up also a sponge in all boundaries with visc_factor and diff_factor up to 10. Maybe we should increase these 2 factors...
And I tried to remove tidal forcing but now, I don't know why, it's worst ! When I start my simulation, I have the following error :
forrtl: severe (64): input conversion error, unit -5, file Internal Formatted Read
27: Image PC Routine Line Source
27: oceanM 000000000085CAD6 for__io_return Unknown Unknown
27: oceanM 000000000087E2BF for_read_int_fmt_ Unknown Unknown
27: oceanM 0000000000839F76 Unknown Unknown Unknown
27: oceanM 0000000000705D6C Unknown Unknown Unknown
27: oceanM 00000000007029A0 Unknown Unknown Unknown
27: oceanM 000000000051799C Unknown Unknown Unknown
27: oceanM 000000000040C7BE Unknown Unknown Unknown
27: oceanM 000000000040BED8 Unknown Unknown Unknown
27: oceanM 000000000040BC9E Unknown Unknown Unknown
27: libc-2.17.so 00002B52D61D6B35 __libc_start_main Unknown Unknown
27: oceanM 000000000040BBA9 Unknown Unknown Unknown
and my log file stops during the reading of input files (apparently just after the reading of the initialisation file whereas I didn't change nothing in my input files !)
And I tried to remove tidal forcing but now, I don't know why, it's worst ! When I start my simulation, I have the following error :
forrtl: severe (64): input conversion error, unit -5, file Internal Formatted Read
27: Image PC Routine Line Source
27: oceanM 000000000085CAD6 for__io_return Unknown Unknown
27: oceanM 000000000087E2BF for_read_int_fmt_ Unknown Unknown
27: oceanM 0000000000839F76 Unknown Unknown Unknown
27: oceanM 0000000000705D6C Unknown Unknown Unknown
27: oceanM 00000000007029A0 Unknown Unknown Unknown
27: oceanM 000000000051799C Unknown Unknown Unknown
27: oceanM 000000000040C7BE Unknown Unknown Unknown
27: oceanM 000000000040BED8 Unknown Unknown Unknown
27: oceanM 000000000040BC9E Unknown Unknown Unknown
27: libc-2.17.so 00002B52D61D6B35 __libc_start_main Unknown Unknown
27: oceanM 000000000040BBA9 Unknown Unknown Unknown
and my log file stops during the reading of input files (apparently just after the reading of the initialisation file whereas I didn't change nothing in my input files !)
Re: ROMS blowing up problem
You can find out what files are being referred to if you recompile with USE_DEBUG. You should get something like what is shown in the wiki.
I have never found a sponge to be helpful at the boundaries. I don't think you need the smoother bathymetry unless the steepest bits are at the boundary and where the boundary trouble is. What are your boundary conditions? Are you using fields from some other source like a global model run? You might want to try that if not.
I have never found a sponge to be helpful at the boundaries. I don't think you need the smoother bathymetry unless the steepest bits are at the boundary and where the boundary trouble is. What are your boundary conditions? Are you using fields from some other source like a global model run? You might want to try that if not.
Re: ROMS blowing up problem
Sorry for the late reply, fix all my errors took me a long time ! I did as you advised me, I turned on USE_DEBUG and I finally found where my errors came from, thank you ! If someone has the same problem I can advice first to update the varinfo.dat file and then, my main error was in my forcing files, my time unit wasn't the same than in the other files (it was 'seconds since 1948-01-01 00:00:0.0' instead of 'seconds since 1948-01-01 00:00:00') which leaded to the following error "Error: forrtl: severe (64): input conversion error, unit -5, file Internal Formatted".
And for my blowing up problem it seems to work also, I just changed the viscosity and the diffusion (I set TNU2=50 instead of TNU2=300 and I set VISC4=-4.0d9 instead of VISC4=-3.0d11) and now I can set dt=150 without any problem.
And for my blowing up problem it seems to work also, I just changed the viscosity and the diffusion (I set TNU2=50 instead of TNU2=300 and I set VISC4=-4.0d9 instead of VISC4=-3.0d11) and now I can set dt=150 without any problem.
-
- Posts: 26
- Joined: Sat Mar 17, 2012 4:54 pm
- Location: CNR - IAMC
- Contact:
Re: ROMS blowing up problem
ElisaSmz wrote:Sorry for the late reply, fix all my errors took me a long time ! I did as you advised me, I turned on USE_DEBUG and I finally found where my errors came from, thank you ! If someone has the same problem I can advice first to update the varinfo.dat file and then, my main error was in my forcing files, my time unit wasn't the same than in the other files (it was 'seconds since 1948-01-01 00:00:0.0' instead of 'seconds since 1948-01-01 00:00:00') which leaded to the following error "Error: forrtl: severe (64): input conversion error, unit -5, file Internal Formatted".
And for my blowing up problem it seems to work also, I just changed the viscosity and the diffusion (I set TNU2=50 instead of TNU2=300 and I set VISC4=-4.0d9 instead of VISC4=-3.0d11) and now I can set dt=150 without any problem.
Hi, I have a similar blowup issue, appearing only when I put some boundaries (closed domain works fine). Did you experience the same?
UPDATE:
Solved, there was an issue in the boundaries I did'nt realize before. Just bad values.
Re: ROMS blowing up problem
Dear uranoscopus,
I have had similar problems, My model blew up at 10 days, apparently due to high water velocity values in historical files near the North and South border.
Have you solved this problem? Could you give me some advice?
Thanks in advance.
Best regards.
zsheng.
I have had similar problems, My model blew up at 10 days, apparently due to high water velocity values in historical files near the North and South border.
Have you solved this problem? Could you give me some advice?
Thanks in advance.
Best regards.
zsheng.
Re: ROMS blowing up problem
I'll say what I always say: try a shorter timestep and look more closely at your solution to see where it is blowing up.
Re: ROMS blowing up problem
Hi Kate,
I had the same problem during running period, my model died after 2 hours forecast.
My error is displayed in the below:
It means that my max speed reach to 20.0066825983225.
How can I do to fix this problem?
Thank you very much!
I had the same problem during running period, my model died after 2 hours forecast.
My error is displayed in the below:
Code: Select all
DIAG speed trouble 20.0066825983225
Found Error: 01 Line: 351 Source: ROMS/Nonlinear/main3d.F
Found Error: 01 Line: 332 Source: ROMS/Drivers/nl_ocean.h
Blowing-up: Saving latest model state into RESTART file
REASON: MaxSpeed = 2.0E+01
How can I do to fix this problem?
Thank you very much!
Re: ROMS blowing up problem
Same answer as above - use a shorter timestep and look more closely at your model output files.
Re: ROMS blowing up problem
Hi Kate,
I have plotted some valuables in the last hst output file but I didn't see somethings went bad in my case or maybe with my limited knowledge about the ROMS model I didn't see that.
Can you kindly help me to find the solution for this blow-up error?
I have tried to use a shorter timestep but it still occurred.
Thank you very much!
I have plotted some valuables in the last hst output file but I didn't see somethings went bad in my case or maybe with my limited knowledge about the ROMS model I didn't see that.
Can you kindly help me to find the solution for this blow-up error?
I have tried to use a shorter timestep but it still occurred.
Thank you very much!
- Attachments
-
- ubar+vbar_in_ocean_hist.png (39.98 KiB) Viewed 31941 times
-
- u_in_ocean_hist.png (70.89 KiB) Viewed 31941 times
-
- temp_in_ocean_hist.png (37.25 KiB) Viewed 31941 times
-
- Lwave_in_ocean_hist.png (123.74 KiB) Viewed 31941 times
-
- Hwave_in_ocean_hist.png (34.13 KiB) Viewed 31941 times
Re: ROMS blowing up problem
How long before the final blow-up is this snapshot? You are running COAWST? I have no experience with modes of blowing up for anything but the ocean component.
For the ocean, it can go from fine to bad in the blink of an eye. A history snapshot some hours before the trouble can look great. I tend to look through the restart record it saves as it's blowing up. Did the velocity get large? Is it u or v or both? It is often just the one, at mid-depth. Looking at surface plots tells you nothing.
For the ocean, it can go from fine to bad in the blink of an eye. A history snapshot some hours before the trouble can look great. I tend to look through the restart record it saves as it's blowing up. Did the velocity get large? Is it u or v or both? It is often just the one, at mid-depth. Looking at surface plots tells you nothing.
Re: ROMS blowing up problem
HI Kate,
Thank you very much for your reply.
My blow-up issue came up after more than 09 hours running as you can see here:
I also displayed u, v momentum component in all of my vertical levels, however I didn't find any abnormal value.
Can you kindly help me to find out the solution here?
I really appreciated your help.
Thank you very much for your reply.
My blow-up issue came up after more than 09 hours running as you can see here:
Code: Select all
Timing for main: time 2017-11-02_21:15:20 on domain 2: 1.68971 elapsed seconds
2230 2017-11-02 21:17:30.00 8.227072E-02 2.282634E+04 2.282642E+04 8.107080E+16
(216,070,02) 1.344279E-04 1.020814E-05 5.272707E-01 2.004695E+01
DIAG speed trouble 20.0469512008091
Found Error: 01 Line: 341 Source: ROMS/Nonlinear/main3d.F
Found Error: 01 Line: 331 Source: ROMS/Drivers/nl_ocean.h, ROMS_run
Blowing-up: Saving latest model state into RESTART file
REASON: MaxSpeed = 2.0E+01
Timing for main: time 2017-11-02_21:15:40 on domain 2: 1.70914 elapsed seconds
WRT_RST - wrote re-start fields (Index=1,1) in record = 3
Can you kindly help me to find out the solution here?
I really appreciated your help.
- Attachments
-
- v_in_ocean_rst.png (101.22 KiB) Viewed 31826 times
-
- u-v_in_ocean_rst.png (102.55 KiB) Viewed 31826 times
-
- u_in_ocean_rst.png (99.59 KiB) Viewed 31826 times
Re: ROMS blowing up problem
Those NE boundary conditions look ghastly. Something is going on there, I don't know what.