My confusion about the model always blow up at the 208 days

[wuting0926]

Dear all,

At first, the model could finish without an obc.nc (open boundary file about temperature and salt). But when I use the obc.nc about temp and salt ( the shape = [39 20 230/300] [temp_time, s_rho, xi_rho] ), the model would always blow up in the 208 days. The temp_time about obc.nc is in the Table 1 (days from initialization). Besides, when I re-start from the RST.nc, the model would blow up immediately. Whatever the time I choose to restart, it would blow up immediately.

Fig 1-8 are the results of the last time before the blowing up time, there are not any abnormal points in these Figs and the ocean.log document see like Fig 9

Can someone give me some suggestions and answer my questions? Thank you very much.

[kate]

Thank you for trying to find your NaN. Even though you didn't find it, it still must be there. Did the model write the crash record to the restart file? Are you plotting that record here? Is your NINFO set to 1 so that you see the trouble as early as possible? Does your plotting software treat NaN like land mask or as something special?

This is what I get with NINFO=1. It doesn't usually get all the way to NaN when it's in trouble:

Code: Select all

  104780 41757 18:33:20  4.673804E-03  1.703768E+04  1.703769E+04  3.288802E+16
         (619,0355,28)  6.821721E-02  2.028960E-01  1.891131E+00  1.749723E+01
  104781 41757 18:35:00  4.677850E-03  1.703768E+04  1.703768E+04  3.288802E+16
         (619,0355,30)  8.245112E-02  2.253907E-01  1.808708E+00  2.143896E+01
 DIAG speed trouble    21.4389575746228      

 Blowing-up: Saving latest model state into  RESTART file

      WRT_RST   - wrote re-start fields (Index=2,2) into time record = 0000003

This happens to me often enough that I made diag.F a little more verbose. In this case, I would not have seen the trouble unless plotting a vector for each point. Here are some relevant plots:

z.png (10.1 KiB) Viewed 2187 times

point.png (10.07 KiB) Viewed 2187 times

For me, the solution is a longer timestep, at least for long enough to get past this.