Option for float output file

[marimy]

Hi all,

I am wondering if there is an option to save float output in several files? Just like the existing option for history files (NDEFHIS) and others.
This would be very useful for long simulations with many particles.

Regards,
Mari

[arango]

No, this is in my to do list for future releases. I assume that you are deploying millions of floats. The floats NetCDF is never that big to require splitting in my applications. The problem with the splitting the float files is that it makes the plotting of the trajectories trickier.

[kate]

I looked into splitting the fish output file, which is similar to the floats output file. One thing that's tricky is the restart. Presumably you want to read the float locations from the last record of the last file. I'll attach the fishy output.F for you to look at.

[marimy]

Currently I am deploying about 500.000 particles. I will probably increase this number when it is possible to split up the float file, which I will be looking forward to!

Thank you Kate, I will take a closer look at your output.F.

Mari

[stone]

Hi, I came across the same problem as Mari met. I follow Kate's suggestion to split float trajectory. And finally it works. Hope it can help rookies like me.

This is what I did.

Code: Select all

In Nonlinear/output.F

ifdef FLOATS  
!
!-----------------------------------------------------------------------
!  If appropriate, process floats NetCDF file.
!-----------------------------------------------------------------------
!
# ifdef MARGINAL_FT
      IF (Nfloats(ng).gt.0.and.(nFLT(ng).gt.0)) THEN
!
!  Create output floats NetCDF file or prepare existing file
!  to append new data to it. Also, notice that it is possible to
!  create several files during a single model run.
!
        IF (LdefFLT(ng)) THEN
          IF (ndefFLT(ng).gt.0) THEN
            IF (idefFLT(ng).lt.0) THEN
              idefFLT(ng)=((ntstart(ng)-1)/ndefFLT(ng))*ndefFLT(ng)
              IF ((ndefFLT(ng).eq.nFLT(ng)).and.                       &
     &                      (idefFLT(ng).le.0)) THEN
                idefFLT(ng)=ndefFLT(ng)         ! one file per record
              ELSE IF (idefFLT(ng).lt.iic(ng)-1) THEN
                idefFLT(ng)=idefFLT(ng)+ndefFLT(ng)
              END IF
            END IF
            IF (ndefFLT(ng).eq.nFLT(ng)) THEN
              ifile=(iic(ng)-1)/ndefFLT(ng)
            ELSE
              ifile=(iic(ng)-1)/ndefFLT(ng)+1
            END IF
!
!  Restart case
!  First, let's check if we need to read from the old file.
!
            IF ((nrrec(ng).ne.0).and.(iic(ng).eq.ntstart(ng))) THEN
              NewFile=.FALSE.
              Ldefine=.TRUE.                  
              IF ((iic(ng)-1).eq.idefFLT(ng)) THEN
                ifile=(iic(ng)-1)/ndefFLT(ng)
              END IF
              idefFLT(ng)=idefFLT(ng)+nFLT(ng)  ! restart offset
            ELSE IF ((iic(ng)-1).eq.idefFLT(ng)) THEN
              idefFLT(ng)=idefFLT(ng)+ndefFLT(ng)
              IF (nFLT(ng).ne.ndefFLT(ng).and.iic(ng).eq.ntstart(ng)) THEN
                idefFLT(ng)=idefFLT(ng)+nFLT(ng)
              END IF
              Ldefine=.TRUE.
              Newfile=.TRUE.
            ELSE
              Ldefine=.FALSE.
            END IF
            IF (Ldefine) THEN
              NrecFLT(ng)=0
              IF (Master) THEN
                lstr=LEN_TRIM(FLTbase(ng))
                WRITE (FLTname(ng),60) FLTbase(ng)(1:lstr-3),ifile
  60            FORMAT (a,'_',i4.4,'.nc')
              END IF
# ifdef DISTRIBUTE
              CALL mp_bcasts (ng, iNLM, FLTname(ng))
# endif
              IF (ncFLTid(ng).ne.-1) THEN
                CALL netcdf_close (ng, iNLM, ncFLTid(ng))
              END IF
              CALL def_floats (ng, Newfile)
              IF (exit_flag.ne.NoError) RETURN
              LwrtFLT(ng)=.TRUE.
            END IF
          ELSE
            IF (iic(ng).eq.ntstart(ng)) THEN
              CALL def_floats (ng, ldefout(ng))
              IF (exit_flag.ne.NoError) RETURN
              LwrtFLT(ng)=.TRUE.
              LdefFLT(ng)=.FALSE.
            END IF
          END IF
        END IF
!
!  Write out data into FLT NetCDF file.
!
        IF ((MOD(iic(ng)-1,nFLT(ng)).eq.0).and.                      &
     &      ((frrec(ng).eq.0).or.(iic(ng).ne.ntstart(ng)))) THEN
          CALL wrt_floats (ng)
          IF (exit_flag.ne.NoError) RETURN
        END IF
      END IF
# else    
      IF (Lfloats(ng).and.                                              &
     &    (Nfloats(ng).gt.0).and.(nFLT(ng).gt.0)) THEN
!
!  Create output floats NetCDF file or prepare existing file to
!  append new data to it.
!
        IF (LdefFLT(ng)) THEN
          IF (frrec(ng).eq.0) THEN
            NewFile=.TRUE.
          ELSE
            NewFile=.FALSE.
          END IF
          CALL def_floats (ng, NewFile)
          IF (exit_flag.ne.NoError) RETURN
          LdefFLT(ng)=.FALSE.
        END IF
!
!  Write out data into floats NetCDF file.
!
        IF ((MOD(iic(ng)-1,nFLT(ng)).eq.0).and.                         &
     &      ((frrec(ng).eq.0).or.(iic(ng).ne.ntstart(ng)))) THEN
          CALL wrt_floats (ng)
          IF (exit_flag.ne.NoError) RETURN
        END IF
      END IF
# endif    ! marginal

# endif    ! floats

In Utility/inp_par.F

ifdef MARGINAL_FT
          ELSE IF (TRIM(KeyWord).eq.'NDEFFLT') THEN
            Npts=load_i(Nval, Rval, Ngrids, ndefFLT)
# endif
          ELSE IF (TRIM(KeyWord).eq.'FLTNAME') THEN
            DO i=1,LEN(FLTname(Nval))
              FLTname(Nval)(i:i)=blank
# ifdef MARGINAL_FT
              FLTbase(Nval)(i:i)=blank
# endif
            END DO
            FLTname(Nval)=TRIM(ADJUSTL(Cval(Nval)))
# ifdef MARGINAL_FT
            FLTbase(Nval)=TRIM(ADJUSTL(FLTname(Nval)))
# endif

In mod_iounits.F
# ifdef MARGINAL_FT
        character (len=256), dimension(Ngrids) :: FLTbase
# endif

In mod_ncparam.F
# ifdef MARGINAL_FT
        integer  :: idefFLT(Ngrids)     ! history file creation flag
# endif
# ifdef MARGINAL_FT
          idefFLT(ng)=-1
# endif

In mod_scalars.F
# ifdef MARGINAL_FT
        logical, dimension(Ngrids) :: LwrtFLT    ! Write history file
# endif
# ifdef MARGINAL_FT
        integer, dimension(Ngrids) :: ndefFLT    ! History file
# endif
# ifdef MARGINAL_FT
        LwrtFLT(ng)=.FALSE.
# endif