DOMAINcfg tool produces crashed domain and mask files

Hello!
I’m trying to create a domain and mask files via DOMAINcfg tool. Coordinate mesh_mask is cut from the ORCA_R025 (min lat ~ 53 N) and I use GEBCO_2022 (min lat ~45 N) as a bathymetry data.

Input coordinates and bathymetry

Screenshot from 2022-12-07 16-36-35

namelist_cfg
!!>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
!! NEMO/OCE :   Configuration namelist_cfg used to overwrite defaults value defined in namelist_ref
!!>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
!! NEMO/OCE  :  1 - Domain & run manager (namrun, namcfg, namdom, namzgr, namzgr_sco )
!!              2 - diagnostics      (namnc4)
!!              3 - miscellaneous    (nammpp, namctl)
!!
!! namelist skeleton : egrep -E '(^/ *$|^! *$|^ *$|&nam.*|!---.*|!! .*|!!==.*|!!>>>.*)' namelist_ref > namelist_skl
!!>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
!-----------------------------------------------------------------------
&namrun        !   parameters of the run
!-----------------------------------------------------------------------
   cn_exp      =  "Arct_ORCA025"  !  experience name

/
!-----------------------------------------------------------------------
&namdom        !   space and time domain (bathymetry, mesh, timestep)
!-----------------------------------------------------------------------
   ln_read_cfg = .false.
   nn_bathy    =    2      ! = 0 compute analyticaly
                           ! = 1 read the bathymetry file
                           ! = 2 compute from external bathymetry
                           ! = 3 compute from parent (if "key_agrif")
   nn_interp   =    1                          ! type of interpolation (nn_bathy =2)
   cn_fcoord   =  'arct_orca025_coord_v2.nc'             ! external coordinates file (jphgr_msh = 0)
   cn_topo     =  'GEBCO_arct_2022.nc'      !  external topo file (nn_bathy =2)
   cn_bath     =  'elevation'          !  topo name in file  (nn_bathy =2)
   cn_lon      =  'lon'                !  lon  name in file  (nn_bathy =2)
   cn_lat      =  'lat'                !  lat  name in file  (nn_bathy =2)
   rn_scale    =   -1                  !  multiplicative factor to account for possibly negative input bathymetry (agrif only) 
   rn_bathy    =    0.                 !  value of the bathymetry. if (=0) bottom flat at jpkm1
   nn_msh      =    1                  !  create (=1) a mesh file or not (=0)
   jphgr_msh   =    0                  !  type of horizontal mesh
   ppglam0     =  999999.0             !  longitude of first raw and column T-point (jphgr_msh = 1)
   ppgphi0     =  999999.0             !  latitude  of first raw and column T-point (jphgr_msh = 1)
   ppe1_deg    =  999999.0             !  zonal      grid-spacing (degrees)
   ppe2_deg    =  999999.0             !  meridional grid-spacing (degrees)
   ppe1_m      =  999999.0             !  zonal      grid-spacing (degrees)
   ppe2_m      =  999999.0             !  meridional grid-spacing (degrees)
   ppsur       =   -4762.96143546300   !  ORCA r4, r2 and r05 coefficients
   ppa0        =     255.58049070440   !  (default coefficients)
   ppa1        =     245.58132232490   !
   ppkth       =      21.43336197938   !
   ppacr       =       3.0             !
   ppdzmin     =  999999.              !  Minimum vertical spacing
   pphmax      =  999999.              !  Maximum depth
   ldbletanh   =  .FALSE.              !  Use/do not use double tanf function for vertical coordinates
   ppa2        =  999999.              !  Double tanh function parameters
   ppkth2      =  999999.              !
   ppacr2      =  999999.              !
/
!-----------------------------------------------------------------------
&namcfg        !   parameters of the configuration
!-----------------------------------------------------------------------
   !
   ln_e3_dep   = .true.    ! =T : e3=dk[depth] in discret sens.
   !                       !      ===>>> will become the only possibility in v4.0
   !                       ! =F : e3 analytical derivative of depth function
   !                       !      only there for backward compatibility test with v3.6
      !                      ! if ln_e3_dep = T
      ln_dept_mid = .false.  ! =T : set T points in the middle of cells
   !                       !
   cp_cfg      =  "orca"   !  name of the configuration
   jp_cfg      =     025   !  resolution of the configuration
   jpidta      =     488   !  1st lateral dimension ( >= jpi )
   jpjdta      =     422   !  2nd    "         "    ( >= jpj )
   jpkdta      =      31   !  number of levels      ( >= jpk )
   Ni0glo      =     488   !  1st dimension of global domain --> i =jpidta
   Nj0glo      =     422   !  2nd    -                  -    --> j  =jpjdta
   jpkglo      =      31
   jperio      =       4   !  lateral cond. type (between 0 and 6)
   ln_use_jattr = .false.  !  use (T) the file attribute: open_ocean_jstart, if present
                           !  in netcdf input files, as the start j-row for reading
   ln_domclo = .false.     ! computation of closed sea masks (see namclo)
/
!-----------------------------------------------------------------------
&namzgr        !   vertical coordinate                                  (default: NO selection)
!-----------------------------------------------------------------------
!-----------------------------------------------------------------------
   ln_zco      = .false.   !  z-coordinate - full    steps
   ln_zps      = .true.    !  z-coordinate - partial steps
   ln_sco      = .false.   !  s- or hybrid z-s-coordinate
   ln_isfcav   = .false.   !  ice shelf cavity             (T: see namzgr_isf)
/
!-----------------------------------------------------------------------
&namzgr_isf    !   isf cavity geometry definition
!-----------------------------------------------------------------------
/
!-----------------------------------------------------------------------
&namzgr_sco    !   s-coordinate or hybrid z-s-coordinate                (default F)
!-----------------------------------------------------------------------
/
!-----------------------------------------------------------------------
&namclo ! (closed sea : need ln_domclo = .true. in namcfg)
!-----------------------------------------------------------------------
/
!-----------------------------------------------------------------------
&namlbc        !   lateral momentum boundary condition                  (default: NO selection)
!-----------------------------------------------------------------------
/
!-----------------------------------------------------------------------
&namagrif      !  AGRIF zoom                                            ("key_agrif")
!-----------------------------------------------------------------------
/
!-----------------------------------------------------------------------
&namnc4        !   netcdf4 chunking and compression settings            ("key_netcdf4")
!-----------------------------------------------------------------------
/

But output files look crashed and ocean.output file has 1 error:

===>>> : E R R O R

      ===========

iom_nf90_check : NetCDF: Index exceeds dimension bound
iom_nf90_g123d , file: GEBCO_arct_2022.nc, var: elevation

bathy_metry in final domain file

umask in final mesh_mask file

Do you have any idea what this error comes from?

Update

We solved error in ocean.output with usage of global GEBCO bathymetry.
But I still have the bad data at the last row of the domain. It looks like that the DOMAINcgf writes wrong coordinates in appropriate variables:

Last (-1) and second to last gphit rows

Screenshot from 2022-12-20 13-44-02

Last (-1) and second to last glamt rows

Screenshot from 2022-12-20 13-49-26

DOMAINcfg run ends with next MPICH message:

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 39159 RUNNING AT hm-vm-nemo
=   EXIT CODE: 9
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Killed (signal 9)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions

Original coordinate mesh is ok:

Original last (-1) and second to last gphit rows

Screenshot from 2022-12-20 14-04-56

Original last (-1) and second to last glamt rows

Screenshot from 2022-12-20 14-02-10

Relevant DOMAINcfg namelist
!!>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
!! NEMO/OCE :   Configuration namelist_cfg used to overwrite defaults value defined in namelist_ref
!!>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
!! NEMO/OCE  :  1 - Domain & run manager (namrun, namcfg, namdom, namzgr, namzgr_sco )
!!              2 - diagnostics      (namnc4)
!!              3 - miscellaneous    (nammpp, namctl)
!!
!! namelist skeleton : egrep -E '(^/ *$|^! *$|^ *$|&nam.*|!---.*|!! .*|!!==.*|!!>>>.*)' namelist_ref > namelist_skl
!!>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
!-----------------------------------------------------------------------
&namrun        !   parameters of the run
!-----------------------------------------------------------------------
   cn_exp      =  "Arct_ORCA025"  !  experience name

/
!-----------------------------------------------------------------------
&namdom        !   space and time domain (bathymetry, mesh, timestep)
!-----------------------------------------------------------------------
   ln_read_cfg = .false.
   nn_bathy    =    2      ! = 0 compute analyticaly
                           ! = 1 read the bathymetry file
                           ! = 2 compute from external bathymetry
                           ! = 3 compute from parent (if "key_agrif")
   nn_interp   =    1                          ! type of interpolation (nn_bathy =2)
   cn_fcoord   =  'arct_orca025_coord_v2.nc'   ! external coordinates file (jphgr_msh = 0)
   cn_topo     =  'GEBCO_2022.nc'      !  external topo file (nn_bathy =2)
   cn_bath     =  'elevation'          !  topo name in file  (nn_bathy =2)
   cn_lon      =  'lon'                !  lon  name in file  (nn_bathy =2)
   cn_lat      =  'lat'                !  lat  name in file  (nn_bathy =2)
   rn_scale    =    -1                  !  multiplicative factor to account for possibly negative input bathymetry (agrif only) 
   rn_bathy    =    0.                 !  value of the bathymetry. if (=0) bottom flat at jpkm1
   nn_msh      =    1                  !  create (=1) a mesh file or not (=0)
   rn_hmin     =   5.                 !  min depth of the ocean (>0) or min number of ocean level (<0)
   rn_e3zps_min=    2.     !  partial step thickness is set larger than the minimum of
   rn_e3zps_rat=    0.1    !  rn_e3zps_min and rn_e3zps_rat*e3t, with 0<rn_e3zps_rat<1
   jphgr_msh   =    0                  !  type of horizontal mesh
!   ppglam0     =  999999.0             !  longitude of first raw and column T-point (jphgr_msh = 1)
!   ppgphi0     =  999999.0             !  latitude  of first raw and column T-point (jphgr_msh = 1)
!   ppe1_deg    =  999999.0             !  zonal      grid-spacing (degrees)
!   ppe2_deg    =  999999.0             !  meridional grid-spacing (degrees)
!   ppe1_m      =  999999.0             !  zonal      grid-spacing (degrees)
!   ppe2_m      =  999999.0             !  meridional grid-spacing (degrees)
   ppsur       =   -4762.96143546300   !  ORCA r4, r2 and r05 coefficients
   ppa0        =     255.58049070440   !  (default coefficients)
   ppa1        =     245.58132232490   !
   ppkth       =      21.43336197938   !
   ppacr       =       3.0             !
   ppdzmin     =  2.                   !  Minimum vertical spacing
   pphmax      =  5000.               !  Maximum depth
   ldbletanh   =  .FALSE.              !  Use/do not use double tanf function for vertical coordinates
!   ppa2        =  999999.              !  Double tanh function parameters
!   ppkth2      =  999999.              !
!   ppacr2      =  999999.              !
/
!-----------------------------------------------------------------------
&namcfg        !   parameters of the configuration
!-----------------------------------------------------------------------
   !
   ln_e3_dep   = .true.    ! =T : e3=dk[depth] in discret sens.
   !                       !      ===>>> will become the only possibility in v4.0
   !                       ! =F : e3 analytical derivative of depth function
   !                       !      only there for backward compatibility test with v3.6
      !                      ! if ln_e3_dep = T
      ln_dept_mid = .false.  ! =T : set T points in the middle of cells
   !                       !
   cp_cfg      =  "orca"   !  name of the configuration
   jp_cfg      =     025   !  resolution of the configuration
   jpidta      =     488   !  1st lateral dimension ( >= jpi )
   jpjdta      =     418   !  2nd    "         "    ( >= jpj )
   jpkdta      =      31   !  number of levels      ( >= jpk )
   Ni0glo      =     488   !  1st dimension of global domain --> i =jpidta
   Nj0glo      =     418   !  2nd    -                  -    --> j  =jpjdta
   jpkglo      =      31
   jperio      =       4   !  lateral cond. type (between 0 and 6)
   ln_use_jattr = .false.  !  use (T) the file attribute: open_ocean_jstart, if present
                           !  in netcdf input files, as the start j-row for reading
   ln_domclo = .false.     ! computation of closed sea masks (see namclo)
/
!-----------------------------------------------------------------------
&namzgr        !   vertical coordinate                                  (default: NO selection)
!-----------------------------------------------------------------------
!-----------------------------------------------------------------------
   ln_zco      = .false.   !  z-coordinate - full    steps
   ln_zps      = .true.    !  z-coordinate - partial steps
   ln_sco      = .false.   !  s- or hybrid z-s-coordinate
   ln_isfcav   = .false.   !  ice shelf cavity             (T: see namzgr_isf)
/
!-----------------------------------------------------------------------
&namzgr_isf    !   isf cavity geometry definition
!-----------------------------------------------------------------------
/
!-----------------------------------------------------------------------
&namzgr_sco    !   s-coordinate or hybrid z-s-coordinate                (default F)
!-----------------------------------------------------------------------
/
!-----------------------------------------------------------------------
&namclo ! (closed sea : need ln_domclo = .true. in namcfg)
!-----------------------------------------------------------------------
/
!-----------------------------------------------------------------------
&namlbc        !   lateral momentum boundary condition                  (default: NO selection)
!-----------------------------------------------------------------------
/
!-----------------------------------------------------------------------
&namagrif      !  AGRIF zoom                                            ("key_agrif")
!-----------------------------------------------------------------------
/
!-----------------------------------------------------------------------
&nammpp        !   Massively Parallel Processing                        ("key_mpp_mpi")
!-----------------------------------------------------------------------
!   ln_listonly =  .false.  !  do nothing else than listing the best domain decompositions (with land domains suppression)
!   !                       !  if T: the largest number of cores tested is defined by max(mppsize, jpni*jpnj)
/
!-----------------------------------------------------------------------
&namctl        !   Control prints                                       (default: OFF)
!-----------------------------------------------------------------------
   sn_cfctl%l_runstat = .TRUE.
   ln_timing   = .true.          !  timing by routine write out in timing.output file
/
!-----------------------------------------------------------------------
&namnc4        !   netcdf4 chunking and compression settings            ("key_netcdf4")
!-----------------------------------------------------------------------
/

gphit, glamt are indeed more or less duplicated in extreme (masked) ghosts points (interpolation from parent is not possible there). This should not matter ?

Thanks for attention)
I guess this shouldn’t be a problem because all previous i-rows are calculated correctly at masked points. Is it possible to run the DOMAINcfg tool in debug mode?

I’ve run the tool with debug flags (attached below) and got the following backtrace:

Assertion failed in file src/mpid/ch3/channels/nemesis/netmod/tcp/socksm.c at line 599: hdr.pkt_type > == MPIDI_NEM_TCP_SOCKSM_PKT_ID_INFO || hdr.pkt_type == MPIDI_NEM_TCP_SOCKSM_PKT_TMPVC_INFO
0x7fe65fe26707 ???
???:0
0x7fe65fe89f74 ???
???:0
0x7fe65fe8c19e ???
???:0
0x7fe65fe7aff1 ???
???:0
0x7fe65fd72e72 ???
???:0
0x7fe65fd664f4 ???
???:0
0x7fe6608447d6 ???
???:0
0x55df9868e4de __lib_mpp_MOD_mpprecv_dp
/home/hydmet/Build_NEMO/nemo_4.2.0/tools/DOMAINcfg/BLD/ppsrc/nemo/lib_mpp.f90:364
0x55df9845a24f __lbclnk_MOD_mpp_lnk_3d_dp
/home/hydmet/Build_NEMO/nemo_4.2.0/tools/DOMAINcfg/BLD/ppsrc/nemo/lbclnk.f90:3221
0x55df983118b0 zgr_zps
/home/hydmet/Build_NEMO/nemo_4.2.0/tools/DOMAINcfg/BLD/ppsrc/nemo/domzgr.f90:1231
0x55df983432c5 __domzgr_MOD_dom_zgr
/home/hydmet/Build_NEMO/nemo_4.2.0/tools/DOMAINcfg/BLD/ppsrc/nemo/domzgr.f90:201
0x55df9826c8fd __domain_MOD_dom_init
/home/hydmet/Build_NEMO/nemo_4.2.0/tools/DOMAINcfg/BLD/ppsrc/nemo/domain.f90:93
0x55df98255c67 __nemogcm_MOD_nemo_init
/home/hydmet/Build_NEMO/nemo_4.2.0/tools/DOMAINcfg/BLD/ppsrc/nemo/nemogcm.f90:296
0x55df98255cb5 __nemogcm_MOD_nemo_gcm
/home/hydmet/Build_NEMO/nemo_4.2.0/tools/DOMAINcfg/BLD/ppsrc/nemo/nemogcm.f90:108
0x55df98251c80 make_domain_cfg
/home/hydmet/Build_NEMO/nemo_4.2.0/tools/DOMAINcfg/BLD/ppsrc/nemo/make_domain_cfg.f90:28
0x55df98251cc2 main
/home/hydmet/Build_NEMO/nemo_4.2.0/tools/DOMAINcfg/BLD/ppsrc/nemo/make_domain_cfg.f90:21
0x7fe660223082 __libc_start_main
…/csu/libc-start.c:308
0x55df98251b8d ???
???:0
0xffffffffffffffff ???
???:0
internal ABORT - process 49

Debug flags:

%FCFLAGS -fdefault-real-8 -O0 -g -fbacktrace -funroll-all-loops -fcray-pointer -ffree-line-length-none -fcheck=all -finit-real=nan

Can you tell me where to look for the error?

I avoided this problem by using a DOMAINcfg tool from DCM from branch “4.2”.

1 Like

Hi, sorry, I’m trying to create my masks-T, U, and V in a 3D grid. I have my domain_cfg.nc file with all the correct variables, including the 3D settings(e3*) and bathy. However, when I use nn_msh to generate my mesh mask file in the DOMAIN_cfg tool, it doesn’t seem to do anything. I noticed that you have your mesh mask files. Could you please tell me how you obtained them?(P.D. I use 4.0.4 NEMO version)

1 Like

Hello, Elsy.
Now we use the DOMAIN_cfg tool within NEMO v.4.2.2 (whithout DCM) and have not encountered any problems using “create mask” option (nn_msh).
You can check if the nn_msh parameter reaches the program in the ocean.output file.
If the program reads the parameter correctly, but does not create files, then the problem is somewhere in the program and the behavior of the program itself should be studied. In particular, with which messages it ends. We usually do something like this when such problems arise.

1 Like

Thank you very much for your answer, yes, in version 4.0.4 there is a missing line of code which is corrected by adding the following line of code at the end of the “dom_init” subroutine in the “domain.F90” file and now it works:

IF(nmsh/=0) CALL dom_wri!Create mesh_mask file

In the new version 4.2 the line is already included.

P.S. you have to recompile the tool after adding the line

1 Like