Unable to fill domain_cfg.nc due to file size limits...?

I am trying to create a domain_cfg.nc file for a high-resolution custom configuration. The domain grid has a size 9000 x 2400 x 31.

Part of the error comes from src/domain.F90 and is shown in the ocean.output file

cfg_write : create the "domain_cfg.nc" file containing all required configuration information
 ~~~~~~~~~
                     iom_nf90_open ~~~ create new file: domain_cfg.nc in WRITE mode
                    ---> domain_cfg.nc OK
           iom_nf90_rp0123d, file: domain_cfg.nc, var: jpiglo define dimension variables done
           iom_nf90_rp0123d, file: domain_cfg.nc, var: jpiglo defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: jpiglo written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: jpjglo defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: jpjglo written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: jpkglo defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: jpkglo written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: jperio defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: jperio written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: ln_zco defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: ln_zco written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: ln_zps defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: ln_zps written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: ln_sco defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: ln_sco written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: ln_isfcav defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: ln_isfcav written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: glamt defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: glamt write dimension variables done
           iom_nf90_rp0123d, file: domain_cfg.nc, var: glamt written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: glamu defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: glamu written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: glamv defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: glamv written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: glamf defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: glamf written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: gphit defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: gphit written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: gphiu defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: gphiu written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: gphiv defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: gphiv written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: gphif defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: gphif written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e1t defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e1t written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e1u defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e1u written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e1v defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e1v written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e1f defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e1f written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e2t defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e2t written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e2u defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e2u written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e2v defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e2v written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e2f defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e2f written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: ff_f defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: ff_f written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: ff_t defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: ff_t written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e3t_1d defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e3t_1d written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e3w_1d defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e3w_1d written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e3t_0 defined ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e3t_0 written ok
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e3u_0 defined ok
 
  ===>>> : E R R O R
 
          ===========
 
 iom_nf90_check : NetCDF: One or more variable sizes violate format constraints
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e3u_0
 
 
  ===>>> : E R R O R
 
          ===========
 
 iom_nf90_check : NetCDF: Operation not allowed in define mode
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e3u_0
 
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e3u_0 written ok
 
  ===>>> : E R R O R
 
          ===========
 
 iom_nf90_check : NetCDF: Operation not allowed in define mode
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e3v_0
 
           iom_nf90_rp0123d, file: domain_cfg.nc, var: e3v_0 defined ok

Using a smaller domain (900 x 240 x 31) is successful.

I have tried adding key_netcdf4 to cpp_DOMAINcfg.fcm and adding the following to the namelist_cfg

!-----------------------------------------------------------------------
&namnc4 ! netcdf4 chunking and compression settings ("key_netcdf4")
!-----------------------------------------------------------------------
nn_nchunks_i = 4 ! number of chunks in i-dimension
nn_nchunks_j = 4 ! number of chunks in j-dimension
nn_nchunks_k = 31 ! number of chunks in k-dimension
!
ln_nc4zip = .true. ! use netcdf4 chunking and compression
!
/

which does not work either.

Is there a work around for this or is it a case of just reducing the domain size?

Run the tool on several processors ?

1 Like

You could also try to change size constraints using ulimit.
This at least solved similiar issues I had with a different tool (REBULD_NEMO) and large grids.