# Segmentation fault on \*nemogcm.f90\* and \*nemo.f90\* **URL:** \ **Category:** Versions\ **Tags:** arch, deps\ **Created:** [4 January 2022 08:06 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169 "2022-01-04T08:06:59Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Jin](https://avatars.discourse-cdn.com/v4/letter/j/4af34b/32.png) [@Jin](https://nemo-ocean.discourse.group/u/Jin)\ **Post date:** [4 January 2022 08:06 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/1 "2022-01-04T08:06:59Z")
Dear all, Hello, I am a beginner and I met some problems when running test case. I’m trying to set up a configuration with GYRE\_PISCES (TOP wasn’t compiled for speed). I encounter this problem. I think it is related with the initialization of the NEMO. Does anyone know how to fix it? ```nohighlight Program received signal SIGSEGV: Segmentation fault - invalid memory reference. Backtrace for this error: #0 0x7f7efb9e3d21 in ??? #1 0x7f7efb9e2ef5 in ??? #2 0x7f7efb68820f in ??? #3 0x7f7efbcf732b in ??? #4 0x55c4fcacb1ff in ??? #5 0x55c4fc6c0736 in ??? #6 0x55c4fc88830e in ??? #7 0x55c4fc48a7ac in ??? #8 0x55c4fbe2fa0c in __nemogcm_MOD_nemo_init at /media/cmlws/Data1/[user]/NEMO/r4.0.6/cfgs/GYRE_testing/BLD/ppsrc/nemo/nemogcm.f90:269 #9 0x55c4fbe329fd in __nemogcm_MOD_nemo_gcm at /media/cmlws/Data1/[user]/NEMO/r4.0.6/cfgs/GYRE_testing/BLD/ppsrc/nemo/nemogcm.f90:165 #10 0x55c4fbe2df61 in nemo at /media/cmlws/Data1/[user]/NEMO/r4.0.6/cfgs/GYRE_testing/WORK/nemo.f90:18 #11 0x55c4fbe2df9a in main at /media/cmlws/Data1/[user]/NEMO/r4.0.6/cfgs/GYRE_testing/WORK/nemo.f90:11 =================================================================================== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = PID 1671418 RUNNING AT cmlws = EXIT CODE: 139 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =================================================================================== YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) This typically refers to a problem with your application. Please see the FAQ page for debugging suggestions ``` For information my arch file: ```perl %NCDF_HOME /home/[user]/anaconda3 %HDF5_HOME /home/[user]/anaconda3 %XIOS_HOME /media/cmlws/Data1/[user]/xios-2.5 %OASIS_HOME /not/defined %NCDF_INC -I%NCDF_HOME/include -I%HDF5_HOME/include %NCDF_LIB -L%NCDF_HOME/lib -lnetcdff -lnetcdf -lstdc++ %XIOS_INC -I%XIOS_HOME/inc %XIOS_LIB -L%XIOS_HOME/lib -lxios -L/usr/lib/gcc/x86_64-linux-gnu/9 -lstdc++ %OASIS_INC -I%OASIS_HOME/build/lib/mct -I%OASIS_HOME/build/lib/psmile.MPI1 %OASIS_LIB -L%OASIS_HOME/lib -lpsmile.MPI1 -lmct -lmpeu -lscrip %CPP /usr/bin/cpp-9 %CPPFLAGS -P -traditional %FC /usr/bin/mpif90 %FCFLAGS -fdefault-real-8 -funroll-all-loops -cpp -fcray-pointer -ffree-line-length-none -g -O0 -fbacktrace %FFLAGS %FCFLAGS #%LD /usr/bin/mpif90 -Wl,-rpath=$HOME/INSTALL/lib:/usr/lib %LD /usr/bin/mpif90 %LDFLAGS -L/usr/lib/x86_64-linux-gnu %FPPFLAGS -P -C -traditional %AR ar %ARFLAGS -rs %MK make %USER_INC %XIOS_INC %OASIS_INC %NCDF_INC %USER_LIB %XIOS_LIB %OASIS_LIB %NCDF_LIB %CC cc %CFLAGS -O0 -fbacktrace ``` Thank you in advance. Best regards, Hwa-Jin Choi ---
**Author:** ![robinson](https://avatars.discourse-cdn.com/v4/letter/r/848f3c/32.png) [@robinson](https://nemo-ocean.discourse.group/u/robinson)\ **Post date:** [5 January 2022 13:37 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/2 "2022-01-05T13:37:25Z")
Hei Jin, What do you mean by “TOP was not compiled for speed” ? Can you run GYRE without PISCES ? Cheers, Robinson ---
**Author:** ![smasson](https://avatars.discourse-cdn.com/v4/letter/s/eb9ed0/32.png) [@smasson](https://nemo-ocean.discourse.group/u/smasson)\ **Post date:** [7 January 2022 08:52 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/3 "2022-01-07T08:52:35Z")
Did you have a look at _ocean.output_? Do you see the `E R R O R` flag? What is the contents of the line 269 of _…/r4.0.6/cfgs/GYRE\_testing/BLD/ppsrc/nemo/nemogcm.f90_ ? Sébastien ---
**Author:** ![Jin](https://avatars.discourse-cdn.com/v4/letter/j/4af34b/32.png) [@Jin](https://nemo-ocean.discourse.group/u/Jin)\ **Post date:** [10 January 2022 05:06 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/4 "2022-01-10T05:06:17Z")
Robinson, I’m sorry. I stand corrected. In _/cfgs/GYRE\_PISCES/cpp\_GYRE\_PISCES.fcm_, I replaced `key_top` with `key_nosignedzero`. I referred to this site. [https://nemo-related.readthedocs.io/en/latest/compilation\_notes/nemo40.html](https://nemo-related.readthedocs.io/en/latest/compilation_notes/nemo40.html) Thank you for your interest. Best regards, Hwa-Jin ---
**Author:** ![Jin](https://avatars.discourse-cdn.com/v4/letter/j/4af34b/32.png) [@Jin](https://nemo-ocean.discourse.group/u/Jin)\ **Post date:** [10 January 2022 09:08 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/5 "2022-01-10T09:08:55Z")
@smasson Thank you for your advices. I can’t find the file… maybe I guess it wasn’t created. Can you tell me where the ocean.output is located? The content of the line 269 of _nemogcm.f90_ is about the `IF( lk_oasis ) THEN` ```fortran SUBROUTINE nemo_init !!---------------------------------------------------------------------- !! ***ROUTINE nemo_init*** !! !! ** Purpose : initialization of the NEMO GCM !!---------------------------------------------------------------------- INTEGER :: ios, ilocal_comm ! local integers !! NAMELIST/namctl/ ln_ctl , sn_cfctl, nn_print, nn_ictls, nn_ictle, & & nn_isplt , nn_jsplt, nn_jctls, nn_jctle, & & ln_timing, ln_diacfl NAMELIST/namcfg/ ln_read_cfg, cn_domcfg, ln_closea, ln_write_cfg, cn_domcfg_out, ln_use_jattr !!---------------------------------------------------------------------- ! cxios_context = 'nemo' ! ! !-------------------------------------------------! ! ! set communicator & select the local rank ! ! ! must be done as soon as possible to get narea ! ! !-------------------------------------------------! ! IF( Agrif_Root() ) THEN IF( lk_oasis ) THEN CALL cpl_init( "oceanx", ilocal_comm ) ! nemo local communicator given by oasis CALL xios_initialize( "not used" , local_comm =ilocal_comm ) ! send nemo communicator to xios ELSE CALL xios_initialize( "for_xios_mpi_id", return_comm=ilocal_comm ) ! nemo local communicator given by xios ENDIF ENDIF ``` Is there any way to fix this problem? Thank you again for your time. Best regards, Hwa-Jin ---
**Author:** ![smasson](https://avatars.discourse-cdn.com/v4/letter/s/eb9ed0/32.png) [@smasson](https://nemo-ocean.discourse.group/u/smasson)\ **Post date:** [11 January 2022 08:31 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/6 "2022-01-11T08:31:08Z")
Very strange… The error appears at the early beginning at the simulation. almost nothing has been done. It looks like the code was not properly compiled. `lk_oasis` should be defined and initialized to .false. in _…/r4.0.6/cfgs/GYRE\_testing/BLD/ppsrc/nemo/sbc\_oce.f90_. Do you see this line: ```fortran LOGICAL , PUBLIC :: lk_oasis = .FALSE. ``` ---
**Author:** ![Jin](https://avatars.discourse-cdn.com/v4/letter/j/4af34b/32.png) [@Jin](https://nemo-ocean.discourse.group/u/Jin)\ **Post date:** [12 January 2022 07:03 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/7 "2022-01-12T07:03:39Z")
Smasson Thanks to your comment, I rechecked the compiled code. I was suspicious of installation of xios, so I reinstalled xios, the segment fault error was passed. I think the xios and NEMO weren’t configured to be consistent with the compilers. Thanks for your help 😀 Hwa-Jin ---
**Author:** ![Phil](https://yyz2.discourse-cdn.com/free1/user_avatar/nemo-ocean.discourse.group/phil/32/103_2.png) [@Phil](https://nemo-ocean.discourse.group/u/Phil)\ **Post date:** [12 August 2022 15:36 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/10 "2022-08-12T15:36:09Z")
Greetings, Jin I faced the same Segmentation Fault due ORCA2\_ICE\_PISCES configration run, the problem described here [https://nemo-ocean.discourse.group/t/segmentation-fault-due-to-new-configuration-run-nemo/342/8](https://nemo-ocean.discourse.group/t/segmentation-fault-due-to-new-configuration-run-nemo/342/8) Could you suggest any steps, based on your experience? ---
**Author:** ![Jin](https://avatars.discourse-cdn.com/v4/letter/j/4af34b/32.png) [@Jin](https://nemo-ocean.discourse.group/u/Jin)\ **Post date:** [16 August 2022 04:26 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/11 "2022-08-16T04:26:37Z")
Hello Phil, I think your code was not properly compiled. As I mentioned above, I reinstalled XIOS, the error was passed. If the code stops before running, I check that XIOS is installed properly, the library paths of NEMO is set correctly. It is also necessary to compile the netcdf and hdf libraries with the same version of the MPI implementation that both NEMO and XIOS have been compiled and linked with. Hope this helps, Hwa Jin Choi ---
**Author:** ![Phil](https://yyz2.discourse-cdn.com/free1/user_avatar/nemo-ocean.discourse.group/phil/32/103_2.png) [@Phil](https://nemo-ocean.discourse.group/u/Phil)\ **Post date:** [16 August 2022 10:38 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/12 "2022-08-16T10:38:40Z")
Thank you for your answer. All parts of puzzle with nemo segmentation fault should be accomplished soon) Btw, the reinstallation of XIOS required full nemo reinstallation (recompilation) or it’s a separate process that does not affect the other structure? ---
**Author:** ![Jin](https://avatars.discourse-cdn.com/v4/letter/j/4af34b/32.png) [@Jin](https://nemo-ocean.discourse.group/u/Jin)\ **Post date:** [17 August 2022 02:53 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/13 "2022-08-17T02:53:29Z")
Hello Phil It is recommended to recompile XIOS and then recompile nemo. Hwa Jin Choi ---
**Author:** ![Phil](https://yyz2.discourse-cdn.com/free1/user_avatar/nemo-ocean.discourse.group/phil/32/103_2.png) [@Phil](https://nemo-ocean.discourse.group/u/Phil)\ **Post date:** [5 August 2022 16:13 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/14 "2022-08-05T16:13:51Z")
Greetings all. I ran successfully the ORCA2\_SAS\_ICE configuration (followed by ARC36 config manual). However, I have a segmentation fault every time when I run ./nemo from a new configuration directory. What I’ve done: 1. New configuration created by following ```shell $ ./makenemo -n 'MY_NEW_TEST' -r 'ORCA2_ICE_PISCES' -m 'gfortran_test' ``` I’ve chosen the ORCA2\_ICE\_PISCES, because i need ocean and ice dynamics together 1. Tried to compile with different CPP KEYS: `key_oce`, `key_mpi2` etc 2. Copied all necessary files in _EXP00_ folder (_domain\_cfg.nc_, forcing, initial etc.) 3. Ran _./nemo_ with command ```shell $ mpirun -n '26' ./nemo ``` 1. Segmentation fault The screenshots are attached here for better understanding. Hoping for your help to newbie in NEMO modelling. Please suggest the ways to find an error, is it in CPP KEYS issue? Or it’s a gfortran compiler issue, maybe there are some special ways - how to compile ICE\_PISCES to run nemo successfully? Kind regards! ![SegmentationFault_Error](https://global.discourse-cdn.com/free1/uploads/nemo_ocean/original/1X/a17a4f0ee6e2f473859a74b4c6f2d7689e569ae0.png) ---
**Author:** ![robinson](https://avatars.discourse-cdn.com/v4/letter/r/848f3c/32.png) [@robinson](https://nemo-ocean.discourse.group/u/robinson)\ **Post date:** [8 August 2022 06:19 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/15 "2022-08-08T06:19:00Z")
Hei, I usually get this kind of error when I lack memory. Hope this helps, Robinson ---
**Author:** ![Phil](https://yyz2.discourse-cdn.com/free1/user_avatar/nemo-ocean.discourse.group/phil/32/103_2.png) [@Phil](https://nemo-ocean.discourse.group/u/Phil)\ **Post date:** [8 August 2022 10:27 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/16 "2022-08-08T10:27:26Z")
Thank you for reply. Unfortunately, I used 26 processors in previous experiment (ORCA\_SAS\_ICE) - everything worked smooth and I had some output at least. However, when i decide to build ocean+ice config (ORCA\_ICE\_PISCES) - i got this issue. I started to think that it might be a structure problem, for example in namelists. What about _namelist\_cfg_ and _namelist\_ref_, should they be similar (i mean all inputs)? ---
**Author:** ![robinson](https://avatars.discourse-cdn.com/v4/letter/r/848f3c/32.png) [@robinson](https://nemo-ocean.discourse.group/u/robinson)\ **Post date:** [8 August 2022 10:41 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/17 "2022-08-08T10:41:12Z")
_namelist\_cfg_ modifies _namelist\_ref_ basically, so the purpose is that they are not the same: one uses ref for 95% of inputs let’s say, and then 5% is modified by cfg. It might be that SAS\_ICE uses a lot less memory than ICE\_PISCES. Can you try to run ORCA\_ICE without PISCES to check ? ---
**Author:** ![Phil](https://yyz2.discourse-cdn.com/free1/user_avatar/nemo-ocean.discourse.group/phil/32/103_2.png) [@Phil](https://nemo-ocean.discourse.group/u/Phil)\ **Post date:** [8 August 2022 10:54 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/18 "2022-08-08T10:54:10Z")
May I include my _namelist\_cfg_ file to make my question clear? ---
**Author:** ![robinson](https://avatars.discourse-cdn.com/v4/letter/r/848f3c/32.png) [@robinson](https://nemo-ocean.discourse.group/u/robinson)\ **Post date:** [8 August 2022 10:55 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/19 "2022-08-08T10:55:56Z")
Sure, or we can chat on zoom if you like and you’ll explain me what you want to do, could be easier… ---
**Author:** ![Phil](https://yyz2.discourse-cdn.com/free1/user_avatar/nemo-ocean.discourse.group/phil/32/103_2.png) [@Phil](https://nemo-ocean.discourse.group/u/Phil)\ **Post date:** [8 August 2022 10:59 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/20 "2022-08-08T10:59:49Z")
Sure, it will be more productive. Appreciate your help. Here is my email: [civilseagull@gmail.com](mailto:civilseagull@gmail.com), type me smth there and ill share a link for zoom if you have time ---
**Author:** ![Phil](https://yyz2.discourse-cdn.com/free1/user_avatar/nemo-ocean.discourse.group/phil/32/103_2.png) [@Phil](https://nemo-ocean.discourse.group/u/Phil)\ **Post date:** [12 August 2022 15:28 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/21 "2022-08-12T15:28:01Z")
There is a problem still exists. Even when I recompiled ORCA2\_ICE\_PISCES. However, test case of this ref. config is also crushing with segmentation fault. Possibly, it’s a problem related to MPI settings, will be grateful for any instruments for MPI diagnostics or some steps to examine the _./nemo_ run. ---
**Author:** ![smasson](https://avatars.discourse-cdn.com/v4/letter/s/eb9ed0/32.png) [@smasson](https://nemo-ocean.discourse.group/u/smasson)\ **Post date:** [18 August 2022 10:27 UTC](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169/22 "2022-08-18T10:27:06Z")
did you look at the lines 158 and 252 in the _…/BLD/ppsrc/nemo/nemogcm.f90_ as suggested by your error message? Which lines is it? [Next page](https://nemo-ocean.discourse.group/t/segmentation-fault-on-nemogcm-f90-and-nemo-f90/169.md?page=2)