forrtl: severe (174): SIGSEGV, segmentation fault occurred

[dhtjtks]

Hi all,

When I tried to run ROMS (example case, 'upwelling'), I had this error. I tried to use both netcdf-3.6.3 and netcdf-4.1.1, however, both cases have same errors. What does it mean?

Thanks,
dhtjtks



####################################################################
# Alabama Supercomputer Center - PBS Prologue
# Your job id is : 622741
# Your job name is : runSCRIPT
# Your job's queue is : large-serial
# Your username for this job is : usajxl
# Your group for this job is : usacveng
# Your job used :
# 1 CPU on dmc87
# Your job started at : Mon Jan 23 13:39:42 CST 2012
####################################################################
setting dmc scratch directory

============================================================
===== Summary of your script job =====
============================================================
The script file is: run
The time limit is 240:00:00 HH:MM:SS.
The target directory is: /home/usajxl/jung/upwelling
The working directory is: /tmp/usajxl.runSCRIPT.622741
The memory limit is: 1gb
The job will start running after: 201201231339.03
Look for: runSCRIPT in queue: large-serial


Model Input Parameters: ROMS/TOMS version 3.6
Monday - January 23, 2012 - 1:39:44 PM
-----------------------------------------------------------------------------

Wind-Driven Upwelling/Downwelling over a Periodic Channel

Operating system : Linux
CPU/hardware : x86_64
Compiler system : ifort
Compiler command : /apps/intel/Compiler/11.1/046/bin/intel64/ifort
Compiler flags : -heap-arrays -fp-model precise -ip -O3 -free -free -free

SVN Root URL : https://www.myroms.org/svn/src/trunk
SVN Revision :

Local Root : /home/usajxl/ROMS/src
Header Dir : /home/usajxl/jung/upwelling
Header file : upwelling.h
Analytical Dir: /home/usajxl/ROMS/src/ROMS/Functionals

Resolution, Grid 01: 0041x0080x016, Parallel Threads: 1, Tiling: 001x001


Physical Parameters, Grid: 01
=============================

1440 ntimes Number of timesteps for 3-D equations.
300.000 dt Timestep size (s) for 3-D equations.
30 ndtfast Number of timesteps for 2-D equations between
each 3D timestep.
1 ERstr Starting ensemble/perturbation run number.
1 ERend Ending ensemble/perturbation run number.
0 nrrec Number of restart records to read from disk.
T LcycleRST Switch to recycle time-records in restart file.
288 nRST Number of timesteps between the writing of data
into restart fields.
1 ninfo Number of timesteps between print of information
to standard output.
T ldefout Switch to create a new output NetCDF file(s).
72 nHIS Number of timesteps between the writing fields
into history file.
1 ntsAVG Starting timestep for the accumulation of output
time-averaged data.
72 nAVG Number of timesteps between the writing of
time-averaged data into averages file.
1 ntsDIA Starting timestep for the accumulation of output
time-averaged diagnostics data.
72 nDIA Number of timesteps between the writing of
time-averaged data into diagnostics file.
0.0000E+00 nl_tnu2(01) NLM Horizontal, harmonic mixing coefficient
(m2/s) for tracer 01: temp
0.0000E+00 nl_tnu2(02) NLM Horizontal, harmonic mixing coefficient
(m2/s) for tracer 02: salt
5.0000E+00 nl_visc2 NLM Horizontal, harmonic mixing coefficient
(m2/s) for momentum.
1.0000E-06 Akt_bak(01) Background vertical mixing coefficient (m2/s)
for tracer 01: temp
1.0000E-06 Akt_bak(02) Background vertical mixing coefficient (m2/s)
for tracer 02: salt
1.0000E-05 Akv_bak Background vertical mixing coefficient (m2/s)
for momentum.
3.0000E-04 rdrg Linear bottom drag coefficient (m/s).
3.0000E-03 rdrg2 Quadratic bottom drag coefficient.
2.0000E-02 Zob Bottom roughness (m).
2 Vtransform S-coordinate transformation equation.
4 Vstretching S-coordinate stretching function.
3.0000E+00 theta_s S-coordinate surface control parameter.
0.0000E+00 theta_b S-coordinate bottom control parameter.
25.000 Tcline S-coordinate surface/bottom layer width (m) used
in vertical coordinate stretching.
1025.000 rho0 Mean density (kg/m3) for Boussinesq approximation.
0.000 dstart Time-stamp assigned to model initialization (days).
0.00 time_ref Reference time for units attribute (yyyymmdd.dd)
0.0000E+00 Tnudg(01) Nudging/relaxation time scale (days)
for tracer 01: temp
0.0000E+00 Tnudg(02) Nudging/relaxation time scale (days)
for tracer 02: salt
0.0000E+00 Znudg Nudging/relaxation time scale (days)
for free-surface.
0.0000E+00 M2nudg Nudging/relaxation time scale (days)
for 2D momentum.
0.0000E+00 M3nudg Nudging/relaxation time scale (days)
for 3D momentum.
0.0000E+00 obcfac Factor between passive and active
open boundary conditions.
F VolCons(1) NLM western edge boundary volume conservation.
F VolCons(2) NLM southern edge boundary volume conservation.
F VolCons(3) NLM eastern edge boundary volume conservation.
F VolCons(4) NLM northern edge boundary volume conservation.
14.000 T0 Background potential temperature (C) constant.
35.000 S0 Background salinity (PSU) constant.
1027.000 R0 Background density (kg/m3) used in linear Equation
of State.
1.7000E-04 Tcoef Thermal expansion coefficient (1/Celsius).
0.0000E+00 Scoef Saline contraction coefficient (1/PSU).
1.000 gamma2 Slipperiness variable: free-slip (1.0) or
no-slip (-1.0).
T Hout(idFsur) Write out free-surface.
T Hout(idUbar) Write out 2D U-momentum component.
T Hout(idVbar) Write out 2D V-momentum component.
T Hout(idUvel) Write out 3D U-momentum component.
T Hout(idVvel) Write out 3D V-momentum component.
T Hout(idWvel) Write out W-momentum component.
T Hout(idOvel) Write out omega vertical velocity.
T Hout(idTvar) Write out tracer 01: temp
T Hout(idTvar) Write out tracer 02: salt

T Aout(idFsur) Write out averaged free-surface.
T Aout(idUbar) Write out averaged 2D U-momentum component.
T Aout(idVbar) Write out averaged 2D V-momentum component.
T Aout(idUvel) Write out averaged 3D U-momentum component.
T Aout(idVvel) Write out averaged 3D V-momentum component.
T Aout(idWvel) Write out averaged W-momentum component.
T Aout(idOvel) Write out averaged omega vertical velocity.
T Aout(idTvar) Write out averaged tracer 01: temp
T Aout(idTvar) Write out averaged tracer 02: salt

T Dout(M2rate) Write out 2D momentun acceleration.
T Dout(M2pgrd) Write out 2D momentum pressure gradient.
T Dout(M2fcor) Write out 2D momentum Coriolis force.
T Dout(M2hadv) Write out 2D momentum horizontal advection.
T Dout(M2xadv) Write out 2D momentum horizontal X-advection.
T Dout(M2yadv) Write out 2D momentum horizontal Y-advection.
T Dout(M2hvis) Write out 2D momentum horizontal viscosity.
T Dout(M2xvis) Write out 2D momentum horizontal X-viscosity.
T Dout(M2yvis) Write out 2D momentum horizontal Y-viscosity.
T Dout(M2sstr) Write out 2D momentum surface stress.
T Dout(M2bstr) Write out 2D momentum bottom stress.

T Dout(M3rate) Write out 3D momentun acceleration.
T Dout(M3pgrd) Write out 3D momentum pressure gradient.
T Dout(M3fcor) Write out 3D momentum Coriolis force.
T Dout(M3hadv) Write out 3D momentum horizontal advection.
T Dout(M3xadv) Write out 3D momentum horizontal X-advection.
T Dout(M3yadv) Write out 3D momentum horizontal Y-advection.
T Dout(M3vadv) Write out 3D momentum vertical advection.
T Dout(M3hvis) Write out 3D momentum horizontal viscosity.
T Dout(M3xvis) Write out 3D momentum horizontal X-viscosity.
T Dout(M3yvis) Write out 3D momentum horizontal Y-viscosity.
T Dout(M3vvis) Write out 3D momentum vertical viscosity.

T Dout(iTrate) Write out rate of change of tracer 01: temp
T Dout(iTrate) Write out rate of change of tracer 02: salt
T Dout(iThadv) Write out horizontal advection, tracer 01: temp
T Dout(iThadv) Write out horizontal advection, tracer 02: salt
T Dout(iTxadv) Write out horizontal X-advection, tracer 01: temp
T Dout(iTxadv) Write out horizontal X-advection, tracer 02: salt
T Dout(iTyadv) Write out horizontal Y-advection, tracer 01: temp
T Dout(iTyadv) Write out horizontal Y-advection, tracer 02: salt
T Dout(iTvadv) Write out vertical advection, tracer 01: temp
T Dout(iTvadv) Write out vertical advection, tracer 02: salt
T Dout(iThdif) Write out horizontal diffusion, tracer 01: temp
T Dout(iThdif) Write out horizontal diffusion, tracer 02: salt
T Dout(iTxdif) Write out horizontal X-diffusion, tracer 01: temp
T Dout(iTxdif) Write out horizontal X-diffusion, tracer 02: salt
T Dout(iTydif) Write out horizontal Y-diffusion , tracer 01: temp
T Dout(iTydif) Write out horizontal Y-diffusion , tracer 02: salt
T Dout(iTvdif) Write out vertical diffusion, tracer 01: temp
T Dout(iTvdif) Write out vertical diffusion, tracer 02: salt

Output/Input Files:

Output Restart File: ocean_rst.nc
Output History File: ocean_his.nc
Output Averages File: ocean_avg.nc
Output Diagnostics File: ocean_dia.nc

Tile partition information for Grid 01: 0041x0080x0016 tiling: 001x001

tile Istr Iend Jstr Jend Npts

0 1 41 1 80 52480

Tile minimum and maximum fractional grid coordinates:
(interior points only)

tile Xmin Xmax Ymin Ymax grid

0 -1.50 42.50 0.50 81.50 RHO-points

0 -2.00 42.00 0.50 81.50 U-points

0 -1.50 42.50 0.00 81.00 V-points

Lateral Boundary Conditions: NLM
============================

Variable Grid West Edge South Edge East Edge North Edge
--------- ---- ---------- ---------- ---------- ----------

zeta 1 Periodic Closed Periodic Closed

ubar 1 Periodic Closed Periodic Closed

vbar 1 Periodic Closed Periodic Closed

u 1 Periodic Closed Periodic Closed

v 1 Periodic Closed Periodic Closed

temp 1 Periodic Closed Periodic Closed

salt 1 Periodic Closed Periodic Closed

Activated C-preprocessing Options:

UPWELLING Wind-Driven Upwelling/Downwelling over a Periodic Channel
ANA_BSFLUX Analytical kinematic bottom salinity flux.
ANA_BTFLUX Analytical kinematic bottom temperature flux.
ANA_GRID Analytical grid set-up.
ANA_INITIAL Analytical initial conditions.
ANA_SMFLUX Analytical kinematic surface momentum flux.
ANA_SSFLUX Analytical kinematic surface salinity flux.
ANA_STFLUX Analytical kinematic surface temperature flux.
ANA_VMIX Analytical vertical mixing coefficients.
ASSUMED_SHAPE Using assumed-shape arrays.
AVERAGES Writing out time-averaged nonlinear model fields.
DIAGNOSTICS_TS Computing and writing tracer diagnostic terms.
DIAGNOSTICS_UV Computing and writing momentum diagnostic terms.
DJ_GRADPS Parabolic Splines density Jacobian (Shchepetkin, 2002).
DOUBLE_PRECISION Double precision arithmetic.
MIX_S_TS Mixing of tracers along constant S-surfaces.
MIX_S_UV Mixing of momentum along constant S-surfaces.
NONLINEAR Nonlinear Model.
!NONLIN_EOS Linear Equation of State for seawater.
POWER_LAW Power-law shape time-averaging barotropic filter.
PROFILE Time profiling activated .
!RST_SINGLE Double precision fields in restart NetCDF file.
SALINITY Using salinity.
SOLVE3D Solving 3D Primitive Equations.
SPLINES Conservative parabolic spline reconstruction.
TS_U3HADVECTION Third-order upstream horizontal advection of tracers.
TS_C4VADVECTION Fourth-order centered vertical advection of tracers.
TS_DIF2 Harmonic mixing of tracers.
UV_ADV Advection of momentum.
UV_COR Coriolis term.
UV_U3HADVECTION Third-order upstream horizontal advection of 3D momentum.
UV_C4VADVECTION Fourth-order centered vertical advection of momentum.
UV_LDRAG Linear bottom stress.
UV_VIS2 Harmonic mixing of momentum.
VAR_RHO_2D Variable density barotropic mode.

Process Information:

Thread # 0 (pid= 26230) is active.

INITIAL: Configuring and initializing forward nonlinear model ...


Vertical S-coordinate System:

level S-coord Cs-curve Z at hmin at hc half way at hmax

16 0.0000000 0.0000000 0.000 0.000 0.000 0.000
15 -0.0625000 -0.0019442 -0.809 -0.806 -1.348 -1.589
14 -0.1250000 -0.0078455 -1.668 -1.661 -2.966 -3.687
13 -0.1875000 -0.0179119 -2.580 -2.568 -4.867 -6.321
12 -0.2500000 -0.0324983 -3.549 -3.531 -7.077 -9.535
11 -0.3125000 -0.0521190 -4.581 -4.558 -9.630 -13.397
10 -0.3750000 -0.0774659 -5.686 -5.656 -12.573 -17.996
9 -0.4375000 -0.1094327 -6.875 -6.837 -15.967 -23.445
8 -0.5000000 -0.1491465 -8.162 -8.114 -19.889 -29.890
7 -0.5625000 -0.1980075 -9.564 -9.506 -24.435 -37.512
6 -0.6250000 -0.2577387 -11.104 -11.034 -29.721 -46.531
5 -0.6875000 -0.3304460 -12.808 -12.724 -35.892 -57.218
4 -0.7500000 -0.4186931 -14.709 -14.609 -43.121 -69.903
3 -0.8125000 -0.5255915 -16.846 -16.726 -51.622 -84.987
2 -0.8750000 -0.6549105 -19.266 -19.124 -61.651 -102.953
1 -0.9375000 -0.8112096 -22.028 -21.859 -73.518 -124.388
0 -1.0000000 -1.0000000 -25.200 -25.000 -87.600 -150.000

Time Splitting Weights: ndtfast = 30 nfast = 42

Primary Secondary Accumulated to Current Step

1-0.0008094437383769 0.0333333333333333-0.0008094437383769 0.0333333333333333
2-0.0014053566728197 0.0333603147912792-0.0022148004111966 0.0666936481246126
3-0.0017877524645903 0.0334071600137066-0.0040025528757869 0.1001008081383191
4-0.0019566842408176 0.0334667517625262-0.0059592371166046 0.1335675599008453
5-0.0019122901320372 0.0335319745705535-0.0078715272486418 0.1670995344713988
6-0.0016548570247459 0.0335957175749547-0.0095263842733877 0.2006952520463536
7-0.0011849025289723 0.0336508794757796-0.0107112868023600 0.2343461315221331
8-0.0005032751608631 0.0336903762267453-0.0112145619632232 0.2680365077488784
9 0.0003887272597151 0.0337071520654408-0.0108258347035081 0.3017436598143192
10 0.0014892209965583 0.0336941944901169-0.0093366137069498 0.3354378543044362
11 0.0027955815694920 0.0336445537902317-0.0065410321374578 0.3690824080946679
12 0.0043042707117221 0.0335513677379153-0.0022367614257356 0.4026337758325831
13 0.0060106451121704 0.0334078920475245 0.0037738836864348 0.4360416678801076
14 0.0079087469427945 0.0332075372104522 0.0116826306292293 0.4692492050905598
15 0.0099910761708920 0.0329439123123590 0.0216737068001213 0.5021931174029188
16 0.0122483446563884 0.0326108764399960 0.0339220514565097 0.5348039938429148
17 0.0146692120341107 0.0322025982847830 0.0485912634906204 0.5670065921276978
18 0.0172400033810439 0.0317136245503127 0.0658312668716643 0.5987202166780105
19 0.0199444086685725 0.0311389577709445 0.0857756755402368 0.6298591744489550
20 0.0227631639997064 0.0304741441486588 0.1085388395399432 0.6603333185976138
21 0.0256737146312911 0.0297153720153352 0.1342125541712342 0.6900486906129490
22 0.0286498597812016 0.0288595815276255 0.1628624139524359 0.7189082721405746
23 0.0316613792205220 0.0279045862015855 0.1945237931729578 0.7468128583421600
24 0.0346736416507075 0.0268492068942347 0.2291974348236653 0.7736620652363948
25 0.0376471948657328 0.0256934188392112 0.2668446296893981 0.7993554840756060
26 0.0405373376992232 0.0244385123436867 0.3073819673886213 0.8237939964192927
27 0.0432936737565710 0.0230872677537126 0.3506756411451924 0.8468812641730054
28 0.0458596469320356 0.0216441452951603 0.3965352880772280 0.8685254094681656
29 0.0481720587108284 0.0201154903974257 0.4447073467880565 0.8886408998655914
30 0.0501605672561820 0.0185097551070648 0.4948679140442384 0.9071506549726561
31 0.0517471682814030 0.0168377361985254 0.5466150823256415 0.9239883911711815
32 0.0528456577069106 0.0151128305891453 0.5994607400325521 0.9391012217603267
33 0.0533610761022577 0.0133513086655816 0.6528218161348098 0.9524525304259084
34 0.0531891349131379 0.0115726061288397 0.7060109510479478 0.9640251365547481
35 0.0522156244733761 0.0097996349650684 0.7582265755213239 0.9738247715198165
36 0.0503158038019030 0.0080591141492892 0.8085423793232269 0.9818838856691057
37 0.0473537721847153 0.0063819206892258 0.8558961515079423 0.9882658063583315
38 0.0431818225418188 0.0048034616164019 0.8990779740497611 0.9930692679747334
39 0.0376397765791564 0.0033640675316746 0.9367177506289175 0.9964333355064080
40 0.0305543017255206 0.0021094083123694 0.9672720523544381 0.9985427438187774
41 0.0217382098544504 0.0010909315881854 0.9890102622088885 0.9996336754069628
42 0.0109897377911118 0.0003663245930371 1.0000000000000004 0.9999999999999999

ndtfast, nfast = 30 42 nfast/ndtfast = 1.40000

Centers of gravity and integrals (values must be 1, 1, approx 1/2, 1, 1):

1.000000000000 1.047601458608 0.523800729304 1.000000000000 1.000000000000

Power filter parameters, Fgamma, gamma = 0.28400 0.18933

Minimum X-grid spacing, DXmin = 1.00000000E+00 km
Maximum X-grid spacing, DXmax = 1.00000000E+00 km
Minimum Y-grid spacing, DYmin = 1.00000000E+00 km
Maximum Y-grid spacing, DYmax = 1.00000000E+00 km
Minimum Z-grid spacing, DZmin = 8.08965824E-01 m
Maximum Z-grid spacing, DZmax = 2.56123321E+01 m

Minimum barotropic Courant Number = 2.22358627E-01
Maximum barotropic Courant Number = 5.42494240E-01
Maximum Coriolis Courant Number = 2.47800000E-02


Maximum grid stiffness ratios: rx0 = 6.931666E-02 (Beckmann and Haidvogel)
rx1 = 8.661243E-01 (Haney)


Initial basin volumes: TotVolume = 3.8843755884E+11 m3
MinVolume = 8.4521383562E+05 m3
MaxVolume = 2.5612332106E+07 m3
Max/Min = 3.0302783777E+01


NL ROMS/TOMS: started time-stepping: (Grid: 01 TimeSteps: 00000001 - 00001440)


STEP Day HH:MM:SS KINETIC_ENRG POTEN_ENRG TOTAL_ENRG NET_VOLUME
C => (i,j,k) Cu Cv Cw Max Speed

0 0 00:00:00 0.000000E+00 6.585677E+02 6.585677E+02 3.884376E+11
(00,00,00) 0.000000E+00 0.000000E+00 0.000000E+00 0.000000E+00
DEF_HIS - creating history file: ocean_his.nc
WRT_HIS - wrote history fields (Index=1,1) into time record = 0000001
DEF_AVG - creating average file: ocean_avg.nc
DEF_DIAGS - creating diagnostics file: ocean_dia.nc
1 0 00:05:00 6.479408E-13 6.585677E+02 6.585677E+02 3.884376E+11
(01,01,01) 1.069435E-09 1.950204E-06 0.000000E+00 9.216783E-06
2 0 00:10:00 9.293195E-12 6.585677E+02 6.585677E+02 3.884376E+11
(01,02,08) 6.735167E-08 1.785829E-06 1.433648E-05 3.660353E-05
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
oceanS 0000000000481977 Unknown Unknown Unknown
oceanS 0000000000475173 Unknown Unknown Unknown
oceanS 000000000045D4B8 Unknown Unknown Unknown
oceanS 0000000000403FE1 Unknown Unknown Unknown
oceanS 0000000000403E2D Unknown Unknown Unknown
oceanS 0000000000403D3C Unknown Unknown Unknown
libc.so.6 00007F80D51CBBC6 Unknown Unknown Unknown
oceanS 0000000000403C39 Unknown Unknown Unknown
####################################################################
# Your job finished at : Mon Jan 23 13:39:50 CST 2012
# Your job requested : cput=240:00:00,mem=1gb,neednodes=1:ppn=1,nodes=1:ppn=1,walltime=336:00:00
# Your job used : cput=00:00:00,mem=0kb,vmem=0kb,walltime=00:00:08
# Your job's cpu utilization : 0%
# Your job's memory utilization (mem) : 0.00%
# Alabama Supercomputer Center - PBS Epilogue
####################################################################

[kate]

I would recompile with the USE_DEBUG flag turned on, then execute the oceanG file. It should be able to tell you line numbers then, perhaps even give array-out-of-bounds info if appropriate.

[dhtjtks]

I would recompile with the USE_DEBUG flag turned on, then execute the oceanG file. It should be able to tell you line numbers then, perhaps even give array-out-of-bounds info if appropriate.

Hi Kate,

When I turned on USE_DEBUG flag, it worked fine both in serial and parallel mode. I don't know why this happens. Anyway, it works now. Thank you so much.

dhtjtks

[kate]

Don't you love it? It could be that the optimizer makes the code bigger so it wants more than 1 GB. Or it could be an optimizer bug. You do want to optimize for non-trivial runs.

[cousins]

Did you ever figure out what this was? I'm getting the same thing now from a recent (608) download using SVN. Using USE_DEBUG allows it to work. Running it without USE_DEBUG but adding -g -traceback (Intel compiler 11.1) in FFLAGS yielded:

NL ROMS/TOMS: started time-stepping: (Grid: 01 TimeSteps: 00000001 - 00001440)


STEP Day HH:MM:SS KINETIC_ENRG POTEN_ENRG TOTAL_ENRG NET_VOLUME
C => (i,j,k) Cu Cv Cw Max Speed

0 0 00:00:00 0.000000E+00 6.585677E+02 6.585677E+02 3.884376E+11
(00,00,00) 0.000000E+00 0.000000E+00 0.000000E+00 0.000000E+00
DEF_HIS - creating history file: ocean_his.nc
WRT_HIS - wrote history fields (Index=1,1) into time record = 0000001
DEF_AVG - creating average file: ocean_avg.nc
DEF_DIAGS - creating diagnostics file: ocean_dia.nc
1 0 00:05:00 6.479408E-13 6.585677E+02 6.585677E+02 3.884376E+11
(01,01,01) 1.069435E-09 1.950204E-06 0.000000E+00 9.216783E-06
2 0 00:10:00 9.293195E-12 6.585677E+02 6.585677E+02 3.884376E+11
(01,02,08) 6.735167E-08 1.785829E-06 1.433648E-05 3.660353E-05
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
oceanS 000000000046685F set_avg_mod_mp_se 558 set_avg.f90
oceanS 000000000045B7F0 set_avg_mod_mp_se 59 set_avg.f90
oceanS 0000000000445F56 main3d_ 185 main3d.f90
oceanS 0000000000403B6F ocean_control_mod 140 ocean_control.f90
oceanS 00000000004039D5 MAIN__ 86 master.f90
oceanS 00000000004037EC Unknown Unknown Unknown
libc.so.6 000000364EC1D994 Unknown Unknown Unknown
oceanS 00000000004036F9 Unknown Unknown Unknown

Line 558 of set_avg.f90 in the Build directory is:

Code: Select all

!-----------------------------------------------------------------------
!  Accumulate time-averaged fields.
!-----------------------------------------------------------------------
!
      ELSE IF (iic(ng).gt.ntsAVG(ng)) THEN
!
!  Accumulate state variables.
!
        IF (Aout(idFsur,ng)) THEN
          DO j=JstrR,JendR
            DO i=IstrR,IendR
558 ->        AVERAGE(ng)%avgzeta(i,j)=AVERAGE(ng)%avgzeta(i,j)+        &
     &                                 OCEAN(ng)%zeta(i,j,Kout)
            END DO
          END DO
        END IF

I have tried:
-O2 instead of -O3
not using -ip
ulimit -s unlimited
-no-heap-arrays
without -fp-model precise

Any ideas?

Maybe it is just something to do with this version of the compiler. I had troubles with later Intel compilers so I just kept this one.

Thanks,

Steve

[colberg]

Yes, I had the same problem with the Intel 11.x compiler recently. Needed to switch to Intel 10.x or Intel 12.x. The segmentation error only occurred when "define AVERAGES" and writing into average file.
Everything worked fine with other compiler versions...

[cousins]

Bizarre. We've had no problem with earlier versions of ROMS (at least a year old) and this compiler,

Thanks for the confirmation.

Steve

[cousins]

Just to close this up, Intel Composer XE 2011 SP1 9.293 is working as expected.

[gaokun227]

I ran into the same issue when 'AVERAGES' is defined (ROMS version 3.7 is used).

forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
oceanM_ave 00000000004D002E set_avg_mod_mp_se 562 set_avg.f90
oceanM_ave 00000000004C47A0 set_avg_mod_mp_se 60 set_avg.f90
oceanM_ave 00000000004A56F1 main3d_ 199 main3d.f90
oceanM_ave 00000000004240EF ocean_control_mod 164 ocean_control.f90
oceanM_ave 0000000000423F05 MAIN__ 108 master.f90
oceanM_ave 0000000000423D4C Unknown Unknown Unknown
libc.so.6 0000003BE1E1D994 Unknown Unknown Unknown
oceanM_ave 0000000000423C59 Unknown Unknown Unknown

Did anyone ever figure out what the problem was and how to resolve it?

[cousins]

What compiler and version are you using? Upgrading to a newer version of the compiler allowed it to work for us previously. It would be good to know what compiler version you are using to know which versions to avoid.

[gaokun227]

What compiler and version are you using? Upgrading to a newer version of the compiler allowed it to work for us previously. It would be good to know what compiler version you are using to know which versions to avoid.

The complier I use is Intel(R) Fortran Intel(R) 64 Compiler Professional for applications running on Intel(R) 64, Version 11.1.

I just found this issue only occurred when 2D averaged fields (ubar,vbar,zeta) were defined to be written out. If I only write out 3D averaged fields (u,v,temp,salt), there is no problem at all.

[cousins]

That is the version we had problems with before (see earlier in the thread). The answer still is to upgrade your compiler.

Good luck,

Steve