Opened 5 years ago

Closed 5 years ago

#1336 closed help (completed)

Job failure: floating point exception (SW diagnostics)

Reported by: jchamber Owned by: um_support
Component: UM Model Keywords:
Cc: Platform:
UM Version: 8.5

Description

My job xkglf is failing on day 3 of a 7-day run. I have looked at the .leave file and the problem appears to be with the SW diagnostics. See below:

Signal received: SIGFPE - Floating-point exception

Signal generated for floating-point exception:

FP invalid operation

Instruction that generated the exception:

fadd fr02,fr03,fr04
Source Operand values:

fr03 = 0.00000000000000e+00
fr04 = -NaNS

Traceback:

Offset 0x000196d4 in procedure glue_rad_, near line 3593 in file /projects/um1/vn8.5/ibm/prebuilds/lam_high_omp/umatmos/ppsrc/UM/atmosphere/radiation_control/glue_rad.f90
Offset 0x000030c0 in procedure ni_rad_ctl_, near line 783 in file /projects/um1/vn8.5/ibm/prebuilds/lam_high_omp/umatmos/ppsrc/UM/atmosphere/radiation_control/ni_rad_ctl.f90
Offset 0x00007598 in procedure atmos_physics1_, near line 2572 in file /projects/cascade/jchamb/um_build/xjwbp/umatmos/ppsrc/UM/control/top_level/atmos_physics1.f90
Offset 0x00019ca0 in procedure atm_step_4a_, near line 6526 in file /projects/cascade/jchamb/um_build/xjwbp/umatmos/ppsrc/UM/control/top_level/atm_step_4A.f90
Offset 0x000ede90 in procedure u_model_4a_, near line 1970 in file /projects/cascade/jchamb/um_build/xjwbp/umatmos/ppsrc/UM/control/top_level/u_model_4A.f90
Offset 0x00003448 in procedure um_shell_, near line 1135 in file /projects/um1/vn8.5/ibm/prebuilds/lam_high_omp/umatmos/ppsrc/UM/control/top_level/um_shell.f90
Offset 0x0000000c in procedure um_main, near line 17 in file /projects/um1/vn8.5/ibm/prebuilds/lam_high_omp/umatmos/ppsrc/UM/control/top_level/um_main.f90
—- End of call chain —-

ERROR: 0031-300 Forcing all remote tasks to exit due to exit code 1 in task 0
xkglf: Run failed

From looking at the UM output in xconv, I can see the net SW down field develops some "issues" at or near sunrise.

I am attempting to run the job from the start of day 3 with the SW diags switched off in STASH so will let you know whether that works or not but I would like the SW diagnostics so was hoping you might be able to suggest a change that would work?!

Many thanks

Jill

Change History (1)

comment:1 Changed 5 years ago by ros

  • Resolution set to completed
  • Status changed from new to closed

Jill managed to resolve this problem so ticket is now being closed.

Note: See TracTickets for help on using tickets.