Opened 10 months ago

Closed 9 months ago

#3310 closed help (answered)

UKESM1 model run failing with no obvious error

Reported by: ajd Owned by: um_support
Component: UKESM Keywords:
Cc: Platform: Monsoon2
UM Version:

Description

Hi CMS,

I am encountering an issue with suite u-bu607, where the 'coupled' task for cycle 20480101 keeps failing (error message copied in below).

I have checked the usual places for any obvious errors (e.g. 'ocean.output') but can't find anything. Also, have tried switching on extra PE diagnostics but that switch in the model doesn't seem to work.

Any ideas on how to resolve this would be much appreciated!

Kind regards,
Andrea

Rank 676 [Mon Jun 22 10:12:50 2020] [c10-0c2s10n2] application called MPI_Abort(comm=0xC4000005, 1) - process 676

atpAppSigHandler: Back-end never delivered its pid. Re-raising signal.
atpAppSigHandler: Back-end never delivered its pid. Re-raising signal.
atpAppSigHandler: Back-end never delivered its pid. Re-raising signal.
_pmiu_daemon(SIGCHLD): [NID 02088] [c10-0c2s10n0] [Mon Jun 22 10:16:47 2020] PE RANK 590 exit signal Aborted
atpAppSigHandler: Back-end never delivered its pid. Re-raising signal.
[NID 02088] 2020-06-22 10:16:47 Apid 108675102: initiated application termination
[FAIL] run_model # return-code=137
2020-06-22T10:16:56Z CRITICAL - failed/EXIT

Change History (3)

comment:1 Changed 10 months ago by grenville

Andrea

In /home/d02/andit/cylc-run/u-bu607/work/20480101T0000Z/coupled/ocean.output:

==⇒>> : E R R O R

===========

stpctl: the zonal velocity is larger than 20 m/s

This common error when the ocean model goes unstable.

Grenville

comment:2 Changed 10 months ago by ajd

Thanks Grenville - I searched for the string 'error' but forgot the space between the letters!

I restarted the model in the year before the failure with a perturbed atmospheric restart dump and that seems to have worked.

Many thanks,
Andrea

comment:3 Changed 9 months ago by grenville

  • Resolution set to answered
  • Status changed from new to closed

Hi Andrea

I'm assuming that it's all worked perfectly.

Grenville

Note: See TracTickets for help on using tickets.