Opened 5 weeks ago

Closed 4 weeks ago

#3452 closed help (answered)

Suite restarting and cycle and failing on wall clock

Reported by: jmw240 Owned by: JamesWeber
Component: UM Model Keywords:
Cc: Platform:
UM Version: 11.5

Description

Hi,

I'm trying to run a copy of the pi-clim control at vn11.5 (u-cb228) using a version of the CRI-STRAT chemistry scheme. I initially tried a 1 month cycling frequency and, after running fine for 2 months, the run failed on the 3rd month. The pe_output file for the 3rd month revealed that the model appeared to have reverted to starting on the 1st month (Jan) rather than the required month (March) despite having run these two months already and archived their output successfully to MASS. After (re)completing Jan and most of Feb in the time slot for March, it failed on the wall clock.

I spoke to Luke Abraham and he mentioned he had had a similar problem when the cycling frequency was 1 month which had been resolved by resetting the cycling frequency back to 3 months. However, this does not appear to be an option for this run as the additional chemistry means 3 months cannot be completed in the maximum 4 hour wallclock period (I tried increasing the OpenMP threads from 1 to 2 and the run only managed about 2.25 months in 4 hours). I also tried a similar suite, u-cb543, with a 2 month cycling frequency and again this was fine for the first 2 months but then hit the same problem.

Any advice would be greatly appreciated.

Many thanks,

James


Change History (3)

comment:1 Changed 5 weeks ago by jmw240

One further thing which I have just noticed. The dump frequency (dumpfreqim) in the suite/app/um/rose-app.conf file was set to 90 days and was not changed when I ran with cycling frequencies of 1 or 2 months. Could this have caused the problem?

comment:2 Changed 5 weeks ago by jmw240

Update: The model ran without issue when I switched the dumpfreqim to 30 days and had the cycling frequency at 1m. I think this ticket can be closed.

comment:3 Changed 4 weeks ago by ros

  • Resolution set to answered
  • Status changed from new to closed

Hi James,

Yes you need to ensure that there is a dump written at the end of the each cycle.

Cheers,
Ros.

Note: See TracTickets for help on using tickets.