Opened 7 months ago

Closed 6 months ago

#3334 closed help (answered)

A question about “restarting if the model blows up”

Reported by: yb19052 Owned by: um_support
Component: UM Model Keywords:
Cc: Platform: NEXCS
UM Version: 10.7

Description (last modified by ros)

Hello CMS,

I have a question about a process for “restarting if the model blows up” in https://code.metoffice.gov.uk/trac/moci/wiki/tips_CRgeneral.

In steps 4 and 5, it worked for UM atmos restart, but not NEMO and CICE restart. I got the following error messages.

[about NEMO restart]

local_moci_lib/perturb_theta.py bv662o_18970101_restart_orig.nc --output ./bv662o_18970101_restart.nc
Perturbing Field - Sec 0, Item 388: ThetaVD After Timestep
Traceback (most recent call last):
  File "local_moci_lib/perturb_theta.py", line 183, in <module>
    main()
  File "local_moci_lib/perturb_theta.py", line 160, in main
    dfile = mule.DumpFile.from_file(dump_in)
  File "/projects/um1/mule/mule-2020.01.1/python-2.7.9_numpy-1.9.1/openmp/lib/mule/__init__.py", line 1254, in from_file
    new_umf._read_file(file_or_filepath)
  File "/projects/um1/mule/mule-2020.01.1/python-2.7.9_numpy-1.9.1/openmp/lib/mule/ff.py", line 502, in _read_file
    super(FieldsFile, self)._read_file(file_or_filepath)
  File "/projects/um1/mule/mule-2020.01.1/python-2.7.9_numpy-1.9.1/openmp/lib/mule/__init__.py", line 1441, in _read_file
    header = headerclass.from_file(source, length)
  File "/projects/um1/mule/mule-2020.01.1/python-2.7.9_numpy-1.9.1/openmp/lib/mule/__init__.py", line 392, in from_file
    values = np.fromfile(source, dtype=cls.DTYPE, count=num_words)
MemoryError

[about CICE restart]

local_moci_lib/perturb_theta.py bv662i.restart.1897-01-01-00000_orig.nc --output ./bv662i.restart.1897-01-01-00000.nc
Perturbing Field - Sec 0, Item 388: ThetaVD After Timestep
Traceback (most recent call last):
  File "local_moci_lib/perturb_theta.py", line 183, in <module>
    main()
  File "local_moci_lib/perturb_theta.py", line 160, in main
    dfile = mule.DumpFile.from_file(dump_in)
  File "/projects/um1/mule/mule-2020.01.1/python-2.7.9_numpy-1.9.1/openmp/lib/mule/__init__.py", line 1254, in from_file
    new_umf._read_file(file_or_filepath)
  File "/projects/um1/mule/mule-2020.01.1/python-2.7.9_numpy-1.9.1/openmp/lib/mule/ff.py", line 502, in _read_file
    super(FieldsFile, self)._read_file(file_or_filepath)
  File "/projects/um1/mule/mule-2020.01.1/python-2.7.9_numpy-1.9.1/openmp/lib/mule/__init__.py", line 1445, in _read_file
    header = headerclass.from_file(source, dim1, dim2)
  File "/projects/um1/mule/mule-2020.01.1/python-2.7.9_numpy-1.9.1/openmp/lib/mule/__init__.py", line 477, in from_file
    values = values.reshape((dim1, dim2), order="F")
ValueError: total size of new array must be unchanged

Can I use the approach with the NEMO and CICE restart dump? How do I fix the errors?

I appreciate your kind cooperation.

Thanks
Kenji

Change History (5)

comment:1 Changed 7 months ago by dcase

Kenji,

I think that you should try to run the model with your perturbed restart for the atmosphere, and leave the others alone.

Hopefully that gets beyond the failure point.

Dave

comment:2 Changed 7 months ago by yb19052

Hi Dave

I appreciate your response.

Even if an error comes from NEMO (e.g., NEMO errors with zonal velocity > 20 m/s or negative salinity), is it okay with my perturbed restart for the atmosphere alone?

Anyway, if that's okay, I'm very happy.

Thanks
Kenji

comment:3 Changed 7 months ago by dcase

I would just try it. And if it doesn't work then possibly reduce the time step for this period of simulation?

Dealing with numerical instabilities is a pain, but hopefully with some luck you can get around this. Let me know how it goes.

comment:4 Changed 6 months ago by ros

  • Description modified (diff)

comment:5 Changed 6 months ago by ros

  • Resolution set to answered
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.