Fail to submit a standard job from PUMA

[jfgu]

Dear CMS helpdesk,

I am sorry to open a new ticket which has the same problem as the previous one: http://cms.ncas.ac.uk/ticket/2224. This problem hasn't been resolved. I have stuck here for days. No one else around me knows what's happening for the suite. Here is a brief desription of my problem. My suite is u-an830. My previous job submitted with this suite ran ok. No errors. After that, I changed three parameters to modify the background fow. When I resubmit the job, it failed. The same procedure had also been done to other suites, this situation had never happened before.

Please could someone help to answer my question? This is really important. Thank you very much.

Attached please see the error information and the following is the information in log file:

[INFO] 2017-07-18T09:27:05+0100 cylc get-global-config -i [hosts][localhost]run\ directory
[INFO] 2017-07-18T09:27:06+0100 cylc get-global-config -i [hosts][localhost]work\ directory
[INFO] 2017-07-18T09:27:06+0100 Configuration: /home/jfgu/roses/u-an830/
[INFO] 2017-07-18T09:27:06+0100 file: rose-suite.conf
[INFO] 2017-07-18T09:27:06+0100 cylc —version
[INFO] 2017-07-18T09:27:06+0100 pgrep -f -u jfgu python.*/bin/cylc-(run|restart)(\ |\ .+\ )u-an830(\ |$)
[INFO] 2017-07-18T09:27:06+0100 pgrep -f -u jfgu python.*/bin/cylc-(run|restart)(\ |\ .+\ )u-an830(\ |$)
[INFO] 2017-07-18T09:27:06+0100 delete: /home/jfgu/cylc-run/u-an830/
[INFO] 2017-07-18T09:27:06+0100 cylc refresh —unregister
[INFO] 2017-07-18T09:27:07+0100 delete: /home/jfgu/.cylc/u-an830
[INFO] 2017-07-18T09:27:07+0100 create: /home/jfgu/cylc-run/u-an830
[INFO] 2017-07-18T09:27:07+0100 create: log.20170718T082707Z
[INFO] 2017-07-18T09:27:07+0100 symlink: log.20170718T082707Z ⇐ log
[INFO] 2017-07-18T09:27:07+0100 create: log/suite
[INFO] 2017-07-18T09:27:07+0100 create: log/rose-conf
[INFO] 2017-07-18T09:27:07+0100 svn info —non-interactive
[INFO] 2017-07-18T09:27:07+0100 svn status —non-interactive
[INFO] 2017-07-18T09:27:07+0100 svn diff —internal-diff —non-interactive
[INFO] 2017-07-18T09:27:07+0100 git describe
[INFO] 2017-07-18T09:27:07+0100 symlink: rose-conf/20170718T092707-run.conf ⇐ log/rose-suite-run.conf
[INFO] 2017-07-18T09:27:07+0100 symlink: rose-conf/20170718T092707-run.version ⇐ log/rose-suite-run.version
[INFO] 2017-07-18T09:27:07+0100 create: share
[INFO] 2017-07-18T09:27:07+0100 create: share/cycle
[INFO] 2017-07-18T09:27:07+0100 create: work
[INFO] 2017-07-18T09:27:07+0100 export CYLC_VERSION=6.11.4
[INFO] 2017-07-18T09:27:07+0100 export ROSE_ORIG_HOST=puma
[INFO] 2017-07-18T09:27:07+0100 export ROSE_VERSION=2016.11.1
[INFO] 2017-07-18T09:27:08+0100 source: /home/jfgu/roses/u-an830/suite-setup.rc
[INFO] 2017-07-18T09:27:08+0100 source: /home/jfgu/roses/u-an830/app
[INFO] 2017-07-18T09:27:08+0100 source: /home/jfgu/roses/u-an830/meta
[INFO] 2017-07-18T09:27:08+0100 source: /home/jfgu/roses/u-an830/Jinja2Filters
[INFO] 2017-07-18T09:27:08+0100 source: /home/jfgu/roses/u-an830/rose-suite.info
[INFO] 2017-07-18T09:27:08+0100 source: /home/jfgu/roses/u-an830/bin
[INFO] 2017-07-18T09:27:08+0100 create: bin
[INFO] 2017-07-18T09:27:08+0100 rsync -a —exclude=.* —timeout=1800 —rsh=ssh\ -oBatchMode=yes —checksum /home/jfgu/roses/u-an830/bin/ bin
[INFO] 2017-07-18T09:27:08+0100 install: bin
[INFO] 2017-07-18T09:27:08+0100 source: /home/jfgu/roses/u-an830/bin
[INFO] 2017-07-18T09:27:08+0100 install: suite-setup.rc
[INFO] 2017-07-18T09:27:08+0100 source: /home/jfgu/roses/u-an830/suite-setup.rc
[INFO] 2017-07-18T09:27:08+0100 create: meta
[INFO] 2017-07-18T09:27:08+0100 rsync -a —exclude=.* —timeout=1800 —rsh=ssh\ -oBatchMode=yes —checksum /home/jfgu/roses/u-an830/meta/ meta
[INFO] 2017-07-18T09:27:08+0100 install: meta
[INFO] 2017-07-18T09:27:08+0100 source: /home/jfgu/roses/u-an830/meta
[INFO] 2017-07-18T09:27:08+0100 install: rose-suite.info
[INFO] 2017-07-18T09:27:08+0100 source: /home/jfgu/roses/u-an830/rose-suite.info
[INFO] 2017-07-18T09:27:08+0100 create: Jinja2Filters
[INFO] 2017-07-18T09:27:08+0100 rsync -a —exclude=.* —timeout=1800 —rsh=ssh\ -oBatchMode=yes —checksum /home/jfgu/roses/u-an830/Jinja2Filters/ Jinja2Filters
[INFO] 2017-07-18T09:27:08+0100 install: Jinja2Filters
[INFO] 2017-07-18T09:27:08+0100 source: /home/jfgu/roses/u-an830/Jinja2Filters
[INFO] 2017-07-18T09:27:08+0100 create: app
[INFO] 2017-07-18T09:27:08+0100 rsync -a —exclude=.* —timeout=1800 —rsh=ssh\ -oBatchMode=yes —checksum /home/jfgu/roses/u-an830/app/ app
[INFO] 2017-07-18T09:27:08+0100 install: app
[INFO] 2017-07-18T09:27:08+0100 source: /home/jfgu/roses/u-an830/app
[INFO] 2017-07-18T09:27:08+0100 install: suite.rc
[INFO] 2017-07-18T09:27:08+0100 cylc get-directory u-an830
[INFO] 2017-07-18T09:27:09+0100 cylc refresh —unregister
[INFO] 2017-07-18T09:27:09+0100 cylc unregister u-an830
[INFO] 2017-07-18T09:27:10+0100 0 suite(s) unregistered.
[INFO] 2017-07-18T09:27:10+0100 cylc register u-an830 /home/jfgu/cylc-run/u-an830
[INFO] 2017-07-18T09:27:10+0100 REGISTER u-an830: /home/jfgu/cylc-run/u-an830
[INFO] 2017-07-18T09:27:10+0100 symlink: /home/jfgu/cylc-run/u-an830 ⇐ /home/jfgu/.cylc/u-an830
[INFO] 2017-07-18T09:27:10+0100 cylc validate -v —strict u-an830
[INFO] 2017-07-18T09:27:11+0100 Loading site/user config files
[INFO] 2017-07-18T09:27:11+0100 Reading file /home/fcm/cylc-6.11.4/conf/global.rc
[INFO] 2017-07-18T09:27:11+0100 Reading file /home/jfgu/cylc-run/u-an830/suite.rc
[INFO] 2017-07-18T09:27:11+0100 Processing with Jinja2
[INFO] 2017-07-18T09:27:11+0100 Writing file /home/jfgu/cylc-run/u-an830/suite.rc.processed
[INFO] 2017-07-18T09:27:11+0100 2017-07-18T09:27:11+01 INFO - Expanding [runtime] namespace lists and parameters
[INFO] 2017-07-18T09:27:11+0100 2017-07-18T09:27:11+01 INFO - Parsing the runtime namespace hierarchy
[INFO] 2017-07-18T09:27:11+0100 2017-07-18T08:27:11Z INFO - Parsing [special tasks]
[INFO] 2017-07-18T09:27:11+0100 2017-07-18T08:27:11Z INFO - Parsing the dependency graph
[INFO] 2017-07-18T09:27:11+0100 2017-07-18T08:27:11Z INFO - Configuring internal queues
[INFO] 2017-07-18T09:27:11+0100 2017-07-18T08:27:11Z INFO - Checking for defined tasks not used in the graph
[INFO] 2017-07-18T09:27:11+0100 2017-07-18T08:27:11Z INFO - Checking [visualization] node attributes
[INFO] 2017-07-18T09:27:11+0100 Instantiating tasks to check trigger expressions
[INFO] 2017-07-18T09:27:11+0100 + atmos_atmos.20000101T0000Z ok
[INFO] 2017-07-18T09:27:11+0100 + atmos_recon.20000101T0000Z ok
[INFO] 2017-07-18T09:27:11+0100 + fcm_make2.20000101T0000Z ok
[INFO] 2017-07-18T09:27:11+0100 + fcm_make.20000101T0000Z ok
[INFO] 2017-07-18T09:27:11+0100 Valid for cylc-6.11.4
[INFO] 2017-07-18T09:27:11+0100 WARNING: deprecated items were automatically upgraded in 'suite definition':
[INFO] 2017-07-18T09:27:11+0100 * (6.11.0) [cylc][event hooks] → [cylc][events] - value unchanged
[INFO] 2017-07-18T09:27:11+0100 * (6.11.0) [runtime][root][event hooks] → [runtime][root][events] - value unchanged
[INFO] 2017-07-18T09:27:11+0100 * (6.11.0) [runtime][XC30][job submission] → [runtime][XC30][job] - value unchanged
[INFO] 2017-07-18T09:27:11+0100 * (6.11.0) [runtime][LINUX][job submission] → [runtime][LINUX][job] - value unchanged
[INFO] 2017-07-18T09:27:11+0100 * (6.11.0) [runtime][XC30][job][method] → [runtime][XC30][job][batch system] - value unchanged
[INFO] 2017-07-18T09:27:11+0100 * (6.11.0) [runtime][LINUX][job][method] → [runtime][LINUX][job][batch system] - value unchanged
[INFO] 2017-07-18T09:27:11+0100 cylc get-config -ao -i [remote]owner -i [remote]host u-an830
[INFO] 2017-07-18T09:27:13+0100 ssh -oBatchMode=yes login.archer.ac.uk bash —login -c \'ROSE_VERSION=2016.11.1\ rose\ suite-run\ -v\ -v\ —name=u-an830\ —new\ —run=run\ —remote=uuid=1cf28e5e-baba-4ebc-9702-7b6caf6fabd3,root-dir=$DATADIR\'

Jian-Feng

[grenville]

Jian-Feng

Duplicating tickets creates an administrative burden for CMS; please avoid doing so.

Grenville

[jfgu]

Sorry for that. I will avoid doing so.

Jian-Feng