Opened 9 months ago

Closed 9 months ago

#3132 closed help (answered)

New tasks aren't passed in cylc

Reported by: luciad Owned by: um_support
Component: UM Model Keywords: task error
Cc: Platform: ARCHER
UM Version: 11.1

Description (last modified by ros)

Hello,

I have added two new tasks in the suite u-bq438: flight_track_sim and warm_rain_out.
These tasks should read input diagnostics from atmos_main and compute offline diagnostics, that won't be further passed to postproc.

When I want to run the suite I get:

[FAIL] cylc validate -o /tmp/tmpqCZ31d --strict u-bq438 # return-code=1, stderr=
[FAIL] WARNING - naked dummy tasks detected (no entry under [runtime]):
[FAIL] 	+	flight_track_sim000
[FAIL] 	+	warm_rain_out000
[FAIL] 'ERROR: strict validation fails naked dummy tasks'

First, I don't understand why the '000' appears after the task name in the error.
Secondly, I might need help in properly scheduling the tasks in the suite.rc. The way I've done it so far is:

[scheduling]
   ...
   [[special tasks]]
        {% if ENS_LOG %}
        sequential = atmos_main<ens>, flight_track_sim<ens>, warm_rain_out<ens>, postproc<ens>, archive_integrity<ens>, pptransfer<ens>, supermean<ens>, housekeeping<ens>, rose_arch_logs<ens>
        {% else %}
        sequential = atmos_main, flight_track_sim, warm_rain_out, postproc, archive_integrity, pptransfer, supermean, housekeeping, rose_arch_logs
        {% endif %}
        {# Set up cycling graph #}
        {% set RESUB_GRAPH = '' %}
        {% if ENS_LOG %}
        {% set RESUB_GRAPH = RESUB_GRAPH ~ 'flight_track_sim<ens> => ' if TASK_POSTPROC else RESUB_GRAPH %}
        {% set RESUB_GRAPH = RESUB_GRAPH ~ 'warm_rain_out<ens> => ' if TASK_POSTPROC else RESUB_GRAPH %}
        {% set RESUB_GRAPH = RESUB_GRAPH ~ 'postproc<ens> => ' if TASK_POSTPROC else RESUB_GRAPH %}
        ...
        {% else %}
        {% set RESUB_GRAPH = RESUB_GRAPH ~ 'flight_track_sim => ' if TASK_POSTPROC else RESUB_GRAPH %}
        {% set RESUB_GRAPH = RESUB_GRAPH ~ 'warm_rain_out => ' if TASK_POSTPROC else RESUB_GRAPH %}
        {% set RESUB_GRAPH = RESUB_GRAPH ~ 'postproc => ' if TASK_POSTPROC else RESUB_GRAPH %}
        ...
        {% endif %}
[runtime]
    ...
    [[flight_track_sim]]
        inherit = RUN_MAIN, ATMOS_RESOURCE, ATMOS
        pre-script = module load anaconda
        [[[environment]]]
            ROSE_TASK_APP = flight_track_sim
            CYCLEPERIOD = {{EXPT_RESUB}}

    [[warm_rain_out]]
        inherit = RUN_MAIN, ATMOS_RESOURCE, ATMOS
        pre-script = module load anaconda
        [[[environment]]]
            ROSE_TASK_APP = warm_rain_out
            CYCLEPERIOD = {{EXPT_RESUB}}

The error might occure because I haven't set the task in the [dependencies] → BUILD_GRAPH section?

Could you help me set up the suite.rc correctly?

Thank you,
Lucia

Change History (8)

comment:1 Changed 9 months ago by dcase

Lucia,

you are running a suite with ensembles (and you've put ENS_LOG as true, so these are being used). This is why there are numbers attached to your jobs (e.g. flight_track_sim<ens>). If you look at the bottom section you'll see that for every app there is an additional section to specify the environment etc in the case of ensembles (e.g. note [[atmos_main]] and also [[atmos_main<ens>]]). You should look at these and make, for e.g., [[flight_track_sim<ens>]] and so on.

If I were you, I would start here and try to follow the previous apps in the suite as guidance.

Dave

comment:2 Changed 9 months ago by luciad

Hi Dave,

Thanks for explaining this.
I've added:

     {% if ENS_LOG %}
     [[flight_track_sim<ens>]]
        inherit = PPE_MEMB<ens>, ATMOS_RESOURCE, ATMOS
        pre-script = module load anaconda
        [[[environment]]]
            ROSE_TASK_APP = flight_track_sim<ens>
            CYCLEPERIOD = {{EXPT_RESUB}}
     {% endif %}

and

    {% if ENS_LOG %}
    [[warm_rain_out]]
        inherit = PPE_MEMB<ens>, ATMOS_RESOURCE, ATMOS
        pre-script = module load anaconda
        [[[environment]]]
            ROSE_TASK_APP = warm_rain_out<ens>
            CYCLEPERIOD = {{EXPT_RESUB}}
    {% endif %}

but now I get the error:

[FAIL] cylc validate -o /tmp/tmpQ4xncS --strict u-bq438 # return-code=1, stderr=
[FAIL] ERROR, parameter 'ens' undefined in 'inherit = PPE_MEMB<ens>, ATMOS_RESOURCE, ATMOS'
Last edited 9 months ago by ros (previous) (diff)

comment:3 Changed 9 months ago by dcase

One thing from the above is that you've put 2 [[warm_rain_out]] sections, and want [[warm_rain_out<ens>]] presumably.

More generally- you'll just have to compare to the apps that are already implemented; there's no better way to change and debug a script other than by doing this, and with lots of trial and error.

comment:4 Changed 9 months ago by luciad

Yes, sorry. That was an obvious mistake.

I also have a question. Firstly, is it doable to make two tasks work in parallel? And if yes, is it feasible to do so when working with ensembles? I am just wondering if there is a way to make the runs faster.

Thank you,
Lucia

comment:5 Changed 9 months ago by dcase

I would guess that the suite will handle this for you. The only thing to note is that if you're running on ARCHER there are limits to the number of jobs that can run concommitantly (16 is a limit, see here for details: https://www.archer.ac.uk/documentation/user-guide/batch.php#sec-5.14). If you need to set limits for cylc, there is documentation here: https://cylc.github.io/doc/built-sphinx/running-suites.html#internalqueues

comment:6 Changed 9 months ago by luciad

Thank you for your time!
Lucia

comment:7 Changed 9 months ago by ros

  • Description modified (diff)

comment:8 Changed 9 months ago by ros

  • Resolution set to answered
  • Status changed from new to closed
Note: See TracTickets for help on using tickets.