Cylc Broadcast with environment variable

[luciana]

Hello.

I'm trying to use cylc broadcast to change the configuration of the next cycle. It works when I use cylc broadcast inside the suite, but it doesn't work when I try to replicate the same idea with a broadcast.rc file. The general issue is to define a variable that depends on the environment inside the broadcast.rc file.

Here is the suite: (also in /home/users/lucy/puma/var-test)

—-
[scheduling]

initial cycle point = 2021
final cycle point = 2023
[ [dependencies] ]

[ [ [R1] ] ] # Initial cycle point.

graph = model ⇒ broadcast-test

[ [ [RP1Y] ] ] # Yearly cycling.

graph = model[-P1Y] ⇒ model ⇒ broadcast-test

[runtime]

[ [model] ]

[ [ [environment] ] ]

VAR_DIR = $CYLC_TASK_WORK_DIR

[ [broadcast-test] ]

script = """

NEXT_CYCLE=$(cylc cycle-point —offset-years=1)
cylc broadcast -n 'model' -s "[environment]VAR_DIR=$CYLC_SUITE_WORK_DIR/$NEXT_CYCLE" "${CYLC_SUITE_NAME}"
# cylc broadcast -n 'model' -F "${CYLC_SUITE_DEF_PATH}/broadcast.rc" "${CYLC_SUITE_NAME}"
"""

—-
[lucy@cylc1 var-test]$ more broadcast.rc
#!jinja2
{% set NEXT_CYCLE='$(cylc cycle-point —offset-years=1)' %}

[environment]

# NEXT_CYCLE=$(cylc cycle-point —offset-years=1)
# VAR_DIR=$CYLC_SUITE_WORK_DIR/${NEXT_CYCLE}
VAR_DIR=$CYLC_SUITE_WORK_DIR/{{NEXT_CYCLE}}

—-

This suite runs and works the way I expect. However, I couldn't reproduce the same results using broadcast.rc. I've tried two different approaches you can see commented in original the file: local variable and jinja2 variable, without success.

Is there a way to make this work? (In general, not only with this specific NEXT_CYCLE variable.)

—-
Output in cylc-run/var-test-working

2021-01-08T00:38:39Z INFO - Broadcast set:

+ [model.*] [environment]VAR_DIR=/home/users/lucy/cylc-run/var-test-working/work/20240101T0000Z

—-
Output in cylc-run/var-test-not-working

2021-01-08T00:52:03Z INFO - Broadcast set:

+ [model.*] [environment]NEXT_CYCLE=$(cylc cycle-point —offset-years=1)
+ [model.*] [environment]VAR_DIR=$CYLC_SUITE_WORK_DIR/${NEXT_CYCLE}

—-
Output in cylc-run/var-test-jinja-not-working

2021-01-08T00:48:54Z INFO - Broadcast set:

+ [model.*] [environment]VAR_DIR=$CYLC_SUITE_WORK_DIR/$(cylc cycle-point —offset-years=1)

—-

Kind regards.

Luciana.

[luciana]

Hi.

I finally figure it out a way to do what I wanted from this ticket. Not the most elegant way, but it works.

—-
[lucy@cylc1 bin]$ more write-broadcast-file.sh
#!/bin/bash

NEXT_CYCLE=$(cylc cycle-point —offset-years=1)

# Create broadcast.rc file
cat > ${CYLC_SUITE_DEF_PATH}/broadcast.rc << EOF
[environment]

VAR_DIR=$CYLC_SUITE_WORK_DIR/${NEXT_CYCLE}

EOF
—-

This example is on: /home/users/lucy/puma/var-test2.

I also tried the idea of writing the variable in a file and then reading the file inside write-broadcast-file.sh. It works and it's good because I can do it with any external variable, not only with something that comes from a Cylc command.

One thing I think it's important to notice is that the following command should have worked:

—-
broadcast.rc
—-
[environment]
NEXT_CYCLE=$(cylc cycle-point —offset-years=1)
VAR_DIR=$CYLC_SUITE_WORK_DIR/${NEXT_CYCLE}
—-

There are several examples in Cylc Tutorial that use this kind of construction:

—-
[ [ [environment] ] ]

CYCLE_TIME = $(cylc cyclepoint —template=%Y%m%d%H)
REFERENCE_TIME = $( cylc util cycletime —offset-hours=6 )
…

—-

That makes me think the problem is in the broadcast implementation in general, and not an extra functionality required by an annoying user (= me! : ) ).

But I'm happy I find a way to make it work and also improved my bash skills. : )

Kind regards.

Luciana.