Cylc Broadcast with environment variable

[luciana]

Hello.

I'm trying to use cylc broadcast to change the configuration of the next cycle. It works when I use cylc broadcast inside the suite, but it doesn't work when I try to replicate the same idea with a broadcast.rc file. The general issue is to define a variable that depends on the environment inside the broadcast.rc file.

Here is the suite: (also in /home/users/lucy/puma/var-test)

—-

[scheduling]
  initial cycle point = 2021
  final cycle point = 2023
  [ [dependencies] ]
    [ [ [R1] ] ]  # Initial cycle point.
      graph = model => broadcast-test
    [ [ [R//P1Y] ] ]  # Yearly cycling.
      graph = model[-P1Y] => model => broadcast-test

[runtime]
  [ [model] ]
    [ [ [environment] ] ]
      VAR_DIR = $CYLC_TASK_WORK_DIR
  [ [broadcast-test] ]
     script = """
        NEXT_CYCLE=$(cylc cycle-point --offset-years=1)
        cylc broadcast -n 'model' -s "[environment]VAR_DIR=$CYLC_SUITE_WORK_DIR/$NEXT_CYCLE" "${CYLC_SUITE_NAME}"
        # cylc broadcast -n 'model' -F "${CYLC_SUITE_DEF_PATH}/broadcast.rc" "${CYLC_SUITE_NAME}"
        """

—-

[lucy@cylc1 var-test]$ more broadcast.rc 
#!jinja2
{% set NEXT_CYCLE='$(cylc cycle-point --offset-years=1)' %}

[environment]
	# NEXT_CYCLE=$(cylc cycle-point --offset-years=1)
        # VAR_DIR=$CYLC_SUITE_WORK_DIR/${NEXT_CYCLE}
        VAR_DIR=$CYLC_SUITE_WORK_DIR/{{NEXT_CYCLE}}

—-

This suite runs and works the way I expect. However, I couldn't reproduce the same results using broadcast.rc. I've tried two different approaches you can see commented in original the file: local variable and jinja2 variable, without success.

Is there a way to make this work? (In general, not only with this specific NEXT_CYCLE variable.)

—-
Output in cylc-run/var-test-working

2021-01-08T00:38:39Z INFO - Broadcast set:
	+ [model.*] [environment]VAR_DIR=/home/users/lucy/cylc-run/var-test-working/work/20240101T0000Z

—-
Output in cylc-run/var-test-not-working

2021-01-08T00:52:03Z INFO - Broadcast set:
	+ [model.*] [environment]NEXT_CYCLE=$(cylc cycle-point --offset-years=1)
	+ [model.*] [environment]VAR_DIR=$CYLC_SUITE_WORK_DIR/${NEXT_CYCLE}

—-
Output in cylc-run/var-test-jinja-not-working

2021-01-08T00:48:54Z INFO - Broadcast set:
	+ [model.*] [environment]VAR_DIR=$CYLC_SUITE_WORK_DIR/$(cylc cycle-point --offset-years=1)

—-

Kind regards.

Luciana.

[luciana]

Hi.

I finally figure it out a way to do what I wanted from this ticket. Not the most elegant way, but it works.

—-

[lucy@cylc1 bin]$ more write-broadcast-file.sh 
#!/bin/bash

NEXT_CYCLE=$(cylc cycle-point --offset-years=1)

# Create broadcast.rc file
cat > ${CYLC_SUITE_DEF_PATH}/broadcast.rc << EOF
[environment]
        VAR_DIR=$CYLC_SUITE_WORK_DIR/${NEXT_CYCLE}
EOF

—-

This example is on: /home/users/lucy/puma/var-test2.

I also tried the idea of writing the variable in a file and then reading the file inside write-broadcast-file.sh. It works and it's good because I can do it with any external variable, not only with something that comes from a Cylc command.

One thing I think it's important to notice is that the following command should have worked:

—-
broadcast.rc
—-

[environment]
NEXT_CYCLE=$(cylc cycle-point —offset-years=1)
VAR_DIR=$CYLC_SUITE_WORK_DIR/${NEXT_CYCLE}

—-

There are several examples in Cylc Tutorial that use this kind of construction:

—-

[ [ [environment] ] ]
            CYCLE_TIME = $(cylc cyclepoint --template=%Y%m%d%H)
            REFERENCE_TIME = $( cylc util cycletime --offset-hours=6 )
            ...

—-

That makes me think the problem is in the broadcast implementation in general, and not an extra functionality required by an annoying user (= me! : ) ).

But I'm happy I find a way to make it work and also improved my bash skills. : )

Kind regards.

Luciana.

[ros]

Glad you found a solution. I will close this ticket now.