UM jobs failing to build the codebase (can't locate ConfigEntry.pm)

[gmann]

Dear NCAS-CMS helpdesk,

This evening when trying to submit UM job from PUMA to ARCHER
I'm encountering the following failure on ARCHER (see error
message below).

It looks like, for some reason, the build script is not able
to locate the necessary files to enable FCM to build the code
(prior to compilation).

Has some part of the FCM build on ARCHER been taken down somehow?

Thanks for any help you can give.

I have given this priority "highest" as I expect it will be
affecting anyone who is submitting UM jobs at this particular
time (for example someone who is working flexible hours during
the evening in lieu of time taken off during the day, to spend
time with young family for example).

Regards,
Graham

Can't locate FCM/Context/ConfigEntry.pm in @INC (@INC contains: /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib /usr/lib/perl5/5.10.0/x86
_64-linux-thread-multi /usr/lib/perl5/5.10.0 /usr/lib/perl5/site_perl/5.10.0/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.10.0 /usr/lib/p
erl5/vendor_perl/5.10.0/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.10.0 /usr/lib/perl5/vendor_perl .) at /fs2/y07/y07/umshared/softwa
re/fcm-2016.02.0/bin/../lib/FCM/Util/ConfigReader.pm line 26.
BEGIN failed--compilation aborted at /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib/FCM/Util/ConfigReader.pm line 26.
Compilation failed in require at /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib/FCM/Util.pm line 30.
BEGIN failed--compilation aborted at /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib/FCM/Util.pm line 30.
Compilation failed in require at /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib/FCM/System.pm line 26.
BEGIN failed--compilation aborted at /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib/FCM/System.pm line 26.
Compilation failed in require at /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib/FCM/CLI.pm line 29.
BEGIN failed--compilation aborted at /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib/FCM/CLI.pm line 29.
Compilation failed in require at /work/y07/y07/umshared/software/fcm/bin/fcm line 25.
BEGIN failed--compilation aborted at /work/y07/y07/umshared/software/fcm/bin/fcm line 25.
COMP_UMSCRIPTS is true - run build command
Can't locate FCM/Context/ConfigEntry.pm in @INC (@INC contains: /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib /usr/lib/perl5/5.10.0/x86
_64-linux-thread-multi /usr/lib/perl5/5.10.0 /usr/lib/perl5/site_perl/5.10.0/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.10.0 /usr/lib/p
erl5/vendor_perl/5.10.0/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.10.0 /usr/lib/perl5/vendor_perl .) at /fs2/y07/y07/umshared/softwa
re/fcm-2016.02.0/bin/../lib/FCM/Util/ConfigReader.pm line 26.
BEGIN failed--compilation aborted at /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib/FCM/Util/ConfigReader.pm line 26.
Compilation failed in require at /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib/FCM/Util.pm line 30.
BEGIN failed--compilation aborted at /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib/FCM/Util.pm line 30.
Compilation failed in require at /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib/FCM/System.pm line 26.
BEGIN failed--compilation aborted at /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib/FCM/System.pm line 26.
Compilation failed in require at /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib/FCM/CLI.pm line 29.
BEGIN failed--compilation aborted at /fs2/y07/y07/umshared/software/fcm-2016.02.0/bin/../lib/FCM/CLI.pm line 29.
Compilation failed in require at /work/y07/y07/umshared/software/fcm/bin/fcm line 25.
BEGIN failed--compilation aborted at /work/y07/y07/umshared/software/fcm/bin/fcm line 25.
UMSCRIPTS build failed

[ros]

Hi Graham,

This looks to be a problem with ARCHER as nothing has changed with our FCM installation. We are investigating.

Regards,
Ros.

[ros]

Hopefully you will have received the following mailing from ARCHER:

ARCHER Serial PP Nodes issue

We are currently working on an issue with the ARCHER Serial post-processing nodes which is causing jobs running on them to fail.

In order to resolve this issue our team will be re-starting the PP nodes later today. Any jobs running when the nodes are re-started will fail (but it is likely that they would fail anyway due to the ongoing issue).

We apologise for the inconvenience caused and will let you know once the restart is completed and the nodes are returned to service.

The ARCHER Helpdesk Team

[ros]

Hi Graham,

The serial PP nodes have been restarted and compilation jobs are now working again.

I'll close this ticket now, but if you are still having problems please reopen it.

Regards,
Ros.