Opened 10 years ago

Closed 10 years ago

#465 closed help (fixed)

MAKEBC with lots of files?

Reported by: danpeake Owned by: willie
Component: UM Tools Keywords: makebc
Cc: g.m.s.lister@… Platform:
UM Version: 6.1

Description

Hi Willie and Chris - I'm emailing you as I wasn't sure who to email, so
I hope (if you can't fix it) you can point me in the right direction as
to who can fix it.

I've been running the MAKEBC program to generate LBC files from global
start dumps, but I'm inputting so many files that when it gets to
opening Port 100 (the first three digit one), it fails.

Is there a way to address this? I attach below the code from the error
message.

OPEN: File /work/n02/n02/danpeake/MYDD/DD_20040715_20 to be Opened on
Unit 99 Exists

this is a Dump

u_field_intfa set to 307200


Model Grid/Levels? in this dump.
row_length = 640
rows = 481
n_rows = 480
model_levels = 50
wet_levels = 50
tr_levels = 0
p_field = 307840
u_field = 307200


JTRACER = -1
No tracer data in this dump.

n_flds= 1012
LBC_CHK_VI : Vertical Interpolation required for LBC Area 1


Dump No 69 : Calling IN_INTF.
IN_INTF calls INTF_HINTC for Dump No 1.
nftout from intf_unit 140


Dump No 69 : Calling GEN_INTF.

Gen_Intf: Timestep 68 : Generate Atmos LBCs for Area 1


Processing dump no 70


OPEN: WARNING: Environment variable FILE not set

Error in opening dump on unit no 100

Error in LOOP_OVER_DUMPS

Change History (7)

comment:1 Changed 10 years ago by danpeake

I've seen Grenville Lister, and he's given me a link to code that deals with this issue. However, not all the dumps I put in the input file are processed into the output file. I input 312 start dumps at hourly intervals, but it only outputs the lateral boundary conditions at 249 hourly timesteps - some 24 hours periods appear to be skipped.

Why is this happening?

comment:2 Changed 10 years ago by willie

  • Owner changed from um_support to willie
  • Status changed from new to accepted

Hi Dan,

I am not sure. It is interesting that 64 are missing. What script are you using?

Regards,

Willie

comment:3 Changed 10 years ago by danpeake

I've tried running it a couple of times, with different number of dumps, and each time only 249 timesteps are output - so it might not be how many its missed out but how many it can physically write to a file?

The most recent run of this example was /work/n02/n02/danpeake/makebc_runscript_lbc3 (submitted using qsub serial3.tcl in the same directory) - the job was 771598 I believe (you can see the .e771598 and .o771598 files in /work/n02/n02/danpeake).

comment:4 Changed 10 years ago by willie

Hi Dan,

I had a look at your script. Could you edit it and replace $F0 … $F359 with

/work/n02/n02/danpeake/MYDD/DD*

and run it again. The wild card expansion will produce the list you want and in the right order. There is also less chance of contravening line lengths in Unix.

Regards,

Willie

comment:5 Changed 10 years ago by willie

Hi Dan,

Even better would be to

cd /work/n02/n02/danpeake/MYDD
makebc_script -n $NL -i DD* -o etc

Regards,

Willie

comment:6 Changed 10 years ago by willie

  • Cc g.m.s.lister@… added

Hi Dan,

I have now extended the vn6.1 makebc to process your files. The resulting LBC file is in /work/n02/n02/wmcginty/makebc_tst. Please copy this to your own workspace.

The original code was limited to 200 start dumps. The extensions required an increase in the size of the domain to 700x500, an increase to 999 start dumps and a modification of the makebc_script to match the code. The code is currently compiled with debug and array bounds checking on and takes about 4 and a half hours on HECToR 2a to process 359 dumps. When the code is compiled with optimization (-O3) it fails (segmentation fault) on the first dump.

Since we have extended the code, it would be wise to examine the output most carefully.

Regards,

Willie

comment:7 Changed 10 years ago by willie

  • Resolution set to fixed
  • Status changed from accepted to closed
Note: See TracTickets for help on using tickets.