advanced debug of job

[swr04ojb]

Hi NCAS,

we have a bug that is proving tricky to chase. What would be ideal would be if we could step through the routine line-by-line, querying variable values as we did so. The UM vm8.5 atmos-only job is on ARCHER, is such a form of debug possible? I've been referred to this page by you before:

​http://www.archer.ac.uk/documentation/best-practice-guide/debug.php

, which I've used for stack-traces, so thank you for that, but I don't know how to implement an interactive debug such as described above. Has anyone attempted this before with the UM on ARCHER? Are there guidance notes somewhere?

cheers,

Oliver

[swr04ojb]

UPDATE:

Simon's found that compiling/running the UM on an interactive ARCHER node significantly reduces the development cycle. From a typical turnaround of ~20minutes using the short-queue we're now down to 5 minutes:

a few minutes for the submit from PUMA
+ 30secs for compile
+ ~30-60sec to get say 100 timesteps in & then stop the model to look for errors.

That is nice, but still doesn't get us *inside* the routine as most modern debuggers would allow.

We can switch on debug-flags on individual files, via hand-edit:

http://cms.ncas.ac.uk/ticket/1449

Presumably we'd now just need a way to attaching a debugger to the process-id?

[grenville]

Oliver

We don't yet have experience with DDT - we previously used totalview.

Grenville

[swr04ojb]

Grenville,

thanks. I've installed a DDT client, and got it talking to ARCHER okay. I'm at 7.5.5 on:

​http://www.archer.ac.uk/documentation/best-practice-guide/debug.php#sec-7.5

which is the step where I would configure it to call the executable, but I'm unsure on what to specify there - should it be the stage_1_submit file, the umuisubmit_run file, the UMScr_Toplevel file, or something else?

cheers,

Oliver

[grenville]

Oliver

It wants the executable - it's not obvious how to specify that. TotalView? worked differently, in that you'd put a wrapper around aprun (that was easy) - it may be that ddt allows the same method of working, but I don't know yet.

Grenville