Opened 7 weeks ago

Closed 5 days ago

#2596 closed help (fixed)

Segmentation fault: address not mapped to object

Reported by: shakka Owned by: willie
Priority: highest Component: UM Model
Keywords: segmentation fault Cc:
Platform: Monsoon2 UM Version: 11.1

Description

Hi CMS,

I'm having more issues with my suite u-ba502. I've tried to add in a new 15 minute time mean STASH usage profile (with the settings shown in the attached image), but am getting a segmentation fault which I don't understand. As this is the only discernible difference between previous configurations that ran successfully, I assume that this is the source of the problem.

The output of job.err looks much like this:

[11] exceptions: An exception was raised:11 (Segmentation fault)
[11] exceptions: the exception reports the extra information: Address not mapped to object.
[11] exceptions: whilst in a serial region
[11] exceptions: Task had pid=10856 on host nid00055
[11] exceptions: Program is "/home/d04/elgil/cylc-run/u-ba502/share/fcm_make/build-atmos/bin/um-atmos.exe"
Warning in umPrintMgr: umPrintExceptionHandler : Handler Invoked


[FAIL] um-atmos # return-code=137

and job.out doesn't give much more detail than this:

WARNING - REQUESTED AND ACTUAL THREADING LEVEL DIFFERENT
THREAD LEVEL REQUESTED is MPL_THREAD_MULTIPLE
THREAD LEVEL SET is MPL_THREAD_SERIALIZED
Application 38500773 exit codes: 139
Application 38500773 exit signals: Killed
Application 38500773 resources: utime ~18s, stime ~63s, Rss ~86784, inblocks ~5149601, outblocks ~1730226
PE0 file exists, but has zero size

Could you please help? I very urgently need to get this suite to run.

Thanks in advance,
Ella

Attachments (1)

Capture.PNG (72.3 KB) - added by shakka 7 weeks ago.
time usage profile set up

Download all attachments as: .zip

Change History (7)

Changed 7 weeks ago by shakka

time usage profile set up

comment:1 Changed 7 weeks ago by grenville

Ella

Your 15 min mean isn't doing much useful - as a quick thing to try, change the sampling frequency to 1 and the unit for sampling frequency to timesteps (cf 1hr-mn).
Grenville

comment:2 Changed 7 weeks ago by shakka

Hi Grenville,

Thanks for the quick response. I did actually try these settings previously, and got the same error. I'll have another go though and see if that changes anything.

I'm also getting an error in the archiving - it's attempting to archive the extra diagnostic files (which I've turned off, so aren't being generated), but failing because they are zero-length, unsurprisingly. Jonathan Wilkinson who developed the suite suggested trying a suite clean in case it was finding old verification files, but that didn't work. Can you think of anything that I could do to fix this?

Ella

comment:3 Changed 7 weeks ago by shakka

Hi Grenville,

I've tried re-running with the setup you suggested, but I'm getting the same error. I think it might be related to only some STASH requests though, because I also tried using the 15 min mean profile only for some STASH profiles, and that seemed to work. I'll try and get to the bottom of which ones are causing the failure.

Ella

comment:4 Changed 5 days ago by willie

  • Owner changed from um_support to willie
  • Status changed from new to accepted

Hi Ella,

What's the current status of this - are you still having problems?

Regards
Willie

comment:5 Changed 5 days ago by shakka

Hi Willie,

I found a STASH variable that was acting up and removed it, which seems to have solved the problem.

Thanks,
Ella

comment:6 Changed 5 days ago by shakka

  • Resolution set to fixed
  • Status changed from accepted to closed
Note: See TracTickets for help on using tickets.