Opened 12 years ago

Closed 11 years ago

#268 closed help (fixed)

Error opening PP files and with byteswapping

Reported by: bethan.white Owned by: willie
Component: UM Tools Keywords: byteswap, pp
Cc: Platform:
UM Version:

Description

I have PP output from a UM job (xdsvt) which I have copied from Hector to my local machine at Leeds. I was able to open all of the the files using xconv on Hector, but I am no longer able to open some of the files now I have copied them across. Also, I have noticed that the files which I can open are already byteswapped (.pp0 and .pp5) while the files which I cannot open are not byteswapped - and byteswapping does not solve the problem. Is there any reason as to why I can open some but not all files, and why some files seem to have been produced byteswapped, but others not? It seems like very odd behaviour and I am not sure how to address the problem.

Thanks,
Bethan.

Change History (7)

comment:1 Changed 12 years ago by willie

  • Component changed from Data to UM Tools
  • Status changed from new to assigned

Hi Bethan,

This depends on the architecture of the computer that you copy the files to. On HECToR xconv shows that the files are all byte swapped 64 bit um ieee format. HECToR is a x86_64, or little endian - try the command 'uname -p' on each computer to find out what architectures you are using. Other little endians are i386 and athlon; big endians are sparc and some others. The thing to do is to make sure you copy to the same kind of computer each time.

Regards,

Willie

comment:2 Changed 12 years ago by bethan.white

Hi Willie,

Thanks for your reply. This is what is confusing me - I copied the files from Hector to the same Leeds computer in consecutive order - there should be no difference in the architecture used at each end. uname -p shows that both machines are x86_64.

I can open every file with xconv on Hector but only some on my local machine. Is there any way the files may have corrupted during the scp process perhaps?

Thanks,
Bethan.

comment:3 Changed 12 years ago by willie

Hi Bethan,

Yes that's possible. Do ls -l on both computers to check that the number of bytes is the same. You can also compute a check sum using the sum -r command on each machine: they should give identical results.

Regards,

Willie

comment:4 Changed 12 years ago by bethan.white

Hi Willie,

The number of bytes is the same for each file at Hector/Leeds?. sum -r also gives the same result.

I don't know if this helps, but I have noticed that the two files I can open in xconv locally are the two smallest files (.pp0 and .pp5, 1013645312 and 6258688 bytes respectively). The rest I cannot open locally in xonv and are between 4209524736 and 6672613376 bytes in size). The error that xconv gives is: "(file name) cannot be read, please reselect filetype from below".

Might there be an issue with filesizes?

Cheers,
Bethan.

comment:5 Changed 12 years ago by willie

Hi Bethan,

Some computers can only address 32 bits worth of bytes, which is 4,294,967,296 and as you say these files are bigger than that. You may have to rebuild your xconv program on your local machine so that it uses 64 bit addressing.

Regards,

Willie

comment:6 Changed 12 years ago by bethan.white

Hi Willie,

Thanks for getting to the bottom of this. I will contact the support people in Leeds and ask them to look at this.

Cheers,
Bethan.

comment:7 Changed 11 years ago by willie

  • Resolution set to fixed
  • Status changed from assigned to closed
Note: See TracTickets for help on using tickets.