Opened 6 weeks ago

Closed 5 weeks ago

Last modified 2 weeks ago

#3370 closed help (fixed)

python 2.7

Reported by: NoelClancy Owned by: jules_support
Component: Land Surface Modelling Keywords: Python2.7, JASMIN, FLUXNET, jaspy
Cc: Platform: JASMIN
UM Version:

Description

Following the change from
jasmin-sci1.ceda.ac.uk
to
sci1.jasmin.ac.uk

iPython was not working

but
module load jasmin-sci
module load jaspy

iPython is running in python 3
this is creating problems with scripts already written in python 2.7
How can I return to the python 2.7 version and use ipython?

Change History (18)

comment:1 Changed 6 weeks ago by ros

Hi Noel,

The default jaspy module on the sci nodes is now python3. Try loading the python2.7 version with module load jaspy/2.7

Cheers,
Ros.

comment:2 Changed 6 weeks ago by NoelClancy

Where can I put module load jaspy/2.7 so that it is executed automatically when I log in?

Will my environment be correct then for running JULES python scripts?

Thanks

comment:3 Changed 6 weeks ago by pmcguire

Hi Noel:
What JULES python scripts are you trying to run? Were they written in Python2.7 or in Python3?

You can try to do a module load jaspy/2.7 and then see if the Python scripts still work or not.
If they don't, then you can do a module unload jaspy/2.7 .

If everything is working, then you can put the module load jaspy/2.7 in your .bashrc file.
This presumes that you aren't running anything else that uses Python3.
Patrick

comment:4 Changed 6 weeks ago by NoelClancy

There are still some problems following your first suggestion

module load jaspy/2.7

File "/apps/contrib/jaspy/miniconda_envs/jaspy2.7/m2-4.6.14/envs/jaspy2.7-m2-4.6.14-r20190715/lib/python2.7/site-packages/nc_time_axis/init.py", line 274, in convert

raise ValueError('The datetime attribute of the CalendarDateTime '

ValueError: The datetime attribute of the CalendarDateTime object must be of type cftime.datetime.

But I am using files based on fluxnet_evaluation.py so I believe it is python2.7 I need to use.

Last edited 6 weeks ago by pmcguire (previous) (diff)

comment:5 Changed 6 weeks ago by pmcguire

  • Component changed from JASMIN to Land Surface Modelling
  • Keywords Python2.7, JASMIN, FLUXNET, jaspy added
  • Owner changed from um_support to jules_support
  • Platform set to JASMIN

comment:6 Changed 6 weeks ago by pmcguire

Hi Noel:
I copied your file ~nmc/scr/new.py into ~pmcguire/test/new.py, and I have the same problem as you with it.

I put an extra line:
print rad_net_H_40_A_expt_cube[0:].coords()
and it looks like the time coordinate is in seconds. Maybe that is why it's not working as cftime.datetime, since maybe the formats are different??

I ran your code with python2.7 after module load jasmin-sci and module load jaspy/2.7.
I haven't used module load jasmin-sci before.

On another note, why do you import iris.quickplot as qplt so many times? You only need to do it once.
This applies to other imports as well.
Patrick

comment:7 Changed 6 weeks ago by NoelClancy

Hi Patrick

Does ticket #2800 explain it?

comment:8 Changed 6 weeks ago by pmcguire

Hi Noel:
That might be it! Good sleuthing!

I note that when I do the following on either sci2.jasmin or cylc1.jasmin:

module load jaspy/2.7
python2.7
 ****             Python 2.7.15 | packaged by conda-forge | (default, May  8 2018, 14:46:53) 
>>> import iris
>>> print(iris.__version__)
1.13.0

you'll see that the version of iris is 1.13.0. This is older than what Azin and I were using 1.5 years ago, which was iris 2.1 or 2.2.

So I am not surprised that you're having problems.

Azin, and V, and I fixed that problem 1.5 years ago by installing a custom conda environment.
We didn't use jaspy, since that didn't exist yet.

You should be able to install iris2.2 by installing a custom conda or miniconda environment, similar to:
http://cms.ncas.ac.uk/ticket/2800#comment:9

There are some clarifying comments after that comment, and maybe the original environment file
/home/users/valeriu/patrick_env.yml needs to be updated to the short version that V (Valeriu) made, that he
pointed us to later in the comments.

Patrick

comment:9 Changed 6 weeks ago by NoelClancy

If I type the either of the following on the command line
conda env create -n plotting_environment -f /home/users/valeriu/patrick_env.yml
conda env create -n plotting_environment -f /home/users/valeriu/patrick_env_short.yml

I get the following message
conda: error: argument command: invalid choice: u'.' (choose from u'clean', u'config', u'create', u'help', u'info', u'init', u'install', u'list', u'package', u'remove', u'uninstall', u'run', u'search', u'update', u'upgrade')

Is this how I am supposed to install iris2.2?

comment:10 Changed 5 weeks ago by NoelClancy

(base) [nmc@sci2 ~]$ conda env create -n pmg_env -f /home/users/valeriu/patrick_env.yml
Collecting package metadata (repodata.json): done
Solving environment: failed

(base) [nmc@sci2 ~]$ conda activate pmg_env
Could not find conda environment: pmg_env

(base) [nmc@sci2 ~]$ conda env create -n pmg_env_short -f /home/users/valeriu/patrick_env_short.yml
Collecting package metadata (repodata.json): done
Solving environment: done

(base) [nmc@sci2 ~]$ conda activate pmg_env_short
(pmg_env_short) [nmc@sci2 ~]$

comment:11 Changed 5 weeks ago by NoelClancy

Now it is working

comment:12 Changed 5 weeks ago by pmcguire

Hi Noel:
Excellent! I am glad it is working! Good job!
I will close the ticket now.
Don't hesitate to create a new ticket if you have questions/issues.
Patrick

comment:13 Changed 5 weeks ago by pmcguire

  • Resolution set to fixed
  • Status changed from new to closed

comment:14 Changed 3 weeks ago by NoelClancy

Patrick, the environment
[nmc@sci2 ~]$ conda env create -n pmg_env_short -f /home/users/valeriu/patrick_env_short.yml

it works well, however, However, sometimes, but not always I get "segmentation fault" and have to re-run the script

Do you know why, or how to fix it, could it be fixed by adding extra libraries?

comment:15 Changed 3 weeks ago by NoelClancy

It's working with out that error, maybe it works if there is a lot of free memory?

sci3.jasmin.ac.uk
Users 46
Free memory 848.9G
CPU 60.0%

comment:16 Changed 3 weeks ago by NoelClancy

Ticket closed

comment:17 Changed 3 weeks ago by pmcguire

Hi Noel:
How much memory does your program use? Can you see that with top? How long does it take for your program to run? How many processors are you using?
Patrick

comment:18 Changed 2 weeks ago by pmcguire

Hi Noel:
Did you figure out how much memory and how many processors your program is using?
Patrick

Note: See TracTickets for help on using tickets.