make & netcdf problem

[dhtjtks]

Hi all,

When I try to generate executable file using 'make' I got an error message like below.
Actually, NETCDF and HDF is not located in '/usr/local', but I am using 'modules' to manage softwares. So, the library is locate in /opt/asn/etc/modulefiles/netcdf. Is this the problem of this? If so, how can I change the path?

Many thanks.
Lee

##############################################
makefile:238: INCLUDING FILE /home/usajxl/make_macros.mk WHICH CONTAINS APPLICATION-DEPENDENT MAKE DEFINITIONS
make: svnversion: Command not found
cp -f /usr/local/include/netcdf.mod Build
cp: cannot stat `/usr/local/include/netcdf.mod': No such file or directory
make: *** No rule to make target `Build/MakeDepend', needed by `all'. Stop.
##############################################

[arango]

Use the build script I have addressed this issue countless times in this forum. See build script information in WikiROMS.

[dhtjtks]

Use the build script I have addressed this issue countless times in this forum. See build script information in WikiROMS.

Thanks!

[fereshteh]

hi
i set
export NETCDF_INCDIR=/home/fkomijani/NETCD4.2.1.1/include
in build file (in gfortran section) and with this path, upwelling example execution does not have error that it shows it can find netcfd.mod path correctly.i change FORT=? pgi to gfortranin makefile, and by executing "make" in trunk path again i got this error
makefile:238: INCLUDING FILE /home/fkomijani/make_macros.mk WHICH CONTAINS APPLICATION-DEPENDENT MAKE DEFINITIONS
cp -f /usr/local/include/netcdf.mod Build
cp: cannot stat `/usr/local/include/netcdf.mod': No such file or directory
make: *** No rule to make target `Build/netcdf.mod', needed by `Build/MakeDepend
but why?

[fereshteh]

i chose gfortran as roms compiler
1-should i change
ifdef USE_NETCDF4
NC_CONFIG ?= nc-config
NETCDF_INCDIR ?= $(shell $(NC_CONFIG) --prefix)/include
LIBS := $(shell $(NC_CONFIG) --flibs)
in linux-gfortran.mk file that located in Compilers forder to my own path?
2- what else i should change in linux-gfortran.mk
3-how can i change it, like these
NETCDF_INCDIR=/home/fkomijani/NETCD4.2.1.1/include
LIBS :=/home/fkomijani/NETCD4.2.1.1/lib
or not?
thanks

[kate]

i chose gfortran as roms compiler
1-should i change
ifdef USE_NETCDF4
NC_CONFIG ?= nc-config
NETCDF_INCDIR ?= $(shell $(NC_CONFIG) --prefix)/include
LIBS := $(shell $(NC_CONFIG) --flibs)
in linux-gfortran.mk file that located in Compilers forder to my own path?

Are you using the build script? If so, you should be changing that, setting USE_NETCDF4 and setting the appropriate NC_CONFIG. The build script is insanely verbose where it is choosing how to set NC_CONFIG - you can clean that up and just set it to point to yours.

2- what else i should change in linux-gfortran.mk

Nothing? Maybe some FFLAGS if you like.

3-how can i change it, like these
NETCDF_INCDIR=/home/fkomijani/NETCD4.2.1.1/include
LIBS :=/home/fkomijani/NETCD4.2.1.1/lib
or not?
thanks

That is not the correct LIBS. Have the script perform "nc-config --flibs", though you can check to see what this returns on your system. Still, have the script do it because it could change with time and versions.

[kate]

Actually, the genius with nc-config is that you want to let it be simply the nc-config that is in your path:

pacman6 292% which nc-config
/usr/local/pkg/netcdf/netcdf-4.3.0.gnu-4.7.3/bin/nc-config

or

pacman3 300% which nc-config
/usr/local/pgi/bin/nc-config

These are both on the system called pacman. The difference is that I've loaded different PrgEnv modules in each window, so that one knows about pgi, the other about gfortran (for purposes of things like mpif90). I don't want to have to change the build script or anything else when changing my programming environment. It should just do the right thing. All that logic in build.bash for setting NC_CONFIG is just clutter and confusing to boot.

So, if this is a small system where you build your own NetCDF, put it in your path. If this is a major supercomputer, poke the system people to use programming environment modules to do the right thing, including for MPI and NetCDF.

[fereshteh]

Hi dear Kate
i found that linux-gfortran.mk should not been changed (thank for this ). with regard build.bash setting (that use USE_NETCDF4=on , FORT=gfortran and USE_DEBUG=on) i defined and active some options in makefile for example:

Code: Select all

[color=#800000]MY_HEADER_DIR ?=/home/fkomijani/projects/upwelling9
 BINDIR ?=/home/fkomijani/projects/upwelling9
MY_ANALYTICAL_DIR ?=/home/fkomijani/projects/upwelling9
USE_DEBUG ?=on
USE_NETCDF4 ?=on
 FORT ?= gfortran[/color]
[color=#800000]and dont change: SCRATCH_DIR ?= Build
[/color]
and also set netcdf4 and hdf5 include and lib paths in .bashrc file (and in buid.bash if required) like:
[color=#800000]export NETCDF_INCDIR=/home/fkomijani/NETCD4.2.1.1/include
export HDF5_LIBDIR=/home/fkomijani/NETCD4.2.1.1/lib
export NETCDF_LIBDIR=/home/fkomijani/NETCD4.2.1.1/lib[/color]
but after typing “make” in command line ([color=#800000][fkomijani@localhost TRUNK]$ make[/color]), no error arise but nothing generate in my path which determined in BINDIR option and one folder called Build generate in trunk folder.
after that i guess that i shoule detemine SCRATCH_DIR ?=  to Build directory that generate after running ./build.bash, but this way can not solve my problem too.
1-Would you please tell me what is my problem   :?: ?
2- should SCRATCH_DIR been  detemined as SCRATCH_DIR =/home/fkomijani/projects/upwelling9/Build  :?: ?
3-also i did not see NCAR_VERSION ?= NCARG441 option in make file :?: 
thanks so  mush  again for your attention and replies, they help me always
cheers :D

[arango]

For inexperienced users the build script (build.bash or build.sh) is the way to go. It is more difficult to edit and modify the makefile. The makefile requires specific format and mistakes can be made easily.

It seem that some people are not big supporters of the build script. It is just a matter of taste. If you are an expert using ROMS, you may choose what you prefer to compile.

Anyway, the build script is very powerful:

• It is very verbose so you have all the sufficient information to make the appropriate decisions. I think that more documentation is always good instead of none. We are always open to suggestions on how to improve the documentation. Check WikiROMS for more information.
• The build script is provided for each applications the we distribute. They are found in the test repository which is separated from the ROMS trunk repository.
• It allows to separate your project from the source code. This is very important since you can overwrite the distributed header files for a test case or project (ROMS_APPLICATION.h), any of the analytical files (ana_*.h), ecosystem models (say nemuro.h; you may customize it), and so on. You just need to put those files in your project directory or subdirectory and tell the build script where they are.
• It allows to select various branches of the code very easily. You can run your project with different versions of ROMS.
• You can set-up all the needed libraries for a particular compiler. System administrators or installers put these libraries in different places. It is impossible for us to guess those places. Mostly all the errors reported in this forum are due to not reading the instructions carefully in the build script or in WikiROMS documentation. I know that most of us don't like to read complex documentation, but this has to be done at least once to be independent. We assume that all the users of ROMS are scientists with a lot of curiosity. This is not a model for an indifferent readers or a lazy users.
• It is very easy to select any of the flavors for parallelism available: distributed-memory or shared-memory. There are several MPI libraries available. We have them all. We need to compile all the different version of the NetCDF library with them. We have serial and parallel NetCDF library for every compiler (ifort, pgi, gfortran) in our computers. When compiling ROMS this has to be consistent. You need to compile ROMS with the same version of the compiler that you used for the NetCDF library. This is where all the novice user fail to notice that this is necessary. We have hundreds od messages in this forum of users panicking about this. Everyone needs to got to this process. It is an important part of the ROMS learning curve.
• You can specify the SCRATH_DIR to any place that you like.
• To compile, you just need to type build.bash or build.sh. All the environmental variables are private to the c-shell process. It does not over-write the environmental variables that you set-up when you login.

[kate]

In answer to fereshteh's question, I'm not sure exactly what error you are getting. However, when setting things in the file build.bash, you have to execute build.bash instead of typing "make". build.bash is a shell script wrapper around the make process, not yet another file for setting make options when using "make" directly.

Again, you no longer have to directly set NETCDF_LIBDIR and NETCDF_INCDIR anywhere. Set NC_CONFIG to your nc-config and let nc-config do the work. The commands "nc-config --prefix" and "nc-config --flibs" are used to set LIBS and NETCDF_INCDIR for you. I'm sure there is out-of-date documentation in abundance on this - ignore it. Maybe we'll fix it someday...

[arango]

Yes, it is true that since NetCDF introduced the nc-conf script the enviromental variables NETCDF_LIBDIR and NETCDF_INCDIR are not longer needed. However, this is not the case for older versions of the library that many users use, including us. This is the reason why the build scripts still have these environmental options. Well, ROMS still have a lot of secrets. Maybe there is stuff that doesn't make sense to users, but there is always a reason for it.

Although the NetCDF 4.x version produces files that compress well, we get better speed up in serial I/O with version 3.x. The HDF-5 library has its advantages but it is slow. The parallel I/O performance is not that good. We still get better speed up with serial I/O. I have been disappointed with this for awhile. Every time that I look at the library in the debugger, the constant recursive calls make me crazy. Then, I can see why it is so inefficient.

There is absolute NOT need to compile ROMS with a NetCDF version having the HDF-5 library if you are not running with parallel I/O and have parallel I/O architecture in your computer. The last time that I tried this it didn't work for me because of changes to the independent and collective I/O access in the parallel library. It is frustrating...

[kate]

It is possible to compile Netcdf4 in classic mode, gaining the benefits of nc-config without the baggage of HDF5. It is also possible to compile Netcdf4 with access to the new formats while still running it in classic mode for ROMS. That's what I do - you need to get all the tools able to read the new files before switching to them.

I've heard that some like to use the compression available in the new files.

[fereshteh]

dear Arango
thanks
i really try to do ROMS compiling step by step base on your constructions.
And i was able to compile it for test case.
My recent question was just about “Installing the ROMS/TOMS Plotting Package” section. As you told in this part i should check the user-defined options section of the makefile and edit the macro variables as needed. And i have done them. Then i move into the src directory and issue the make command for getting plotting executables ccnt, cnt, csec, and sec.
[fkomijani@localhost TRUNK]$ make
makefile:234: INCLUDING FILE /home/fkomijani/projects/upwelling9/Build/make_macros.mk WHICH CONTAINS APPLICATION-DEPENDENT MAKE DEFINITIONS
rm -f -r /home/fkomijani/make_macros.mk
and nothing arised in PLT_BINDIR directory (one option in makefile).
i like to say that i use netcdf4.2.1.1 which installed with zlib and hdf5.
I was just asking about reason of this? And the way for solve this problem?
thanks again for your reply
cheers
fereshte