Availability/Download/Installation of the EDSS/Models-3 I/O API

Contents

Back to the I/O API User Manual

Foreword

Please read the copyright, licensing, and acknowledgements page: The I/O API library is available under the Lesser Gnu Public License, version 2.1; the "tool" and example programs are available under the Gnu Public License, version 2.

The What's New page documents bug-fixes, new features, new compiler-support, etc.

Note that a large amount of effort has been expended during the whole of I/O API development to ensure upward compatibility: programs written for the original Version 0.9 pre-release of November, 1992 should still work correctly with the latest (August 2014) I/O API Version 3.1, unlike some other software we will refrain from naming here. There are very few upward compatibility issues between Versions 3.1 and 3.2, mostly coming from the renaming and generalization of MODULE MATXATTS as MODULE MODATTS3.

In general, you are best off if you can build your whole modeling system (libnetcdf.a, libpvm3.a, libioapi.a, and your model(s) CMAQ, SMOKE, etc. with a common compiler set and common set of compile-flags.

If you run into troubles with I/O API related programs, it is useful to know the versions of all the software components. The CVS-related program ident can report to you versioning keywords in the various components of source, object, library, or executable files. For example: 

% cd $HOME/apps/${BIN}
% ident init3.o
init3.o:
     $Id:: init3.F 1 2014-03-14 20:22:54Z coats                  $
% ident libioapi.a
libioapi.a:
     $Id:: init3.F 1 2014-03-14 20:22:54Z coats                  $
     $Id:: m3utilio.f 30 2014-08-07 14:34:44Z coats              $
% ident m3stat
m3stat:
     $Id:: m3stat.f 1 2014-03-14 20:22:54Z coats                   $
     $Id:: init3.F 1 2014-03-14 20:22:54Z coats                    $
     $Id: @(#) netcdf library version 3.6.1 of Feb  7 2014 15:05:22 $
Go to Contents

Build and installation instructions for I/O API Versions 3.0, 3.1, 3.2, and 4.0

As of this writing (June 2019), I/O API Versions 2.2 and earlier are obsolete; 3.0 and 3.1 obsolescent, in critical-fixes-only mode (and does not work with netCDF version 4 because of version-to-version incompatibilites in that product); 3.2 is the current stable/production version. 4.0 is the new (beta, as of 10/21/2021) "e;all Fortran converted to Standard conforming .f90 source code format. See the Change Log / "What's New" / Research Directions page for further information.

Basically, the I/O API is developed as a "rolling release" software system, with bug-fixes available as soon as they are made. This is most easily accessed by way of the GitHub version of the release, since git will automatically update exactly what needs to be updated, allowing you to do an as-simple-as-possible make to install the update.

To set up the installation:
  1. Download the gzipped tar-file of the source code for the version you want from the CMAS web-site:
    For the "rolling release" version
    ioapi-4.0.tar.gz
    ioapi-3.2.tar.gz
    ioapi-3.1.tar.gz
    , or ioapi-3.0.tar.gz.
    or for the April 20, 2020, August 7, 2020, or August 28, 2020 versions of I/O API Version 3.2:
    ioapi-3.2-2020111.tar.gz ioapi-3.2-2020220.tar.gz ioapi-3.2-20200828.tar.gz
    This "tar-ball" contains directories ioapi for the I/O API library source code, HTML for the documentation, m3tools for the related tool programs, and for version 3.0, iotest for various test-programs for I/O API developers, and notcdf for the NCEP libnotcdf.a library (a dummy library for installation sites that don't allow netCDF on-site).

    For CMAQ-DDM, ioapi-3.2-large has expanded maximum-number-of-files and maximum-variables-per-file parameter-values. This version should be kept carefully isolated from the regular version of the I/O API, and should be used only for CMAQ-DDM and CMAQ-ISAM:
    ioapi-3.2-large.tar.gz
    The frozen-for-CMAQ August 7, 2020 version of I/O API-large can be downloaded from the CMAS web-site:
    ioapi-3.2-large-2020220.tar.gz

    Versions 3.2 (and 4.0 to come) are also available on GitHub from <https://github.com/cjcoats/ioapi-3.2> or <https://github.com/cjcoats/ioapi-4.0>. To install a copy of this source code from GitHub, go to the directory under which do the installation, (This directory will be the BASEDIR for subsequent steps in the installation.) and then do the command 

        git clone https://github.com/cjcoats/ioapi-4.0
    or 
        git clone https://github.com/cjcoats/ioapi-3.2
    or, for the April 20, 2020 or August 7, 2020 versions 
        git clone --branch 2020111 https://github.com/cjcoats/ioapi-3.2
    or 
        git clone --branch 2020220 https://github.com/cjcoats/ioapi-3.2
    or 
        git clone --branch 20200828 https://github.com/cjcoats/ioapi-3.2
    Note that these latter give you fixed-release versions, for which the "rolling release" bug-fixes are not easily available.

    To build ioapi-3.2-large or ioapi-4.0-large from GitHub (using the GitHub rolling-release version), copy PARMS3-LARGE.EXT to PARMS3.EXT, and then proceed as normal, making sure that you keep ioapi-3.2-large strictly sequestered from normal ioapi-3.2.

    Note also that the ioapi-4.0 version should not be installed "on top of" previous versions, because of name-collisions between older .f source files and their newer .f90 counterparts, and the resulting make problems that causes.

  2. If you downoaded a ioapi-*.tar.gz:
    cd to the directory under which you wish to build the I/O API. gunzip and untar it (gtar xvfz ioapi-3.1.tar.gz [etc...] does this in one operation).
    This directory will be the BASEDIR for subsequent steps in the installation.

  3. NOTE: Makefile, ioapi/Makefile, and m3tools/Makefile will need to be customized for your particular installation. Make sure you back them up when you do this.
    For I/O API 3.2, these are no longer in the tar-file; for previous versions, note that the untar-operation above will undo your customization.

    You will need to copy the relevant Makefile.* templates to Makefile in the ${BASEDIR}/ioapi and ${BASEDIR}/m3tools directories. For most users, you want to start with Makefile.nocpl.

    In particular, you will need to customize make-variables BASEDIR in all Makefiles, and LIBS in m3tools/Makefile to deal with netCDF-4 library issues. For MS-Windows/Cygwin systems using the Cygwin netCDF libraries, you will need to change the m3tools/Makefile library-names slightly, from -lnetcdff -lnetcdf to -lnetcdff.dll -lnetcdf.dll, as well as not using the flag -fsecond-underscore in the Makeinclude.${BIN}.

    Note that the Makefiles and environment-variable BIN-structure support the simultaneous building and maintenance of multiple versions of both the I/O API and m3tools programs using the bins, bins, binclean, bindirs, and binrelink make-targets, which may be customized to support the versions you want. Examples of such versions (each of which has its own Makeinclude.${BIN}) are:

    • Choice of compilers
    • Choice of different compiler-options (optimized, debug, profiling)
    • Choice of memory model: small (normal) vs medium
    • "normal" vs global-climate vs pnetCDF/MPI distributed-output vs coupling-mlode

    gfortran Warning: Starting with version 10, gfortran is taking a particular indiosyncraticinterpretation of the Fortran-2018 Standard. In order to build any of the modeling codes, starting with netCDF-Fortran, you will need to include the compile-flag 

                           -fallow-argument-mismatch
    in all compiles (for netCDF-Fortran, I/O API, M3Tools, SMOKE, CMAQ, and all their pre- and post-processors and utility programs). As of Aug. 28, 2020, there are now new BIN=Linux*gfort10* types and corresponding Makeinclude.Linux*gfort10* that incorporate this flag for the I/O API and M3Tools.

    Intel-compiler Warning: For whatever reason, Intel has chosen to change some flags from version to version, making it difficult to have generic Makeinclude.*ifort* files. For versions before March 25, 2020, in particular, you will need to edit those files so that make-variables OMPFLAGS AND OMPLIBS use the following:

    • For compiler-versions 15 and earlier: -openmp
    • For compiler-versions 16-18 -qopenmp is preferred, but -openmp is accepted (with a warning-message)
    • For compiler-version 19 and later: -qopenmp required
    After that date, the Makeinclude.*ifort* files contain both forms of the flags, and work although they generate "compile flag not recognized" warnings for the unused form.

    Note that if you update the I/O API library, then cd ../m3tools; make relink will efficiently re-build the m3tools programs.

    Additional definitions in make-variable DEFINEFLAGS (in Makefile orARCHFLAGS (in Makeinclude.${BIN} turn on various optional capabilities:

    -DIOAPI_NO_STDOUT=1
    suppresses write-operations to the screen in routines INIT3(), M3MSG2(), M3MESG(), M3ABORT().
    -DIOAPICPL=1
    turns on (PVM based) I/O  coupling mode.
    -DIOAPI_SNOOP=1
    turns on snoop mode for read-operations: iff timestep-flag not available, sleep for SNOOPSECS3 seconds. then re-try, for up to SNOOPTRY3 attempts.
    for I/O API-3.2 and later only
    -DIO_360=1, -DIO_365=1
    create the 360-day or 365-day global climate versions of the library
    365-day version for I/O API-3.2 and later only
    -DIOAPI_PNCF=1
    turns on PnetCDF/MPI distributed-file mode
    for I/O API-3.2 and later only
    -DIOAPI_NCF4=1
    turns on netCDF-4 INTEGER*8 operations, and requires HDF-enabled netCDF-4 libraries, instead of netCDF-3 (for the rest of your Makefiles).
    Required if your netCDF library is built without the recommended --disable-netcdf-4

    You may also need to customize the Makeinclude.* configuration-files for the configurations) you want to build. It may be useful to make a new copy with its own ${BIN} extension, so that your changes don't get wiped out if you do an update—e.g.,

    cp Makeinclude.Linux2_x86_64ifort Makeinclude.Linux2_x86_64intel
    setenv BIN Linux2_x86_64intel
    ...
    This is especially true of PnetCDF enabled I/O API configurations (below), for which the names (etc.) of the MPI based compilers are highly dependent upon the details of individual installations. Note that there are already a number of specialized Makeinclude.${BIN}s for various purposes; they should serve as templates if you need these capabilities for different compilers:
    Makeinclude.*dbg
    Debug version of library
    Makeinclude.*360
    360-day (no leap-year) climatology-year version of library
    Makeinclude.*365
    365-day (no leap-year) climatology-year version of library
    Makeinclude.*MPI
    PnetCDF/MPI enabled distributed-I/O version of library
    Makeinclude.*_medium*
    medium-memory-model (large stack/large-array) version of library

    Note also that starting with their Version 16 (and later) compilers, Intel has introduced a new compiler directive -qopenmp to enable OpenMP, and has deprecated the previous -openmp (which now results in a "deprecated flag" warning).
    I/O API-3.2 only: Changing the OMPFLAGS and OMPLIBS make-variables in the Makeinclude.*ifort* to match this compiler-change (as indicated in the comments) can eliminate this compile warning for current Intel compilers.

  4. If you want to disable the coupling-mode configuration (which is enabled by default), cd ioapi and then cp Makefile.nocpl Makefile and customize it, and similarly for the m3tools directory.

  5. I/O API 3.2 only: If you want to enable the PnetCDF distributed I/O configuration (for CMAQ CCTM), cd ioapi and then cp Makefile.pncf Makefile and setenv environment variable BIN to one of the supported distributed-I/O types Linux2_x86_64gfortmpi, Linux2_x86_64ifortmpi,Linux2_x86_64pgmpi, Linux2_x86_64sunmpi
    NOTES
    1. PnetCDF enabled I/O API libraries are not compatible with "normal" libraries. Make sure you keep them separate.
      You will want to build both, since the PnetCDF enabled libraries are for the CMAQ CCTM only.
    2. What is called mpif90 is really a script that understands about an underlying compiler-set (e.g., Intel ifort/icc, PGI pgf90/pgcc, etc.), a set of (MPI-installation specific) INCLUDE files, and a set of$quot;.a" library-files. These are not compatible from MPI-implementation to MPI-implementation (OpenMPI, MVAPICH, etc.) nor from underlying compiler-set to underlying compiler-set.
    3. There may even be compatibility problems between different versions of the "same" implementation: This author has found that OpenMPI-1.4.x is not even source-code compatible with OpenMPI-1.6.x ;-(
    4. This part of the I/O API source code had to be hacked badly to deal with the above.

  6. If you want to enable the 360-day or 365-day (no-leap-year) climatology year configuration (365-day for I/O API 3.2 only), use one of the various Makeinclude.*_360 or Makeinclude.*_365 (and its corresponding ${BIN}), or else customize your ioapi/Makefile by adding -DIO_360=1 or -DIO_365=1 respectively to the DEFINEFLAGS make-variable.
    Note that you need to keep the resulting libraries separate from the normal (leap-year enabled) libraries.

  7. For each desired configuration of machine, compiler-set, and other options, setenv BIN <type>, where <machinetype> matches the extension on one of the ioapi/Makeinclude.* (creating a new one, if necessary). Then do a top-level make dirs to build the object-directories.

    The usual pattern for generating BIN is

    setenv BIN `uname -s``uname -r | cut -d. -f1`
    and then modify the result by appending any compiler specifics, e.g., for F90, debug- or profile-compiled configurations.

    A list of the "standard" supported platforms can be found here.

    Linux versions 2, 3, and 4 are binary-compatible, so Linux2 is used for all of these (and similarly for recent versions of SunOS).

    Note, for example, that for Linux2_x86*, there are distinct (and incompatible) variants for the GNU gfortran/gcc, Intel ifort/icc, Lahey-Fujitsu lf95/GNU gcc, and Portland Group pgf90/pgcc; moreover, the latter is not compatible with Portland Group pgf90/GNU gcc (when the latter is used to build libnetcdf.a, as in the vendor supplied libraries on RedHat (that incompatibility being the largest single generator of installation-questions in recent years!).

    INTEL and SGI NOTES:

    • At some point in its 8.0.x sequence of compiler versions, Intel has changed the names of its compilers: ecc becomes icc for the Itanium (IA64) platform, and both efc and ifc become ifort; you may need to edit v3.0 and earlier Makeinclude.${BIN} accordingly.
    • There are problems with the Fortran pre-processing for some versions of the Intel Fortran compilers; these can to some extent alleviated by separating the preprocessing from the compile step for the *.F files. Makefile.cpphack is a sample Makefile for this purpose (as provided, it assumes OpenMP and coupling mode).
    • There are problems with SGI f90 Version 7.4. Use Version 7.4.1 or later instead.
    • There are command line incompatibilities between SGI f90 Version 7.3.x and 7.4.1; for 7.3.x, you need to edit the MAKEINCLUDE.${BIN} to replace -TARG:platform=ip27 -TARG:processor=r12000 by -TARG:platform=ip27,processor=r12000, (etc.)

  8. Acquire and build the netCDF library, libnetcdf.a (and libnetcdff.a for netCDF-4.1 or later), for each configuration. The netCDF home page is https://www.unidata.ucar.edu/software/netcdf/ For each configuration,copy or link the resulting libnetcdf.a (and libnetcdff.a, if you're building netCDF version 4.1 or later) to the appropriate object directory $BASEDIR/${BIN}.
    Vendor supplied netCDF libraries frequently cause problems, in part because they are compiled with compilers that are not link-compatible with the Fortran compilers used by the I/O API and the various models (CMAQ, SMOKE, etc.) that use it. In particular, Deeksha Rastogi reported very-obscure errors caused by a vendor supplied netcdf/4.6.1 (and fixed by compiling and using his own netcdf-c-4.7.4 and netcdf-fortran-4.5.2
    See NOTES on netCDF-4 for configuring netCDF-4.

    For I/O API-3.2 and netCDF-4: Some configurations of netCDF-4 have INTEGER*8 (64-bit integer) support, and some do not; the I/O API tries to make it possible to use this support if it is available, but needs to include "hacks" to cover for its absence when it isn't. You can find out if it is included in netCDF using the command 

            nm libnetcdff.a | grep nf_get_var_int64_
    If present, you need to add the definition -DIOAPI_NCF4=1 to the make-variable ARCHFLAGS in your MAKEINCLUDE.${BIN}. Otherewise, you will get "multiply defined symbol" errors when you attempt to compile programs.

    For I/O API-3.2, if you are building the PnetCDF/MPI distributed-I/O version for the CMAQ CCTM, then install the appropriate MPI development package, and download PnetCDF from https://trac.mcs.anl.gov/projects/parallel-netcdf/wiki/Download then build and install libpnetcdf.a and copy or link it to the appropriate object directory.
    Note that to a very large extent, different MPI implementations are not compatible: Intel MPI does not play well with OpenMPI does not play well with MPICH does not play well with MVAPICH... ;-(

    If you plan to build a coupling-mode configuration, do the same with PVM's libpvm3.a. See http://www.csm.ornl.gov/pvm/pvm_home.html (Only the C interface to PVM is used; the resulting library tends to be more compiler-independent than for netCDF, where both C and Fortran interfaces are used.)

  9. Edit the relevant files ioapi/Makeinclude.${BIN} for your system specific compiler options, PVM installation location, etc., or create new ones, as appropriate.

  10. Edit the Makefiles to set make-variable BASEDIR, etc., for your system.

    If you are doing a multi-configuration installation, do the same for each */Makefile*.

    If you are doing a single-configuration installation, you may customize make-variables in the top level Makefile and do the following from the top level directory as a short-cut:

    make configure
    make dir
    [cd ../${BIN} and copy in the needed netCDF etc. libraries]
    cd ..
    make

  11. setenv BIN ..., followed by one of the make <target> commands below, as appropriate for your configuration(s).
    Alternatively, make BIN=<machinetype> (where note that a command-line variable-definition overrides the environement variable definition).
    Intel compiler note: After version 16, Intel changed the OpenMP compile-flags. After March 25, 2020, the Makeinclude files for Intel compile-types contain both forms of the flags, so that all versions of the compile will succeed, at the cost of a warning for not recognizing the other-version copy of the flag.

  12. Note that a top-level make will generate messages about installing the notCDF library. Almost all users can ignore these messages; libnotcdf.a is only required at NCEP, where it is politically forbidden to have any copy of libnetcdf.a on the system.

Top-Level make targets include the following:
make dir
Construct the build-directory $OBJDIR You only need to do this once for each configuration ${BIN}
make all
Build all libraries and executables (assumes the Fortran compiler is Fortran-90 compliant) for the default mode.
make test (for I/O API 3.2 post-March 25, or I/O API 4.0)
Subsequent to make all, the command make test from the installations ${BASEDIR} runs a series of tests that ensures the I/O API and m3tools programs are correctly installed, and demonstrate various I/O API capabilities.
make cpl
Build all libraries and executables for PVM-enabled coupling mode.
make nocpl
... for no-PVM no-coupling mode. This is probably the best target for regulatory users who need multiple configurations.
make relink
Rebuild all the M3TOOLS executables, using the existing M3TOOLS object files (e.g.,if you have changed/updated the libraries).
make clean
removes all executable, object (.o), module (.mod), and library (.a) files.
make configure
allows you to set up make-parameters in the top-level Makefile and then run a set of sed commands on the templates */Makefile*.sed in order to generate lower level Makefiles for a particular configuration.
Writes over any existing Makefile customizations!
make bins
builds everything for several choices of (Linux) architectures and compilers.
Note: easily customized for your system.
make binclean
cleans everything for several choices of architectures and compilers.
make fix or
cd ioapi; make fixed_src
For CMAQ and SMOKE Users:
builds SMOKE and CMAQ "fixed-132" non-ISO-Standard Fortran source form INCLUDE-files and places them in directory ${BASEDIR}/ioapi/fixed_src/.

Back to Contents

NOTES on gfortran

GNU gfortran seems to be insistent upon incompatibility: not only incompatibility with existing industry standards, but even inconsistency with itself from version to version. In some cases, not even Makefile will continue to work from one version to the next. At this time, gfortran should be used only with trepidation.
--Carlie J. Coats, Jr., Ph.D.
I/O API author

Finding MODULE files
Fortran compilers use various ways to find directories containing "module files" such as the I/O API's m3utilio.mod. Suppose the I/O API's OBJDIR is something like /foo/bar/${BIN}. Then for most F90 compilers, the way to tell the compiler to search that directory for module files is to use the following as a compiler-flag:
-I/foo/bar/${BIN}
. Some other compilers (e.g., Sun, Absoft, older versions of gfortran) use
-M/foo/bar/${BIN}
or (PGI)
-module /foo/bar/${BIN}
. Recent versions of gfortran use
-J /foo/bar/${BIN}
. Needless to say, this makes it more difficult to write Makefiles ;-(

IARGC and COMMAND_ARGUMENT_COUNT, etc.
Recent versions of gfortran break compatibility with older versions, and with standards such as Mil-Spec 1753, in their handling of command line arguments: they no longer support IARGC() and GETARG(), insisting rather on Fortran-2003 constructs COMMAND_ARGUMENT_COUNT() and GET_COMMAND_ARGUMENT().

I/O API file init3.F now has "hacks" to get around this incompatibility: if preprocessor-symbol NEED_ARGS is defined, it generates IARGC() and GETARG() in terms of COMMAND_ARGUMENT_COUNT() and GET_COMMAND_ARGUMENT(). By default, Makeinclude.Linux2_x86, and Makeinclude.Linux2_x86_64 and Makeinclude.Linux2_x86_64gfort (which specify how to build the I/O API (etc.) for 32-bit or 64-bit gfortran/gcc) now define this symbol as part of make-variable ARCHFLAGS.

If you get "symbol not found" errors when building I/O API programs like those in m3tools, remove this -DNEED_ARGS=1 from the relevant Makeinclude file.

List-directed READ( DEV,*, ...)s
Basically, the Fortran standard says that for something like the following example 
    ...
    READ( DEV,*,IOSTAT=ISTAT) ITEM1, ITEM2, ITEM3, ITEM4, ITEM5
the program should read the list ITEM1 through ITEM5 from the file at unit DEV, using as many lines as necessary to find all these values, and treating commas and whitespace—including blank lines— as item-separators; then discard the remainder (if any) of the last line read.

gfortran has its own idiosyncratic interpretation that says blank lines within such a list should be treated as a list-terminator, leading to an I/O error.

Back to Contents

NOTE for users of CMAQ and SMOKE

According to the ISO Fortran Standard, there are two "source formats" for Fortran source code -- the older (f77) "fixed" format with code in columns 7-72 and continuation markers in column 6, and the (f90) "free" format, with source in columns 1-132 and ampersand ("&") for trailing continuation markers, as well as a specification for code which is acceptable to both forms.

Since it is used by various models that use "free" format, the I/O API carefully follows the Standard in this regard, and uses the ISO-app[roved "both-form" format for its Fortran INCLUDE files.

Some of the models—in particular, CMAQ and SMOKE however, do not—instead using a "fixed-132" format. The I/O API Makefile has a way of dealing with this; the commands

cd <installation directory>/ioapi
make fixed_src
will create a new directory ioapi/fixed_src and populate it with copies of the INCLUDE files that have been edited to work with the non-Standard "fixed-132" format. You will need to use these INCLUDE files with CMAQ and SMOKE.

ASIDE on FILE EXTENSIONS:

The "industry standard" for Fortran source-file naming is that the extensions for file names should be as follows: Virtually all UNIX/Linux Fortran compilers will automatically use the file-extension to determine which source form to expect, unless you override this expectation with explicit flags specifying otherwise. Sadly, what these flags are varies from compiler to compiler.

The findent program is an excellent resource for converting code back and forth between the two forms (and even the non-Standard one); see https://sourceforge.net/projects/findent/.

It is also much more convenient for make dependencies if Fortran MODULEs are one per file, with file name the lower case of the MODULE name— MODULE MODFOO will be in the file named modfoo.f or modfoo.f90, as appropriate (depending upon source-form).

Back to Contents

NOTES on CygWin under MS Windows

I/O API Version 3.2 can be successfully installed under the CygWin64 unix-emulation environment for Microsoft Windows. You will need to make sure you install a good development environment with it; fortunately, what you need is available in the Cygwin repositories.

My recommendation is to install CygWin64 with at least the following additional packages:

binutils cmake coreutils diffutils dos2unix findutils gcc-core gcc-gfortran gcc-g++ gdb git gnuplot grep libcurl-devel libcurl4 libgd-devel libgd3 libhdf5-devel libhdf5_10 libjpeg-devel libjpeg8 libnetcdf libnetcdf-devel libnetcdf-fortran libnetcdf-fortran-devel libpng16-devel libpng16 libtool netcdf netcdf-fortran ncview openmpi tar tcsh tree util-linux vim wget xhost xinit xorg-server xorg-x11-fonts-dpi100 xterm xxdiff zlib zlib-devel zlib0
Note that this installation will include Version 4.4 of the netCDF libraries, for both Fortran and C; you will not need to build these libraries yourself "from scratch". On the other hand, when you build programs using the I/O API (and therefore netCDF), you will need to refer to them a bit differently than normal usage, as
-lnetcdff.dll -lnetcdf.dll
For example, you will have to edit the definition of LIBS in m3tools/Makefile:
LIBS = -L${OBJDIR} -lioapi -lnetcdff.dll -lnetcdf.dll $(OMPLIBS) $(ARCHLIB) $(ARCHLIBS)

The Cygwin64 package list is at

https://cygwin.com/packages/package_list.html
To install additional packages after the initial installation, open the File Explorer on Downloads and run setupx86_64.exe found there. When you get to package selection, choose "uninstalled" and pick your selections from the menu.

You may also wish to install a GUI editor that understands UNIX line conventions (which are different from Windows; Windows notepad screws this up badly). I've found that the Crimson Editor is a free and workable choice for such an editor. (For CygWin graphical programs such as the nedit editor, you will need to install and run the X-windows server...)

Back to Contents

NOTE on netCDF Version 4

Evidently, starting with netCDF-Fortran 4.4.2, UCAR in their wisdom decided to silently eliminate the CALL NC*() Fortran interfaces in terms of which the I/O API was originally implemented. [If you can find any documentation indicating this intent on the web, I would enjoy seeing it; I can't find it.]

I/O API Version 3.2 was tediously re-coded to replace all 943 of these calls by the newer IERR=NF_*() Fortran interfaces that are still supported. Prior editions of the I/O API will have link errors with netCDF-Fortran 4.4.2 or later, because routines such as NCCLOS() are no longer in that library.

NetCDF Version 4.x have lots of additional build options, that will require a complex set of additional libraries in your Makefiles and all other model-building systems. It is recommended that you disable these options by adding the command-line flags below to your netCDF configure command:

--disable-netcdf4 --disable-dap
or (depending upon netCDF version; don't you love consistency and compatibility !?? )
--disable-netcdf-4 --disable-dap
If you build netCDF-4 without the --disable-netcdf4, you will also need to add -DIOAPI_NCF4=1 either to the ARCHFLAGS in your Makeinclude.${BIN} or to the DEFINEFLAGS in your Makefile, since defining netCDF-4 causes netCDF to change parts of the netCDF-3 interface.

NetCDF Version 4.1 and later also split the Fortran interfaces into a separate library libnetcdff.a As a result, much compatiblity with the infrastructure (Makefiles, etc.) is broken: you must modify all Makefiles, replacing -lnetcdf with -lnetcdff -lnetcdf. If you don't build netCDF as indicated above, you will need additional libraries that may be discovered by running the netCDF commands

nc-config --libs
nf-config --flibs

Starting with I/O API Version 3.1 of October 11, 2011, the m3tools/Makefiles are configured by default to use the -lnetcdff -lnetcdf netCDF-v4-style libraries. If using a previous version of netCDF, you must modify them manually.

You must also modify the build-systems for CMAQ, MCIP, SMOKE, and all your other models that use netCDF similarly.

On the other hand, you may find it much easier to revert to a previous edition of netCDF (such as version 3.6.3) and revert your Makefiles and build systems to use just -lnetcdf instead of the -lnetcdff -lnetcdf required by netCDF version 4 and later.

Back to Contents

Notes on Linux2_x86_64 memory models

There are three different binary-incompatible "memory models" available for 64-bit x86 Linux:
small
At most 2 GB each for static data (COMMONs, etc.), ALLOCATEd arrays, program-stack, and program machine-code.

medium
Program machine-code at most 2 GB; no restrictions on data (large arrays, large stack, large data are OK).

large
No restrictions on code or data.

The small memory model basically is a 32-bit programming model that uses the full 64-bit instruction set (which has more registers, and is in other ways more powerful than the 32-bit x86 instruction set). It is the default for most (Intel, GNU, Portland Group) compilers. Since most programs don't use more than 2 GB of total memory, it is adequate for most environmental modeling needs.

The medium memory model is fully 64-bit for data, but 32-bit for the instruction-part of programs. Given enough RAM, it can handle virtually all environmental programming needs. It is the default for Sun/Oracle's Linux compilers; for Intel, GNU, Portland Group compilers, you need to add the following option to the compile and link commands

-mcmodel=medium

The large memory model is rarely needed (who writes models that compile into object files larger than 2 GB, anyway??), and seems to be supported only by the Intel compilers, and the latest versions of the GNU compilers provided that the compiler itself is compiled with this option. You will need the following compile-option for it:

-mcmodel=large

Note that program-codes (object files and libraries) compiled with the different memory models are binary-incompatible. All model-components need to be compiled with the same memory-model flags. . Binary types ${BIN}=Linux2_x86_64gfort_medium, Linux2_x86_64ifort_medium, and Linux2_x86_64pg_medium are provided for medium-memory-model use with the Intel, GNU, or PGI compilers. Note also that the Intel and PGI compilers do not support static linking for the medium (or large, for Intel) memory model.

See http://eli.thegreenplace.net/2012/01/03/understanding-the-x64-code-models for a somewhat readable and very detailed account of the issues, and also https://software.intel.com/en-us/node/525162

Back to Contents

netCDF and Large Files

Starting with Version 3.3.1, netCDF can be compiled with "large file support" for most 32-bit platforms (and for 64-bit platforms like Cray, IRIX64, Alpha, IA64, and x86_64). This suffices for most environmental modeling applications; the limitation is that (the totality of variables in) an individual time step may not exceed 4 GBytes. Starting with Version 3.6, netCDF supports a new variant of the underlying data format, with essentially no (software based) limitations on file, record, or array size. This format is not yet supported by older versions of the I/O API (partly because the old and new netCDF formats are not fully compatible); design analysis was in progress in 2005, and for I/O API versions 3.1 and later, large-file support is automatic for input files and can be turned on for output files by doing the following before running the relevant model (this is the default for I/O 3.2):
setenv IOAPI_OFFSET_64 YES
See Standard I/O API environment variables in the documentation.

For more information, see URL http://my.unidata.ucar.edu/content/software/netcdf/docs/netcdf/Large-File-Support.html#Large-File-Support

Back to Contents

Other Notes and Warnings

Strong Recommendation  1
Don't change any I/O API INCLUDE files.
Any changes are likely to result in very obscure, difficult-to-diagnose bugs caused by an inconsistency between standard libioapi.a object-libraries and whatever code is compiled with the resulting modified INCLUDE-file. Note that the distribution license specifies that by making any changes to any of these INCLUDE files, the user explicitly agrees that in the case any assistance is required of UNC CEMPD or of the I/O API author, Carlie J. Coats, Jr. as a result of such changes, the user and/or his project or contract agrees to reimburse UNC CEMPD and/or the I/O API author, Carlie J. Coats, Jr., at a rate triple the normal contract rate for the services required.

Strong Recommendation  2
You are best off if you can build the whole modeling system (libnetcdf.a, libpvm3.a, libioapi.a, and your model(s) CMAQ, SMOKE, etc. with a common compiler set and common set of compile-flags. In particular, Linux-distribution-vendor supplied libnetcdf.a (and libnetcdff.a, for netCDF-4) rarely works with CMAS-supported compiler sets. Deeksha Rastogi points out some extremely-obscure errors caused by using vendor-supplied netCDF-4.6.1, probably related to (very technical and obscure) differences in the compiler's struct-padding.

Note 3
For 64-bit platforms, it turns out that many compilers by default still only allow individual arrays and program-stack sizes smaller than 2 GB. For the vast majority of current modeling applications, this is not a problem. This is documented in detail in a section above.
To use "large" arrays, i.e., larger than 2 GB, or very-deep recursion, you will need to build your entire model (libraries and all) using compile flags that tell the compiler to use the "medium 64-bit" memory model. The I/O API Makeinclude.Linux2_x86_64ifort_medium (for BIN=Linux2_x86_64ifort_medium) has the flags you need in order to do this with Intel compilers. (Portland Group and Gnu compilers also default to "small 64-bit" memory model in the same way as Intel; Sun's Linux compilers default to medium 64-bit memory model. For others, check your compiler documentation.)

Note 4 (applicable for v3.0 and earlier only)
On at least Sun Solaris and SGI IRIX platforms, f77 and f90 are known not to be link compatible; you will need to keep builds for them separate (which is the original distinction for BIN=IRIX64f77 vs BIN=IRIX64f90, for example). It is recommended that for other systems you likewise keep different compiler systems (and sometimes even different versions of the same compiler--Intel ifort versions 10, 11, and 12, for example) separate.

Back to Contents

I/O API Version 3.2 Availability

NOTE, January 15, 2015: Version 3.2 of the I/O API library source code and documentation is available from GitHub from <https://github.com/cjcoats/ioapi-3.2>. or for download in tar-gzipped form from the CMAS Center web site <ioapi-3.2.tar.gz>. To install it from GitHub, go to the directory under which do the installation, and then do the command 
    git clone https://github.com/cjcoats/ioapi-3.2
git clone... will create a directory ioapi-3.2 with subdirectories ioapi, m3tools, and HTML, and various other files, including the LICENSE and top-level Makefile; unpacking the tar-ball will create everything under that ioapi-3.2 directory.

Due to INTERFACE changes in MODULE M3UTILIO, new MODULE MODGCTP, and new module MODULE MODATTS3 (which replaces MODULE MATXATTS), there will be minor source-code changes in programs that use it; re-compilation and re-linking will be required: see here for more details.

Back to Contents

I/O API Version 3.1 Availability

Version 3.1 of the I/O API library source code is available here for download in tar-gzipped form. This version is upwardly source-code compatible with previous versions; however, it is not object-code compatible, nor is it compatible with object files built using INCLUDE files derived from previous versions. New features of this release are documented here, and include

The build procedure is given above..

Version 3.1 can read all files generated by Versions 3.0 and earlier; Version 3.0 can read all files generated by Version 3.1, provided that the number if variables in the file NVARS3D < 121.

Due to the changes in MXVARS3. Version 3.1 is a "Jubilee Day" release: it should completely replace previous versions; object and library files produced using Version 3.1 should not be mixed with previous versions.

Note that proper use of MODULE M3UTILIO ensures that the library and user object code use consistent parameter values (as well as ensuring correctness of most call interfaces).

Note also that changing MXVARS3 was never supported, is strongly discouraged, and may lead to invocation of the triple-time clause (see header of PARMS3.EXT) if support of such a modified installation is requested.

This release is current as of October 1, 2015 and includes all bug-fixes up through that date.

Back to Contents

I/O API Version 3.0 Availability

Version 3.0 (or earlier) of the I/O API library source code should be considered obsolete, and used only if absolutely necessary.

Version 3.0 of the I/O API library source code is available here for download in tar-gzipped form. New features of this release are documented here, and include additional support for WRF, a new native-binary-format lower layer, a file-set lower layer, some new routines, and enhancements for portability and OpenMP thread-safety.

This release is current as of June 11, 2007 and includes all bug-fixes up through that date. It should be considered the production release of Version 3.0 for netCDF files and for PVM-based virtual files, but should probably still be considered "beta" for native-binary layer, and for the multi-file file-list layer, since each of these latter has basically a single-user community rather than widespread beta-test usage.

The build procedure also has been considerably enhanced, by the addition of a top-level Makefile which allows one to build various configurations of the entire (I/O API + M3TOOLS) system with one make command. Depending upon what you want to do, there are two main options for how you will build the system:

To build multiple configurations from one installation directory (e.g., multiple compilers, etc.)
The procedure is much as before, except that one can use just a single make bins command to build all the libraries and all the executables for a configuration.

Before you do so, yhou should edit the ioapi/Makefile and m3tools/Makefile commands so that the make bins and make binclean commands properly reflect the configurations you want to build.

To build a single configuration from one installation directory
Follow the build procedure given above..

To set up the installation:
  1. Download the gzipped tar-file. It contains directories ioapi for the I/O API library source code, HTML for the documentation, m3tools for the related tool programs, iotest for various test-programs, and notcdf for the NCEP libnotcdf.a library (a dummy library for installation sites that don't allow netCDF on-site).

  2. cd to the directory under which you wish to build the I/O API. gunzip and untar (gtar xvfz ioapi-3.0.tar.gz does this in one operation).

  3. If you want to disable the coupling-mode configuration (which is enabled by default), cd ioapi and then cp Makefile.nocpl Makefile.

  4. For each desired configuration, setenv BIN <machinetype>, where <machinetype> matches the extension on one of the ioapi/Makeinclude.*. Then do a top-level make dirs to build the object-directories.

    The usual pattern for generating BIN is

    setenv BIN `uname -s``uname -r | cut -d. -f1`
    and then modify the result by appending any compiler specifics, e.g., for F90 or debug compiled configurations.

    Note, for example, that for Linux2_x86, there are distinct (and incompatible) variants for the GNU g77/gcc, Intel ifc/icc, Lahey-Fujitsu lf95/GNU gcc, and Portland Group pgf90/pgcc; moreover, the latter is not compatible with Portland Group pgf90/GNU gcc (when the latter is used to build libnetcdf.a, as in the vendor supplied libraries on RedHat (that incompatibility being the largest single generator of installation-questions in recent years!).

    INTEL and SGI NOTES:

    • At some point in its 8.0.x sequence of compiler versions, Intel has changed the names of its compilers: ecc becomes icc for the Itanium (IA64) platform, and both efc and ifc become ifort; you may need to edit the MAKEINCLUDE.${BIN} accordingly.
    • There are problems with the Fortran pre-processing for some versions of the Intel Fortran compilers; these can to some extent alleviated by separating the preprocessing from the compile step for the *.F files. Makefile.cpphack is a sample Makefile for this purpose (as provided, it assumes OpenMP and coupling mode).
    • There are problems with SGI f90 Version 7.4. Use Version 7.4.1 or later instead.
    • There are command line incompatibilities between SGI f90 Version 7.3.x and 7.4.1; for 7.3.x, you need to edit the MAKEINCLUDE.${BIN} to replace -TARG:platform=ip27 -TARG:processor=r12000 by -TARG:platform=ip27,processor=r12000, (etc.)

  5. Acquire and build the netCDF library, libnetcdf.a, for each configuration. The netCDF home page is https://www.unidata.ucar.edu/software/netcdf/ For each configuration,copy or link the resulting libnetcdf.a ( and libnetcdff.a, if you're building netCDF version 4.1 or later) to the appropriate object directory.

    If you plan to build a coupling-mode configuration, do the same with PVM's libpvm3.a. See http://www.csm.ornl.gov/pvm/pvm_home.html (Only the C interface to PVM is used; the resulting library tends to be more compiler-independent than for netCDF, where both C and Fortran interfaces are used.)

  6. Edit the relevant files ioapi/Makeinclude.${BIN} for your system specific compiler options and PVM installation location, as appropriate.

  7. Edit the top-level Makefile to set variables BASEDIR, etc., for your system.

    If you are doing a multi-configuration installation, do the same for each */Makefile*.

    If you are doing a single-configuration installation, do make configure

  8. setenv BIN ..., followed by one of the make <target> commands below, as appropriate for your configuration(s),

  9. Note that a top-level make will generate messages about installing the notCDF library. Almost all users can ignore these messages; libnotcdf.a is only required at NCEP, where it is politically forbidden to have any copy of libnetcdf.a on the system.

Back to Contents

I/O API Version 2.2 Availability

Version 2.2 (and all older versions) of the I/O API library source code should be considered badly obsolete.

Version 2.2 of the I/O API library source code is available here for download in tar-gzipped form. New features of this release are documented here. The build procedure is the same as that for Version 2.1, documented below, except for the following:

This download is current as of October 18, 2003

Back to Contents

I/O API Version 2.0 and 2.1 Availability

Updated May 7,2002 for SMOKE release: now produces a library named libioapi_v2.1.a instead of Version 2.0's libioapi_new.a

Source code for Release Version 2.1 of the I/O API and the I/O -related tool and sample programs is available here for download in tar-gzipped form. New features of this release are documented here. Makeinclude files are available for the following platforms (and some debug- and other variants, denoted by dbg i8, and r8 suffixes):

From these examples and from a knowledge of the compiler user manual, it should be relatively easy to build Makeinclude files for most UNIX or UNIX-like platforms and compilers. Instructions for building the I/O API library libioapi_v2.1.a and the I/O API tool executable programs are as given below.

NOTE: It is important to ensure that the compile flags used for building the I/O API library are consistent with both the compile flags used for building your program and the compile flags used for building the netCDF, PVM, and other libraries used by your programs. Failure to do so may result in link-failures or (in the case of the Linux Portland Group compilers) run-time program deadlocks (program hangs, doing no work while consuming all available CPU resources).

  1. Download the gzipped tar-file ioapi_21.tar.gz. It contains directories ioapi for the I/O API library source code, ioapi_doc for the HTML documentation, and ioapi_tools for the related tool programs.

  2. cd to the directory under which you wish to build the I/O API. gunzip and untar the ioapi_21.tar.gz (with Gnu tar,
    tar xvfz ioapi_v2.1.tar.gz
    does unzip-untar all in one step).

  3. setenv BIN <machinetype>
    where <machinetype> matches the extension on one of the Makeinclude.* (building your own Makeinclude if yours is not one of the supported systems).
    The usual pattern for generating BIN is
    setenv BIN `uname -s``uname -r | cut -d. -f1`
    although there are exceptions where more work is needed for Cray, SGI and Linux systems, and F90 or DEBUG compiles.

  4. The default directory for both executables and object libraries is in directory ../${BIN} relative to the source code directories for the I/O API and tools. Edit the Makefile to put OBJDIR wherever you want it (if you want somewhere other than the default ../${BIN} location).
    NOTE 1: Different compilers generate linker-visible object names in different ways (some with multiple options...). It is important that all of the compiles for an entire executable program (including the compiles for netCDF, PVM, and other libraries) use the same scheme; this is controlled by various parts of the ARCHFLAGS variable in the (compiler-system dependent) Makeinclude.${BIN} files.
    NOTE 2: By default on most systems, OpenMP parallelism is enabled; see the OMPFLAGS variable in Makeinclude.${BIN}. The I/O API does not have parallel sections of its own; however, enabling OpenMP does allow the activation of critical sections allowing the I/O API to be thread-safe for OpenMP-parallel programs (like the MAQSIP-RT air quality model, the WRF or MCPL-enabled MM5 meteorology models, research versions of SMOKE, and others. You may set OMPFLAGS and OMPLIBS to empty-strings in order to disable this thread-protection, if you wish.

  5. In the I/O API library source directory ioapi, type make to build the object library. The current build process will generate $OBJDIR/libioapi_v2.1.a (to distinguish it from earlier Version 1 $OBJDIR/libioapi.a); mv it to libioapi.a or libm3io.a, if desired.

  6. If necessary, get netCDF and build libnetcdf.a; if you're building with Coupling Mode active, do the same for PVM.
  7. Copy or link (ln -s ...) libnetcdf.a (and libpvm3.a if you built it) to your OBJDIR.

  8. In the I/O API tool source directory ioapi_tools, type either make (if you have a F90-compliant Fortran compiler), or make f77 (if you don't) to build the tool-program executables (adjusting the LIB make-variable if you renamed $OBJDIR/libioapi_v2.1.a in the preceeding step). Note that there are a number of these programs that do require F90: dayagg, ginterp, m3cple, m3agmax, m3agmask, m3combo, m3merge, mtxblend, mtxbuild, mtxcalc, mtxcple, presterp, and selmrg2d.

Binary executables and object libraries are no longer available (June 11, 2012 ).

NOTE: On the Sun SunOS and SGI IRIX platforms, there are link incompatibilities between f77 and f90, caused by the fact that these vendors' f77 and f90 by default use different system libraries. Compaq, Cray, and IBM compilers do not seem to do this; however, it is probably advisable not to mix f77 and f90 compiles on a not-well-understood new platform.

Back to Contents

NOTE on File Types

The "industry standard" for Fortran source-file naming is that the extensions for file names should be as follows: Virtually all UNIX/Linux Fortran compilers will automatically use the file-extension to determine which source form to expect, unless you override this expectation with explicit flags specifying otherwise. Sadly, what these flags are varies from compiler to compiler.

The findent program is an excellent resource for converting code back and forth between the two forms (and even the non-Standard CMAQ/SMOKE "fixed-132" one); see https://sourceforge.net/projects/findent/.

It is also much more convenient for make dependencies if Fortran MODULEs are one per file, with file name the lower case of the MODULE name— MODULE MODFOO should be in the file named modfoo.f or modfoo.f90, as appropriate (depending upon source-form).

Files with extension ".gz" or ".tgz" are compressed with the GNU gzip utility. Source for gzip is available from a number of FTP sites; executables for a variety of platforms are available from several sources, including ftp://ftp.netscape.com/pub/unsupported/gnu/. On windows platforms, either winzip or the Cygnus windows port of gzip can deal with ".gz" and ".tgz" files.

Back to Contents

DISCLAIMER

This source code and these object libraries are available in unsupported form only, and neither MCNC nor Baron Advanced Meteorological Systems LLC accepts any liability for its usability or correctness for any purpose. See the "Notices" document for acknowledgements, copyright and licensing information.

On the other hand, its author thinks it is good stuffTM and welcomes comments and suggestions.

Back to Contents

I/O API Documents

Documentation for the EDSS/Models-3/BAMS I/O API is available only as HTML -- it was designed from the start as a heavily cross-linked hyperdocument, and is not available in linear dead-tree document-forms such as PDF. The Documentation for the EDSS/Models-3 I/O API is copyright © 1992-2002 MCNC, © 1995-2017 Carlie J. Coats Jr, © 2003-2011 Baron Advanced Meteorological Systems, and © 2014- UNC Institute for the Environment.

A gzipped tar-file snapshot for I/O API Version 2.1 source code and its documentation is available here.;

Back to Contents

UP: I/O API home page

Back to the I/O API User Manual

Send comments to

Carlie J. Coats, Jr.
carlie@jyarborough.com 

    $Id: AVAIL.html 206 2021-10-21 15:46:35Z coats $