Singularity Amazon AWS

HOST MPI Method

Singularity Container was prepared by Carlie Coats on his machine as root, then packaged to a tar.gz file.

/proj/ie/apps/dogwood/singularity

I copied this to a Red Hat AMI

1) Installed Singularity

2) Uploaded the benchmark case: CMAQv5.3.1_Benchmark_2Day_Input/2016_12SE1

3) Launched c4.xlarge instance for testing. C4.xlarge has 4 processors.

I was able to run using the mpi versions mpich and mvapich, but the openmpi version did not run correctly.

/home/training/Scripts-CMAQ/singularity-cctm.csh

The run_cctm_Bench_2016_12SE1.csh script calls the following script:

/21dayscratch/scr/l/i/lizadams/singularity/cmaq.sif/opt/CMAQ_REPO/CCTM/scripts/run_cctm_Bench_2016_12SE1.csh

I modified the run_cctm.csh to use, so that the output from each mpi version did not overwrite each other.

if ( ! $?RUNID      )   setenv RUNID        ${VRSN}_${MPIVERSION}_${compiler}_${APPL}
set OUTDIR  = $DATADIR/cctm/$VRSN/${MPIVERSION}

Fixed the OMI File name:

    set OMIfile   = OMI_1979_to_2019.dat

Fixed the date, using date conversion, Julian to Gregorian

      set YESTERDAYG = `jul2greg ${YESTERDAY}`
      setenv ICFILE CCTM_CGRID_${RUNID}_${YESTERDAYG}.nc
      setenv INIT_MEDC_1 $ICpath/CCTM_MEDIA_CONC_${RUNID}_${YESTERDAYG}.nc
      setenv SOILINP    $OUTDIR/CCTM_SOILOUT_${RUNID}_${YESTERDAYG}.nc
      setenv ISAM_PREVDAY "$OUTDIR/CCTM_SA_CGRID_${RUNID}_${YESTERDAYG}.nc"

Fixed the path to the MCIP files

 set METpath    =  $DATADIR/met/mcipv5.0

Table of run time

PE Configuration	2x2pe
Wall Time Openmpi	only ran on 1 pe, did not finish
Wall Time Mvapich	7088.1 second
Wall Time Mpich	7204.1 second

Tried running the ISAM version using cctm_isam.csh.

This is picking up the correct execubable in the container.

The ISAM output files are not being generated and the isam_control.txt file isn't being read.

I think that the executable needs to be rebuilt.