LensPix compile with large integers

Use of Healpix, camb, CLASS, cosmomc, compilers, etc.
Post Reply
Victor Chan
Posts: 1
Joined: October 04 2018
Affiliation: University of Toronto, Department of Astronomy and Astrophysics

LensPix compile with large integers

Post by Victor Chan » October 11 2018

Hello,

My issue started when I tried to run LensPix to simulate maps at NSIDE=4096 on the Niagara supercomputer at Scinet. I had previously successfully simulated lensed CMB maps with LensPix at NSIDE=2048, but upon running a simulation for NSIDE=4096, I receive this output:

Code: Select all

 Random seeds:   175, 29218 rand_inst:   0
 POLALM2MAP: Sending to farm
 POLALM2MAP Time:    1679.84955756902
 SCALALM2MAP: Sending to farm
 SCALALM2MAP Time :   683.105898983078
 ALM2GRADIENTMAP: Sending to farm
 ALM2GRADIENTMAP Time:    941.056164598092
 ALM2LENSEDMAPINTERPCYL: Sending to farm
Large npix would need compilation (and MPI library) with long integers
The version of LensPix I had working was compiled with mpif90 with openmpi 3.1.0 and the following Makefile:

Code: Select all

#default settings for ifort

F90C     = mpif90
#F90C    = ifort

healpix = /home/p/pen/victorc/packages/Healpix_3.40
LAPACKL = -mkl=sequential -lmpi -lhealpix -qopenmp

#remove -xHost if cluster is not homogeneous
#add -DHEALPIXI4B if using older healpix and get errors about arguments not matching
FFLAGS = -O3 -xHost -ip -fpp -error-limit 500 -DMPIPIX -DMPI -heap-arrays

#cfitsio = /usr/local/Cluster-Apps/cfitsio/intel/3.300
cfitsio = ${SCINET_CFITSIO_ROOT}


F90FLAGS = $(FFLAGS) -I$(healpix) -I$(healpix)/include -L$(cfitsio)/lib -L$(healpix)/lib $(LAPACKL) -lcfitsio

OBJFILES= toms760.o inifile.o utils.o spin_alm_tools.o \
   HealpixObj.o HealpixVis.o

LENSPIX = $(OBJFILES) SimLens.o

default: simlens
all: simlens recon

spin_alm_tools.o:  utils.o toms760.o
HealpixObj.o: spin_alm_tools.o
HealpixVis.o: HealpixObj.o
SimLens.o: HealpixVis.o inifile.o

.f.o:
	f77 $(F90FLAGS) -c $<

%.o: %.f90
	$(F90C) -g $(F90FLAGS) -c $*.f90

%.o: %.F90
	$(F90C) -g $(F90FLAGS) -c $*.F90


simlens: $(LENSPIX) 	
	$(F90C) -g -o simlens $(LENSPIX) $(F90FLAGS) $(LINKFLAGS)

recon: $(OBJFILES) LensReconExample.o
	$(F90C) -g -o recon $(OBJFILES) LensReconExample.o $(F90FLAGS) $(LINKFLAGS)

clean:
	rm -f *.o* *.e* *.mod *.d *.pc *.obj core* *.il
To start, I tried re-compiling LensPix with the -i8 flag, but I receive similar errors to the following:

Code: Select all

spin_alm_tools.f90(7488): error #6633: The type of the actual argument differs from the type of the dummy argument.   [NLMAX]
    call SyncInts(nlmax,L1_Max,lmax_bi)
------------------^
spin_alm_tools.f90(7488): error #6633: The type of the actual argument differs from the type of the dummy argument.   [LMAX_BI]
    call SyncInts(nlmax,L1_Max,lmax_bi)
-------------------------------^
and

Code: Select all

spin_alm_tools.f90(7914): warning #6075: The data type of the actual argument does not match the definition.   [I]
            call  maparray2packedpolalms(H,i, dummymaps, dummyalms, 1, .false.)
-------------------------------------------^
spin_alm_tools.f90(7916): warning #6075: The data type of the actual argument does not match the definition.   [I]
            call  scalalm2bispectrum(H, i, i, i, dummyalm, dummyslice)
----------------------------------------^
spin_alm_tools.f90(7916): warning #6075: The data type of the actual argument does not match the definition.   [I]
            call  scalalm2bispectrum(H, i, i, i, dummyalm, dummyslice)
-------------------------------------------^
spin_alm_tools.f90(7916): warning #6075: The data type of the actual argument does not match the definition.   [I]
            call  scalalm2bispectrum(H, i, i, i, dummyalm, dummyslice)
----------------------------------------------^
spin_alm_tools.f90(7918): warning #6075: The data type of the actual argument does not match the definition.   [I]
            call packedscalalms2maparray(H,i,dummyscalalms,dummymaps,1,.false.)
-------------------------------------------^
compilation aborted for spin_alm_tools.f90 (code 1)
make: *** [spin_alm_tools.o] Error 1
I've contacted the support for Niagara, and they believe it isn't an issue with the modules I have loaded. I have also checked that HealPix is working for high resolution by creating maps at NSIDE=4096 with healpy and taking their power spectra. Has anyone else tried to simulate high resolution maps, and run into this?

Thanks,
Victor Chan

Antony Lewis
Posts: 1485
Joined: September 23 2004
Affiliation: University of Sussex
Contact:

Re: LensPix compile with large integers

Post by Antony Lewis » October 12 2018

You may be able to change the MPI_INTEGER passes to MPI_INTEGER8 and update other variables accordingly. I've not tried it (github pull request welcome if you find a general solution).

Post Reply