[Rd] cleanup and Makevars

Prof Brian Ripley ripley at stats.ox.ac.uk
Thu Jul 12 11:05:02 CEST 2007 

You seem extraordinarily reluctant to tell us what you are actually doing: 
no command appears here.  I presume this is a call to R CMD INSTALL, but 
why do I have to keep guessing?

Since you have taken over the make action via the all: target in your 
Makevars, you need also to take over responsibility for cleaning.  I 
should think

all: myclean $(SHLIB)

myclean:
 	rm -f $(OBJS)

should do it.  And BTW, the use of $(OBJS) to set the objects is 
undocumented, at least in 'Writing R Extensions'.

On Wed, 11 Jul 2007, Kasper Daniel Hansen wrote:

>
> On Jul 11, 2007, at 9:30 AM, Prof Brian Ripley wrote:
>
>> On Mon, 9 Jul 2007, Kasper Daniel Hansen wrote:
>> 
>>> Hi
>>> 
>>> This is a question prompted by the mac version of R, but as I see it,
>>> it should have broader interest.
>>> 
>>> These days the CRAN Mac binary per default compiles every package for
>>> two architectures. First i386 and then ppc. In between the two
>>> compilation runs, any object files in pkgname/src is removed. This
>>> cleanup is necessary since otherwise Make would not recompile the
>>> object files under the next architecture.
>>> 
>>> I have a package which includes a large SDK which I have put in
>>>  src/fusion_sdk
>>> (and subdirectories). I take care of that by using a Makevars file
>>> with essentially (the whole Makevars file is reproduced below)
>>> 
>>> PKG_SOURCES = \
>>>        fusion_sdk/calvin_files/data/src/CDFData.cpp
>>> 
>>> OBJS=$(PKG_SOURCES:.cpp=.o)
>>> 
>>> all: $(SHLIB)
>>> 
>>> However, this setup does not remove the object files in the
>>> fusion_sdk subdirectory. From a posting on R-sig-mac by Simon Urbanek
>>> I learned that I need to clean up these directories myself.
>>> 
>>> I have - per R extensions - attempted to do so using a pkgname/
>>> cleanup script. While this works for cleaning up the sub directories
>>> when I do R CMD build, it seems as if this script is not being run
>>> between compilation runs (is this intentional btw.?).
>> 
>> What do you mean by 'compilation runs'?  The 'cleanup' script will be run 
>> by R CMD INSTALL --cleanup (and without --cleanup it is intentionally not 
>> run).
>> 
>>> So my question is now: how do I in a simple way clean up my
>>> subdirectories? I would prefer it to be as simple as possible because
>>> so far I have not really needed anything besides a Makevars file. In
>>> fact the impression I have right now from looking at the SHLIB and
>>> INSTALL scripts is that I might need a Makefile that removes the
>>> object files _before_ starting the compilation, as I don't really see
>>> any cleanup process (eg. making a target clean) - but this may very
>>> well be due to my limited understanding of these scripts.
>> 
>> I see R CMD INSTALL --clean that runs a clean up afterwards, and 2.6.0 will 
>> also have R CMD INSTALL --preclean in case you forgot --clean on the last 
>> run.
>> 
>>> Any help/hints on how to proceed? Especially if I want it to be
>>> portable?
>> 
>> What does R CMD INSTALL --clean not do that you want?
>> 
>> If you are talking about R CMD SHLIB, that currently does not clean up but 
>> this has been raised and it will most likely have a --clean option in 2.6.0 
>> that removes $(OBJECTS).
>
> Perhaps I was not clear enough: The CRAN binary of R for Mac has two 
> architectures inside $R_HOME/bin/exec, namely "ppc" and "i386". This means 
> that whenever anyone installs a source package I see two compilation runs - 
> the following is a shortened output:
>
> * Installing to library '/Library/Frameworks/R.framework/Resources/library'
> * Installing *source* package 'affxparser' ...
> ** libs
> ** arch - i386
> g++-4.0 -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -no-cpp-precomp 
> -I/Library/Frameworks/R.framework/Resources/include 
> -I/Library/Frameworks/R.framework/Resources/include/i386 -imacros 
> R_affx_constants.h -Ifusion_sdk/calvin_files/array/src 
> -Ifusion_sdk/calvin_files/data/src -Ifusion_sdk/calvin_files/exception/src 
> -Ifusion_sdk/calvin_files/fusion/src 
> -Ifusion_sdk/calvin_files/fusion/src/GCOSAdapter 
> -Ifusion_sdk/calvin_files/fusion/src/CalvinAdapter 
> -Ifusion_sdk/calvin_files/parameter/src -Ifusion_sdk/calvin_files/parsers/src 
> -Ifusion_sdk/calvin_files/portability/src 
> -Ifusion_sdk/calvin_files/template/src -Ifusion_sdk/calvin_files/utils/src 
> -Ifusion_sdk/calvin_files/writers/src -Ifusion_sdk/file 
> -Ifusion_sdk/portability -D_USE_MEM_MAPPING_ -msse3    -fPIC  -g -O2 -Wall 
> -O0 -c fusion_sdk/calvin_files/data/src/CDFData.cpp -o 
> fusion_sdk/calvin_files/data/src/CDFData.o
>
> (Now a lot of other files gets compiled)
>
> g++-4.0 -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk -dynamiclib 
> -Wl,-macosx_version_min -Wl,10.3 -undefined dynamic_lookup -single_module 
> -multiply_defined suppress -L/usr/local/lib -o affxparser.so 
> fusion_sdk/calvin_files/data/src/CDFData.o 
> fusion_sdk/calvin_files/data/src/CDFProbeGroupInformation.o 
> fusion_sdk/calvin_files/data/src/CDFProbeInformation.o 
> fusion_sdk/calvin_files/data/src/CDFProbeSetInformation.o 
> fusion_sdk/calvin_files/data/src/CDFQCProbeInformation.o 
> fusion_sdk/calvin_files/data/src/CDFQCProbeSetInformation.o 
> fusion_sdk/calvin_files/data/src/CELData.o 
> fusion_sdk/calvin_files/data/src/DataGroup.o 
> fusion_sdk/calvin_files/data/src/DataGroupHeader.o 
> fusion_sdk/calvin_files/data/src/DataSet.o 
> fusion_sdk/calvin_files/data/src/DataSetHeader.o 
> fusion_sdk/calvin_files/data/src/FileHeader.o 
> fusion_sdk/calvin_files/data/src/GenericData.o 
> fusion_sdk/calvin_files/data/src/GenericDataHeader.o 
> fusion_sdk/calvin_files/exception/src/ExceptionBase.o 
> fusion_sdk/calvin_files/fusion/src/CalvinAdapter/CalvinCELDataAdapter.o 
> fusion_sdk/calvin_files/fusion/src/FusionBPMAPData.o 
> fusion_sdk/calvin_files/fusion/src/FusionCDFData.o 
> fusion_sdk/calvin_files/fusion/src/FusionCDFQCProbeSetNames.o 
> fusion_sdk/calvin_files/fusion/src/FusionCELData.o 
> fusion_sdk/calvin_files/fusion/src/GCOSAdapter/GCOSCELDataAdapter.o 
> fusion_sdk/calvin_files/parameter/src/ParameterNameValueType.o 
> fusion_sdk/calvin_files/parsers/src/CDFFileReader.o 
> fusion_sdk/calvin_files/parsers/src/CelFileReader.o 
> fusion_sdk/calvin_files/parsers/src/DataGroupHeaderReader.o 
> fusion_sdk/calvin_files/parsers/src/DataGroupReader.o 
> fusion_sdk/calvin_files/parsers/src/DataSetHeaderReader.o 
> fusion_sdk/calvin_files/parsers/src/DataSetReader.o 
> fusion_sdk/calvin_files/parsers/src/FileHeaderReader.o 
> fusion_sdk/calvin_files/parsers/src/FileInput.o 
> fusion_sdk/calvin_files/parsers/src/GenericDataHeaderReader.o 
> fusion_sdk/calvin_files/parsers/src/GenericFileReader.o 
> fusion_sdk/calvin_files/utils/src/AffymetrixGuid.o 
> fusion_sdk/calvin_files/utils/src/DateTime.o 
> fusion_sdk/calvin_files/utils/src/FileUtils.o 
> fusion_sdk/calvin_files/utils/src/StringUtils.o 
> fusion_sdk/calvin_files/utils/src/checksum.o fusion_sdk/file/BPMAPFileData.o 
> fusion_sdk/file/BPMAPFileWriter.o fusion_sdk/file/CDFFileData.o 
> fusion_sdk/file/CELFileData.o fusion_sdk/file/FileIO.o 
> fusion_sdk/file/FileWriter.o R_affx_cel_parser.o R_affx_cdf_parser.o 
> R_affx_cdf_extras.o R_affx_bpmap_parser.o 
> -F/Library/Frameworks/R.framework/.. -framework R
> ** arch - ppc
> g++-4.0 -arch ppc -isysroot /Developer/SDKs/MacOSX10.4u.sdk -no-cpp-precomp 
> -I/Library/Frameworks/R.framework/Resources/include 
> -I/Library/Frameworks/R.framework/Resources/include/ppc -imacros 
> R_affx_constants.h -Ifusion_sdk/calvin_files/array/src 
> -Ifusion_sdk/calvin_files/data/src -Ifusion_sdk/calvin_files/exception/src 
> -Ifusion_sdk/calvin_files/fusion/src 
> -Ifusion_sdk/calvin_files/fusion/src/GCOSAdapter 
> -Ifusion_sdk/calvin_files/fusion/src/CalvinAdapter 
> -Ifusion_sdk/calvin_files/parameter/src -Ifusion_sdk/calvin_files/parsers/src 
> -Ifusion_sdk/calvin_files/portability/src 
> -Ifusion_sdk/calvin_files/template/src -Ifusion_sdk/calvin_files/utils/src 
> -Ifusion_sdk/calvin_files/writers/src -Ifusion_sdk/file 
> -Ifusion_sdk/portability -D_USE_MEM_MAPPING_ -I/usr/local/include    -fPIC 
> -g -O2 -Wall -O0 -c R_affx_cel_parser.cpp -o R_affx_cel_parser.o
>
> As you see from this (and sorry for all the include statements), when R 
> starts compiling for arch = "ppc", it starts with a with in /src whereas the 
> first compilation run (or more precise the first architecture) starts from 
> /src/fusion_sdk/calvin_files/data/src. This is of course because the object 
> files in the subdirectories do not get removed between the two architectures.
>
> This has - probably - something to do with R CMD SHLIB not cleaning up after 
> itself. A method of removing $(OBJECTS) would be exactly what I need, and may 
> I suggest that this is done per default instead of being set by an option. 
> Anyone using the CRAN binary for the Mac will have the same problem as I.
>
> I am happy to see that this may be fixed for R-2.6.0. Is there anything I can 
> do in the meantime for getting my package to work under R-2.5.1? (I can 
> always move my source files from the /src/fusion_sdk subdirectories directly 
> into /src, but I would prefer to avoid that).
>
>>> 
>>> Kasper
>>> 
>>> The full Makevars file:
>>> 
>>> MYCXXFLAGS=-O0
>> 
>> You do realize that is highly non-portable?
>
> Yes, but I do not know of any other way to downgrade the optimization level. 
> The SDK I am using breaks (due to memory alignment issues if a specific 
> optimization flag for GCC is include - the flag is certainly implied by -O2 
> and as far as I recall also -O1). I guess the solution is to switch to an 
> autoconfigure script. The main reason for not doing this is 1) time and 
> actually more 2) the fact that the SDK does not come with an ac script.
>
> Thanks for the feedback, Kasper
>
>
>>> %.o: %.cpp
>>> 	$(CXX) $(ALL_CPPFLAGS) $(ALL_CXXFLAGS) $(MYCXXFLAGS) -c $< -o $@
>>> 
>>> PKG_CPPFLAGS=\
>>> -imacros R_affx_constants.h\
>>> -Ifusion_sdk/calvin_files/array/src\
>>> -Ifusion_sdk/calvin_files/data/src\
>>> -Ifusion_sdk/calvin_files/exception/src\
>>> -Ifusion_sdk/calvin_files/fusion/src\
>>> -Ifusion_sdk/calvin_files/fusion/src/GCOSAdapter\
>>> -Ifusion_sdk/calvin_files/fusion/src/CalvinAdapter\
>>> -Ifusion_sdk/calvin_files/parameter/src\
>>> -Ifusion_sdk/calvin_files/parsers/src\
>>> -Ifusion_sdk/calvin_files/portability/src\
>>> -Ifusion_sdk/calvin_files/template/src\
>>> -Ifusion_sdk/calvin_files/utils/src\
>>> -Ifusion_sdk/calvin_files/writers/src\
>>> -Ifusion_sdk/file\
>>> -Ifusion_sdk/portability\
>>> -D_USE_MEM_MAPPING_
>>> 
>>> PKG_SOURCES = \
>>> 	fusion_sdk/calvin_files/data/src/CDFData.cpp\
>>> 	fusion_sdk/calvin_files/data/src/CDFProbeGroupInformation.cpp\
>>> 	fusion_sdk/calvin_files/data/src/CDFProbeInformation.cpp\
>>> 	fusion_sdk/calvin_files/data/src/CDFProbeSetInformation.cpp\
>>> 	fusion_sdk/calvin_files/data/src/CDFQCProbeInformation.cpp\
>>> 	fusion_sdk/calvin_files/data/src/CDFQCProbeSetInformation.cpp\
>>> 	fusion_sdk/calvin_files/data/src/CELData.cpp\
>>> 	fusion_sdk/calvin_files/data/src/DataGroup.cpp\
>>> 	fusion_sdk/calvin_files/data/src/DataGroupHeader.cpp\
>>> 	fusion_sdk/calvin_files/data/src/DataSet.cpp\
>>> 	fusion_sdk/calvin_files/data/src/DataSetHeader.cpp\
>>> 	fusion_sdk/calvin_files/data/src/FileHeader.cpp\
>>> 	fusion_sdk/calvin_files/data/src/GenericData.cpp\
>>> 	fusion_sdk/calvin_files/data/src/GenericDataHeader.cpp\
>>> 	fusion_sdk/calvin_files/exception/src/ExceptionBase.cpp\
>>> 	fusion_sdk/calvin_files/fusion/src/CalvinAdapter/
>>> CalvinCELDataAdapter.cpp\
>>> 	fusion_sdk/calvin_files/fusion/src/FusionBPMAPData.cpp\
>>> 	fusion_sdk/calvin_files/fusion/src/FusionCDFData.cpp\
>>> 	fusion_sdk/calvin_files/fusion/src/FusionCDFQCProbeSetNames.cpp\
>>> 	fusion_sdk/calvin_files/fusion/src/FusionCELData.cpp\
>>> 	fusion_sdk/calvin_files/fusion/src/GCOSAdapter/GCOSCELDataAdapter.cpp\
>>> 	fusion_sdk/calvin_files/parameter/src/ParameterNameValueType.cpp\
>>> 	fusion_sdk/calvin_files/parsers/src/CDFFileReader.cpp\
>>> 	fusion_sdk/calvin_files/parsers/src/CelFileReader.cpp\
>>> 	fusion_sdk/calvin_files/parsers/src/DataGroupHeaderReader.cpp\
>>> 	fusion_sdk/calvin_files/parsers/src/DataGroupReader.cpp\
>>> 	fusion_sdk/calvin_files/parsers/src/DataSetHeaderReader.cpp\
>>> 	fusion_sdk/calvin_files/parsers/src/DataSetReader.cpp\
>>> 	fusion_sdk/calvin_files/parsers/src/FileHeaderReader.cpp\
>>> 	fusion_sdk/calvin_files/parsers/src/FileInput.cpp\
>>> 	fusion_sdk/calvin_files/parsers/src/GenericDataHeaderReader.cpp\
>>> 	fusion_sdk/calvin_files/parsers/src/GenericFileReader.cpp\
>>> 	fusion_sdk/calvin_files/utils/src/AffymetrixGuid.cpp\
>>> 	fusion_sdk/calvin_files/utils/src/DateTime.cpp\
>>> 	fusion_sdk/calvin_files/utils/src/FileUtils.cpp\
>>> 	fusion_sdk/calvin_files/utils/src/StringUtils.cpp\
>>> 	fusion_sdk/calvin_files/utils/src/checksum.cpp\
>>> 	fusion_sdk/file/BPMAPFileData.cpp\
>>> 	fusion_sdk/file/BPMAPFileWriter.cpp\
>>> 	fusion_sdk/file/CDFFileData.cpp\
>>> 	fusion_sdk/file/CELFileData.cpp\
>>> 	fusion_sdk/file/FileIO.cpp\
>>> 	fusion_sdk/file/FileWriter.cpp\
>>> 	R_affx_cel_parser.cpp\
>>> 	R_affx_cdf_parser.cpp\
>>> 	R_affx_cdf_extras.cpp\
>>> 	R_affx_bpmap_parser.cpp
>>> 
>>> OBJS=$(PKG_SOURCES:.cpp=.o)
>>> 
>>> all: $(SHLIB)
>> 
>> -- 
>> Brian D. Ripley,                  ripley at stats.ox.ac.uk
>> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
>> University of Oxford,             Tel:  +44 1865 272861 (self)
>> 1 South Parks Road,                     +44 1865 272866 (PA)
>> Oxford OX1 3TG, UK                Fax:  +44 1865 272595

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

More information about the R-devel mailing list