[Bioc-devel] Compilation flags, CHECK errors and BiocNeighbors

[Shian Su]

From what I can see in scPipe, it looks to me like there’s difference in flags for tokay1 and tokay2 (release and development).

http://bioconductor.org/checkResults/release/bioc-LATEST/scPipe/tokay1-buildbin.html
-O2 -Wall -mtune=generic

http://bioconductor.org/checkResults/devel/bioc-LATEST/scPipe/tokay2-buildbin.html
-O3 -march=native -mtune=native

On 18 Dec 2018, at 12:05 pm, Aaron Lun <infinite.monkeys.with.keyboards using gmail.com<mailto:infinite.monkeys.with.keyboards using gmail.com>> wrote:

Thanks Val. I don�t think it�s a BiocNeighbors thing, as it doesn�t try to customize the compilation flags or have its own Makevars. Moreover, the �-O3 -mtune=native -mtune=generic� flags seem to show up on all of my packages containing C++11 code. Some cursory checks of other packages suggest that the correct flags (�-O2 -mtune=generic�) are used for C++98 code.

-A

> On 17 Dec 2018, at 17:47, Obenchain, Valerie <Valerie.Obenchain using RoswellPark.org<mailto:Valerie.Obenchain using RoswellPark.org>> wrote:
>
> Hi Aaron,
>
> The only compilation flags that are different for tokay1 (release) and
> tokay2 (devel) are C++14 flags. BiocNeighbors is not using C++14 but
> C++11 so I think the changes we discussed previously actually don't
> apply to your case.
>
> All compilation flags we use are listed at the top of the build report,
> e.g., for tokay2:
>
> https://www.bioconductor.org/checkResults/devel/bioc-LATEST/tokay2-NodeInfo.html <https://www.bioconductor.org/checkResults/devel/bioc-LATEST/tokay2-NodeInfo.html>
>
> I can look into this further but right now I'm not sure where the '-O3
> -march=native -mtune=native' is coming from in the check output for
> BiocNeighbors. We don't use 'native' on the builders for build/check or
> for creating binaries.
>
> Herve might have more insight on this.
>
> Val
>
>
>
>
>
>
>
> On 12/15/18 10:56 PM, Aaron Lun wrote:
>> Sometime between 6-18 November, BiocNeighbors� BioC-devel builds began failing on Windows 64-bit, and have continued to fail since:
>>
>> http://bioconductor.org/checkResults/devel/bioc-LATEST/BiocNeighbors/ <http://bioconductor.org/checkResults/devel/bioc-LATEST/BiocNeighbors/> <http://bioconductor.org/checkResults/devel/bioc-LATEST/BiocNeighbors/ <http://bioconductor.org/checkResults/devel/bioc-LATEST/BiocNeighbors/>>
>>
>> The most interesting part is the nature of the failures. They are not segmentation faults but rather �incorrect� output in the unit tests:
>>
>> - BiocNeighbors uses the Annoy algorithm for approximate nearest neighbor search, which is provided as a header-only C++ library in the RcppAnnoy package.
>>
>> - I have compiled the BiocNeighhbors C++ code with an �#include" for these libraries to use the Annoy routines. For testing, I compared the output of my C++ code to the output of the code in the RcppAnnoy package.
>>
>> - It is these tests that are failing (i.e., the output does not match up) during CHECK on Windows 64-bit only, despite the fact that the same library is being �#include�d in both the BiocNeighbors and RcppAnnoy sources!
>>
>> What makes this particularly intriguing is that the differences between BiocNeighbors and RcppAnnoy are very minor. Less than 1% of the neighbor identities differ, and only for some of the scenarios, so it�s not an obvious bug that would be changing the output en masse. Now, the package also uses/tests Annoy in BioC-release but builds fine on tokay1:
>>
>> http://bioconductor.org/checkResults/release/bioc-LATEST/BiocNeighbors/ <http://bioconductor.org/checkResults/release/bioc-LATEST/BiocNeighbors/> <http://bioconductor.org/checkResults/release/bioc-LATEST/BiocNeighbors/ <http://bioconductor.org/checkResults/release/bioc-LATEST/BiocNeighbors/>>
>>
>> The major difference between the Bioc-release/devel builds is the compilation flags, which have changed from �-O2 -mtune=generic� to �-O3 -march=native -mtune=native� in tokay2. I am told (thanks Val) that the timing of this change is consistent with the start of the BiocNeighbors build failures on tokay2. I would guess that RcppAnnoy is also compiled with �-O2 -mtune=generic� on the CRAN build systems, introducing differences in optimization levels between the BiocNeighbors and RcppAnnoy binaries. These could be responsible for the discrepancies in the search results.
>>
>> I was able to reproduce this on my Unix cluster (gcc 6.5.0) where setting �-march=native� with either �-O3� or �-O2� caused a difference in the calculations. After much trial and error, I eventually narrowed this down to the �-mfma� flag, which seems to change the precision of multiply-and-add operations and thus the search results. This occurs even when AVX support is turned off; I guess the compiler tries to be smart if it detects you are doing some kind of simultaneous multiply and addition, which is a pretty common thing to do when computing Euclidean distances.
>>
>> In summary: can we not use �-march=native� on tokay2? (Val, I know we discussed this, but whatever changes you made to the compilation flags don�t seem to have propagated to the build machines.) As the case study with BiocNeighbors shows, this leads to inconsistencies between the CRAN and BioC-devel binaries for the same code, which unnecessarily complicates downstream usage and unit tests. I also wonder how binaries specialized for tokay2�s architecture would behave on other CPUs with different instruction sets, if they would run at all.
>>
>> Cheers,
>>
>> Aaron
>>       [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> <mailto:Bioc-devel using r-project.org> mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel <https://stat.ethz.ch/mailman/listinfo/bioc-devel>
>>
>
>
>
> This email message may contain legally privileged and/or confidential information.  If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited.  If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you.


        [[alternative HTML version deleted]]

_______________________________________________
Bioc-devel using r-project.org<mailto:Bioc-devel using r-project.org> mailing list
https://stat.ethz.ch/mailman/listinfo/bioc-devel


_______________________________________________

The information in this email is confidential and intended solely for the addressee.
You must not disclose, forward, print or use it without the permission of the sender.

The Walter and Eliza Hall Institute acknowledges the Wurundjeri people of the Kulin
Nation as the traditional owners of the land where our campuses are located and
the continuing connection to country and community.
_______________________________________________

	[[alternative HTML version deleted]]