[Bioc-devel] error in loading pbdMPI on tokay2

[Martin Morgan]

On 02/07/2018 05:01 PM, Armen Abnousi wrote:
> Hi Valerie,
> 
> Thanks for your reply. I can make it work using snow I believe. But the
> problem is that all functions in these packages that you have suggested
> (including snow; as far as I have seen) return the answer to one node and
> then if I want all nodes to have the same answer I will need to broadcast
> it to all nodes or repeat the same operation. This is very prohibitive,
> requiring a lot of communication. On the other hand if I could use MPI
> functionality, allGather would cut the operation to half (if not more,
> given the size of the communication that will be needed. I haven't been
> able to find a package with a functionality similar to allGather that will
> work without MPI. With larger dataset sizes, this problem becomes more
> threatening and that is just the opposite of what drives us to use
> parallelism. I don't know if there is any other option, maybe we can't do
> better, but I thought I'll let you know what my problem is and maybe you'll
> suggest a better way to deal with it. Thank you again.

as a sub-optimal solution one could use the file system for data 
transfer, sending tempfile() or app-specific 
BiocFileCache::BiocFileCache() file paths between nodes; access could be 
synchronized using BiocParallel::ipc* mutex.

Although the build system does not support MPI, sufficiently motivated 
users might go to the trouble of installing it; you could make your 
parallel code conditional on presence of MPI, defaulting to serial 
(maybe with an encouraging message()) otherwise. MPI depedencies would 
be moved to the Enhances: field, I think...

Martin

> 
> Thanks,
> Armen
> 
> On Wed, Jan 31, 2018 at 2:37 PM, Obenchain, Valerie <
> Valerie.Obenchain at roswellpark.org> wrote:
> 
>> Hi Armen,
>>
>> MPI is not installed on the Windows builders so pbdMPI is not supported.
>> Was there a specific reason you chose pbdMPI? There are several other
>> options for parallel evaluation which are supported across all 3 platforms,
>> e.g., BiocParallel, doParallel and batchtools.
>>
>> I took a brief look at what naddaR is importing from pdbMPI and didn't see
>> anything unusual - I believe you can find all functionality in one of the
>> alternatives. Please have a look and let us know if one of these will work
>> for you.
>>
>> Thanks.
>> Valerie
>>
>>
>>
>> On 01/27/2018 12:35 PM, Armen Abnousi wrote:
>>
>> Hi,
>>
>> I'm submitting a package for build, but for windows it goes to tokay2 and
>> fails during the build with the following message:
>>
>> ----------------------------------------------------------------------------------
>>
>> * checking for file 'naddaR/DESCRIPTION' ... OK
>> * preparing 'naddaR':
>> * checking DESCRIPTION meta-information ... OK
>> * installing the package to build vignettes
>>        -----------------------------------
>> * installing *source* package 'naddaR' ...
>> ** R
>> ** inst
>> ** preparing package for lazy loading
>> Error : .onLoad failed in loadNamespace() for 'pbdMPI', details:
>>    call: inDL(x, as.logical(local), as.logical(now), ...)
>>    error: unable to load shared object
>> 'C:/Users/pkgbuild/packagebuilder/workers/jobs/626/R-libs/pbdMPI/libs/x64/pbdMPI.dll':
>>    LoadLibrary failure:  The specified module could not be found.
>>
>> ERROR: lazy loading failed for package 'naddaR'
>> * removing 'C:/Users/pkgbuild/AppData/Local/Temp/Rtmp4kcgED/Rinst1ffc2775216d/naddaR'
>> In R CMD INSTALL
>>        -----------------------------------
>> ERROR: package installation failed
>>
>> ------------------------------------------------------------
>> -----------------------
>>
>> How can I change this? It builds for Linux and Mac on Merida2 and Malbec2.
>>
>> Thanks,
>> Armen
>>
>> 	[[alternative HTML version deleted]]
>>
>> _______________________________________________Bioc-devel at r-project.org mailing listhttps://stat.ethz.ch/mailman/listinfo/bioc-devel
>>
>>
>>
>> This email message may contain legally privileged and/or confidential
>> information. If you are not the intended recipient(s), or the employee or
>> agent responsible for the delivery of this message to the intended
>> recipient(s), you are hereby notified that any disclosure, copying,
>> distribution, or use of this email message is prohibited. If you have
>> received this message in error, please notify the sender immediately by
>> e-mail and delete this email message from your computer. Thank you.
> 
> 	[[alternative HTML version deleted]]
> 
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
> 


This email message may contain legally privileged and/or...{{dropped:2}}