[Bioc-devel] error in loading pbdMPI on tokay2

[Armen Abnousi]

Hi Martin,

Thanks for your answer. What do you mean by "might go to the trouble of
installing it", is there a way for us to install software on the build
machines?
As for the other suggestion, I'm not sure whether using
BiocFileCache::BiocFileCache() will be more efficient than using parallel
apply for example returning all the results to one processor and then
broadcasting them out to all. If every processor is to write its
intermediate results to a file and those files are to be read by every
other processor then there will be "n" file reads, where each file is
possibly on a different location on the disk. Do you happen to have any
performance evaluation results for this method. But in any case I did not
know about BiocFileCache, thanks for sharing it.

Thanks,
Armen

On Thu, Feb 8, 2018 at 7:58 AM, Martin Morgan <martin.morgan at roswellpark.org
> wrote:

>
>
> On 02/07/2018 05:01 PM, Armen Abnousi wrote:
>
>> Hi Valerie,
>>
>> Thanks for your reply. I can make it work using snow I believe. But the
>> problem is that all functions in these packages that you have suggested
>> (including snow; as far as I have seen) return the answer to one node and
>> then if I want all nodes to have the same answer I will need to broadcast
>> it to all nodes or repeat the same operation. This is very prohibitive,
>> requiring a lot of communication. On the other hand if I could use MPI
>> functionality, allGather would cut the operation to half (if not more,
>> given the size of the communication that will be needed. I haven't been
>> able to find a package with a functionality similar to allGather that will
>> work without MPI. With larger dataset sizes, this problem becomes more
>> threatening and that is just the opposite of what drives us to use
>> parallelism. I don't know if there is any other option, maybe we can't do
>> better, but I thought I'll let you know what my problem is and maybe
>> you'll
>> suggest a better way to deal with it. Thank you again.
>>
>
> as a sub-optimal solution one could use the file system for data transfer,
> sending tempfile() or app-specific BiocFileCache::BiocFileCache() file
> paths between nodes; access could be synchronized using BiocParallel::ipc*
> mutex.
>
> Although the build system does not support MPI, sufficiently motivated
> users might go to the trouble of installing it; you could make your
> parallel code conditional on presence of MPI, defaulting to serial (maybe
> with an encouraging message()) otherwise. MPI depedencies would be moved to
> the Enhances: field, I think...
>
> Martin
>
>
>> Thanks,
>> Armen
>>
>> On Wed, Jan 31, 2018 at 2:37 PM, Obenchain, Valerie <
>> Valerie.Obenchain at roswellpark.org> wrote:
>>
>> Hi Armen,
>>>
>>> MPI is not installed on the Windows builders so pbdMPI is not supported.
>>> Was there a specific reason you chose pbdMPI? There are several other
>>> options for parallel evaluation which are supported across all 3
>>> platforms,
>>> e.g., BiocParallel, doParallel and batchtools.
>>>
>>> I took a brief look at what naddaR is importing from pdbMPI and didn't
>>> see
>>> anything unusual - I believe you can find all functionality in one of the
>>> alternatives. Please have a look and let us know if one of these will
>>> work
>>> for you.
>>>
>>> Thanks.
>>> Valerie
>>>
>>>
>>>
>>> On 01/27/2018 12:35 PM, Armen Abnousi wrote:
>>>
>>> Hi,
>>>
>>> I'm submitting a package for build, but for windows it goes to tokay2 and
>>> fails during the build with the following message:
>>>
>>> ------------------------------------------------------------
>>> ----------------------
>>>
>>> * checking for file 'naddaR/DESCRIPTION' ... OK
>>> * preparing 'naddaR':
>>> * checking DESCRIPTION meta-information ... OK
>>> * installing the package to build vignettes
>>>        -----------------------------------
>>> * installing *source* package 'naddaR' ...
>>> ** R
>>> ** inst
>>> ** preparing package for lazy loading
>>> Error : .onLoad failed in loadNamespace() for 'pbdMPI', details:
>>>    call: inDL(x, as.logical(local), as.logical(now), ...)
>>>    error: unable to load shared object
>>> 'C:/Users/pkgbuild/packagebuilder/workers/jobs/626/R-libs/
>>> pbdMPI/libs/x64/pbdMPI.dll':
>>>    LoadLibrary failure:  The specified module could not be found.
>>>
>>> ERROR: lazy loading failed for package 'naddaR'
>>> * removing 'C:/Users/pkgbuild/AppData/Local/Temp/Rtmp4kcgED/Rinst1ffc27
>>> 75216d/naddaR'
>>> In R CMD INSTALL
>>>        -----------------------------------
>>> ERROR: package installation failed
>>>
>>> ------------------------------------------------------------
>>> -----------------------
>>>
>>> How can I change this? It builds for Linux and Mac on Merida2 and
>>> Malbec2.
>>>
>>> Thanks,
>>> Armen
>>>
>>>         [[alternative HTML version deleted]]
>>>
>>> _______________________________________________Bioc-devel at r-project.org
>>> mailing listhttps://stat.ethz.ch/mailman/listinfo/bioc-devel
>>>
>>>
>>>
>>> This email message may contain legally privileged and/or confidential
>>> information. If you are not the intended recipient(s), or the employee or
>>> agent responsible for the delivery of this message to the intended
>>> recipient(s), you are hereby notified that any disclosure, copying,
>>> distribution, or use of this email message is prohibited. If you have
>>> received this message in error, please notify the sender immediately by
>>> e-mail and delete this email message from your computer. Thank you.
>>>
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> Bioc-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>
>>
>
> This email message may contain legally privileged and/or confidential
> information.  If you are not the intended recipient(s), or the employee or
> agent responsible for the delivery of this message to the intended
> recipient(s), you are hereby notified that any disclosure, copying,
> distribution, or use of this email message is prohibited.  If you have
> received this message in error, please notify the sender immediately by
> e-mail and delete this email message from your computer. Thank you.
>

	[[alternative HTML version deleted]]