[Bioc-devel] package vignette error : external data can't be captured when compiling package vignette

[Hervé Pagès]

On 12/02/2016 09:26 AM, Leonardo Collado Torres wrote:
> Dear Jurat,
>
> Maybe it'll be helpful to you if you see how other packages have their
> data. For example, take a look at one of my packages:
> https://github.com/lcolladotor/derfinder
>
> You'll see that I have two directories inside /inst/extdata with some
> files that I use in my vignette code. See
> https://github.com/lcolladotor/derfinder/tree/master/inst/extdata for
> the files and search system.file() inside
> https://github.com/lcolladotor/derfinder/blob/master/vignettes/derfinder-quickstart.Rmd
> to see how I use it in the vignette.
>
> Regarding your permissions issue, I don't recall right now how to get
> this to work with Windows. But basically you can see that error at:
>
> Warning in dir.create(pkgoutdir, mode = "0755") :
>   cannot create dir 'C:\Program Files\R\R-devel\bin\x64\MSPC.Rcheck',
> reason 'Permission denied'
>
>
> Is your user in your windows machine part of the "administrators"
> group? If not, that could be why.

Just to clarify, you don't need (and shouldn't) need administrator
privileges to 'R CMD check' a package. The error message above (which
appears as a warning) indicates that Jurat is trying to run
'R CMD check' from inside the C:\Program Files\R\R-devel\bin\x64\
folder, which is of course a very bad idea. As Dan said, 'R CMD check'
should be run in a directory where you have write permissions. Typically
in your home or in a directory below it.

H.

> You could otherwise run R CMD build
> and upload your tar ball (a .tar.gz file) to
> https://builder.r-hub.io/.
>
> Finally, you might want to search the manual of how to create R
> extensions for other details.
> https://cran.r-project.org/doc/manuals/r-release/R-exts.html
>
>
> Best,
> Leo
>
> On Fri, Dec 2, 2016 at 12:11 PM, Jurat Shayidin <juratbupt at gmail.com> wrote:
>> Dear Dan :
>>
>> I forgot to commit vignette file,  now I pushed new changes, so now error
>> could be something else. People in stackoverflow suggest me to put all my
>> external data (a.k.a, bed files) into vignette folder, try to run R CMD
>> check and see what happen, I did this solution, but it doesn't work. Plus,
>> I checked file permission of my package parent directory, seems fine to me.
>> How I check file permission of my package ?
>> How can I fix this write permission on my machine ? This is not intuitive
>> to me, Any hint please ?
>>
>> Herve suggest me don't create inst/extdata directory manually, but I still
>> can't let external data available for vignette code. FYI, my objective in
>> vignette file : let external data available for package use, compile
>> trivial version of vignette with no error, then continue to make it
>> perfect.  Here is updated session of R CMD check :
>>
>>
>> Microsoft Windows [Version 6.3.9600]
>> (c) 2013 Microsoft Corporation. All rights reserved.
>>
>> C:\Users\jvrat>cd C:\Program Files\R\R-devel\bin\x64
>>
>> C:\Program Files\R\R-devel\bin\x64>R CMD INSTALL
>> "C:\Users\jvrat\Documents\MSPC"
>> * installing to library 'C:/Users/jvrat/Documents/R/win-library/3.4'
>> * installing *source* package 'MSPC' ...
>> ** R
>> ** inst
>> ** preparing package for lazy loading
>> ** help
>> *** installing help indices
>> ** building package indices
>> ** installing vignettes
>> ** testing if installed package can be loaded
>> * DONE (MSPC)
>>
>> C:\Program Files\R\R-devel\bin\x64>R CMD check
>> "C:\Users\jvrat\Documents\MSPC"
>> Warning in dir.create(pkgoutdir, mode = "0755") :
>>   cannot create dir 'C:\Program Files\R\R-devel\bin\x64\MSPC.Rcheck',
>> reason 'Permission denied'
>> ERROR: cannot create check dir 'C:/Program
>> Files/R/R-devel/bin/x64/MSPC.Rcheck'
>>
>> C:\Program Files\R\R-devel\bin\x64>R CMD build
>> "C:\Users\jvrat\Documents\MSPC"
>> * checking for file 'C:\Users\jvrat\Documents\MSPC/DESCRIPTION' ... OK
>> * preparing 'MSPC':
>> * checking DESCRIPTION meta-information ... OK
>> * installing the package to build vignettes
>> * creating vignettes ...Warning: running command
>> '"C:/PROGRA~1/R/R-devel/bin/x64/Rscript" --vanilla --default-packages= -e
>> "tools::buildVignettes(dir = '.', tangle = TRUE)"' had status 1
>>  ERROR
>> Warning in engine$weave(file, quiet = quiet, encoding = enc) :
>>   Pandoc (>= 1.12.3) and/or pandoc-citeproc not available. Falling back to
>> R Markdown v1.
>> Quitting from lines 60-64 (vignette.Rmd)
>> Error: processing vignette 'vignette.Rmd' failed with diagnostics:
>> subscript out of bounds
>> Execution halted
>>
>> C:\Program Files\R\R-devel\bin\x64>
>>
>>
>> How I fix R CMD check error above ? system.file() still can't detect
>> external data. Any idea please ? Thanks a lot
>>
>> Best regards :
>>
>> Jurat
>>
>>
>> On Fri, Dec 2, 2016 at 5:43 PM, Dan Tenenbaum <dtenenba at fredhutch.org>
>> wrote:
>>
>>> See below.
>>>
>>> ----- Original Message -----
>>>> From: "Jurat Shayidin" <juratbupt at gmail.com>
>>>> To: "Dan Tenenbaum" <dtenenba at fredhutch.org>, "bioc-devel" <
>>> bioc-devel at r-project.org>
>>>> Sent: Friday, December 2, 2016 8:32:35 AM
>>>> Subject: Re: [Bioc-devel] package vignette error : external data can't
>>> be captured when compiling package vignette
>>>
>>>> Dear Dan :
>>>>
>>>> Really appreciated for your quick respond. Instead, I am using R CMD
>>> check
>>>> on my packages, I have an error. Here is whole session detail :
>>>>
>>>> Microsoft Windows [Version 6.3.9600]
>>>> (c) 2013 Microsoft Corporation. All rights reserved.
>>>>
>>>> C:\Users\jvrat>cd C:\Program Files\R\R-devel\bin\x64
>>>>
>>>> C:\Program Files\R\R-devel\bin\x64>R CMD INSTALL
>>>> "C:\Users\jvrat\Documents\MSPC"
>>>> * installing to library 'C:/Users/jvrat/Documents/R/win-library/3.4'
>>>> * installing *source* package 'MSPC' ...
>>>> ** R
>>>> ** inst
>>>> ** preparing package for lazy loading
>>>> ** help
>>>> *** installing help indices
>>>> ** building package indices
>>>> ** installing vignettes
>>>> ** testing if installed package can be loaded
>>>> * DONE (MSPC)
>>>>
>>>> C:\Program Files\R\R-devel\bin\x64>R CMD check
>>>> "C:\Users\jvrat\Documents\MSPC"
>>>> Warning in dir.create(pkgoutdir, mode = "0755") :
>>>>  cannot create dir 'C:\Program Files\R\R-devel\bin\x64\MSPC.Rcheck',
>>> reason
>>>> 'Permission denied'
>>>> ERROR: cannot create check dir 'C:/Program
>>>> Files/R/R-devel/bin/x64/MSPC.Rcheck'
>>>
>>> You need to run R CMD check in a directory where you have write
>>> permissions.
>>>
>>>
>>>>
>>>> C:\Program Files\R\R-devel\bin\x64>R CMD build
>>>> "C:\Users\jvrat\Documents\MSPC"
>>>> * checking for file 'C:\Users\jvrat\Documents\MSPC/DESCRIPTION' ... OK
>>>> * preparing 'MSPC':
>>>> * checking DESCRIPTION meta-information ... OK
>>>> * installing the package to build vignettes
>>>> * creating vignettes ...Warning: running command
>>>> '"C:/PROGRA~1/R/R-devel/bin/x64/Rscript" --vanilla --default-packages=
>>> -e
>>>> "tools::buildVignettes(dir = '.', tangle = TRUE)"' had status 1
>>>> ERROR
>>>> Warning in engine$weave(file, quiet = quiet, encoding = enc) :
>>>>  Pandoc (>= 1.12.3) and/or pandoc-citeproc not available. Falling back to
>>>> R Markdown v1.
>>>> Quitting from lines 62-66 (vignette.Rmd)
>>>> Error: processing vignette 'vignette.Rmd' failed with diagnostics:
>>>> object 'inputData' not found
>>>> Execution halted
>>>>
>>>
>>> Line 61 of the vignette (as it is in Github; it's apparently different on
>>> yur machine based on the error message) is:
>>>
>>> total.ERs <- denoise_ERs(peakGRs = inputData, tau.w = 1.0E-04, .fileName =
>>> "noiseER", outDir = "", verbose = FALSE)
>>>
>>> As you can see, the peakGRs argument is set to inputData, but inputData is
>>> not defined anywhere.
>>>
>>> Dan
>>>
>>>
>>>
>>>> C:\Program Files\R\R-devel\bin\x64>
>>>>
>>>>
>>>> How can I fix this R CMD check error ? Instead people in stackoverflow
>>>> remind me that using devtools packages is not stable some times, so I go
>>>> for old fashion : use CMD. Plus, still my external data can't available
>>> for
>>>> vignette data, and vignette compilation is failed again. I used
>>>> system.file() to do this, but not working. Any idea please ? How can I
>>>> overcome this problem? Thanks a lot to Bioconductor project team.
>>>>
>>>> Best regards :
>>>>
>>>> Jurat
>>>>
>>>>
>>>> On Fri, Dec 2, 2016 at 5:20 PM, Dan Tenenbaum <dtenenba at fredhutch.org>
>>>> wrote:
>>>>
>>>>> If your package is in github at https://github.com/julaiti/MSPC , it
>>>>> looks like there is no inst or extdata folder in that repository.
>>>>>
>>>>> Maybe it has not yet been added/committed/pushed to git?
>>>>>
>>>>> Note that everything _under_ inst gets installed when you install the
>>>>> package, but the inst directory itself goes away.
>>>>>
>>>>> If in your package source you have the following structure:
>>>>>
>>>>> inst/extdata/
>>>>> inst/foo.txt
>>>>>
>>>>> In the installed package you end up with:
>>>>>
>>>>> extdata/
>>>>> foo.txt
>>>>>
>>>>> HTH
>>>>> Dan
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Jurat Shayidin" <juratbupt at gmail.com>
>>>>>> To: "Hervé Pagès" <hpages at fredhutch.org>, "bioc-devel" <
>>>>> bioc-devel at r-project.org>
>>>>>> Sent: Friday, December 2, 2016 5:36:28 AM
>>>>>> Subject: Re: [Bioc-devel] package vignette error : external data can't
>>>>> be captured when compiling package vignette
>>>>>
>>>>>> Dear Hervé :
>>>>>>
>>>>>> Thanks again for your response on my issue. I've read your message
>>> very
>>>>>> carefully and did all you suggested to me, still can't fix the
>>> vignette
>>>>>> compilation error. I have developed my package on windows machine
>>> under
>>>>>> devel version of R and Bioc, all unit test works fine to me. I don't
>>>>>> understand why inst/ directory was not created when I build and
>>> install
>>>>> my
>>>>>> packages, external data can't be read during vignette compilation.
>>>>>> However, my objective is, to build my package vignette with no error
>>> in
>>>>> the
>>>>>> first place, then continue to edit the context until getting final
>>>>> version
>>>>>> of vignette file. When I install packages using devtools::install(), I
>>>>> got
>>>>>> this :
>>>>>>
>>>>>>> devtools::install()
>>>>>> Installing MSPC
>>>>>> "C:/PROGRA~1/R/R-devel/bin/x64/R" \
>>>>>> --no-site-file --no-environ --no-save \
>>>>>> --no-restore --quiet CMD INSTALL \
>>>>>> "C:/Users/jvrat/Documents/MSPC" \
>>>>>> --library="C:/Users/jvrat/Documents/R/win-library/3.4" \
>>>>>> --install-tests
>>>>>> * installing *source* package 'MSPC' ...
>>>>>> ** R
>>>>>> ** tests
>>>>>> ** preparing package for lazy loading
>>>>>> ** help
>>>>>> *** installing help indices
>>>>>> ** building package indices
>>>>>> ** installing vignettes
>>>>>> ** testing if installed package can be loaded
>>>>>> * DONE (MSPC)
>>>>>> Reloading installed MSPC
>>>>>>
>>>>>>
>>>>>> I believe doing this is right in vignette file after I follow your
>>>>> advise :
>>>>>>
>>>>>> ```{r}
>>>>>> library(MSPC)
>>>>>> inputBed <- list.files(system.file("extdata", package = "MSPC"),
>>>>> full.names
>>>>>> = TRUE)
>>>>>> inputData <- readPeakFile(peakFolder = inputBed)
>>>>>>
>>>>>> inputData
>>>>>> ```
>>>>>>
>>>>>>
>>>>>> However, I should not add inst/extdata manually , without this I can't
>>>>>> allow to load external data for package use, and vignette compilation
>>>>> keep
>>>>>> failed. To be honest, this is my very first time building R packages,
>>> I
>>>>>> don't know how to fix this error. Just out of curiosity, during
>>> package
>>>>>> installation, everything is under the inst/ , but I don't have this
>>>>> folder.
>>>>>> Why is that ? Is that possible to get further help from Bioconductor
>>>>>> project team to review the package source in github ?
>>>>>>
>>>>>> I am stuck with this problem, I searched all possible answer from all
>>>>> over
>>>>>> the place, cannot fix my issues. I am very sorry if my question is
>>> naive
>>>>> to
>>>>>> ask here. Could you point me out how to possibly solve this problem?
>>>>> Thanks
>>>>>> again for your great favor.
>>>>>>
>>>>>> Best regards :
>>>>>>
>>>>>> Jurat
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Dec 2, 2016 at 2:04 AM, Hervé Pagès <hpages at fredhutch.org>
>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Make sure you understand the difference between the *package source
>>>>>>> tree*, which you control and where you must create the inst/ folder,
>>>>>>> and the *package installation folder*, which gets created and
>>>>>>> populated by 'R CMD INSTALL'. The exact location of the *package
>>>>>>> installation folder* doesn't really matter but if you are curious
>>>>>>> you can use the .libPaths() command to see it. The *package
>>> installation
>>>>>>> folder* is the subfolder of .libPaths() that has the name of the
>>>>>>> package. You can also see it with find.package("mypackage").
>>>>>>>
>>>>>>> During installation, everything that is under the inst/ folder will
>>>>>>> get installed *directly* under the *package installation folder*.
>>>>>>> So if you created the extdata/ folder under inst/ (as it should be),
>>>>>>> extdata/ will end up being installed at
>>> .libPaths()/mypackage/extdata/
>>>>>>> The code in your examples can get the path to the *installed*
>>> extdata/
>>>>>>> folder with
>>>>>>>
>>>>>>>   system.file(package="mypackage", "extdata")
>>>>>>>
>>>>>>> or get the path to any file under extdata/ with e.g.
>>>>>>>
>>>>>>>   system.file(package="mypackage", "extdata", "data1.bed")
>>>>>>>
>>>>>>> Don't do
>>>>>>>
>>>>>>>   system.file(package="mypackage", "extdata/data1.bed")
>>>>>>>
>>>>>>> even if it works for you because it's not portable (the use of / as
>>>>>>> a separator is a platform-dependent thing).
>>>>>>>
>>>>>>> Finally note that you should never try to modify anything that is
>>> under
>>>>>>> .libPaths() by hand. Furthermore, and that's a mistake we see
>>> sometimes
>>>>>>> with contributed packages, the code in your package should always
>>>>>>> treat the *package installation folder* as a read-only folder.
>>>>>>>
>>>>>>> Hope this helps,
>>>>>>>
>>>>>>> H.
>>>>>>>
>>>>>>>
>>>>>>> On 12/01/2016 04:34 PM, Jurat Shayidin wrote:
>>>>>>>
>>>>>>>> Hi BiocDevel :
>>>>>>>>
>>>>>>>> I am getting vignette error when I building my packages, and
>>> external
>>>>> data
>>>>>>>> can't be captured by system.file() . I did unit test all function
>>> of my
>>>>>>>> packages, it works fine. When I am going to compile package
>>> vignette,
>>>>> test
>>>>>>>> input bed file can't be detected. However, I used
>>>>>>>> devtools::install()command
>>>>>>>> to install my packages, but installation directory inst/ was not
>>>>> created
>>>>>>>> automatically. By R package convention, all external data must be
>>>>> located
>>>>>>>> in inst/, so I manually created this folder  inst/extdata and paste
>>> my
>>>>>>>> data, but vignette compilation still failed. According to convention
>>>>> of R
>>>>>>>> package, my package structure will be:
>>>>>>>>
>>>>>>>> myPackage
>>>>>>>> `- inst
>>>>>>>> `- extdata
>>>>>>>> `- data1.bed
>>>>>>>> `- data2.bed
>>>>>>>> `- R
>>>>>>>> `- ...
>>>>>>>> `- NAMESPACE
>>>>>>>> `- DESCRIPTION
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> Hervé Pagès
>>>>>>>
>>>>>>> Program in Computational Biology
>>>>>>> Division of Public Health Sciences
>>>>>>> Fred Hutchinson Cancer Research Center
>>>>>>> 1100 Fairview Ave. N, M1-B514
>>>>>>> P.O. Box 19024
>>>>>>> Seattle, WA 98109-1024
>>>>>>>
>>>>>>> E-mail: hpages at fredhutch.org
>>>>>>> Phone:  (206) 667-5791
>>>>>>> Fax:    (206) 667-1319
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Jurat Shahidin
>>>>>>
>>>>>> Dipartimento di Elettronica, Informazione e Bioingegneria
>>>>>> Politecnico di Milano
>>>>>> Piazza Leonardo da Vinci 32 - 20133 Milano, Italy
>>>>>> Mobile : +39 3279366608
>>>>>>
>>>>>>       [[alternative HTML version deleted]]
>>>>>>
>>>>>> _______________________________________________
>>>>>> Bioc-devel at r-project.org mailing list
>>>>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Jurat Shahidin
>>>>
>>>> Dipartimento di Elettronica, Informazione e Bioingegneria
>>>> Politecnico di Milano
>>>> Piazza Leonardo da Vinci 32 - 20133 Milano, Italy
>>>> Mobile : +39 3279366608
>>>
>>
>>
>>
>> --
>> Jurat Shahidin
>>
>> Dipartimento di Elettronica, Informazione e Bioingegneria
>> Politecnico di Milano
>> Piazza Leonardo da Vinci 32 - 20133 Milano, Italy
>> Mobile : +39 3279366608
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> Bioc-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>
> _______________________________________________
> Bioc-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>

-- 
Hervé Pagès

Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024

E-mail: hpages at fredhutch.org
Phone:  (206) 667-5791
Fax:    (206) 667-1319