[Bioc-devel] package vignette error : external data can't be captured when compiling package vignette

[Dan Tenenbaum]

I tried building your package, there are a number of issues with it.

I recommend that you 1) always be aware of what is and is not in github, by using the 'git status' command. Bear in mind that if files are not committed to git then we will not be able to see them and not be able to help you. Specifically, the inst folder and everything under it does not exist in the github repository. Therefore I get an error in the chunk at lines 45-50, I don't get as far as you (error in lines 60-64).

As Herve suggests, please run R CMD check from a directory that you have write permission in, such as your home directory.

R CMD check will alert you about various problems, including referring to packages in NAMESPACE but not DESCRIPTION, relying on BiocStyle but not mentioning it in DESCRIPTION, etc.

Dan



----- Original Message -----
> From: "Jurat Shayidin" <juratbupt at gmail.com>
> To: "Dan Tenenbaum" <dtenenba at fredhutch.org>, "bioc-devel" <bioc-devel at r-project.org>
> Sent: Friday, December 2, 2016 9:11:29 AM
> Subject: Re: [Bioc-devel] package vignette error : external data can't be captured when compiling package vignette

> Dear Dan :
> 
> I forgot to commit vignette file,  now I pushed new changes, so now error
> could be something else. People in stackoverflow suggest me to put all my
> external data (a.k.a, bed files) into vignette folder, try to run R CMD
> check and see what happen, I did this solution, but it doesn't work. Plus,
> I checked file permission of my package parent directory, seems fine to me.
> How I check file permission of my package ?
> How can I fix this write permission on my machine ? This is not intuitive
> to me, Any hint please ?
> 
> Herve suggest me don't create inst/extdata directory manually, but I still
> can't let external data available for vignette code. FYI, my objective in
> vignette file : let external data available for package use, compile
> trivial version of vignette with no error, then continue to make it
> perfect.  Here is updated session of R CMD check :
> 
> 
> Microsoft Windows [Version 6.3.9600]
> (c) 2013 Microsoft Corporation. All rights reserved.
> 
> C:\Users\jvrat>cd C:\Program Files\R\R-devel\bin\x64
> 
> C:\Program Files\R\R-devel\bin\x64>R CMD INSTALL
> "C:\Users\jvrat\Documents\MSPC"
> * installing to library 'C:/Users/jvrat/Documents/R/win-library/3.4'
> * installing *source* package 'MSPC' ...
> ** R
> ** inst
> ** preparing package for lazy loading
> ** help
> *** installing help indices
> ** building package indices
> ** installing vignettes
> ** testing if installed package can be loaded
> * DONE (MSPC)
> 
> C:\Program Files\R\R-devel\bin\x64>R CMD check
> "C:\Users\jvrat\Documents\MSPC"
> Warning in dir.create(pkgoutdir, mode = "0755") :
>  cannot create dir 'C:\Program Files\R\R-devel\bin\x64\MSPC.Rcheck',
> reason 'Permission denied'
> ERROR: cannot create check dir 'C:/Program
> Files/R/R-devel/bin/x64/MSPC.Rcheck'
> 
> C:\Program Files\R\R-devel\bin\x64>R CMD build
> "C:\Users\jvrat\Documents\MSPC"
> * checking for file 'C:\Users\jvrat\Documents\MSPC/DESCRIPTION' ... OK
> * preparing 'MSPC':
> * checking DESCRIPTION meta-information ... OK
> * installing the package to build vignettes
> * creating vignettes ...Warning: running command
> '"C:/PROGRA~1/R/R-devel/bin/x64/Rscript" --vanilla --default-packages= -e
> "tools::buildVignettes(dir = '.', tangle = TRUE)"' had status 1
> ERROR
> Warning in engine$weave(file, quiet = quiet, encoding = enc) :
>  Pandoc (>= 1.12.3) and/or pandoc-citeproc not available. Falling back to
> R Markdown v1.
> Quitting from lines 60-64 (vignette.Rmd)
> Error: processing vignette 'vignette.Rmd' failed with diagnostics:
> subscript out of bounds
> Execution halted
> 
> C:\Program Files\R\R-devel\bin\x64>
> 
> 
> How I fix R CMD check error above ? system.file() still can't detect
> external data. Any idea please ? Thanks a lot
> 
> Best regards :
> 
> Jurat
> 
> 
> On Fri, Dec 2, 2016 at 5:43 PM, Dan Tenenbaum <dtenenba at fredhutch.org>
> wrote:
> 
>> See below.
>>
>> ----- Original Message -----
>> > From: "Jurat Shayidin" <juratbupt at gmail.com>
>> > To: "Dan Tenenbaum" <dtenenba at fredhutch.org>, "bioc-devel" <
>> bioc-devel at r-project.org>
>> > Sent: Friday, December 2, 2016 8:32:35 AM
>> > Subject: Re: [Bioc-devel] package vignette error : external data can't
>> be captured when compiling package vignette
>>
>> > Dear Dan :
>> >
>> > Really appreciated for your quick respond. Instead, I am using R CMD
>> check
>> > on my packages, I have an error. Here is whole session detail :
>> >
>> > Microsoft Windows [Version 6.3.9600]
>> > (c) 2013 Microsoft Corporation. All rights reserved.
>> >
>> > C:\Users\jvrat>cd C:\Program Files\R\R-devel\bin\x64
>> >
>> > C:\Program Files\R\R-devel\bin\x64>R CMD INSTALL
>> > "C:\Users\jvrat\Documents\MSPC"
>> > * installing to library 'C:/Users/jvrat/Documents/R/win-library/3.4'
>> > * installing *source* package 'MSPC' ...
>> > ** R
>> > ** inst
>> > ** preparing package for lazy loading
>> > ** help
>> > *** installing help indices
>> > ** building package indices
>> > ** installing vignettes
>> > ** testing if installed package can be loaded
>> > * DONE (MSPC)
>> >
>> > C:\Program Files\R\R-devel\bin\x64>R CMD check
>> > "C:\Users\jvrat\Documents\MSPC"
>> > Warning in dir.create(pkgoutdir, mode = "0755") :
>> >  cannot create dir 'C:\Program Files\R\R-devel\bin\x64\MSPC.Rcheck',
>> reason
>> > 'Permission denied'
>> > ERROR: cannot create check dir 'C:/Program
>> > Files/R/R-devel/bin/x64/MSPC.Rcheck'
>>
>> You need to run R CMD check in a directory where you have write
>> permissions.
>>
>>
>> >
>> > C:\Program Files\R\R-devel\bin\x64>R CMD build
>> > "C:\Users\jvrat\Documents\MSPC"
>> > * checking for file 'C:\Users\jvrat\Documents\MSPC/DESCRIPTION' ... OK
>> > * preparing 'MSPC':
>> > * checking DESCRIPTION meta-information ... OK
>> > * installing the package to build vignettes
>> > * creating vignettes ...Warning: running command
>> > '"C:/PROGRA~1/R/R-devel/bin/x64/Rscript" --vanilla --default-packages=
>> -e
>> > "tools::buildVignettes(dir = '.', tangle = TRUE)"' had status 1
>> > ERROR
>> > Warning in engine$weave(file, quiet = quiet, encoding = enc) :
>> >  Pandoc (>= 1.12.3) and/or pandoc-citeproc not available. Falling back to
>> > R Markdown v1.
>> > Quitting from lines 62-66 (vignette.Rmd)
>> > Error: processing vignette 'vignette.Rmd' failed with diagnostics:
>> > object 'inputData' not found
>> > Execution halted
>> >
>>
>> Line 61 of the vignette (as it is in Github; it's apparently different on
>> yur machine based on the error message) is:
>>
>> total.ERs <- denoise_ERs(peakGRs = inputData, tau.w = 1.0E-04, .fileName =
>> "noiseER", outDir = "", verbose = FALSE)
>>
>> As you can see, the peakGRs argument is set to inputData, but inputData is
>> not defined anywhere.
>>
>> Dan
>>
>>
>>
>> > C:\Program Files\R\R-devel\bin\x64>
>> >
>> >
>> > How can I fix this R CMD check error ? Instead people in stackoverflow
>> > remind me that using devtools packages is not stable some times, so I go
>> > for old fashion : use CMD. Plus, still my external data can't available
>> for
>> > vignette data, and vignette compilation is failed again. I used
>> > system.file() to do this, but not working. Any idea please ? How can I
>> > overcome this problem? Thanks a lot to Bioconductor project team.
>> >
>> > Best regards :
>> >
>> > Jurat
>> >
>> >
>> > On Fri, Dec 2, 2016 at 5:20 PM, Dan Tenenbaum <dtenenba at fredhutch.org>
>> > wrote:
>> >
>> >> If your package is in github at https://github.com/julaiti/MSPC , it
>> >> looks like there is no inst or extdata folder in that repository.
>> >>
>> >> Maybe it has not yet been added/committed/pushed to git?
>> >>
>> >> Note that everything _under_ inst gets installed when you install the
>> >> package, but the inst directory itself goes away.
>> >>
>> >> If in your package source you have the following structure:
>> >>
>> >> inst/extdata/
>> >> inst/foo.txt
>> >>
>> >> In the installed package you end up with:
>> >>
>> >> extdata/
>> >> foo.txt
>> >>
>> >> HTH
>> >> Dan
>> >>
>> >>
>> >> ----- Original Message -----
>> >> > From: "Jurat Shayidin" <juratbupt at gmail.com>
>> >> > To: "Hervé Pagès" <hpages at fredhutch.org>, "bioc-devel" <
>> >> bioc-devel at r-project.org>
>> >> > Sent: Friday, December 2, 2016 5:36:28 AM
>> >> > Subject: Re: [Bioc-devel] package vignette error : external data can't
>> >> be captured when compiling package vignette
>> >>
>> >> > Dear Hervé :
>> >> >
>> >> > Thanks again for your response on my issue. I've read your message
>> very
>> >> > carefully and did all you suggested to me, still can't fix the
>> vignette
>> >> > compilation error. I have developed my package on windows machine
>> under
>> >> > devel version of R and Bioc, all unit test works fine to me. I don't
>> >> > understand why inst/ directory was not created when I build and
>> install
>> >> my
>> >> > packages, external data can't be read during vignette compilation.
>> >> > However, my objective is, to build my package vignette with no error
>> in
>> >> the
>> >> > first place, then continue to edit the context until getting final
>> >> version
>> >> > of vignette file. When I install packages using devtools::install(), I
>> >> got
>> >> > this :
>> >> >
>> >> >> devtools::install()
>> >> > Installing MSPC
>> >> > "C:/PROGRA~1/R/R-devel/bin/x64/R" \
>> >> > --no-site-file --no-environ --no-save \
>> >> > --no-restore --quiet CMD INSTALL \
>> >> > "C:/Users/jvrat/Documents/MSPC" \
>> >> > --library="C:/Users/jvrat/Documents/R/win-library/3.4" \
>> >> > --install-tests
>> >> > * installing *source* package 'MSPC' ...
>> >> > ** R
>> >> > ** tests
>> >> > ** preparing package for lazy loading
>> >> > ** help
>> >> > *** installing help indices
>> >> > ** building package indices
>> >> > ** installing vignettes
>> >> > ** testing if installed package can be loaded
>> >> > * DONE (MSPC)
>> >> > Reloading installed MSPC
>> >> >
>> >> >
>> >> > I believe doing this is right in vignette file after I follow your
>> >> advise :
>> >> >
>> >> > ```{r}
>> >> > library(MSPC)
>> >> > inputBed <- list.files(system.file("extdata", package = "MSPC"),
>> >> full.names
>> >> > = TRUE)
>> >> > inputData <- readPeakFile(peakFolder = inputBed)
>> >> >
>> >> > inputData
>> >> > ```
>> >> >
>> >> >
>> >> > However, I should not add inst/extdata manually , without this I can't
>> >> > allow to load external data for package use, and vignette compilation
>> >> keep
>> >> > failed. To be honest, this is my very first time building R packages,
>> I
>> >> > don't know how to fix this error. Just out of curiosity, during
>> package
>> >> > installation, everything is under the inst/ , but I don't have this
>> >> folder.
>> >> > Why is that ? Is that possible to get further help from Bioconductor
>> >> > project team to review the package source in github ?
>> >> >
>> >> > I am stuck with this problem, I searched all possible answer from all
>> >> over
>> >> > the place, cannot fix my issues. I am very sorry if my question is
>> naive
>> >> to
>> >> > ask here. Could you point me out how to possibly solve this problem?
>> >> Thanks
>> >> > again for your great favor.
>> >> >
>> >> > Best regards :
>> >> >
>> >> > Jurat
>> >> >
>> >> >
>> >> >
>> >> > On Fri, Dec 2, 2016 at 2:04 AM, Hervé Pagès <hpages at fredhutch.org>
>> >> wrote:
>> >> >
>> >> >> Hi,
>> >> >>
>> >> >> Make sure you understand the difference between the *package source
>> >> >> tree*, which you control and where you must create the inst/ folder,
>> >> >> and the *package installation folder*, which gets created and
>> >> >> populated by 'R CMD INSTALL'. The exact location of the *package
>> >> >> installation folder* doesn't really matter but if you are curious
>> >> >> you can use the .libPaths() command to see it. The *package
>> installation
>> >> >> folder* is the subfolder of .libPaths() that has the name of the
>> >> >> package. You can also see it with find.package("mypackage").
>> >> >>
>> >> >> During installation, everything that is under the inst/ folder will
>> >> >> get installed *directly* under the *package installation folder*.
>> >> >> So if you created the extdata/ folder under inst/ (as it should be),
>> >> >> extdata/ will end up being installed at
>> .libPaths()/mypackage/extdata/
>> >> >> The code in your examples can get the path to the *installed*
>> extdata/
>> >> >> folder with
>> >> >>
>> >> >>   system.file(package="mypackage", "extdata")
>> >> >>
>> >> >> or get the path to any file under extdata/ with e.g.
>> >> >>
>> >> >>   system.file(package="mypackage", "extdata", "data1.bed")
>> >> >>
>> >> >> Don't do
>> >> >>
>> >> >>   system.file(package="mypackage", "extdata/data1.bed")
>> >> >>
>> >> >> even if it works for you because it's not portable (the use of / as
>> >> >> a separator is a platform-dependent thing).
>> >> >>
>> >> >> Finally note that you should never try to modify anything that is
>> under
>> >> >> .libPaths() by hand. Furthermore, and that's a mistake we see
>> sometimes
>> >> >> with contributed packages, the code in your package should always
>> >> >> treat the *package installation folder* as a read-only folder.
>> >> >>
>> >> >> Hope this helps,
>> >> >>
>> >> >> H.
>> >> >>
>> >> >>
>> >> >> On 12/01/2016 04:34 PM, Jurat Shayidin wrote:
>> >> >>
>> >> >>> Hi BiocDevel :
>> >> >>>
>> >> >>> I am getting vignette error when I building my packages, and
>> external
>> >> data
>> >> >>> can't be captured by system.file() . I did unit test all function
>> of my
>> >> >>> packages, it works fine. When I am going to compile package
>> vignette,
>> >> test
>> >> >>> input bed file can't be detected. However, I used
>> >> >>> devtools::install()command
>> >> >>> to install my packages, but installation directory inst/ was not
>> >> created
>> >> >>> automatically. By R package convention, all external data must be
>> >> located
>> >> >>> in inst/, so I manually created this folder  inst/extdata and paste
>> my
>> >> >>> data, but vignette compilation still failed. According to convention
>> >> of R
>> >> >>> package, my package structure will be:
>> >> >>>
>> >> >>> myPackage
>> >> >>> `- inst
>> >> >>> `- extdata
>> >> >>> `- data1.bed
>> >> >>> `- data2.bed
>> >> >>> `- R
>> >> >>> `- ...
>> >> >>> `- NAMESPACE
>> >> >>> `- DESCRIPTION
>> >> >>>
>> >> >>>
>> >> >> --
>> >> >> Hervé Pagès
>> >> >>
>> >> >> Program in Computational Biology
>> >> >> Division of Public Health Sciences
>> >> >> Fred Hutchinson Cancer Research Center
>> >> >> 1100 Fairview Ave. N, M1-B514
>> >> >> P.O. Box 19024
>> >> >> Seattle, WA 98109-1024
>> >> >>
>> >> >> E-mail: hpages at fredhutch.org
>> >> >> Phone:  (206) 667-5791
>> >> >> Fax:    (206) 667-1319
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Jurat Shahidin
>> >> >
>> >> > Dipartimento di Elettronica, Informazione e Bioingegneria
>> >> > Politecnico di Milano
>> >> > Piazza Leonardo da Vinci 32 - 20133 Milano, Italy
>> >> > Mobile : +39 3279366608
>> >> >
>> >> >       [[alternative HTML version deleted]]
>> >> >
>> >> > _______________________________________________
>> >> > Bioc-devel at r-project.org mailing list
>> >> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >>
>> >
>> >
>> >
>> > --
>> > Jurat Shahidin
>> >
>> > Dipartimento di Elettronica, Informazione e Bioingegneria
>> > Politecnico di Milano
>> > Piazza Leonardo da Vinci 32 - 20133 Milano, Italy
>> > Mobile : +39 3279366608
>>
> 
> 
> 
> --
> Jurat Shahidin
> 
> Dipartimento di Elettronica, Informazione e Bioingegneria
> Politecnico di Milano
> Piazza Leonardo da Vinci 32 - 20133 Milano, Italy
> Mobile : +39 3279366608