[Bioc-devel] package vignette error : external data can't be captured when compiling package vignette

Sun Dec 4 19:36:22 CET 2016

I don't really understand your question. But, create an inst/extdata directory:

md inst
md inst\extdata
... add your files to inst\extdata ...
git add inst\extdata
git commit
git push

Dan

----- Original Message -----
> From: "Jurat Shayidin" <juratbupt at gmail.com>
> To: "Dan Tenenbaum" <dtenenba at fredhutch.org>, "bioc-devel" <bioc-devel at r-project.org>
> Sent: Sunday, December 4, 2016 5:12:50 AM
> Subject: Re: [Bioc-devel] package vignette error : external data can't be captured when compiling package vignette

> Dear Dan :
> 
> Thanks again for your kind help. I did clean up my package git working
> tree, make every commit readable, tractable. I've read about R writing
> extension manual and that helped a lot. Many thanks to Herve' insightful
> answer to overcome file permission issue, R CMD check working perfectly
> now. Through the long list of R CMD check report, CMD check alarmed me
> several possible issues on my packages, and I am working on it.  Plus, R
> CMD check complained about some hidden file such as .gitignore,
> .Rproj.user, .git . Therefore I did strip away these files following this
> way :
> 
> 
> C:\Windows\system32>cd C:\Program Files\R\R-devel\bin\x64
> 
> C:\Program Files\R\R-devel\bin\x64>R CMD build
> "C:\Users\jvrat\Documents\MSPC"
> * checking for file 'C:\Users\jvrat\Documents\MSPC/DESCRIPTION' ... OK
> * preparing 'MSPC':
> * checking DESCRIPTION meta-information ... OK
> * installing the package to build vignettes
> * creating vignettes ... OK
> * checking for LF line-endings in source and make files
> * checking for empty or unneeded directories
> Removed empty directory 'MSPC/tests/testthat'
> * building 'MSPC_0.99.0.tar.gz'
> 
> 
> C:\Program Files\R\R-devel\bin\x64>R CMD check "MSPC_0.99.0.tar.gz"
> * using log directory 'C:/Program Files/R/R-devel/bin/x64/MSPC.Rcheck'
> * using R Under development (unstable) (2016-11-09 r71642)
> * using platform: x86_64-w64-mingw32 (64-bit)
> * using session charset: ISO8859-1
> * checking for file 'MSPC/DESCRIPTION' ... OK
> * checking extension type ... Package
> * this is package 'MSPC' version '0.99.0'
> * checking package namespace information ... OK
> 
> Now inst/ directory is created in MSPC_0.99.0.tar.gz, I could add my
> external data in extdata (I think I need manually create extdata folder)
> and make available these data for vignette code. But How can I commit these
> changes to git automatically ? Because MSPC_0.99.0.tar.gz was not in my
> package directory, I cannot push these changes to git. Could you give me
> possible idea please ? Many thanks to Bioconductor Project team.
> 
> Best regards :
> Jurat
> 
> On Fri, Dec 2, 2016 at 7:05 PM, Dan Tenenbaum <dtenenba at fredhutch.org>
> wrote:
> 
>> I tried building your package, there are a number of issues with it.
>>
>> I recommend that you 1) always be aware of what is and is not in github,
>> by using the 'git status' command. Bear in mind that if files are not
>> committed to git then we will not be able to see them and not be able to
>> help you. Specifically, the inst folder and everything under it does not
>> exist in the github repository. Therefore I get an error in the chunk at
>> lines 45-50, I don't get as far as you (error in lines 60-64).
>>
>> As Herve suggests, please run R CMD check from a directory that you have
>> write permission in, such as your home directory.
>>
>> R CMD check will alert you about various problems, including referring to
>> packages in NAMESPACE but not DESCRIPTION, relying on BiocStyle but not
>> mentioning it in DESCRIPTION, etc.
>>
>> Dan
>>
>>
>>
>> ----- Original Message -----
>> > From: "Jurat Shayidin" <juratbupt at gmail.com>
>> > To: "Dan Tenenbaum" <dtenenba at fredhutch.org>, "bioc-devel" <
>> bioc-devel at r-project.org>
>> > Sent: Friday, December 2, 2016 9:11:29 AM
>> > Subject: Re: [Bioc-devel] package vignette error : external data can't
>> be captured when compiling package vignette
>>
>> > Dear Dan :
>> >
>> > I forgot to commit vignette file,  now I pushed new changes, so now error
>> > could be something else. People in stackoverflow suggest me to put all my
>> > external data (a.k.a, bed files) into vignette folder, try to run R CMD
>> > check and see what happen, I did this solution, but it doesn't work.
>> Plus,
>> > I checked file permission of my package parent directory, seems fine to
>> me.
>> > How I check file permission of my package ?
>> > How can I fix this write permission on my machine ? This is not intuitive
>> > to me, Any hint please ?
>> >
>> > Herve suggest me don't create inst/extdata directory manually, but I
>> still
>> > can't let external data available for vignette code. FYI, my objective in
>> > vignette file : let external data available for package use, compile
>> > trivial version of vignette with no error, then continue to make it
>> > perfect.  Here is updated session of R CMD check :
>> >
>> >
>> > Microsoft Windows [Version 6.3.9600]
>> > (c) 2013 Microsoft Corporation. All rights reserved.
>> >
>> > C:\Users\jvrat>cd C:\Program Files\R\R-devel\bin\x64
>> >
>> > C:\Program Files\R\R-devel\bin\x64>R CMD INSTALL
>> > "C:\Users\jvrat\Documents\MSPC"
>> > * installing to library 'C:/Users/jvrat/Documents/R/win-library/3.4'
>> > * installing *source* package 'MSPC' ...
>> > ** R
>> > ** inst
>> > ** preparing package for lazy loading
>> > ** help
>> > *** installing help indices
>> > ** building package indices
>> > ** installing vignettes
>> > ** testing if installed package can be loaded
>> > * DONE (MSPC)
>> >
>> > C:\Program Files\R\R-devel\bin\x64>R CMD check
>> > "C:\Users\jvrat\Documents\MSPC"
>> > Warning in dir.create(pkgoutdir, mode = "0755") :
>> >  cannot create dir 'C:\Program Files\R\R-devel\bin\x64\MSPC.Rcheck',
>> > reason 'Permission denied'
>> > ERROR: cannot create check dir 'C:/Program
>> > Files/R/R-devel/bin/x64/MSPC.Rcheck'
>> >
>> > C:\Program Files\R\R-devel\bin\x64>R CMD build
>> > "C:\Users\jvrat\Documents\MSPC"
>> > * checking for file 'C:\Users\jvrat\Documents\MSPC/DESCRIPTION' ... OK
>> > * preparing 'MSPC':
>> > * checking DESCRIPTION meta-information ... OK
>> > * installing the package to build vignettes
>> > * creating vignettes ...Warning: running command
>> > '"C:/PROGRA~1/R/R-devel/bin/x64/Rscript" --vanilla --default-packages=
>> -e
>> > "tools::buildVignettes(dir = '.', tangle = TRUE)"' had status 1
>> > ERROR
>> > Warning in engine$weave(file, quiet = quiet, encoding = enc) :
>> >  Pandoc (>= 1.12.3) and/or pandoc-citeproc not available. Falling back to
>> > R Markdown v1.
>> > Quitting from lines 60-64 (vignette.Rmd)
>> > Error: processing vignette 'vignette.Rmd' failed with diagnostics:
>> > subscript out of bounds
>> > Execution halted
>> >
>> > C:\Program Files\R\R-devel\bin\x64>
>> >
>> >
>> > How I fix R CMD check error above ? system.file() still can't detect
>> > external data. Any idea please ? Thanks a lot
>> >
>> > Best regards :
>> >
>> > Jurat
>> >
>> >
>> > On Fri, Dec 2, 2016 at 5:43 PM, Dan Tenenbaum <dtenenba at fredhutch.org>
>> > wrote:
>> >
>> >> See below.
>> >>
>> >> ----- Original Message -----
>> >> > From: "Jurat Shayidin" <juratbupt at gmail.com>
>> >> > To: "Dan Tenenbaum" <dtenenba at fredhutch.org>, "bioc-devel" <
>> >> bioc-devel at r-project.org>
>> >> > Sent: Friday, December 2, 2016 8:32:35 AM
>> >> > Subject: Re: [Bioc-devel] package vignette error : external data can't
>> >> be captured when compiling package vignette
>> >>
>> >> > Dear Dan :
>> >> >
>> >> > Really appreciated for your quick respond. Instead, I am using R CMD
>> >> check
>> >> > on my packages, I have an error. Here is whole session detail :
>> >> >
>> >> > Microsoft Windows [Version 6.3.9600]
>> >> > (c) 2013 Microsoft Corporation. All rights reserved.
>> >> >
>> >> > C:\Users\jvrat>cd C:\Program Files\R\R-devel\bin\x64
>> >> >
>> >> > C:\Program Files\R\R-devel\bin\x64>R CMD INSTALL
>> >> > "C:\Users\jvrat\Documents\MSPC"
>> >> > * installing to library 'C:/Users/jvrat/Documents/R/win-library/3.4'
>> >> > * installing *source* package 'MSPC' ...
>> >> > ** R
>> >> > ** inst
>> >> > ** preparing package for lazy loading
>> >> > ** help
>> >> > *** installing help indices
>> >> > ** building package indices
>> >> > ** installing vignettes
>> >> > ** testing if installed package can be loaded
>> >> > * DONE (MSPC)
>> >> >
>> >> > C:\Program Files\R\R-devel\bin\x64>R CMD check
>> >> > "C:\Users\jvrat\Documents\MSPC"
>> >> > Warning in dir.create(pkgoutdir, mode = "0755") :
>> >> >  cannot create dir 'C:\Program Files\R\R-devel\bin\x64\MSPC.Rcheck',
>> >> reason
>> >> > 'Permission denied'
>> >> > ERROR: cannot create check dir 'C:/Program
>> >> > Files/R/R-devel/bin/x64/MSPC.Rcheck'
>> >>
>> >> You need to run R CMD check in a directory where you have write
>> >> permissions.
>> >>
>> >>
>> >> >
>> >> > C:\Program Files\R\R-devel\bin\x64>R CMD build
>> >> > "C:\Users\jvrat\Documents\MSPC"
>> >> > * checking for file 'C:\Users\jvrat\Documents\MSPC/DESCRIPTION' ...
>> OK
>> >> > * preparing 'MSPC':
>> >> > * checking DESCRIPTION meta-information ... OK
>> >> > * installing the package to build vignettes
>> >> > * creating vignettes ...Warning: running command
>> >> > '"C:/PROGRA~1/R/R-devel/bin/x64/Rscript" --vanilla
>> --default-packages=
>> >> -e
>> >> > "tools::buildVignettes(dir = '.', tangle = TRUE)"' had status 1
>> >> > ERROR
>> >> > Warning in engine$weave(file, quiet = quiet, encoding = enc) :
>> >> >  Pandoc (>= 1.12.3) and/or pandoc-citeproc not available. Falling
>> back to
>> >> > R Markdown v1.
>> >> > Quitting from lines 62-66 (vignette.Rmd)
>> >> > Error: processing vignette 'vignette.Rmd' failed with diagnostics:
>> >> > object 'inputData' not found
>> >> > Execution halted
>> >> >
>> >>
>> >> Line 61 of the vignette (as it is in Github; it's apparently different
>> on
>> >> yur machine based on the error message) is:
>> >>
>> >> total.ERs <- denoise_ERs(peakGRs = inputData, tau.w = 1.0E-04,
>> .fileName =
>> >> "noiseER", outDir = "", verbose = FALSE)
>> >>
>> >> As you can see, the peakGRs argument is set to inputData, but inputData
>> is
>> >> not defined anywhere.
>> >>
>> >> Dan
>> >>
>> >>
>> >>
>> >> > C:\Program Files\R\R-devel\bin\x64>
>> >> >
>> >> >
>> >> > How can I fix this R CMD check error ? Instead people in stackoverflow
>> >> > remind me that using devtools packages is not stable some times, so I
>> go
>> >> > for old fashion : use CMD. Plus, still my external data can't
>> available
>> >> for
>> >> > vignette data, and vignette compilation is failed again. I used
>> >> > system.file() to do this, but not working. Any idea please ? How can I
>> >> > overcome this problem? Thanks a lot to Bioconductor project team.
>> >> >
>> >> > Best regards :
>> >> >
>> >> > Jurat
>> >> >
>> >> >
>> >> > On Fri, Dec 2, 2016 at 5:20 PM, Dan Tenenbaum <dtenenba at fredhutch.org
>> >
>> >> > wrote:
>> >> >
>> >> >> If your package is in github at https://github.com/julaiti/MSPC , it
>> >> >> looks like there is no inst or extdata folder in that repository.
>> >> >>
>> >> >> Maybe it has not yet been added/committed/pushed to git?
>> >> >>
>> >> >> Note that everything _under_ inst gets installed when you install the
>> >> >> package, but the inst directory itself goes away.
>> >> >>
>> >> >> If in your package source you have the following structure:
>> >> >>
>> >> >> inst/extdata/
>> >> >> inst/foo.txt
>> >> >>
>> >> >> In the installed package you end up with:
>> >> >>
>> >> >> extdata/
>> >> >> foo.txt
>> >> >>
>> >> >> HTH
>> >> >> Dan
>> >> >>
>> >> >>
>> >> >> ----- Original Message -----
>> >> >> > From: "Jurat Shayidin" <juratbupt at gmail.com>
>> >> >> > To: "Hervé Pagès" <hpages at fredhutch.org>, "bioc-devel" <
>> >> >> bioc-devel at r-project.org>
>> >> >> > Sent: Friday, December 2, 2016 5:36:28 AM
>> >> >> > Subject: Re: [Bioc-devel] package vignette error : external data
>> can't
>> >> >> be captured when compiling package vignette
>> >> >>
>> >> >> > Dear Hervé :
>> >> >> >
>> >> >> > Thanks again for your response on my issue. I've read your message
>> >> very
>> >> >> > carefully and did all you suggested to me, still can't fix the
>> >> vignette
>> >> >> > compilation error. I have developed my package on windows machine
>> >> under
>> >> >> > devel version of R and Bioc, all unit test works fine to me. I
>> don't
>> >> >> > understand why inst/ directory was not created when I build and
>> >> install
>> >> >> my
>> >> >> > packages, external data can't be read during vignette compilation.
>> >> >> > However, my objective is, to build my package vignette with no
>> error
>> >> in
>> >> >> the
>> >> >> > first place, then continue to edit the context until getting final
>> >> >> version
>> >> >> > of vignette file. When I install packages using
>> devtools::install(), I
>> >> >> got
>> >> >> > this :
>> >> >> >
>> >> >> >> devtools::install()
>> >> >> > Installing MSPC
>> >> >> > "C:/PROGRA~1/R/R-devel/bin/x64/R" \
>> >> >> > --no-site-file --no-environ --no-save \
>> >> >> > --no-restore --quiet CMD INSTALL \
>> >> >> > "C:/Users/jvrat/Documents/MSPC" \
>> >> >> > --library="C:/Users/jvrat/Documents/R/win-library/3.4" \
>> >> >> > --install-tests
>> >> >> > * installing *source* package 'MSPC' ...
>> >> >> > ** R
>> >> >> > ** tests
>> >> >> > ** preparing package for lazy loading
>> >> >> > ** help
>> >> >> > *** installing help indices
>> >> >> > ** building package indices
>> >> >> > ** installing vignettes
>> >> >> > ** testing if installed package can be loaded
>> >> >> > * DONE (MSPC)
>> >> >> > Reloading installed MSPC
>> >> >> >
>> >> >> >
>> >> >> > I believe doing this is right in vignette file after I follow your
>> >> >> advise :
>> >> >> >
>> >> >> > ```{r}
>> >> >> > library(MSPC)
>> >> >> > inputBed <- list.files(system.file("extdata", package = "MSPC"),
>> >> >> full.names
>> >> >> > = TRUE)
>> >> >> > inputData <- readPeakFile(peakFolder = inputBed)
>> >> >> >
>> >> >> > inputData
>> >> >> > ```
>> >> >> >
>> >> >> >
>> >> >> > However, I should not add inst/extdata manually , without this I
>> can't
>> >> >> > allow to load external data for package use, and vignette
>> compilation
>> >> >> keep
>> >> >> > failed. To be honest, this is my very first time building R
>> packages,
>> >> I
>> >> >> > don't know how to fix this error. Just out of curiosity, during
>> >> package
>> >> >> > installation, everything is under the inst/ , but I don't have this
>> >> >> folder.
>> >> >> > Why is that ? Is that possible to get further help from
>> Bioconductor
>> >> >> > project team to review the package source in github ?
>> >> >> >
>> >> >> > I am stuck with this problem, I searched all possible answer from
>> all
>> >> >> over
>> >> >> > the place, cannot fix my issues. I am very sorry if my question is
>> >> naive
>> >> >> to
>> >> >> > ask here. Could you point me out how to possibly solve this
>> problem?
>> >> >> Thanks
>> >> >> > again for your great favor.
>> >> >> >
>> >> >> > Best regards :
>> >> >> >
>> >> >> > Jurat
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > On Fri, Dec 2, 2016 at 2:04 AM, Hervé Pagès <hpages at fredhutch.org>
>> >> >> wrote:
>> >> >> >
>> >> >> >> Hi,
>> >> >> >>
>> >> >> >> Make sure you understand the difference between the *package
>> source
>> >> >> >> tree*, which you control and where you must create the inst/
>> folder,
>> >> >> >> and the *package installation folder*, which gets created and
>> >> >> >> populated by 'R CMD INSTALL'. The exact location of the *package
>> >> >> >> installation folder* doesn't really matter but if you are curious
>> >> >> >> you can use the .libPaths() command to see it. The *package
>> >> installation
>> >> >> >> folder* is the subfolder of .libPaths() that has the name of the
>> >> >> >> package. You can also see it with find.package("mypackage").
>> >> >> >>
>> >> >> >> During installation, everything that is under the inst/ folder
>> will
>> >> >> >> get installed *directly* under the *package installation folder*.
>> >> >> >> So if you created the extdata/ folder under inst/ (as it should
>> be),
>> >> >> >> extdata/ will end up being installed at
>> >> .libPaths()/mypackage/extdata/
>> >> >> >> The code in your examples can get the path to the *installed*
>> >> extdata/
>> >> >> >> folder with
>> >> >> >>
>> >> >> >>   system.file(package="mypackage", "extdata")
>> >> >> >>
>> >> >> >> or get the path to any file under extdata/ with e.g.
>> >> >> >>
>> >> >> >>   system.file(package="mypackage", "extdata", "data1.bed")
>> >> >> >>
>> >> >> >> Don't do
>> >> >> >>
>> >> >> >>   system.file(package="mypackage", "extdata/data1.bed")
>> >> >> >>
>> >> >> >> even if it works for you because it's not portable (the use of /
>> as
>> >> >> >> a separator is a platform-dependent thing).
>> >> >> >>
>> >> >> >> Finally note that you should never try to modify anything that is
>> >> under
>> >> >> >> .libPaths() by hand. Furthermore, and that's a mistake we see
>> >> sometimes
>> >> >> >> with contributed packages, the code in your package should always
>> >> >> >> treat the *package installation folder* as a read-only folder.
>> >> >> >>
>> >> >> >> Hope this helps,
>> >> >> >>
>> >> >> >> H.
>> >> >> >>
>> >> >> >>
>> >> >> >> On 12/01/2016 04:34 PM, Jurat Shayidin wrote:
>> >> >> >>
>> >> >> >>> Hi BiocDevel :
>> >> >> >>>
>> >> >> >>> I am getting vignette error when I building my packages, and
>> >> external
>> >> >> data
>> >> >> >>> can't be captured by system.file() . I did unit test all function
>> >> of my
>> >> >> >>> packages, it works fine. When I am going to compile package
>> >> vignette,
>> >> >> test
>> >> >> >>> input bed file can't be detected. However, I used
>> >> >> >>> devtools::install()command
>> >> >> >>> to install my packages, but installation directory inst/ was not
>> >> >> created
>> >> >> >>> automatically. By R package convention, all external data must be
>> >> >> located
>> >> >> >>> in inst/, so I manually created this folder  inst/extdata and
>> paste
>> >> my
>> >> >> >>> data, but vignette compilation still failed. According to
>> convention
>> >> >> of R
>> >> >> >>> package, my package structure will be:
>> >> >> >>>
>> >> >> >>> myPackage
>> >> >> >>> `- inst
>> >> >> >>> `- extdata
>> >> >> >>> `- data1.bed
>> >> >> >>> `- data2.bed
>> >> >> >>> `- R
>> >> >> >>> `- ...
>> >> >> >>> `- NAMESPACE
>> >> >> >>> `- DESCRIPTION
>> >> >> >>>
>> >> >> >>>
>> >> >> >> --
>> >> >> >> Hervé Pagès
>> >> >> >>
>> >> >> >> Program in Computational Biology
>> >> >> >> Division of Public Health Sciences
>> >> >> >> Fred Hutchinson Cancer Research Center
>> >> >> >> 1100 Fairview Ave. N, M1-B514
>> >> >> >> P.O. Box 19024
>> >> >> >> Seattle, WA 98109-1024
>> >> >> >>
>> >> >> >> E-mail: hpages at fredhutch.org
>> >> >> >> Phone:  (206) 667-5791
>> >> >> >> Fax:    (206) 667-1319
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Jurat Shahidin
>> >> >> >
>> >> >> > Dipartimento di Elettronica, Informazione e Bioingegneria
>> >> >> > Politecnico di Milano
>> >> >> > Piazza Leonardo da Vinci 32 - 20133 Milano, Italy
>> >> >> > Mobile : +39 3279366608
>> >> >> >
>> >> >> >       [[alternative HTML version deleted]]
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > Bioc-devel at r-project.org mailing list
>> >> >> > https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Jurat Shahidin
>> >> >
>> >> > Dipartimento di Elettronica, Informazione e Bioingegneria
>> >> > Politecnico di Milano
>> >> > Piazza Leonardo da Vinci 32 - 20133 Milano, Italy
>> >> > Mobile : +39 3279366608
>> >>
>> >
>> >
>> >
>> > --
>> > Jurat Shahidin
>> >
>> > Dipartimento di Elettronica, Informazione e Bioingegneria
>> > Politecnico di Milano
>> > Piazza Leonardo da Vinci 32 - 20133 Milano, Italy
>> > Mobile : +39 3279366608
>>
> 
> 
> 
> --
> Jurat Shahidin
> 
> Dipartimento di Elettronica, Informazione e Bioingegneria
> Politecnico di Milano
> Piazza Leonardo da Vinci 32 - 20133 Milano, Italy
> Mobile : +39 3279366608