[Bioc-devel] vignette problems

Martin Morgan martin.morgan at roswellpark.org
Mon Apr 9 16:44:34 CEST 2018 

I'll try to provide you with a pull request addressing issues. Martin

On 04/09/2018 08:42 AM, campos wrote:
> Dear devel team,
> 
> I am still puzzled with the problems with mac compiling. I am really 
> lost and have no idea how to continue or how to be able to check about 
> this problems with my linux machine in order to fix it faster. Could you 
> please help me with that??
> 
> Best,
> 
> Rafael
> 
> 
> On 05.04.2018 14:29, Shepherd, Lori wrote:
>>
>> In order for changes to be propagated a version bump in the 
>> DESCRIPTION file is needed.  Please bump the version in the 
>> DESCRIPTION file to 2.7.2.
>>
>>
>>
>>
>> Lori Shepherd
>>
>> Bioconductor Core Team
>>
>> Roswell Park Cancer Institute
>>
>> Department of Biostatistics & Bioinformatics
>>
>> Elm & Carlton Streets
>>
>> Buffalo, New York 14263
>>
>> ------------------------------------------------------------------------
>> *From:* Bioc-devel <bioc-devel-bounces at r-project.org> on behalf of 
>> campos <campos at mpipz.mpg.de>
>> *Sent:* Thursday, April 5, 2018 7:45:57 AM
>> *To:* Morgan, Martin; bioc-devel
>> *Subject:* Re: [Bioc-devel] vignette problems
>> Hey Martin,
>>
>> I pushed new changes since last friday but in
>> https://bioconductor.org/checkResults/3.7/bioc-LATEST/STAN/ says that
>> the last change date was friday. Any idea what is the problem?
>>
>> I have tried to fix the problems with memory and all you told me.
>>
>> Best,
>>
>> Rafael
>>
>>
>> On 03.04.2018 17:06, Martin Morgan wrote:
>> > Please use 'reply all' so that the mailing list remains engaged.
>> >
>> > Check out the release schedule
>> >
>> > http://bioconductor.org/developers/release-schedule/
>> >
>> > in particular
>> >
>> > Wednesday April 25
>> >
>> > - Deadline for packages passing ‘‘R CMD build’’ and ‘‘R CMD check’’
>> > without errors or warnings.
>> >
>> > so you still have time to get your package in order.
>> >
>> > Using the same techniques as before, I still see valgrind problems,
>> > the first being
>> >
>> > > hmm_fitted_poilog = fitHMM(trainRegions, hmm_poilog,
>> > sizeFactors=sizeFactors, maxIters=10)
>> > ==24916== Invalid write of size 4
>> > ==24916==    at 0x4BA93FD7:
>> > TransitionMatrix::updateAuxiliaries(double**, double***, double*,
>> > int*, int, int**, int, double, int) (TransitionMatrix.cpp:292)
>> > ==24916==    by 0x4BA77934: HMM::updateSampleAux(double***, int*, int,
>> > double**, double**, double**, double***, double*, int*, int*, int*,
>> > int**, double***, SEXPREC*, SEXPREC*, int, double, int, int, int)
>> > (HMM.cpp:771)
>> > ==24916==    by 0x4BA7896D: HMM::BaumWelch[abi:cxx11](double***, int*,
>> > int, int, int**, int*, int*, int*, int, int, int**, double***,
>> > SEXPREC*, SEXPREC*, int, double, double, int, int) (HMM.cpp:1076)
>> > ==24916==    by 0x4BA8FEFD: RHMMFit (RWrapper.cpp:1494)
>> > ==24916==    by 0x4F2992D: R_doDotCall (dotcode.c:692)
>> > ==24916==    by 0x4F339D5: do_dotcall (dotcode.c:1252)
>> > ==24916==    by 0x4F81BA6: bcEval (eval.c:6771)
>> > ==24916==    by 0x4F6E963: Rf_eval (eval.c:624)
>> > ==24916==    by 0x4F71188: R_execClosure (eval.c:1764)
>> > ==24916==    by 0x4F70E7C: Rf_applyClosure (eval.c:1692)
>> > ==24916==    by 0x4F6F18B: Rf_eval (eval.c:747)
>> > ==24916==    by 0x4F74B12: do_set (eval.c:2774)
>> > ==24916==  Address 0x2e73a294 is 4 bytes inside a block of size 5 
>> alloc'd
>> > ==24916==    at 0x4C2DB8F: malloc (in
>> > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
>> > ==24916==    by 0x4BA93FA6:
>> > TransitionMatrix::updateAuxiliaries(double**, double***, double*,
>> > int*, int, int**, int, double, int) (TransitionMatrix.cpp:289)
>> > ==24916==    by 0x4BA77934: HMM::updateSampleAux(double***, int*, int,
>> > double**, double**, double**, double***, double*, int*, int*, int*,
>> > int**, double***, SEXPREC*, SEXPREC*, int, double, int, int, int)
>> > (HMM.cpp:771)
>> > ==24916==    by 0x4BA7896D: HMM::BaumWelch[abi:cxx11](double***, int*,
>> > int, int, int**, int*, int*, int*, int, int, int**, double***,
>> > SEXPREC*, SEXPREC*, int, double, double, int, int) (HMM.cpp:1076)
>> > ==24916==    by 0x4BA8FEFD: RHMMFit (RWrapper.cpp:1494)
>> > ==24916==    by 0x4F2992D: R_doDotCall (dotcode.c:692)
>> > ==24916==    by 0x4F339D5: do_dotcall (dotcode.c:1252)
>> > ==24916==    by 0x4F81BA6: bcEval (eval.c:6771)
>> > ==24916==    by 0x4F6E963: Rf_eval (eval.c:624)
>> > ==24916==    by 0x4F71188: R_execClosure (eval.c:1764)
>> > ==24916==    by 0x4F70E7C: Rf_applyClosure (eval.c:1692)
>> > ==24916==    by 0x4F6F18B: Rf_eval (eval.c:747)
>> > ==24916==
>> >
>> > This seems to be the exact same code as in the problem that you fixed
>> > at another location. Actually, I would guess that all of these
>> >
>> > grep --color -nH -e ")\*ncores+1" *
>> > HMM.cpp:784:    int *myStateBuckets = 
>> (int*)malloc(sizeof(int)*ncores+1);
>> > MultivariateGaussian.cpp:295:    int *myDimBuckets =
>> > (int*)malloc(sizeof(int)*ncores+1);
>> > MultivariateGaussian.cpp:475:    int *myDimBuckets =
>> > (int*)malloc(sizeof(int)*ncores+1);
>> > TransitionMatrix.cpp:132:        int *myStateBuckets =
>> > (int*)malloc(sizeof(int)*ncores+1);
>> > TransitionMatrix.cpp:289:    int *myStateBuckets =
>> > (int*)malloc(sizeof(int)*ncores+1);
>> >
>> > are the same problem. Also, usually code that has been copy/pasted
>> > like this can instead be refactored to  a single function call, so a
>> > bug can be fixed in one place.
>> >
>> > I still see a number of compiler warnings, the first of which is
>> >
>> > STAN master$ R CMD INSTALL .
>> > Bioconductor version 3.7 (BiocInstaller 1.29.5), ?biocLite for help
>> > * installing to library
>> > '/home/mtmorgan/R/x86_64-pc-linux-gnu-library/3.5-Bioc-3.7'
>> > * installing *source* package 'STAN' ...
>> > ** libs
>> > g++  -I"/home/mtmorgan/bin/R-3-5-branch/include" -DNDEBUG
>> > -I/usr/local/include  -D_RDLL_ -fopenmp  -fpic  -g -O0 -Wall -pedantic
>> > -c HMM.cpp -o HMM.o
>> > HMM.cpp: In member function ‘virtual void
>> > HMM::calcEmissionProbs(double***, double**, int*, int, int**, int*,
>> > int*, int**, int, int, int*)’:
>> > HMM.cpp:112:15: warning: unused variable ‘j’ [-Wunused-variable]
>> >          int i,j,t,k;
>> >                ^
>> > It really pays to clean these up; most are harmless, but they obscure
>> > the more important warnings.
>> >
>> > Martin
>> >
>> >
>> > On 04/03/2018 09:58 AM, campos wrote:
>> >> Hi Martin,
>> >>
>> >> when I run now valgrind seems that there are no problems. I pushed
>> >> yesterday the changes, was I still on time for the new release? We
>> >> want to publish the changes and it would be really helpful if the
>> >> package is running on Bioconductor.
>> >>
>> >> Thank you very much,
>> >>
>> >> Rafael
>> >>
>> >>
>> >> On 02.04.2018 02:51, Martin Morgan wrote:
>> >>>
>> >>>
>> >>> On 04/01/2018 08:06 PM, Martin Morgan wrote:
>> >>>>
>> >>>>
>> >>>> On 04/01/2018 03:53 PM, campos wrote:
>> >>>>> Dear Martin,
>> >>>>>
>> >>>>> I am trying to fix this problem but I am really lost... Do you
>> >>>>> mean C++ code? Becasue there is no C code in the whole package. I
>> >>>>> really don't know what the problem might be.
>> >>>>
>> >>>> Please keep the conversation on the bioc-devel mailing list, so
>> >>>> that others can learn or help.
>> >>>>
>> >>>> I use Linux, not Mac, but C (which I used to mean your C++ code)
>> >>>> errors often occur on all platforms but are only visible as a
>> >>>> segfault on one. I created the vignette R code with
>> >>>>
>> >>>>    cd vignettes
>> >>>>    R CMD Stangle STAN-knitr.Rmd
>> >>>>
>> >>>> This produces a file STAN-knitr.R. I then ran your R code with
>> >>>> valgrind
>> >>>>
>> >>>>    R -d valgrind -f STAN-knitr.R
>> >>>>
>> >>>> this runs much slower than without valgrind. The first error
>> >>>> reported by valgrind was
>> >>>>
>> >>>>
>> >>>>  > ##
>> >>>> 
>> ----STAN-PoiLog----------------------------------------------------------- 
>>
>> >>>>
>> >>>>  > nStates = 10
>> >>>>  > hmm_poilog = initHMM(trainRegions, nStates, "PoissonLogNormal",
>> >>>> sizeFactors)
>> >>>>  > hmm_fitted_poilog = fitHMM(trainRegions, hmm_poilog,
>> >>>> sizeFactors=sizeFactors, maxIters=10)
>> >>>> [1] 6
>> >>>> ==22304== Invalid write of size 4
>> >>>> ==22304==    at 0x4B489316: HMM::BaumWelch[abi:cxx11](double***,
>> >>>> int*, int, int, int**, int*, int*, int*, int, int, int**,
>> >>>> double***, SEXPREC*, SEXPREC*, int, double, double, int, int)
>> >>>> (HMM.cpp:998)
>> >>>> ==22304==    by 0x4B4A0EFF: RHMMFit (RWrapper.cpp:1494)
>> >>>> ==22304==    by 0x4F2992D: R_doDotCall (dotcode.c:692)
>> >>>> ==22304==    by 0x4F339D5: do_dotcall (dotcode.c:1252)
>> >>>> ==22304==    by 0x4F81BA6: bcEval (eval.c:6771)
>> >>>> ==22304==    by 0x4F6E963: Rf_eval (eval.c:624)
>> >>>> ==22304==    by 0x4F71188: R_execClosure (eval.c:1764)
>> >>>> ==22304==    by 0x4F70E7C: Rf_applyClosure (eval.c:1692)
>> >>>> ==22304==    by 0x4F6F18B: Rf_eval (eval.c:747)
>> >>>> ==22304==    by 0x4F74B12: do_set (eval.c:2774)
>> >>>> ==22304==    by 0x4F6EDF5: Rf_eval (eval.c:699)
>> >>>> ==22304==    by 0x4FB7BEE: Rf_ReplIteration (main.c:258)
>> >>>> ==22304==  Address 0x238b28f4 is 4 bytes inside a block of size 5
>> >>>> alloc'd
>> >>>> ==22304==    at 0x4C2DB8F: malloc (in
>> >>>> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
>> >>>> ==22304==    by 0x4B4892E5: HMM::BaumWelch[abi:cxx11](double***,
>> >>>> int*, int, int, int**, int*, int*, int*, int, int, int**,
>> >>>> double***, SEXPREC*, SEXPREC*, int, double, double, int, int)
>> >>>> (HMM.cpp:995)
>> >>>> ==22304==    by 0x4B4A0EFF: RHMMFit (RWrapper.cpp:1494)
>> >>>> ==22304==    by 0x4F2992D: R_doDotCall (dotcode.c:692)
>> >>>> ==22304==    by 0x4F339D5: do_dotcall (dotcode.c:1252)
>> >>>> ==22304==    by 0x4F81BA6: bcEval (eval.c:6771)
>> >>>> ==22304==    by 0x4F6E963: Rf_eval (eval.c:624)
>> >>>> ==22304==    by 0x4F71188: R_execClosure (eval.c:1764)
>> >>>> ==22304==    by 0x4F70E7C: Rf_applyClosure (eval.c:1692)
>> >>>> ==22304==    by 0x4F6F18B: Rf_eval (eval.c:747)
>> >>>> ==22304==    by 0x4F74B12: do_set (eval.c:2774)
>> >>>> ==22304==    by 0x4F6EDF5: Rf_eval (eval.c:699)
>> >>>>
>> >>>> 'Invalid write' suggests that you are writing after the end of
>> >>>> memory that you'd allocated. I looked at the C code at the line
>> >>>> where the error occurs as indicated in the stack trace, HMM.cpp:998
>> >>>> which is the assigment myStateBucks[i] = 0 in the loop
>> >>>>
>> >>>>      int *myStateBuckets = (int*)malloc(sizeof(int)*ncores+1);
>> >>>>      for(i=0; i<=ncores; i++)
>> >>>>      {
>> >>>>          myStateBuckets[i] = 0;
>> >>>>      }
>> >>>>
>> >>>> The argument to malloc (where he memory was allocated, at line 995)
>> >>>> should be the number of bytes to allocate and it should have been
>> >>>> memory for ncores + 1 'int'
>> >>>>
>> >>>>    malloc(sizeof(int) * (ncores + 1))
>> >>>>
>> >>>> rather than what you wrote, which is memory for ncores ints plus 1
>> >>>> byte.
>> >>>>
>> >>>> C++ code would avoid the need for such explicit memory management,
>> >>>> e.g., using a vector from the standard template library
>> >>>>
>> >>>>    std::vector<int> myStateBuckets(ncores);
>> >>>
>> >>> oops, std::vector<int> myStateBuckets(ncores + 1); !
>> >>>>
>> >>>> There were may other valgrind errors, but I do not know whether
>> >>>> these are from similar programming errors, or a consequence of this
>> >>>> one.
>> >>>>
>> >>>> Martin
>> >>>>
>> >>>>> Thanks,
>> >>>>> Rafael
>> >>>>>>
>> >>>>>> On 03/29/2018 01:07 PM, campos wrote:
>> >>>>>>> Dear bioc-devel team,
>> >>>>>>>
>> >>>>>>> I have made some changes in the package STAN and although it
>> >>>>>>> seems to install correctly, I have problems with timeout and
>> >>>>>>> error in windows... Could someone help me to improve the time?
>> >>>>>>>
>> >>>>>>> https://bioconductor.org/checkResults/3.7/bioc-LATEST/STAN/
>> >>>>>>
>> >>>>>> it looks like, with your most recent commit (at the top of the
>> >>>>>> page, 'Snapshot Date', 'Last Commit', 'Last Changed Date'), the
>> >>>>>> package built on Windows and Linux.
>> >>>>>>
>> >>>>>> There is a segfault on Mac, which is likely a programming error
>> >>>>>> in your C code. It could be debugged perhaps using valgrind or
>> >>>>>> similar tools, but the first step would be to isolate the code to
>> >>>>>> something more easily reproduced than the full vignette. It would
>> >>>>>> also help to clean up the C code so that it compiles without
>> >>>>>> warnings with the -Wall -pedantic flags
>> >>>>>>
>> >>>>>> Martin
>> >>>>>>
>> >>>>>>>
>> >>>>>>> Best,
>> >>>>>>>
>> >>>>>>> Rafael
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On 28.03.2018 01:08, Martin Morgan wrote:
>> >>>>>>>> When I try and install the version on the master branch of the
>> >>>>>>>> Bioconductor git repository I get
>> >>>>>>>>
>> >>>>>>>> STAN master$ Rdev --vanilla CMD INSTALL .
>> >>>>>>>> * installing to library
>> >>>>>>>> ‘/home/mtmorgan/R/x86_64-pc-linux-gnu-library
>> >>>>>>>> ...
>> >>>>>>>> ** testing if installed package can be loaded
>> >>>>>>>> Error: package or namespace load failed for 'STAN' in
>> >>>>>>>> namespaceExport(ns, exports):
>> >>>>>>>>  undefined exports: viterbi2Gviz
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> This comes about in a rather interesting way because the body
>> >>>>>>>> of plotViterbi is not defined
>> >>>>>>>>
>> >>>>>>>> plotViterbi <- function(viterbi, regions, gen, chrom, from, to,
>> >>>>>>>> statecols, col)
>> >>>>>>>>
>> >>>>>>>> #'
>> >>>>>>>> ...
>> >>>>>>>>
>> >>>>>>>> Can you please commit a version of the package that installs?
>> >>>>>>>>
>> >>>>>>>> Martin
>> >>>>>>>>
>> >>>>>>>> On 03/27/2018 06:42 PM, campos wrote:
>> >>>>>>>>> Dear bioc-devel team,
>> >>>>>>>>>
>> >>>>>>>>> I am developing the STAN packages and I am running into
>> >>>>>>>>> problems when trying to build my package. The problem is the
>> >>>>>>>>> following:
>> >>>>>>>>>
>> >>>>>>>>> Error in vignette_type(Outfile) :
>> >>>>>>>>>    Vignette product 'STAN.tex' does not have a known filename
>> >>>>>>>>> extension ('NA')
>> >>>>>>>>> ERROR: installing vignettes failed
>> >>>>>>>>> * removing '/tmp/Rtmp925Iru/Rinst63471ff1efdc/STAN'
>> >>>>>>>>> I tried to build the package in old versions (which they used
>> >>>>>>>>> to work) and I run in other problems but in this case is:
>> >>>>>>>>>
>> >>>>>>>>> Error in texi2dvi(file = file, pdf = TRUE, clean = clean,
>> >>>>>>>>> quiet = quiet,  :
>> >>>>>>>>>    Running 'texi2dvi' on 'STAN.tex' failed.
>> >>>>>>>>> LaTeX errors:
>> >>>>>>>>> ! LaTeX Error: File `beramono.sty' not found.
>> >>>>>>>>>
>> >>>>>>>>> Type X to quit or <RETURN> to proceed,
>> >>>>>>>>> or enter new name. (Default extension: sty)
>> >>>>>>>>>
>> >>>>>>>>> Could you help me with this problem?
>> >>>>>>>>>
>> >>>>>>>>> Thank you very much,
>> >>>>>>>>>
>> >>>>>>>>> Rafael
>> >>>>>>>>>
>> >>>>>>>>> _______________________________________________
>> >>>>>>>>> Bioc-devel at r-project.org mailing list
>> >>>>>>>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> This email message may contain legally privileged and/or
>> >>>>>>>> confidential information.  If you are not the intended
>> >>>>>>>> recipient(s), or the employee or agent responsible for the
>> >>>>>>>> delivery of this message to the intended recipient(s), you are
>> >>>>>>>> hereby notified that any disclosure, copying, distribution, or
>> >>>>>>>> use of this email message is prohibited.  If you have received
>> >>>>>>>> this message in error, please notify the sender immediately by
>> >>>>>>>> e-mail and delete this email message from your computer. Thank
>> >>>>>>>> you.
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> This email message may contain legally privileged and/or
>> >>>>>> confidential information.  If you are not the intended
>> >>>>>> recipient(s), or the employee or agent responsible for the
>> >>>>>> delivery of this message to the intended recipient(s), you are
>> >>>>>> hereby notified that any disclosure, copying, distribution, or
>> >>>>>> use of this email message is prohibited. If you have received
>> >>>>>> this message in error, please notify the sender immediately by
>> >>>>>> e-mail and delete this email message from your computer. Thank you.
>> >>>>>
>> >>>>
>> >>>>
>> >>>> This email message may contain legally privileged
>> >>>> and/or...{{dropped:2}}
>> >>>>
>> >>>> _______________________________________________
>> >>>> Bioc-devel at r-project.org mailing list
>> >>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>> >>>
>> >>>
>> >>> This email message may contain legally privileged and/or
>> >>> confidential information.  If you are not the intended recipient(s),
>> >>> or the employee or agent responsible for the delivery of this
>> >>> message to the intended recipient(s), you are hereby notified that
>> >>> any disclosure, copying, distribution, or use of this email message
>> >>> is prohibited.  If you have received this message in error, please
>> >>> notify the sender immediately by e-mail and delete this email
>> >>> message from your computer. Thank you.
>> >>
>> >
>> >
>> > This email message may contain legally privileged and/or confidential
>> > information.  If you are not the intended recipient(s), or the
>> > employee or agent responsible for the delivery of this message to the
>> > intended recipient(s), you are hereby notified that any disclosure,
>> > copying, distribution, or use of this email message is prohibited.  If
>> > you have received this message in error, please notify the sender
>> > immediately by e-mail and delete this email message from your
>> > computer. Thank you.
>>
>> _______________________________________________
>> Bioc-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>
>> This email message may contain legally privileged and/or confidential 
>> information. If you are not the intended recipient(s), or the employee 
>> or agent responsible for the delivery of this message to the intended 
>> recipient(s), you are hereby notified that any disclosure, copying, 
>> distribution, or use of this email message is prohibited. If you have 
>> received this message in error, please notify the sender immediately 
>> by e-mail and delete this email message from your computer. Thank you. 
> 


This email message may contain legally privileged and/or...{{dropped:2}}

More information about the Bioc-devel mailing list