Do note that these p-values are corrected for multiple testing because of
the way they are computed.  So 0.001 is not so bad.

Kasper


On Mon, Feb 24, 2014 at 9:37 AM, James W. MacDonald <jmacdon@uw.edu> wrote:

> Hi Giovanni,
>
> Tim advised you to do the exact opposite. If you set B = 100, then the
> smallest p-value you can achieve is 0.01, which as he says, 'sucks'. If you
> use B = 1000, the smallest p-value will be 0.001 which is marginally better.
>
> Best,
>
> Jim
>
>
>
> On 2/24/2014 5:46 AM, Giovanni Calice wrote:
>
>> Hi Tim,
>> I'm already in the order of ideas to set the n of resamples below about
>> 100 as you say.
>> I await news about your comments for a better marginal test statistic and
>> the reference discussion about this topic.
>>
>> For now I'll follow your advice and previous ones.
>>
>> Thank you very much.
>>
>> Best,
>> Giovanni
>>
>>
>> Laboratory of Preclinical and Translational Research
>> IRCCS - CROB Oncology Referral Center of Basilicata
>> Rionero in Vulture - Basilicata - Italy
>>
>>
>> 2014-02-23 5:47 GMT+01:00 Tim Triche, Jr. <tim.triche@gmail.com <mailto:
>> tim.triche@gmail.com>>:
>>
>>
>>     Don't do that.  B is the number of permutations.  Your
>>     significance estimates will suck if you set it below about 100.
>>      If you can wait (a day or so) for 1000 or more permutations, they
>>     will be quite good.  I have some ideas for a better marginal test
>>     statistic to capture the omnibus null of "no difference" against
>>     any reasonable alternative, but that's a topic for another
>>     discussion.  It won't change the fact that you need a lot of
>>     permutations to get solid significance estimates from a biased
>>     sample (450k chip).
>>
>>     Best,
>>
>>
>>     Statistics is the grammar of science.
>>     Karl Pearson <http://en.wikipedia.org/wiki/The_Grammar_of_Science>
>>
>>
>>     On Sat, Feb 22, 2014 at 10:02 AM, Giovanni Calice
>>     <giovcalice@gmail.com <mailto:giovcalice@gmail.com>> wrote:
>>
>>         Hi,
>>
>>         I missed to ask advice about B parameter.
>>         In my first bumphunter runs, I set B=1 (default setting)
>>         having thus
>>         results within a reasonable time.
>>
>>         Now, as mentioned in the previous mail, I set B=1000 according
>>         to the reference
>>         manual and
>>         I thought so in order to have a statistically significant results.
>>
>>         In my next bumphunter call, have I to change this setting
>>         (B=1000)?
>>
>>         Thanks,
>>         Giovanni
>>
>>
>>         Laboratory of Preclinical and Translational Research
>>         IRCCS - CROB Oncology Referral Center of Basilicata
>>         Rionero in Vulture - Basilicata - Italy
>>
>>
>>         2014-02-21 18:53 GMT+01:00 Giovanni Calice
>>         <giovcalice@gmail.com <mailto:giovcalice@gmail.com>>:
>>
>>
>>         > Hi Jim,
>>         >
>>         > Ok!
>>         > My next bumphunter call will be on 1/3 of total ncores (I
>>         share compute
>>         > cycles with co-workers too)
>>         > with cutoff and cutoffQ = 0.95.
>>         >
>>         > Many Thanks.
>>         >
>>         > Best,
>>         > Giovanni
>>         >
>>         > Laboratory of Preclinical and Translational Research
>>         > IRCCS - CROB Oncology Referral Center of Basilicata
>>         > Rionero in Vulture - Basilicata - Italy
>>         >
>>         >
>>         >
>>         > 2014-02-21 18:29 GMT+01:00 James W. MacDonald
>>         <jmacdon@uw.edu <mailto:jmacdon@uw.edu>>:
>>
>>         >
>>         > Hi Giovanni,
>>         >>
>>         >> I don't know about the number of cores. I share compute
>>         cycles with
>>         >> co-workers, so can't just hog them all up. A recent
>>         bumphunter run I did on
>>         >> our shared server (a single multi-core machine with I
>>         believe 32
>>         >> processors) was:
>>         >>
>>         >> library(doParallel)
>>         >> registerDoParallel(cores = 12)
>>         >> bmps.main <- bumphunter(eset, design.cd <http://design.cd>,
>>
>>         cutoff = 0.99)
>>         >>
>>         >> Which resulted in something like 2500 bumps and finished
>>         in, I don't
>>         >> know, say 15-20 minutes?
>>         >>
>>         >> Best,
>>         >>
>>         >> Jim
>>         >>
>>         >>
>>         >>
>>         >> On 2/21/2014 12:21 PM, Giovanni Calice wrote:
>>         >>
>>         >>> Hi Jim,
>>         >>>
>>         >>> the call for bumphunter is:
>>         >>>
>>         >>> bumphunter(myGenomicRatioSet, design=designMatrix,
>>         cutoff=0.05, B=1000,
>>         >>> verbose=TRUE)
>>         >>>
>>         >>> I set cutoff to 0.05 not cutoffQ.
>>         >>> Next bumphunter call I will follow your advice;
>>         >>> regard to parallelize the process what is your suggestion?
>>         >>>
>>         >>> Would it be better to parallelize only on some of the ncores?
>>         >>>
>>         >>> Best,
>>         >>> Giovanni
>>         >>>
>>         >>> Laboratory of Preclinical and Translational Research
>>         >>> IRCCS - CROB Oncology Referral Center of Basilicata
>>         >>> Rionero in Vulture - Basilicata - Italy
>>         >>>
>>         >>>
>>         >>> 2014-02-21 17:52 GMT+01:00 James W. MacDonald
>>         <jmacdon@uw.edu <mailto:jmacdon@uw.edu> <mailto:
>>
>>         >>> jmacdon@uw.edu <mailto:jmacdon@uw.edu>>>:
>>         >>>
>>         >>>
>>         >>>     Hi Giovanni,
>>         >>>
>>         >>>     You should give the actual call you used for
>>         bumphunter, rather
>>         >>>     than simply saying that you used 'recommended
>>         parameters'. It
>>         >>>     looks like you set cutoffQ to 0.05, which is exactly
>>         opposite to
>>         >>>     what you should be doing. Instead you should be using
>>         something
>>         >>>     like cutoffQ = 0.95.
>>         >>>
>>         >>>     Best,
>>         >>>
>>         >>>     Jim
>>         >>>
>>         >>>
>>         >>>
>>         >>>
>>         >>>     On Friday, February 21, 2014 11:36:32 AM, Giovanni
>>         Calice wrote:
>>         >>>
>>         >>>         Hi Kasper,
>>         >>>
>>         >>>         Thank you very much for the immediate response!
>>         >>>
>>         >>>           So,
>>         >>>           What do you suggest?
>>         >>>
>>         >>>         Giovanni
>>         >>>
>>         >>>         Laboratory of Preclinical and Translational Research
>>         >>>         IRCCS - CROB Oncology Referral Center of Basilicata
>>         >>>         Rionero in Vulture - Basilicata - Italy
>>         >>>
>>         >>>
>>         >>>         2014-02-21 17:27 GMT+01:00 Kasper Daniel Hansen
>>         >>>         <khansen@jhsph.edu <mailto:khansen@jhsph.edu>
>>         <mailto:khansen@jhsph.edu <mailto:khansen@jhsph.edu>>>:
>>
>>         >>>
>>         >>>
>>         >>>
>>         >>>             On Fri, Feb 21, 2014 at 11:25 AM, Giovanni Calice
>>         >>>             <giovcalice@gmail.com
>>         <mailto:giovcalice@gmail.com> <mailto:giovcalice@gmail.com
>>
>>         <mailto:giovcalice@gmail.com>>>wrote:
>>         >>>
>>         >>>
>>         >>>                 273952
>>         >>>
>>         >>>
>>         >>>
>>         >>>             This is a crazy number of putative bumps
>>         >>>
>>         >>>             Kasper
>>         >>>
>>         >>>
>>         >>>                 [[alternative HTML version deleted]]
>>         >>>
>>         >>> _______________________________________________
>>         >>>         Bioconductor mailing list
>>         >>> Bioconductor@r-project.org
>>         <mailto:Bioconductor@r-project.org>
>>         <mailto:Bioconductor@r-project.org
>>
>>         <mailto:Bioconductor@r-project.org>>
>>         >>>
>>         >>> https://stat.ethz.ch/mailman/listinfo/bioconductor
>>         >>>         Search the archives:
>>         >>> http://news.gmane.org/gmane.science.biology.informatics.
>>         >>> conductor
>>         >>>
>>         >>>
>>         >>>     --
>>         >>>     James W. MacDonald, M.S.
>>         >>>     Biostatistician
>>         >>>     University of Washington
>>         >>>     Environmental and Occupational Health Sciences
>>         >>>     4225 Roosevelt Way NE, # 100
>>         >>>     Seattle WA 98105-6099
>>         >>>
>>         >>>
>>         >>>
>>         >> --
>>         >> James W. MacDonald, M.S.
>>         >> Biostatistician
>>         >> University of Washington
>>         >> Environmental and Occupational Health Sciences
>>         >> 4225 Roosevelt Way NE, # 100
>>         >> Seattle WA 98105-6099
>>         >>
>>         >>
>>         >
>>
>>                 [[alternative HTML version deleted]]
>>
>>         _______________________________________________
>>         Bioconductor mailing list
>>         Bioconductor@r-project.org <mailto:Bioconductor@r-project.org>
>>         https://stat.ethz.ch/mailman/listinfo/bioconductor
>>         Search the archives:
>>         http://news.gmane.org/gmane.science.biology.informatics.conductor
>>
>>
>>
>>
> --
> James W. MacDonald, M.S.
> Biostatistician
> University of Washington
> Environmental and Occupational Health Sciences
> 4225 Roosevelt Way NE, # 100
> Seattle WA 98105-6099
>
> _______________________________________________
> Bioconductor mailing list
> Bioconductor@r-project.org
> https://stat.ethz.ch/mailman/listinfo/bioconductor
> Search the archives: http://news.gmane.org/gmane.
> science.biology.informatics.conductor
>

	[[alternative HTML version deleted]]

