The notes about GRanges/CHiPpeakAnnotate/etc. are separate.  For the BioC
packages, I just used Illumina's supplied accessions to map the probes to
GO/KEGG/etc. and added bimaps similar to the 27k package's maps for
Illumina's chromosome (CHR37 and CHR36, for the respective builds) and cpg
coordinate (CPG36 and CPG37 respectively), along with NuID and CpGs maps
generated from Illumina's SourceSeq.

The annotation of the probes from Issa was a separate exercise; I should
have made it clearer that, in some cases, I want to confirm or specific
annotations for myself, and I suspect that will always be a special case
(for the reasons you outlined here).

Sorry for the unclear phrasing.  The 27k and 450k .db packages are all stock
Illumina manifest data, mapped to the human db0 schema as provided by
makeDBPackage().



On Tue, Jan 18, 2011 at 12:28 PM, Sean Davis <sdavis2@mail.nih.gov> wrote:

>
>
> On Tue, Jan 18, 2011 at 3:13 PM, Tim Triche, Jr. <ttriche@usc.edu> wrote:
>
>> Hi all,
>>
>> Thanks for all the tips and suggestions!   I have updated the 27k and 450k
>> methylation annotation packages, and will re-run the packaging script in a
>> few days to see how much of a difference this makes.  I need to update the
>> 27k annotations anyways, now that I've added some fields and bimaps for my
>> own purposes (decoding IDAT files and mapping the probes properly).   But,
>> I've got working builds of the packages now -- whee.
>>
>> The new 27k and 450k packages are ready (or at least, everything works --
>> I am only just learning how to use RUnit properly so the formal tests have
>> not yet been updated).  Eventually I would like to automate the merging
>> process for information from Illumina and that which is included from other
>> sources (Entrez, UCSC, etc).  What is the accepted way of doing this?  I
>> think I could handle maintenance of these packages if there's a means to
>> properly automate the merging of manufacturer-specific and assembly-specific
>> data.
>>
>> For what it's worth, I tried using CHiPpeakAnno, GenomicRanges, and some
>> other tools (but mostly the former) to annotate a few probes, and I'm not
>> 100% certain that I did it right.  Absent strand information, which I find
>> almost indecipherable from Illumina's "TOP/BOT" vs the usual "+/-"
>> watson/crick notation, the results suggest that a handful of allegedly
>> upstream CpG island probes are... not.  The re-annotation results do
>> correspond fairly closely to the actual assay results, though (Jean-Pierre
>> Issa wanted to compare some Illumina methylation results with those from the
>> MCA platform in use at MD Anderson, so I said I would try and figure out why
>> some Illumina probes seem concordant with the MCA probes and some don't...
>> the reason, if GRanges is correct, is that some of the Illumina probes don't
>> seem to map where I expected them to).
>>
>> What's the best way to do reannotation of a platform for specific
>> features?  It took a long time just to map the accessions on the 450k
>> platform, I'm not real excited about BLATting everything in sight whenever a
>> string changes somewhere, if I don't have to.
>>
>>
> Hi, Tim.
>
> I would suggest using the annotations supplied by the manufacturer without
> modification.  Doing otherwise will very likely be a non-trivial exercise,
> particularly with regard to documenting what has been done.  Of course, feel
> free to do that with your own versions (or even separate parallel versions)
> of the packages, but for those for release on bioconductor, I suggest that
> sticking to the manufacturer's annotations is the best way to go.  Of
> course, if you find annotation problems, bring those up with Illumina
> directly, as they have an interest in maintaining relevant and correct
> annotations.
>
> Sean
>
>
>
>> Thanks again,
>>
>> Tim Triche, Jr.
>>
>>
>>
>> 2011/1/14 Hervé Pagès <hpages@fhcrc.org>
>>
>>> Hi Tim,
>>>
>>> FWIW also make sure you use the latest version of RSQLite (currently
>>> 0.9-4). The SQLite engine shipped with RSQLite was updated a couple
>>> of months ago and seems to be significantly faster than the previous
>>> versions in some situations.
>>>
>>> Cheers,
>>> H.
>>>
>>>
>>>
>>> On 01/13/2011 12:20 PM, Marc Carlson wrote:
>>>
>>>> Hi Tim,
>>>>
>>>> It certainly is doing something.  There are a couple of different things
>>>> that can cause slowness with this.  The 1st is that SQLForge is doing a
>>>> lot of busy work so that the packages it produces can "know" how many
>>>> mappings are expected from each (there is a table called map_counts that
>>>> requires this information).  The second problem is that you are mapping
>>>> over an order of magnitude more probes than we usually would ever need
>>>> to do.  In the past, performance has not been a big problem, in part
>>>> because you only need to do this step once and in part because normally
>>>> people only want to map a few tens of thousands of probes.  But you seem
>>>> to be pushing the envelope pretty hard and the wait time has become
>>>> pretty extreme as a result.  My guess is that I need to add some
>>>> temporary indices into the initial mapping process to speed up this step
>>>> when there are a lot of probes.  I will take a look and see what I can
>>>> do.
>>>>
>>>>   Marc
>>>>
>>>>
>>>>
>>>> On 01/13/2011 09:45 AM, Tim Triche, Jr. wrote:
>>>>
>>>>> Hi Sean, (and others)
>>>>>
>>>>> Does it usually take an obscenely long time for makeDBPackage to run
>>>>> when
>>>>> given a bunch of refseq IDs?  I ran the following:
>>>>>
>>>>>
>>>>>  makeDBPackage("HUMANCHIP_DB", affy=FALSE,
>>>>>>
>>>>>> prefix="IlluminaHumanMethylation450k",fileName="acc450k.txt",baseMapType='refseq',version='1.0.0',manufacturer='Illumina',chipName='Human
>>>>> Methylation 450k', manufacturerUrl='http://illumina.com/')
>>>>> baseMapType is refseq # time passes...
>>>>>
>>>>> and it's doing SOMETHING, because the table temp_probe_map in the
>>>>> sqlite
>>>>> file it created is filling up.  But it's been grinding away at this for
>>>>> the
>>>>> past 12 hours, which seems a bit excessive for mapping 806,334 refseq
>>>>> accessions.  Really, all I want is for my bimap objects to work as
>>>>> expected,
>>>>> the annotation integration is just gravy.
>>>>>
>>>>> The idea was to release updated 27k and completed 450k packages to
>>>>> handle
>>>>> the IDAT mappings, immediately followed by a methylumIDAT release, and
>>>>> start
>>>>> merging that and my preprocessing stuff into the [methy]lumi toolchain.
>>>>>  The
>>>>> 450k probes are split between two designs, so I kind of had to bite the
>>>>> bullet and roll my own schema to do the mappings efficiently, plus
>>>>> between
>>>>> 450k and FFPE samples there has been a LOT of weirdness lately that I'd
>>>>> like
>>>>> not to depend on Illumina's software to handle.  So...
>>>>>
>>>>> I started out doing this on my laptop (gave up after a few hours and
>>>>> moved
>>>>> it to the server):
>>>>>
>>>>>
>>>>>  sessionInfo()
>>>>>>
>>>>>>  R version 2.13.0 Under development (unstable) (2010-12-21 r53879)
>>>>> Platform: x86_64-unknown-linux-gnu (64-bit)
>>>>>
>>>>> locale:
>>>>>  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
>>>>>  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
>>>>>  [5] LC_MONETARY=C              LC_MESSAGES=en_US.UTF-8
>>>>>  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
>>>>>  [9] LC_ADDRESS=C               LC_TELEPHONE=C
>>>>> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>>>>>
>>>>> attached base packages:
>>>>> [1] grid      stats     graphics  grDevices datasets  utils     methods
>>>>> [8] base
>>>>>
>>>>> other attached packages:
>>>>>  [1] human.db0_2.4.1                      ChIPpeakAnno_1.7.0
>>>>>
>>>>>  [3] limma_3.5.20                         GO.db_2.4.1
>>>>>
>>>>>  [5] BSgenome.Ecoli.NCBI.20080805_1.3.16  BSgenome_1.19.2
>>>>>
>>>>>  [7] GenomicRanges_1.3.7                  Biostrings_2.19.2
>>>>>
>>>>>  [9] IRanges_1.9.17                       multtest_2.6.0
>>>>>
>>>>> [11] biomaRt_2.7.1                        groupedPMA_0.2
>>>>>
>>>>> [13] betareg_2.2-3                        Formula_1.0-0
>>>>>
>>>>> [15] PMA_1.0.7                            huge_0.9
>>>>>
>>>>> [17] MASS_7.3-9                           igraph_0.5.5-1
>>>>>
>>>>> [19] glasso_1.4                           glmnet_1.5.1
>>>>>
>>>>> [21] Matrix_0.999375-46                   lattice_0.19-17
>>>>>
>>>>> [23] grplasso_0.4-2                       impute_1.25.0
>>>>>
>>>>> [25] rGammaGamma_1.0                      methylumIDAT_0.1
>>>>>
>>>>> [27] IlluminaHumanMethylation27k.db_1.4.0 org.Hs.eg.db_2.4.6
>>>>>
>>>>> [29] RSQLite_0.9-4                        DBI_0.2-5
>>>>>
>>>>> [31] AnnotationDbi_1.13.0                 ggplot2_0.8.9
>>>>>
>>>>> [33] proto_0.3-8                          lumi_2.3.5
>>>>>
>>>>> [35] nleqslv_1.8                          matrixStats_0.2.2
>>>>>
>>>>> [37] R.methodsS3_1.2.1                    gsl_1.9-8
>>>>>
>>>>> [39] methylumi_1.3.3                      Biobase_2.11.7
>>>>>
>>>>> [41] gtools_2.6.2                         reshape_0.8.3
>>>>>
>>>>> [43] plyr_1.4
>>>>>
>>>>> loaded via a namespace (and not attached):
>>>>>  [1] affy_1.27.2           affyio_1.17.4         annotate_1.27.1
>>>>>  [4] digest_0.4.2          hdrcde_2.15           KernSmooth_2.23-4
>>>>>  [7] lmtest_0.9-27         mgcv_1.7-2            nlme_3.1-97
>>>>> [10] preprocessCore_1.11.0 RCurl_1.5-0           sandwich_2.2-6
>>>>> [13] splines_2.13.0        survival_2.36-2       tools_2.13.0
>>>>> [16] XML_3.2-0             xtable_1.5-6
>>>>>
>>>>>        [[alternative HTML version deleted]]
>>>>>
>>>>> _______________________________________________
>>>>> Bioc-devel@r-project.org mailing list
>>>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Bioc-devel@r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/bioc-devel
>>>>
>>>
>>>
>>> --
>>> Hervé Pagès
>>>
>>> Program in Computational Biology
>>> Division of Public Health Sciences
>>> Fred Hutchinson Cancer Research Center
>>> 1100 Fairview Ave. N, M2-B876
>>> P.O. Box 19024
>>> Seattle, WA 98109-1024
>>>
>>> E-mail: hpages@fhcrc.org
>>> Phone:  (206) 667-5791
>>> Fax:    (206) 667-1319
>>>
>>
>>
>>
>> --
>> Pure reason shows that reason is not enough.
>>
>> Dennis V. Lindley<http://www.york.ac.uk/depts/maths/histstat/people/lindley.gif>
>>
>>
>


-- 
Pure reason shows that reason is not enough.

Dennis V. Lindley<http://www.york.ac.uk/depts/maths/histstat/people/lindley.gif>

	[[alternative HTML version deleted]]

