Yea, I was planning on adding a warning.


On Wed, Jul 31, 2013 at 11:49 AM, Janet Young <jayoung@fhcrc.org> wrote:

> Makes sense - thanks.  I was also playing with iterating through my
> queries 1000 at a time, but I'm sure the downloaded bigWig would be more
> efficient (and more palatable to the UCSC people).
>
> From a user point of view, it would be helpful if ucscTableQuery could
> issue a warning if we try to query with >1000 ranges, rather than silently
> returning a partial result - does that make sense to you?  I spent a little
> time down a rabbit hole with that one.
>
> Janet
>
>
>
> On Jul 31, 2013, at 11:39 AM, Michael Lawrence wrote:
>
> Right, the table browser is the limitation. It only allows 1000 query
> regions, and limits the output for "whole genome" as well to something like
> 100,000.
>
> If you've got a lot of data to query, it's best to just download the
> bigWig or other file and query it locally. These scores should be available
> via the FTP as bigWig.
>
>
>
>
> On Tue, Jul 30, 2013 at 1:06 PM, Janet Young <jayoung@fhcrc.org> wrote:
>
>> I'm guessing this issue might come from the UCSC end:  using their online
>> Table Browser, there is a query limit of 1000 regions.
>>
>> Janet
>>
>> On Jul 30, 2013, at 12:15 PM, Janet Young wrote:
>>
>> > Hi again,
>> >
>> > I think I've found another issue with rtracklyer, where if I run a
>> query with range specified as a large number of small GRanges, it seems to
>> only be getting results for the first 1000 ranges. Again, hopefully the
>> code below will explain all.
>> >
>> > thanks,
>> >
>> > Janet
>> >
>> > -------------------------------------------------------------------
>> >
>> > Dr. Janet Young
>> >
>> > Malik lab
>> > http://research.fhcrc.org/malik/en.html
>> >
>> > Fred Hutchinson Cancer Research Center
>> > 1100 Fairview Avenue N., A2-025,
>> > P.O. Box 19024, Seattle, WA 98109-1024, USA.
>> >
>> > tel: (206) 667 4512
>> > email: jayoung  ...at...  fhcrc.org
>> >
>> > -------------------------------------------------------------------
>> >
>> >
>> > ##### works:
>> >
>> > library(rtracklayer)
>> > library(GenomicRanges)
>> >
>> > session <- browserSession("UCSC")
>> > genome(session) <- "hg19"
>> >
>> >
>> > #### make some sample ranges - a large number of small ranges:
>> > numRanges <- 5000
>> > rangeWidths <- 50
>> > myRanges <- GRanges( seqnames=rep("chr1",numRanges),
>> >
>> ranges=IRanges(start=1:numRanges*rangeWidths*2,width=rangeWidths) )
>> >
>> > #### run a query
>> > query <- ucscTableQuery (session, "cons46way", range=myRanges)
>> > tableName(query) <- "phyloP46wayPrimates"
>> > scores <- track(query)
>> > ### seems to work with no apparent error
>> >
>> > ##### but the number of scores returned is much smaller than the number
>> of positions queried:
>> > length(scores)
>> > # [1] 41749
>> > sum(width(myRanges))
>> > # [1] 250000
>> >
>> >
>> > #### use findOverlaps to see which of the ranges had scores - only 837
>> of the 5000 regions had scores. It's ranges 109-1000 that have scores. Not
>> a problem that ranges 1-108 had missing scores (they're probably in an
>> assembly gap, or don't have aligned primate seqs), but at least some of
>> ranges 1001-5000 should have scores.
>> >
>> > myOverlaps <- findOverlaps(myRanges, scores)
>> > length(unique(queryHits(myOverlaps)))
>> > # [1] 837
>> > range(queryHits(myOverlaps))
>> > # [1]  109 1000
>> >
>> > ### a separate query shows that at least ranges 1001-1010 indeed should
>> have scores (haven't checked the rest):
>> > query2 <- ucscTableQuery (session, "cons46way",
>> range=myRanges[1001:1010])
>> > tableName(query2) <- "phyloP46wayPrimates"
>> > scores2 <- track(query2)
>> >
>> > ### every position queried has a score:
>> > sum(width(myRanges[1001:1010]))
>> > # [1] 500
>> > length(scores2)
>> > # [1] 500
>> >
>> > myOverlaps2 <- findOverlaps(myRanges, scores2)
>> > range(queryHits(myOverlaps2))
>> > # [1] 1001 1010
>> >
>> >
>> >
>> > ##################
>> >
>> >
>> > sessionInfo()
>> >
>> > R version 3.0.1 Patched (2013-07-29 r63455)
>> > Platform: x86_64-unknown-linux-gnu (64-bit)
>> >
>> > locale:
>> >  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
>> >  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
>> >  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
>> >  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
>> >  [9] LC_ADDRESS=C               LC_TELEPHONE=C
>> > [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>> >
>> > attached base packages:
>> > [1] parallel  stats     graphics  grDevices utils     datasets  methods
>> > [8] base
>> >
>> > other attached packages:
>> > [1] rtracklayer_1.21.9    GenomicRanges_1.13.35 XVector_0.1.0
>> > [4] IRanges_1.19.19       BiocGenerics_0.7.3
>> >
>> > loaded via a namespace (and not attached):
>> > [1] Biostrings_2.29.14 bitops_1.0-5       BSgenome_1.29.1
>>  RCurl_1.95-4.1
>> > [5] Rsamtools_1.13.26  stats4_3.0.1       tools_3.0.1
>>  XML_3.98-1.1
>> > [9] zlibbioc_1.7.0
>> >
>>
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> Bioconductor mailing list
>> Bioconductor@r-project.org
>> https://stat.ethz.ch/mailman/listinfo/bioconductor
>> Search the archives:
>> http://news.gmane.org/gmane.science.biology.informatics.conductor
>>
>
>
>

	[[alternative HTML version deleted]]

