[R-sig-DB] rmysql warning and its associated mysql error

[Jeffrey Horner]

At this point, I would suggest providing a reproducible example that you 
can share with the list, as I cannot troubleshoot your pseudo code. Your 
problem could possibly come from biglm and not RMySQL.

Also, I presume you tried to answer my previous questions, which were:

What do you mean when you "add some code to the biglm package"?

So, you're really not adding code to biglm, only utilizing it with the 
below pseudo code?

Also, I presume you can't provide an example table schema that you are 
using, but how about the same schema with the names changed?

Best,

Jeff

christophe dutang wrote on 11/04/2008 11:21 AM:
> The fit of the GLM is something like this:
> 
> for(i in <iterations>)
> {
>    while(<data to read>)
>    {
>         <read data and create model matrix>
>         <update the current solution>
>         *
>         <test if the tolerance is reached>
>    }
>    <print traces>
> }
> <return the results>
> 
> 
> * I add here code to compute the con/discordant pair.
> 
> It is so annoying I can't send you my whole code...
> 
> 
> 2008/11/4 Jeffrey Horner <jeff.horner using vanderbilt.edu 
> <mailto:jeff.horner using vanderbilt.edu>>
> 
>     christophe dutang wrote on 11/04/2008 07:07 AM:
> 
>         Hi all,
> 
>         Here are some details about my code:
> 
>         - I use a mysql server where my whole database is stored,
>         - I'm doing a logistic regression with the package biglm
>         - my 'data function' just does a SQL request for the selected
>         variables and then I read part of it with the function fetch:
> 
>         MySQLdatafun<-function(reset=FALSE)
>         {
>          if(reset)
>          {
>            beginRead <<- 0
>            endRead <<- 0
>            dbClearResult(resSQL)
>            resSQL <<- dbSendQuery(con, SQLSelectExplVar)
>            return(NULL)
>          }
> 
>          if(endRead >= lengthData)
>              return(NULL)
> 
>          beginRead<<-endRead+1
>          endRead<<-endRead+min(chunksize, lengthData-endRead)
> 
>          mydata <<- data.frame(fetch(resSQL, n=endRead-beginRead+1),
>         stringsAsFactors=TRUE)
>          # BTW the last argument 'stringAsFactor' does not seem to work.
> 
>          for(i in 1:NCOL(mydata))
>            if(class(mydata[1,i]) == "character")
>              mydata[,i] <<- factor(mydata[,i])
> 
>          mydata
>         }
> 
>         - what's really strange is the problem occurs when I add some
>         code to the biglm package to compute the con/discordant percentage.
> 
> 
>     What do you mean when you "add some code to the biglm package"?
> 
>     Also, can you provide a table schema?
> 
>     Jeff
> 
> 
>         I can not yet upgrade to 2.8.0 since there is no version of the
>         (d)com server for this version of R.
> 
>         Thanks in advance
> 
>         Christophe Dutang
> 
>         2008/11/3 Jeffrey Horner <jeff.horner using vanderbilt.edu
>         <mailto:jeff.horner using vanderbilt.edu>
>         <mailto:jeff.horner using vanderbilt.edu
>         <mailto:jeff.horner using vanderbilt.edu>>>
> 
> 
>            christophe dutang wrote on 10/31/2008 03:28 PM:
> 
>                Hi,
> 
>                I'm currently experiencing problem with the combination
>         of mysql
>                / Rmysql /
>                R when reading the result of a 'big' query. If I select
>         only 4
>                variables of
>                my mysql table, the result dimension has 56972 rows, I
>         read by
>                pack of
>                50000, namely the first of 50000 and the second of 6972.
>         In this
>                cas I do
>                not get any DBI warning telling an error of mysql server.
>         But If
>                I read the
>                21 variables of my table, the result dimension is then
>         56972 x
>                21. In R, the
>                first read of 50000 rows is fine but second stops after
>         reading
>                2182 rows...
>                and a DBI warning is raised
> 
>                RS-DBI driver warning: (error while fetching rows)
> 
>                This problem was raised in 2003, cf.
>                https://stat.ethz.ch/pipermail/r-help/2003-April/032708.html
> 
>                But I found here
>                http://www.mysqlperformanceblog.com/2007/07/06/, that "If
>                you do not check for error it can look as you've done with
>                result set while
>                you only processed a portion of it, which can cause
>         rather hard
>                to catch
>                errors."
> 
>                Does anyone experience this problem? and know how to
>         solve it?
> 
> 
>            Try upgrading R to 2.8.0. <http://2.8.0.> <http://2.8.0.>
> 
> 
>            Can you provide your code to the list... or pseudo code so
>         that we
>            can troubleshoot? Specifically are you calling fetch() or
>         dbApply()?
> 
>            Jeff
> 
> 
>                Thanks in advance
> 
>                Christophe Dutang
> 
> 
>                PS : I use R 2.7.2 on windows XP pro with RMySQL_0.6-1 and a
>                MySQL community
>                server 5.0
> 
>                       [[alternative HTML version deleted]]
> 
>                _______________________________________________
>                R-sig-DB mailing list -- R Special Interest Group
>                R-sig-DB using stat.math.ethz.ch
>         <mailto:R-sig-DB using stat.math.ethz.ch>
>         <mailto:R-sig-DB using stat.math.ethz.ch
>         <mailto:R-sig-DB using stat.math.ethz.ch>>
> 
>                https://stat.ethz.ch/mailman/listinfo/r-sig-db
> 
> 
> 
>            --    http://biostat.mc.vanderbilt.edu/JeffreyHorner
> 
> 
> 
> 
> 
> 
>     -- 
>     http://biostat.mc.vanderbilt.edu/JeffreyHorner
> 
> 


-- 
http://biostat.mc.vanderbilt.edu/JeffreyHorner