[R] Automatic saving of many regression's output

[arun]

Hi,
No problem,

You could try:
library(tseries)

res6 <- do.call(rbind,lapply(lst1[sapply(lst1,function(x) !(all(rowSums(is.na(x))>0)))],function(x) {resid <- residuals(lm(rate~.,data=x)); unlist(jarque.bera.test(resid)[1:3])}) )


 A.K.




On Wednesday, November 27, 2013 7:47 PM, Tomasz Schabek <schabek.tomasz at gmail.com> wrote:

Great!

Thank you for help one more time!
yes, you are right - jarque.bera.test() should be applied to a vector, so the deal is: residuals from each of those 123 regressions captured by e.g:
"resid <-residuals(model)"  and "jarque.bera.test(resid)" are tested in jarque.bera.test(). Could you manage it?

You are really helpful and kind person!




Kind regards,
Atenciosamente,
Pozdrawiam,

T. S.


On 28 November 2013 01:33, arun <smartpink111 at yahoo.com> wrote:


>
>Hi,
>In that case:
>
> lst5 <- lapply(lst1[sapply(lst1,function(x) !(all(rowSums(is.na(x))>0)))],function(x) vif(lm(rate~., x)))
>res5 <- do.call(rbind,lst5)
>
>
>As I mentioned earlier, it is not clear how you wanted to test jarque.bera.test().  Also, the results from lst3,lst4,lst5 etc could be saved using capture.output() (not tested though).  Or if you wanted to modify it and wanted only specific categories, for example:
> res4 <- do.call(rbind,lapply(lst4,function(x) unlist(x[-4])))
>
>
>
> 
>
>On Wednesday, November 27, 2013 7:21 PM, nooldor <nooldor at gmail.com> wrote:
>
>Thank you for fast answer!
>
>and big THANK for help!
>
>I found error in the previous script (it was doing 334 regressions on 123 length vectors and it should be opposite: 123 regressions on 334 length vector) anyway I modify it:
>
>data<-read.table("reg3-dane.csv", head=T, sep=";", dec=",")
>>data$indx <- as.numeric(gl(123*334,334,123*334))
>>lst1 <- split(data[,-16],data[,16])
>>any(sapply(lst1,nrow)!=123)
>>#[1] FALSE
>>lapply(lst1[sapply(lst1,function(x) !(all(rowSums(is.na(x))>0)))],function(x) summary(lm(rate~cap.log,data=x)) )
>>capture.output(lst2,file="nooldor.txt")
>>it's ok now (at least when I compared regression summary from excel and R it was the same :-) )
>
>
>capture.output(lst2,file="nooldor.txt") works fine!
>
>packages:
>vif {car}
>jarque.bera.test {tseries}
>
>ncvTest {car}
>durbinWatsonTest {car}
>
>
>R version 3.0.2 (2013-09-25)
>
>
>T.S.
>
>
>On 28 November 2013 00:38, arun <smartpink111 at yahoo.com> wrote:
>
>Hi,
>>
>>2. You need to tell which package you are using.
>>
>>3. Does this work for you?
>>capture.output(lst2,file="nooldor.txt")
>>
>>4.
>>
>>
>>
>>lst2
>> <- lapply(lst1[sapply(lst1,function(x)
>>!(all(rowSums(is.na(x))>0)))],function(x)
>>print(summary(lm(rate~.,data=x)))  ###prints the output on R console
>>
>>A.K.
>>
>>
>>
>>Hi,
>>
>>Thank you for patience and help :-)
>>
>>now the code looks like that:
>>
>>
>>data<-read.table("reg3-dane.csv", head=T, sep=";", dec=",")
>>>data$indx <- as.numeric(gl(334*123,123,334*123))
>>>lst1
>>
>> <- split(data[,-16],data[,16]) # 1. by changing "16" parameter I can
>> add or remove variables (also by modyfing the "reg3-dane.csv" file),
>>right?
>>>any(sapply(lst1,nrow)!=123)
>>>#[1] FALSE
>>
>>>lst2 <- lapply(lst1[sapply(lst1,function(x) !(all(rowSums(is.na(x))>0)))],function(x) summary(lm(rate~cap.log+liqamih.log+pbv,data=x)) )
>>>length(lst2)
>> # 2.where I can place the test for each (from 123) regression like
>>jarque.bera.test() 
>>vif() 
>>ncvTest() 
>>durbinWatsonTest() to have it saved with regression summary? and 3. how
>>to get those list with results more user-friendly? I would like to get
>>the report 
>>>#[1] 334 
>>>
>>
>>is it ok?
>>
>>Could you help me with the questions in remarks above?
>>
>>And could you modify the script to also print the summary (and tests) of each regression (each of 123) in console?
>>
>>
>>Best wishes!
>>T.S.
>>
>>
>>
>>
>>On Wednesday, November 27, 2013 5:49 PM, arun <smartpink111 at yahoo.com> wrote:
>>
>>
>>
>>Hi,
>>
>>lst1[[1]][,2] <- NA
>>lst2 <- lapply(lst1,function(x) summary(lm(rate~.,data=x)))
>>Error in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) :
>>  0 (non-NA) cases
>>
>>
>>
>>lst2 <- lapply(lst1[sapply(lst1,function(x) !(all(rowSums(is.na(x))>0)))],function(x) summary(lm(rate~.,data=x)) )
>>A.K.
>>
>>
>>
>>Hi,
>>
>>thank you for help. :-)
>>
>>I applied your script to the data but I have got the error:
>>
>>Error
>>in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) :  0
>>(non-NA) casesI forget to write that some of the data are NA.
>>
>>I executed this code:
>>
>>lst1 <- split(data[,-16],data[,16])
>>>any(sapply(lst1,nrow)!=123)
>>>#[1] FALSE
>>>lst2
>><- lapply(lst1,function(x)
>>summary(lm(rate~cap.log+liqamih.log+pbv,data=x))) # here I can set the
>>dependent variables if I  want to test different versions of the model
>>(e.g with only e dependent variables), right?
>>>length(lst2)
>>>#[1] 334
>>>
>>
>>
>>
>>
>>
>>On Wednesday, November 27, 2013 5:27 PM, arun <smartpink111 at yahoo.com> wrote:
>>Hi,
>>Try:
>>set.seed(49)
>>dat1 <- as.data.frame(matrix(sample(c(NA,1:50),41082*15,replace=TRUE),ncol=15))
>> dat1$indx <- as.numeric(gl(334*123,123,334*123))
>>names(dat1)[1] <- "rate"
>> lst1 <- split(dat1[,-16],dat1[,16])
>>any(sapply(lst1,nrow)!=123)
>>#[1] FALSE
>>lst2 <- lapply(lst1,function(x) summary(lm(rate~.,data=x)))
>> length(lst2)
>>#[1] 334
>>
>>A.K.
>>
>>Hi all!
>>
>>I am very beginner in R so please excuse me some of the naive questions. I am learning.
>>Here is description of my problem:
>>
>>I have database (in single csv file)
>>                   characteristic_1    characteristic_2               ...          characteristic_49
>>subject_1     |      c1_1_t=1             |   c2_1_t=1             ... |     c49_1_t=1
>>subject_2     |      c1_2_t=1             |   c2_2_t=1             ... |     c49_2_t=1
>>subject_3     |      c1_3_t=1             |   c2_3_t=1             ... |     c49_3_t=1
>>...
>>subject_334  |      c1_334_t=1         |   c2_334_t=1          ... |     c49_334_t=1
>>subject_1     |      c1_1_t=2            |   c2_1_t=2              ... |     c49_1_t=2
>>subject_2     |      c1_2_t=2            |   c2_2_t=2              ... |     c49_2_t=2
>>subject_3     |      c1_3_t=2            |   c2_3_t=2              ... |     c49_3_t=2
>>...
>>subject_334  |      c1_3_t=2            |   c2_3_t=2              ... |     c49_3_t=2
>>
>>and so on ... till t (time) = 123
>>
>>so I have 334 subjects with 49 characteristics measured in 123 points of time.
>>
>>I would like to run 123 regressions (three kinds: lm, rlm and
>>lmrob - for comparison reasons) each one for 334 subjects and 49
>>dependent variables and after each regression (actually after conducting
>>each of the three regressions:lm, rlm and lmrob) I would like to save
>>txt (or csv) file with results (summary) and some test* (each regression
>>can be named reg_1, reg_2 ... reg_123) for those regressions.
>>
>>To make things more clear:
>>regressions would look like that:
>>
>>summary(lm(rate~cap.log+liqamih.log+liqwol.log+pbv.log+mom.log+
>>             +beta.wig+beta.wig.eq
>>           +beta.sp
>>           +beta.wig.macro
>>           +beta.sp.macro
>>           +beta.sentim.pl+beta.sentim.pl.ort
>>           +beta.sentim.usa+beta.sentim.usa.ort, data=data))
>>
>>the problem is how to make this lm() above for "rolling window"
>>id est for first 334 observations? (total observations: 123*334) and so
>>on.
>>I need to run regression_1 for first 334 observations, regression_2
>>for next 334 obs (from 335 to 669) and so on till regression_123 (from
>>last 40748 till 41082).
>>And each time I run such regression I would like to save results (summary and mentioned tests).
>>
>>Then I would like to repeat the same procedure but for rlm() and lmrob() if possible.
>>
>>I think I can write "tests" part of the script alone (could you
>>write me some comments where exactly I should put it in script to have
>>the test automatically repeated the results saved), but 'saving' and
>>'repeating 123 times' procedures are quite complicated for me, at least
>>now. So here I am asking for help with it.
>>
>>In the end I would like to have three txt (or csv) files:
>>one containing 123 "summaries" and test results of lm,
>>one containing 123 "summaries" and test results of rlm
>>and one containing 123 "summaries" and test results of lmrob.
>>
>>Could someone help me with this task?
>>I am grateful for your help and support.
>>
>>________________
>>*like:
>>jarque.bera.test()
>>vif()
>>ncvTest()
>>durbinWatsonTest()
>>
>>---some of them are not applicable for rlm and lmrob - so in
>>this case I would like to have "test NA" in the three output txt (or
>>csv) files
>>Some of them are also not applicable to cross-sectional regressions
>>... but still I would like to keep them in script for later
>>modifications
>>
>