[R-meta] Help with interpretation please?

[Michael Dewey]

Comments in-line.

Note this is a plain text list and sending in HTML magles your message. 
In this case I think it has not done too much harm.

On 23/07/2017 11:23, Paul Cuckoo wrote:
> I'm using the metafor package in R to attempt to model the impact various
> covariates have on an uplift score.
>
>> head(data.test)
> Cell.No. brand_lift_ppt experiment_variance           vertical_name
> Gender age_targeting_min1     0.01002470         0.000251804
> [Entertainment & Media]      A                352     0.01343524
>   0.000119650 [Entertainment & Media]      M                183
> 0.01601813         0.000352114 [Entertainment & Media]      F
>       184     0.17279558         0.000275272 [Entertainment & Media]
>    F                185     0.14091904         0.000203351
> [Entertainment & Media]      M                186     0.13449727
>   0.000202170 [Entertainment & Media]      A                35
>
> I specify the model as follows:
>
> model1 <- rma(data=data.test, yi=brand_lift_ppt, vi=experiment_variance,
>               mods = ~ vertical_name)
> summary(model1)
>
> and get the following output:
>
> Mixed-Effects Model (k = 375; tau^2 estimator: REML)
>
>     logLik    deviance         AIC         BIC        AICc
>   500.9043  -1001.8086   -985.8086   -954.5439   -985.4075
>
> tau^2 (estimated amount of residual heterogeneity):     0.0031 (SE = 0.0003)
> tau (square root of estimated tau^2 value):             0.0558
> I^2 (residual heterogeneity / unaccounted variability): 86.70%
> H^2 (unaccounted variability / sampling variability):   7.52
> Test for Residual Heterogeneity:
> QE(df = 368) = 2703.1344, p-val < .0001
> Test of Moderators (coefficient(s) 1:7):
> QM(df = 7) = 253.9762, p-val < .0001
> Model Results:
>
>                                         estimate      se     zval
> pval    ci.lb   ci.ub
> vertical_name[Automotive]                 0.0398  0.0081   4.9171
> <.0001   0.0239  0.0556  ***
> vertical_name[Consumer Packaged Goods]    0.0510  0.0050  10.1589
> <.0001   0.0412  0.0609  ***
> vertical_name[Entertainment & Media]      0.0642  0.0071   9.0057
> <.0001   0.0502  0.0782  ***
> vertical_name[Retail]                     0.0559  0.0087   6.4059
> <.0001   0.0388  0.0730  ***
> vertical_name[Technology]                 0.0063  0.0335   0.1875
> 0.8513  -0.0593  0.0719
> vertical_name[Telecom]                    0.0226  0.0112   2.0273
> 0.0426   0.0008  0.0445    *
> vertical_name[Travel]                     0.0239  0.0428   0.5580
> 0.5768  -0.0600  0.1078
>
> I'm having trouble interpreting the output to get to what I need. Some
> questions:
>
> 1) Entertainment and Media appears to be significant. How do I interpret
> the coefficient? I haven't logged or otherwise transformed any of the data.
> Is it simply that when I select Entertainment and Media, I see on average a
> 6.42% brand uplift?

It means that the score is 0.0642 higher on average compared to your 
reference category.
>
> 2) How can I get an overall estimate of the impact of vertical media,
> relative to other covariates, say gender? A ratio of QM score?
>

You would presumably need a model with both covariates in but note that 
the whole issue of relative importance in regression (and by extension 
in meta-regression) is fraught with problems. You could look at the 
effect of each covariate in the model with both in the model by using 
the btt parameter.

> 3) Should I be combining all factors together in mods or is it acceptable
> to test separately?
>

If you want to know how they affect your score over and above the others 
then you need a full model.

> 4) Under what circumstances should I be considering a transformation to
> brand_lift before modelling?

Try looking at the residuals. Any advice about diagnostics in regression 
would apply here. You might want to look at the plots of influence 
statistics too, try influence(model1) and its plot method.

>
> 	[[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis at r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis
>
> ---
> This email has been checked for viruses by AVG.
> http://www.avg.com
>
>

-- 
Michael
http://www.dewey.myzen.co.uk/home.html