[R-meta] Help with interpretation please?
lists at dewey.myzen.co.uk
Sun Jul 23 13:13:09 CEST 2017
Note this is a plain text list and sending in HTML magles your message.
In this case I think it has not done too much harm.
On 23/07/2017 11:23, Paul Cuckoo wrote:
> I'm using the metafor package in R to attempt to model the impact various
> covariates have on an uplift score.
> Cell.No. brand_lift_ppt experiment_variance vertical_name
> Gender age_targeting_min1 0.01002470 0.000251804
> [Entertainment & Media] A 352 0.01343524
> 0.000119650 [Entertainment & Media] M 183
> 0.01601813 0.000352114 [Entertainment & Media] F
> 184 0.17279558 0.000275272 [Entertainment & Media]
> F 185 0.14091904 0.000203351
> [Entertainment & Media] M 186 0.13449727
> 0.000202170 [Entertainment & Media] A 35
> I specify the model as follows:
> model1 <- rma(data=data.test, yi=brand_lift_ppt, vi=experiment_variance,
> mods = ~ vertical_name)
> and get the following output:
> Mixed-Effects Model (k = 375; tau^2 estimator: REML)
> logLik deviance AIC BIC AICc
> 500.9043 -1001.8086 -985.8086 -954.5439 -985.4075
> tau^2 (estimated amount of residual heterogeneity): 0.0031 (SE = 0.0003)
> tau (square root of estimated tau^2 value): 0.0558
> I^2 (residual heterogeneity / unaccounted variability): 86.70%
> H^2 (unaccounted variability / sampling variability): 7.52
> Test for Residual Heterogeneity:
> QE(df = 368) = 2703.1344, p-val < .0001
> Test of Moderators (coefficient(s) 1:7):
> QM(df = 7) = 253.9762, p-val < .0001
> Model Results:
> estimate se zval
> pval ci.lb ci.ub
> vertical_name[Automotive] 0.0398 0.0081 4.9171
> <.0001 0.0239 0.0556 ***
> vertical_name[Consumer Packaged Goods] 0.0510 0.0050 10.1589
> <.0001 0.0412 0.0609 ***
> vertical_name[Entertainment & Media] 0.0642 0.0071 9.0057
> <.0001 0.0502 0.0782 ***
> vertical_name[Retail] 0.0559 0.0087 6.4059
> <.0001 0.0388 0.0730 ***
> vertical_name[Technology] 0.0063 0.0335 0.1875
> 0.8513 -0.0593 0.0719
> vertical_name[Telecom] 0.0226 0.0112 2.0273
> 0.0426 0.0008 0.0445 *
> vertical_name[Travel] 0.0239 0.0428 0.5580
> 0.5768 -0.0600 0.1078
> I'm having trouble interpreting the output to get to what I need. Some
> 1) Entertainment and Media appears to be significant. How do I interpret
> the coefficient? I haven't logged or otherwise transformed any of the data.
> Is it simply that when I select Entertainment and Media, I see on average a
> 6.42% brand uplift?
It means that the score is 0.0642 higher on average compared to your
> 2) How can I get an overall estimate of the impact of vertical media,
> relative to other covariates, say gender? A ratio of QM score?
You would presumably need a model with both covariates in but note that
the whole issue of relative importance in regression (and by extension
in meta-regression) is fraught with problems. You could look at the
effect of each covariate in the model with both in the model by using
the btt parameter.
> 3) Should I be combining all factors together in mods or is it acceptable
> to test separately?
If you want to know how they affect your score over and above the others
then you need a full model.
> 4) Under what circumstances should I be considering a transformation to
> brand_lift before modelling?
Try looking at the residuals. Any advice about diagnostics in regression
would apply here. You might want to look at the plots of influence
statistics too, try influence(model1) and its plot method.
> [[alternative HTML version deleted]]
> R-sig-meta-analysis mailing list
> R-sig-meta-analysis at r-project.org
> This email has been checked for viruses by AVG.
More information about the R-sig-meta-analysis