[R] Test for equality of complicatedly related average correlations

[Adam D. I. Kramer]

Hi Ralph,

My approach provides the same answer by asking a different question.
Effectively, my approach tests whether the difference between timepoints is
larger for X than for Y (and also gives you the main effects of whether X
has a higher mean than Y and whether scores increase or decrease over time,
which may or may not be of interest but as control variables serve the same
purpose as using correlations rather than difference scores).

One objection to this mode of comparing test-retest reliabilities is that
one test may be more variable over time (at the item level) than the other,
but that the overall means don't differ...but the correlational approch you
asked about also suffers from this problem.

--Adam

On Sun, 7 Sep 2008, Ralph79 wrote:

> I have to get a bit more familiar with the model you propose in order to
> understand if it applies to my problem as well.
>
> My question is not really "does time show a different effect" but "which
> one of two measures is more reliable": My respondents have completed
> exactly the same questionnaire twice (t=1 and t=2). The questionnaire
> consisted of two ways of measuring attribute importance, and the "better"
> method of measuring these importances is the one that gives the same
> importances for each respondent in t=1 and t=2. In other words: I want to
> examine test-retest reliability of the two measures. Naturally, if
> X(t=1,t=2)-correlation is higher for a specific respondent than the
> Y(t=1,t=2)-corralation, than for this respondent the method that yields
> the X-importances is more reliable. All I want to do is to see if this
> holds for the whole sample as well...

> Anyway, thank you again, I will think of your approach.
>
> Ralph
>
>
>
> Adam D. I. Kramer-3 wrote:
>>
>> Hi Ralph,
>>
>>  	I had the same problem you do a few months ago, and realized that
>> the question I had (does time show a different effect for X than Y) was
>> not
>> best modeled as differences between correlations across individuals, but
>> as
>> whether time interacts with condition.
>>
>>  	I answered this question with
>>> library(nlme)
>>> lme(obs ~ cond*time, random=~cond*time|subj)
>>
>> ...where obs is the responses on the X or Y variable, cond is a factor of
>> either X or Y, and subj is your subject variable. This fits a heirarchical
>> linear model to the data. The relationship between X and time is sig.
>> diff.
>> from the relationship between Y and time if the cond:time fixed effect is
>> true.
>>
>> This approach makes better use of your data, because when you correlate
>> the
>> observations, you're effectively "losing" variability (because
>> correlations
>> are doubly standardized) as well as degrees of freedom (you have 9 df
>> within
>> each individual, but each correlation is only one number).
>>
>> --Adam
>>
>> On Sat, 6 Sep 2008, Ralph79 wrote:
>>
>>>
>>> Dear R-Users,
>>>
>>> I am currently looking for a way to test the equality of two correlations
>>> that are related in a very special way. Let me describe the situation
>>> with
>>> an example.
>>>
>>> - There are 100 respondents, and there are 2 points in time, t=1 and t=2.
>>>
>>> - For each of the respondents and at each of the time points, I have
>>> information on 10 X-variables and on 10 Y-variables.
>>>
>>> - Based on this information, I calculate two correlations for each
>>> respondent: cor(X[t=1],X[t=2]) and cor(Y[t=1],Y[t=2]), with X and Y being
>>> the vectors of the corresponding 10 variables.
>>>
>>> - Now I get the average correlations over the whole sample using Fishers
>>> Z-transformation, i.e. I have mean(cor(X[t=1],X[t=2])) and
>>> mean(cor(X[t=1],X[t=2])) and want to know if the mean correlations are
>>> significantly different!
>>>
>>>
>>> I haven't found any test that deals with exactly my situation. Therefore,
>>> I
>>> "simply" apply a paired t-test based on the individual z-correlations.
>>> From
>>> my point of view this should be ok, because of the z's normality.
>>> However, I
>>> am unsure if there is a better way to test the hypothesis that I am
>>> interested in?
>>>
>>> I'd be grateful for any comment or hint.
>>>
>>> Thank you very much,
>>>
>>> Ralph
>>>
>>> -----
>>> Ralph Wirth
>>> University Erlangen-Nuremberg, Chair of Statistics
>>> GfK Group, Department of Methods and Product Development
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/Test-for-equality-of-complicatedly-related-average-correlations-tp19346312p19346312.html
>>> Sent from the R help mailing list archive at Nabble.com.
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
>
>
> -----
> Ralph Wirth
> University Erlangen-Nuremberg, Chair of Statistics
> GfK Group, Department of Methods and Product Development
>
> -- 
> View this message in context: http://www.nabble.com/Test-for-equality-of-complicatedly-related-average-correlations-tp19346312p19355825.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>