[R] Vectorized expression to extrapolate matrix columns with columns of another matrix

Gabor Grothendieck ggrothendieck at gmail.com
Thu May 13 01:23:18 CEST 2010

```I have managed to reduce it to two statements using the trick that the
elements of x-x are NA or 0 according to whether the corresponding
element of x is NA or not.  You could probably extend this to your
situation too.

> library(zoo)
> m <- zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
>
> r <- replace(m[,1], TRUE,
+                  na.locf(m[,1]) * m[,2] / na.locf(m[,2] + (m[,1]-m[,1])))
> cbind(V1 = r, V2 = m[,2])
V1 V2
1970-01-02 1.0  1
1970-01-03 2.0  4
1970-01-04 4.5  9
1970-01-05 8.0 16
1970-01-06 5.0 25
1970-01-07 7.2 36
1970-01-08 9.8 49

On Wed, May 12, 2010 at 4:58 PM, Abiel X Reinhart
<abiel.x.reinhart at jpmchase.com> wrote:
> Thanks Gabor, this looks like it serves my needs. I've extended the code to work with an example where we have two multicolumn zoo objects, one with the original data and another that has the growth rates.
>
> # mat1 = zoo object to extend
> # mat2 = zoo object whose growth rate is used to extend mat1
> mergeGrowth <- function(mat1, mat2)
> {
>        ix <- is.na(mat1)
>        mat1.locf <- na.locf(mat1, na.rm=F)
>        mat2.locf <- mat2
>        mat2.locf[ix] <- NA
>        mat2.locf <- na.locf(mat2.locf, na.rm=F)
>        coredata(mat1)[ix] <- coredata(mat1.locf * mat2 / mat2.locf)[ix]
>        mat1
> }
>
>
> Abiel Reinhart
>
> -----Original Message-----
> From: Gabor Grothendieck [mailto:ggrothendieck at gmail.com]
> Sent: Wednesday, May 12, 2010 12:00 PM
> To: Abiel X Reinhart
> Cc: r-help at r-project.org
> Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
>
> Yes, that is what it does.  Note that na.approx interpolates and does
> not work precisely as you discussed but its easy, does use m[,2] and
> may be good enough.   If you really do want something precisely as you
> discussed try this.  It NAs out the rows of m for which column 1 is NA
> and then uses na.locf to move the prior non-NA into it.  Then we apply
> the formula:
>
>> library(zoo)
>> m <- zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
>>
>> # mm will hold result; m.locf
>> m.locf <- mm <- m
>> ix <- is.na(mm[,1])
>> m.locf[ix,] <- NA
>> m.locf <- na.locf(m.locf)
>> mm[ix, 1] <- m.locf[ix, 1] * mm[ix,2] / m.locf[ix,2]
>> mm
>
> 1970-01-02 1.0  1
> 1970-01-03 2.0  4
> 1970-01-04 4.5  9
> 1970-01-05 8.0 16
> 1970-01-06 5.0 25
> 1970-01-07 7.2 36
> 1970-01-08 9.8 49
>
> 1970-01-02 1.0  1
> 1970-01-03 2.0  4
> 1970-01-04 4.5  9
> 1970-01-05 8.0 16
> 1970-01-06 5.0 25
> 1970-01-07 7.2 36
> 1970-01-08 9.8 49
>
>
>
>
>
> Yes, that is what it does. Please read the help file for na.approx and
> approx.  If you want something different you will have to special case
> the end values.
>
> On Wed, May 12, 2010 at 11:11 AM, Abiel X Reinhart
> <abiel.x.reinhart at jpmchase.com> wrote:
>> Gabor,
>>
>> Maybe I am doing this wrong, but rule=2 does not look like it is growing the series out, but rather just carrying the last value forward. It looks like na.approx() followed by na.locf(). For instance:
>>
>> m <- zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
>> na.approx(m[, 1], x = m[, 2], rule=2)
>>
>> 1970-01-02 1970-01-03 1970-01-04 1970-01-05 1970-01-06 1970-01-07 1970-01-08
>>       1.0        2.0        2.7        3.7        5.0        5.0        5.0
>>
>> Abiel Reinhart
>>
>> -----Original Message-----
>> From: Gabor Grothendieck [mailto:ggrothendieck at gmail.com]
>> Sent: Wednesday, May 12, 2010 10:40 AM
>> To: Abiel X Reinhart
>> Cc: r-help at r-project.org
>> Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
>>
>> Use rule = 2 as in the extrapolation examples in the na.approx help file.
>>
>> On Wed, May 12, 2010 at 10:10 AM, Abiel X Reinhart
>> <abiel.x.reinhart at jpmchase.com> wrote:
>>> Gabor,
>>>
>>> This comes close to solving my problem, but I am still left with the problem of how I can extrapolate, not just interpolate. In our example, if I define m as,
>>>
>>> m <- zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
>>>
>>>
>>> m <- zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
>>>
>>> then I will only get five values back, when I really want m[,1] to be fully extrapolated so that there are seven values. Is there a workaround?
>>>
>>> Abiel Reinhart
>>>
>>> -----Original Message-----
>>> From: Gabor Grothendieck [mailto:ggrothendieck at gmail.com]
>>> Sent: Wednesday, May 12, 2010 8:37 AM
>>> To: Abiel X Reinhart
>>> Cc: r-help at r-project.org
>>> Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
>>>
>>> Try this using the zoo package.  See ?na.approx for more and note that
>>> this functionality requires zoo 1.6-3 or later.
>>>
>>> .> m <- zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
>>>> na.approx(m[, 1], x = m[, 2])
>>> 1970-01-02 1970-01-03 1970-01-04 1970-01-05 1970-01-06 1970-01-07 1970-01-08
>>>  1.000000   2.000000   2.714286   3.714286   5.000000   5.916667   7.000000
>>>> na.approx(m[, 1])
>>> 1970-01-02 1970-01-03 1970-01-04 1970-01-05 1970-01-06 1970-01-07 1970-01-08
>>>         1          2          3          4          5          6          7
>>>
>>>
>>> On Tue, May 11, 2010 at 4:48 PM, Abiel X Reinhart
>>> <abiel.x.reinhart at jpmchase.com> wrote:
>>>> I have two identically sized matrices of data that represent time series (I am storing the data in zoo objects, but the idea should apply to any matrix of data). The time series in the second matrix extend further than in the first matrix, and I would like to use the data in matrix 2 to extrapolate the data in matrix 1. In other words, if mat1[i,j] == NA, then mat1[i,j] <- mat1[i-1, j]*mat2[i,j]/mat2[i-1,j]. Of course, before we can calculate mat1[i,j] we may need to calculate mat1[i-1,j], and that in turn may require the computation of mat1[i-2,j], etc. This could all clearly be done with loops, but I am wondering if anyone can think of a vectorized expression or other efficient method that would work.
>>>>
>>>> Thanks very much.
>>>>
>>>> Abiel Reinhart
>>>> This communication is for informational purposes only. It is not
>>>> intended as an offer or solicitation for the purchase or sale of
>>>> any financial instrument or as an official confirmation of any
>>>> transaction. All market prices, data and other information are not
>>>> warranted as to completeness or accuracy and are subject to change
>>>> necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
>>>> and affiliates.
>>>>
>>>> This transmission may contain information that is privileged,
>>>> confidential, legally privileged, and/or exempt from disclosure
>>>> under applicable law. If you are not the intended recipient, you
>>>> are hereby notified that any disclosure, copying, distribution, or
>>>> use of the information contained herein (including any reliance
>>>> thereon) is STRICTLY PROHIBITED. Although this transmission and any
>>>> attachments are believed to be free of any virus or other defect
>>>> that might affect any computer system into which it is received and
>>>> opened, it is the responsibility of the recipient to ensure that it
>>>> is virus free and no responsibility is accepted by JPMorgan Chase &
>>>> Co., its subsidiaries and affiliates, as applicable, for any loss
>>>> or damage arising in any way from its use. If you received this
>>>> transmission in error, please immediately contact the sender and
>>>> destroy the material in its entirety, whether in electronic or hard
>>>> copy format. Thank you.
>>>>
>>>> Please refer to http://www.jpmorgan.com/pages/disclosures for
>>>> disclosures relating to European legal entities.
>>>>
>>>> ______________________________________________
>>>> R-help at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-help