[R] NFFT on a Zoo?

[Bob Cunningham]

FWIW:  It turns out I dove into a rabbit hole:

1. Though the gaps in my 3-axis accelerometer data represent 10% data 
loss (OMG!), the number of gaps represents only 0.1% of the 3 million 
data points (BFD).

2. The data is noisy enough that 0.1% discontinuity can't affect an 
FFT.  Each gap was removed simply by adjusting subsequent timestamps.

3. With the gaps removed, the remaining jitter in the timestamps is both 
small and nearly normally distributed (no systematic errors).  So the 
timestamps were eliminated from further processing, and the mean 
inter-sample time was used as the sampling period.

So, neither NFFT nor Zoo are needed, since a regular FFT now works just 
fine.

The Moral of the Story is: "Take a closer look at the data before 
deciding difficult processing is needed."

Homer Simpson translation: "Doh!"

A big "Thanks!" to all who responded to my newbie posts:  The R 
Community is richly blessed with wisdom, kindness and patience.

-BobC



On 11/03/2010 01:22 PM, Mike Marchywka wrote:
> ----------------------------------------
>    
>> From: ggrothendieck at gmail.com
>> Date: Wed, 3 Nov 2010 15:27:13 -0400
>> To: FlyMyPG at gmail.com
>> CC: r-help at r-project.org; rpy-list at lists.sourceforge.net
>> Subject: Re: [R] NFFT on a Zoo?
>>
>> On Wed, Nov 3, 2010 at 2:59 PM, Bob Cunningham  wrote:
>>      
>>> I have an irregular time series in a Zoo object, and I've been unable to
>>> find any way to do an FFT on it.  More precisely, I'd like to do an NFFT
>>> (non-equispaced / non-uniform time FFT) on the data.
>>>
>>> The data is timestamped samples from a cheap self-logging accelerometer.
>>>   The data is weakly regular, with the following characteristics:
>>> - short gaps every ~20ms
>>> - large gaps every ~200ms
>>> - jitter/noise in the timestamp
>>>
>>> The gaps cover ~10% of the acquisition time.  And they occur often enough
>>> that the uninterrupted portions of the data are too short to yield useful
>>> individual FFT results, even without timestamp noise.
>>>
>>> My searches have revealed no NFFT support in R, but I'm hoping it may be
>>> known under some other name (just as non-uniform time series are known as
>>> 'zoo' rather than 'nts' or 'nuts').
>>>
>>> I'm using R through RPy, so any solution that makes use of numpy/scipy would
>>> also work.  And I care more about accuracy than speed, so a non-library
>>> solution in R or Python would also work.
>>>
>>> Alternatively, is there a technique by which multiple FFTs over smaller
>>> (incomplete) data regions may be combined to yield an improved view of the
>>> whole?  My experiments have so far yielded only useless results, but I'm
>>> getting ready to try PCA across the set of partial FFTs.
>>>
>>>        
>>      
>
> I'm pretty sure all of this is in Oppenheim and Shaffer meaning it
> is also in any newer books. I recall something about averaging
> but you'd need to look at details. Alternatively, and this is from
> distant memory so maybe someone else can comment, you can just
> feed a regularly spaced time series to anyone, go get FFTW for example,
> and insert zeroes for missing data. This is equivalent to multiplying
> your real data with a window function that is zero at missing points.
> I think you can prove that multiplication
> in time domain is convolution in FT domain so you can back this out
> by deconvolving with your window function spectrum. This probably is not
> painless, the window spectrum will have badly placed zeroes etc, but it
> may be helpful.
> Apaprently this is still a bit of an open issue,
>
> http://books.google.com/books?id=BW1PdOqZo6AC&pg=PA2&lpg=PA2&dq=dft+window+missing+data&source=bl&ots=fSY-iRoCNN&sig=30cC0SdkrDcp62iWc-Mv26mfNjI&hl=en&ei=AMTRTNmyMYP88AauxtzKDA&sa=X&oi=book_result&ct=result&resnum=6&ved=0CDEQ6AEwBTgK#v=onepage&q&f=false
>
>
>
> You should be able to do the case of a sine wave with pencil and paper
> and see if or how this really would work.
>
>
>    
>> Check out the entire thread that starts here.
>>
>> http://www.mail-archive.com/r-help@r-project.org/msg36349.html
>>
>> --
>> Statistics&  Software Consulting
>> GKX Group, GKX Associates Inc.
>> tel: 1-877-GKX-GROUP
>> email: ggrothendieck at gmail.com
>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>      
>   		 	   		
>