[R] Temperature Prediction Model
bartjoosen
bartjoosen at hotmail.com
Fri Oct 30 10:38:25 CET 2009
Hi,
also interested...
If you are checking for non-normal behaviour, first let us define normal
behaviour: only small temperature changes and no steep ramps?
If so, maybe you can make an rolling average of the last x points, and check
if the following point deviates more than ... ?
Or is it more like a trend analysis, to see whether the last variation is a
trend or normal variation?
Then you can take a look at change point analysis.
gl
Bart
Aneeta wrote:
>
> Thank you Clint for your response. I am happy to know that you have gotten
> interested in this analysis.:-)
>
> Let me give you some details about sensor networks that would help you
> understand my goal better. Sensor nodes run on small batteries which have
> limited life. So communication amongst various nodes is kept at a minimum
> to preserve battery life. Hence, I am working on a localized analysis
> which each node could perform on its own.
>
> The disturbances caused by the malicious nodes would be abrupt and for a
> short span. But please note that the data set that I have supplied is from
> a perfectly working sensor network which is not under any attack. The
> attack model will be simulated separately. I apologize if I have given you
> the idea that the data set consists of noisy data.
>
> Ideally I would first like to build a model for the normal behaviour of
> each node from the data set which is at hand. Once we are able to define
> the normal behaviour I could introduce some gorillas and see if we can
> detect that some node is under attack. I have this part figured out.
>
> I am not looking at a day-of-week dependence. Rather I am looking for a
> time-of-day dependence. We could take into consideration more than 7 days
> of data if that gives us a stronger model. What I was hoping to do was to
> plot the data for the past 7-days on top of each other to get a general
> idea on how the temperature varies throughout the day and thus build an
> equation which would calculate the temperature given the time of the day.
>
> Thank you again for your help.
>
> Best Regards,
> Aneeta
>
>
>
>
>
> Clint Bowman wrote:
>>
>> Aneeta,
>>
>> My "gorilla and mouse" analogies were referring to the magnitude of
>> the disturbance and also to its time signature. Are you only
>> interested in the large disturbance which is abrupt (the gorilla)?
>> Or do you also want to be able to detect the more surreptitious
>> attack which may be quite gradual (the mouse)?
>>
>> You will want to define the magnitude (and perhaps the associated
>> duration) of the smallest disturbance that would be important. I
>> would look at the entire data set to see what would be the
>> likelihood of detecting such a change given the noise in the
>> temperature data. Or alternatively, use the global analysis to
>> help define the minimum disturbance that could be detected.
>>
>> Then see what can be done with just the first 7 days of data (or
>> for matter the past 7 days regardless of when they occur).
>>
>> I applaude your goal of looking at each sensor without referring to
>> other nodes but I think I would develop the analysis by looking for
>> anomalies in one sensor's data when compared with other sensors and
>> then focusing on those periods to determine an approach for
>> detecting a disturbance.
>>
>> Because you are looking at 7 days, should we assume that you expect
>> a day-of-week dependence? If so, I'd be more comfortable if you
>> used more than one week to develop it.
>>
>> I fear that you've gotten me quite interested in this analysis,
>> good luck.
>>
>> Clint
>>
>> --
>> Clint Bowman INTERNET: clint at ecy.wa.gov
>> Air Quality Modeler INTERNET: clint at math.utah.edu
>> Department of Ecology VOICE: (360) 407-6815
>> PO Box 47600 FAX: (360) 407-7534
>> Olympia, WA 98504-7600
>>
>> On Sun, 25 Oct 2009, Aneeta wrote:
>>
>>>
>>> Thank you everyone for all the responses.
>>>
>>> Clint you are correct in assuming that the problem deals with sensors in
>>> a
>>> lab setup which can be assumed to be isolated from outside temperature
>>> changes. And, I am only dealing with temperature so the other parameters
>>> are
>>> not important.
>>>
>>> There will be no gorillas or mouses in the picture but rather some
>>> malicious
>>> attacker who would try to cause disturbances in the normal readings.
>>> That is
>>> why it is important to have an equation that defines 'normal behaviour'.
>>>
>>> The data-sets contain readings for multiple days. I want to take the
>>> first 7
>>> days for each node and establish a relationship between time(column 2)
>>> and
>>> temperature(column 4).
>>>
>>> My objective is not to model temperature variation throughout the year
>>> and
>>> take into consideration climatic changes. Rather, it is to define a
>>> model
>>> for the given data which happens to be temperature recorded by nodes. In
>>> a
>>> simple way we may look at it as a set of X(time) and Y(temperature)
>>> values
>>> where I am trying to define Y in terms of X.
>>>
>>> How should I approach this problem?
>>>
>>> Many Thanks,
>>> Aneeta
>>>
>>>
>>> Clint Bowman wrote:
>>>>
>>>> Aneeta,
>>>>
>>>> If I understand the figure at
>>>> <http://db.csail.mit.edu/labdata/labdata.html> this problem deals
>>>> with sensors in a lab that is probably isolated from outdoor
>>>> temperature changes.
>>>>
>>>> I assume the predictive model must detect when a "rampaging 800
>>>> pound gorilla" messes with a sensor. Do we also have to detect the
>>>> pawing of a "micro-mouse" as well?
>>>>
>>>> The collected data also seem to have other parameters which would
>>>> be valuable--are you limited to just temperature?
>>>>
>>>> Clint
>>>>
>>>> --
>>>> Clint Bowman INTERNET: clint at ecy.wa.gov
>>>> Air Quality Modeler INTERNET: clint at math.utah.edu
>>>> Department of Ecology VOICE: (360) 407-6815
>>>> PO Box 47600 FAX: (360) 407-7534
>>>> Olympia, WA 98504-7600
>>>>
>>>> On Thu, 22 Oct 2009, Thomas Adams wrote:
>>>>
>>>>> Aneeta,
>>>>>
>>>>> You will have to have a seasonal component built into your model,
>>>>> because
>>>>> the
>>>>> seasonal variation does matter, particularly -where- you are
>>>>> geographically
>>>>> (San Diego, Chicago, Denver, Miami are very different). Generally,
>>>>> there
>>>>> is a
>>>>> sinusoidal daily temperature variation, but frontal passages and
>>>>> thunderstorms, etc., can and will disrupt this nice pattern. You may
>>>>> have
>>>>> to
>>>>> tie this into temperature predictions from a mesoscale numerical
>>>>> weather
>>>>> prediction model. Otherwise, you will end up with lots of misses and
>>>>> false
>>>>> alarms…
>>>>>
>>>>> Regards,
>>>>> Tom
>>>>>
>>>>> Aneeta wrote:
>>>>>> The data that I use has been collected by a sensor network deployed
>>>>>> by
>>>>>> Intel.
>>>>>> You may take a look at the network at the following website
>>>>>> http://db.csail.mit.edu/labdata/labdata.html
>>>>>>
>>>>>> The main goal of my project is to simulate a physical layer attack
>>>>>> on a
>>>>>> sensor network and to detect such an attack. In order to detect an
>>>>>> attack
>>>>>> I
>>>>>> need to have a model that would define the normal behaviour. So the
>>>>>> actual
>>>>>> variation of temperature throughout the year is not very important
>>>>>> out
>>>>>> here.
>>>>>> I have a set of data for a period of 7 days which is assumed to be
>>>>>> the
>>>>>> correct behaviour and I need to build a model upon that data. I may
>>>>>> refine
>>>>>> the model later on to take into account temperature variations
>>>>>> throughout
>>>>>> the year.
>>>>>>
>>>>>> Yes I am trying to build a model that will predict the temperature
>>>>>> just
>>>>>> on
>>>>>> the given time of the day so that I am able to compare it with the
>>>>>> observed
>>>>>> temperature and determine if there is any abnormality. Each node
>>>>>> should
>>>>>> have
>>>>>> its own expectation model (i.e. there will be no correlation between
>>>>>> the
>>>>>> readings of the different nodes).
>>>>>>
>>>>>>
>>>>>> Steve Lianoglou-6 wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> On Oct 21, 2009, at 12:31 PM, Aneeta wrote:
>>>>>>>
>>>>>>>
>>>>>>>> Greetings!
>>>>>>>>
>>>>>>>> As part of my research project I am using R to study temperature
>>>>>> data
>>>>>>>> collected by a network. Each node (observation point) records
>>>>>>>> temperature of
>>>>>>>> its surroundings throughout the day and generates a dataset. Using
>>>>>> the
>>>>>>>> recorded datasets for the past 7 days I need to build a prediction
>>>>>>>> model for
>>>>>>>> each node that would enable it to check the observed data against
>>>>>> the
>>>>>>>> predicted data. How can I derive an equation for temperature using
>>>>>> the
>>>>>>>> datasets?
>>>>>>>> The following is a subset of one of the datasets:-
>>>>>>>>
>>>>>>>> Time Temperature
>>>>>>>>
>>>>>>>> 07:00:17.369668 17.509
>>>>>>>> 07:03:17.465725 17.509
>>>>>>>> 07:04:17.597071 17.509
>>>>>>>> 07:05:17.330544 17.509
>>>>>>>> 07:10:47.838123 17.5482
>>>>>>>> 07:14:16.680696 17.5874
>>>>>>>> 07:16:46.67457 17.5972
>>>>>>>> 07:29:16.887654 17.7442
>>>>>>>> 07:29:46.705759 17.754
>>>>>>>> 07:32:17.131713 17.7932
>>>>>>>> 07:35:47.113953 17.8324
>>>>>>>> 07:36:17.194981 17.8324
>>>>>>>> 07:37:17.227013 17.852
>>>>>>>> 07:38:17.809174 17.8618
>>>>>>>> 07:38:48.00011 17.852
>>>>>>>> 07:39:17.124362 17.8618
>>>>>>>> 07:41:17.130624 17.8912
>>>>>>>> 07:41:46.966421 17.901
>>>>>>>> 07:43:47.524823 17.95
>>>>>>>> 07:44:47.430977 17.95
>>>>>>>> 07:45:16.813396 17.95
>>>>>>>>
>>>>>>> I think you/we need much more information.
>>>>>>>
>>>>>>> Are you really trying to build a model that predicts the
>>>>>>> temperature
>>>>>>> just given the time of day?
>>>>>>>
>>>>>>> Given that you're in NY, I'd say 12pm in August sure feels much
>>>>>>> different than 12pm in February, no?
>>>>>>>
>>>>>>> Or are you trying to predict what one sensor readout would be at a
>>>>>>> particular time given readings from other sensors at the same time?
>>>>>>>
>>>>>>> Or ... ?
>>>>>>>
>>>>>>> -steve
>>>>>>>
>>>>>>> --
>>>>>>> Steve Lianoglou
>>>>>>> Graduate Student: Computational Systems Biology
>>>>>>> | Memorial Sloan-Kettering Cancer Center
>>>>>>> | Weill Medical College of Cornell University
>>>>>>> Contact Info: http://cbio.mskcc.org/~lianos/contact
>>>>>>>
>>>>>>> ______________________________________________
>>>>>>> R-help at r-project.org mailing list
>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>>>>> PLEASE do read the posting guide
>>>>>>> http://www.R-project.org/posting-guide.html
>>>>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>> ______________________________________________
>>>> R-help at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>>> PLEASE do read the posting guide
>>>> http://www.R-project.org/posting-guide.html
>>>> and provide commented, minimal, self-contained, reproducible code.
>>>>
>>>>
>>>
>>>
>> ______________________________________________
>> R-help at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
>
>
--
View this message in context: http://old.nabble.com/Temperature-Prediction-Model-tp25995874p26127240.html
Sent from the R help mailing list archive at Nabble.com.
More information about the R-help
mailing list