[R-sig-Geo] Parameter tuning of the bfastlite function

Fri Sep 27 10:45:03 CEST 2024

Please follow up on https://github.com/bfast2/bfast/issues/113.

Roger

---
Roger Bivand
Emeritus Professor
Department of Economics
Norwegian School of Economics, Bergen, Norway

________________________________
Fra: R-sig-Geo <r-sig-geo-bounces using r-project.org> p� vegne av Roger Bivand <Roger.Bivand using nhh.no>
Sendt: fredag, september 27, 2024 10:33:25 a.m.
Til: Hugo Costa <hugoagcosta using gmail.com>
Kopi: r-sig-geo <r-sig-geo using r-project.org>
Emne: Re: [R-sig-Geo] Parameter tuning of the bfastlite function

The last update was 2021-05-10; the package is available on CRAN, so no worries, but also only one small correction to documentation since then. On that basis, I'd reach out to the maintainer directly.

Roger

--
Roger Bivand
Emeritus Professor
Norwegian School of Economics
Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway
Roger.Bivand using nhh.no

________________________________________
From: Hugo Costa <hugoagcosta using gmail.com>
Sent: 27 September 2024 10:00
To: Roger Bivand
Cc: Nikolaos Tziokas; r-sig-geo
Subject: Re: [R-sig-Geo] Parameter tuning of the bfastlite function

You don't often get email from hugoagcosta using gmail.com. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
It's a pity if BFAST is not developed and maintained...
Hugo

Roger Bivand <Roger.Bivand using nhh.no<mailto:Roger.Bivand using nhh.no>> escreveu (sexta, 27/09/2024 �(s) 08:47):
Could I suggest waiting until https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbfast2%2Fbfast%2Fissues%2F113&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055257289%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=ABYk2bkfc06Mg2%2FjDRQ%2FVyF2Ip1t25i9Pk05kMD0nWg%3D&reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbfast2%2Fbfast%2Fissues%2F113&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055277013%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=70FJ6V8Tbwys%2FGgzXeEqcLGvZ4AAYOFp05l3noQobts%3D&reserved=0><https://github.com/bfast2/bfast/issues/113> which contains the same content has been attended to? Alternatively link to the list archives in the github issue to ensure that any responses to one channel do not get overlooked on the other channel (I've linked for now)? Emailing directly or github pinging the package developers may also help if this is urgent. Also consider reaching out to authors/maintainers of packages using bfast https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcran.r-project.org%2Fpackage%3Dbfast&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055287053%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=AimFwdWsVSLJPtJam5bP8pTX85fv1IdgGgeX6pSnb%2Bs%3D&reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcran.r-project.org%2Fpackage%3Dbfast&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055298284%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=P%2B%2BSDER6GodY%2BUwdIQnheP6q%2B8iHi75lccuUMDRy6tY%3D&reserved=0><https://cran.r-project.org/package=bfast>, and look (I'm sure you already have looked) among the many articles citing the underlying work. The package description mentions a forthcoming paper on BFAST Lite - this seems to be https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.3390%2Frs13163308&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055308702%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=v2NugDxN7r7kI6RWHCkCQy%2BnuNAYFv3EqvNc71wiM3M%3D&reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.3390%2Frs13163308&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055318828%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=sfeir%2BQ4PTWJ1h2JS%2F8uns5V9pNAFY4iPC2hIdtm8sc%3D&reserved=0><https://doi.org/10.3390/rs13163308>. This isn't my field, but I think that bfast isn't being developed actively, and you may need to search broadly to try to resolve your issues, so looking at how other users handle this may help.

Hope this helps,

 Roger

--
Roger Bivand
Emeritus Professor
Norwegian School of Economics
Postboks 3490 Ytre Sandviken, 5045 Bergen, Norway
Roger.Bivand using nhh.no<mailto:Roger.Bivand using nhh.no>

________________________________________
From: R-sig-Geo <r-sig-geo-bounces using r-project.org<mailto:r-sig-geo-bounces using r-project.org>> on behalf of Nikolaos Tziokas <nikos.tziokas using gmail.com<mailto:nikos.tziokas using gmail.com>>
Sent: 27 September 2024 02:19
To: r-sig-geo
Subject: [R-sig-Geo] Parameter tuning of the bfastlite function

[You don't often get email from nikos.tziokas using gmail.com<mailto:nikos.tziokas using gmail.com>. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

I am using the bfastlite() function from the BFAST package to run a
time-series analysis. From the author's paper (BFAST Lite: A Lightweight
Break Detection Method for Time Series Analysis) (table 2), I quote:

"Needs parameter tuning to optimise performance, does not differentiate
between breaks in seasonality and trend"

So far, I was fine-tuning the model manually, that is, I was changing the
parameters one by one, which is time-consuming. Does someone have a better
solution regarding the fine-tuning of the model?

To see which parameters of the model achieve the best results, I was
checking the dates in the detected breakpoints (visual inspection). I am
not sure if that method (visual inspection) is appropriate.

I apologize if this question sound a bit vague, so let me expand a little
bit. After running the bfastlite() using the default parameters (i.e., bp =
bfastlite(datats)), we get a result. Is there a way to measure (something
like rmse, or r-squared) how well the algorithm modeled the ts? What I
basically mean is that if there is an index equivalent to let's say rmse
when someone is running a linear regression. For example, what if the
parameter breaks with BIC instead of LWZ detects more accurate the
breakpoints (by visually inspecting the detected breakpoints)? Apart from
the visual inspection, shouldn't be some other way to measure the
performance of the model?

Based on the above, is there a more efficient way to optimize the
parameters of the model (based on some metric)? What do I mean by
optimizing the parameters? I think with an example I can explain it better.
When someone is tuning a random forest model, he/she can perform a full
grid search to find the optimal parameters of the model (mtry, number of
trees, etc) by searching all the possible combinations and for each
combination he/she checks the rmse (or mse, r-squared). Is this what the
authors of the paper meant when they said "Needs parameter tuning to
optimise performance"? And if so, how did they do it?

library(bfast)

plot(simts) # stl object containing simulated NDVI time series
datats <- ts(rowSums(simts$time.series))

# sum of all the components (season,abrupt,remainder)
tsp(datats) <- tsp(simts$time.series) # assign correct time series
attributes
plot(datats)

# Detect breaks. default parameters
bp = bfastlite(datats)
plot(bp)

# optimized model ??????
bp_opt <- bfastlite()

R 4.4.1, bfast 1.6.1, Windows 11.

        [[alternative HTML version deleted]]

_______________________________________________
R-sig-Geo mailing list
R-sig-Geo using r-project.org<mailto:R-sig-Geo using r-project.org>
https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-geo&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055328661%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=X%2FmG6ekEMUibqmKjwvz8UR9nhbCHdJ1z6OeXYVt1xHE%3D&reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-geo&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055338301%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=SuOLT4Vf2VMd3o1ZC6BmuYE%2FDvZ44AAAfxn15o%2FZbQ0%3D&reserved=0><https://stat.ethz.ch/mailman/listinfo/r-sig-geo>

_______________________________________________
R-sig-Geo mailing list
R-sig-Geo using r-project.org<mailto:R-sig-Geo using r-project.org>
https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-geo&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055353091%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=GrzbjPKmla%2FLhevetGcO4HIQCOKXgKE5NwctDq%2BxdY0%3D&reserved=0<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-geo&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055365368%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=KuVSz78osdOp7zKPXsjBCi%2F4zXORxLuyRMnDVVMxAfM%3D&reserved=0><https://stat.ethz.ch/mailman/listinfo/r-sig-geo>

_______________________________________________
R-sig-Geo mailing list
R-sig-Geo using r-project.org
https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-geo&data=05%7C02%7CRoger.Bivand%40nhh.no%7C57a12d2f8ad64b7a256408dcdecf0bd3%7C33a15b2f849941998d56f20b5aa91af2%7C0%7C0%7C638630228055377528%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=p5kzfUhUW0Ju78MaYLQsXfciAbzLsViEKyLIP2nluOc%3D&reserved=0<https://stat.ethz.ch/mailman/listinfo/r-sig-geo>

	[[alternative HTML version deleted]]