[R] subject: Log likelihood above 0

[Ravi Varadhan]

Yes, of course!

So, the complete answer is:  the log-likelihood can be in (-Inf, Inf), regardless of whether the random variable is continuous or discrete or mixed.

Ravi.
____________________________________________________________________

Ravi Varadhan, Ph.D.
Assistant Professor,
Division of Geriatric Medicine and Gerontology
School of Medicine
Johns Hopkins University

Ph. (410) 502-2619
email: rvaradhan at jhmi.edu


----- Original Message -----
From: peter dalgaard <pdalgd at gmail.com>
Date: Tuesday, October 5, 2010 9:49 am
Subject: Re: [R] subject: Log likelihood above 0
To: Ravi Varadhan <rvaradhan at jhmi.edu>
Cc: Daniel Haugstvedt <daniel.haugstvedt at gmail.com>, r-help at r-project.org


>  On Oct 5, 2010, at 15:36 , Ravi Varadhan wrote:
>  
>  > Likelihood is a function of the parameters, conditioned upon the 
> data.  It is not the same as a probability density function.  Terms or 
> factors which do not involve parameters can be omitted from the 
> likelihood function.  For continuous random variables, the density 
> function can be in (0, Inf).  Therefore, the likelihood function can 
> assume any value between 0 and Inf.  Hence the log-likelihood can be 
> in (-Inf, Inf).  
>  > 
>  > When the random variable is discrete, the density or probability 
> mass function cannot be greater than 1.   Hence the likelihood cannot 
> be greater than 1, in which case, the log-likelihood cannot be positive.
>  
>  ...unless one of the above mentioned terms that do not involve 
> parameters is omitted. E.g. the Poisson likelihood is
>  
>  x log lambda - lambda - log(x!)
>  
>  and the sum of the first two terms can easily be positive.
>  
>  
>  > 
>  > Ravi.
>  > ____________________________________________________________________
>  > 
>  > Ravi Varadhan, Ph.D.
>  > Assistant Professor,
>  > Division of Geriatric Medicine and Gerontology
>  > School of Medicine
>  > Johns Hopkins University
>  > 
>  > Ph. (410) 502-2619
>  > email: rvaradhan at jhmi.edu
>  > 
>  > 
>  > ----- Original Message -----
>  > From: Daniel Haugstvedt <daniel.haugstvedt at gmail.com>
>  > Date: Tuesday, October 5, 2010 9:16 am
>  > Subject: [R] subject: Log likelihood above 0
>  > To: r-help at r-project.org
>  > 
>  > 
>  >> Hi -
>  >> 
>  >> In an effort to learn some basic arima modeling in R i went through
>  >> the tutorial found at
>  >> 
>  >> 
>  >> One of the examples gave me a log likelihood of 77. Now I am simply
>  >> wondering if this is the expected behavior? Looking in my text book
>  >> this should not be possible. I have actually spent some time on this
>  >> but neither the documentation ?arima or google gave me a satisfying
>  >> answer.
>  >> 
>  >> 
>  >> 
>  >> Data and code:
>  >> 
>  >> gTemp.raw = c(-0.11, -0.13, -0.01, -0.04, -0.42, -0.23, -0.25, -0.45,
>  >> -0.23, 0.04, -0.22, -0.55
>  >> , -0.40,  -0.39, -0.32, -0.32, -0.27, -0.15, -0.21, -0.25, -0.05,
>  >> -0.05, -0.30, -0.35
>  >> , -0.42,  -0.25, -0.15, -0.41, -0.30, -0.31, -0.21, -0.25, -0.33,
>  >> -0.28, -0.02,  0.06
>  >> , -0.20,  -0.46, -0.33, -0.09, -0.15, -0.04, -0.09, -0.16, -0.11,
>  >> -0.15,  0.04, -0.05
>  >> ,  0.01,  -0.22, -0.03,  0.03,  0.04, -0.11,  0.05, -0.08,  0.01,
>  >> 0.12,  0.15, -0.02
>  >> ,  0.14,   0.11,  0.10,  0.06,  0.10, -0.01,  0.01,  0.12, -0.03,
>  >> -0.09, -0.17, -0.02
>  >> ,  0.03,   0.12, -0.09, -0.09, -0.18,  0.08,  0.10,  0.05, -0.02,
>  >> 0.10,  0.05,  0.03
>  >> , -0.25,  -0.15, -0.07, -0.02, -0.09,  0.00,  0.04, -0.10, -0.05,
>  >> 0.18, -0.06, -0.02
>  >> , -0.21,   0.16,  0.07,  0.13,  0.27,  0.40,  0.10,  0.34,  0.16,
>  >> 0.13,  0.19,  0.35
>  >> ,  0.42,   0.28,  0.49,  0.44,  0.16,  0.18,  0.31,  0.47,  0.36,
>  >> 0.40,  0.71,  0.43
>  >> ,  0.41,   0.56,  0.70,  0.66,  0.60)
>  >> 
>  >> gTemp.ts = ts(gTemp.raw, start=1880, freq=1)
>  >> 
>  >> gTemp.model = arima(diff(gTemp.ts), order=c(1,0,1))
>  >> 
>  >> 
>  >> 
>  >> Results:
>  >> 
>  >>> gTemp.model
>  >> 
>  >> Call:
>  >> arima(x = diff(gTemp.ts), order = c(1, 0, 1))
>  >> 
>  >> Coefficients:
>  >>          ar1      ma1         intercept
>  >>        0.2695  -0.8180     0.0061
>  >> s.e.  0.1122   0.0624     0.0030
>  >> 
>  >> sigma^2 estimated as 0.01680:  log likelihood = 77.05,  aic = -146.11
>  >> 
>  >> ______________________________________________
>  >> R-help at r-project.org mailing list
>  >> 
>  >> PLEASE do read the posting guide 
>  >> and provide commented, minimal, self-contained, reproducible code.
>  > 
>  > ______________________________________________
>  > R-help at r-project.org mailing list
>  > 
>  > PLEASE do read the posting guide 
>  > and provide commented, minimal, self-contained, reproducible code.
>  
>  -- 
>  Peter Dalgaard
>  Center for Statistics, Copenhagen Business School
>  Solbjerg Plads 3, 2000 Frederiksberg, Denmark
>  Phone: (+45)38153501
>  Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com
>