[R] discrete entropy is not rotation invariant?

context grey mobygeek at yahoo.com
Mon Mar 20 02:58:58 CET 2006


suppose one is forming a probability p(x,y), where the
axes are somewhat accidental and rotation is possible.

I'm thinking about whether the discrete entropy H(x,y)
should change
if the probability is rotated in the x,y plane.  

My current conclusion is that it _does_ change, at
least if the
entropy is estimated via bins.    As a simple example,
the probability mass is concentrated in a single bin
(that is not
close to the center of rotation), but it is not a true
delta function,
merely concentrated.

After rotation, the probability mass will often be
spread into two bins,
simply because it does not exactly align into one bin.
 This clearly
changes the estimated entropy.

My question:  should one regard this as an artifact of
the binning,
or is it correct to consider that the entropy changes?

More information about the R-help mailing list