[R] optimize linear function

[Prof Brian Ripley]

This is just a linear programming problem.  So the packages which do 
linear programming are `particularly well suited to this sort of task'
and theory tells you a lot about the solution.

Resarching how L_1 regression is done (that is that sum of the absolute 
values of the errors) should give you a lot of help.  This exact 
problem is a special case of one formulation of support vector machines.

On Sat, 12 Jun 2004, Michaell Taylor wrote:

> I am attempting to optimize a regression model's parameters to meet a specific 
> target for the sum of positive errors over sum of the dependent variable 
> (minErr below).  

The sum of the dependent variable is a constant so can be ignored.

> I see two courses of action , 1) estimate a linear model then iteratively 
> reduce the regressors to achieve the desired positive error threshold 
> (naturally the regressors and predicted values are biased - but this is 
> acceptable).  Were the problem only a single independent variable problem, 
> this approach would be fairly trivial.  But the two variable model proves 
> more troublesome from an efficiency point of view (efficiency being the mean 
> negative error for a given minErr).
> 
> The second method, which seems more efficient, is to optimize the linear 
> regressors until the criteria is met.  There are several optimizer packages 
> out there, but none seem particularly well suited to this sort of task.  
> 
> Any suggestions out there?
> 
> ===========  Sample Problem = first approach  ================
> 
> # if we say... 
> B1 <- .8
> B2 <- .2
> 
> # then construct some data..
> x1 <- rnorm(100,mean=100,sd=20)
> x2 <- rnorm(100,mean=10,sd=2)
> y <- B1*x1+B2*x2+rnorm(100,mean=5,sd=10)
> # normally of course B1 and B2 are unknown to begin with
> 
> # linear model  (presumably yielding B1=.8 and B2=.2)
> m<-lm(y ~ x1+ x2)
> # test on summation of positive errors.
> e <- resid(m)
> minErr <- (sum(ifelse(e<0,0,e))/sum(y))-.03
> while (minErr>.03){
> 	Make some adjustment to B1 and B2
> 	minErr <- (sum(ifelse(e<0,0,e))/sum(y))-.03
> 	}
> 
> ======== Second Approach ===========
> 
> construct a function to minimize minErr with beginning values from 'm'.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595