# [R] optimize linear function

Prof Brian Ripley ripley at stats.ox.ac.uk
Sat Jun 12 23:35:07 CEST 2004

```This is just a linear programming problem.  So the packages which do
linear programming are `particularly well suited to this sort of task'
and theory tells you a lot about the solution.

Resarching how L_1 regression is done (that is that sum of the absolute
values of the errors) should give you a lot of help.  This exact
problem is a special case of one formulation of support vector machines.

On Sat, 12 Jun 2004, Michaell Taylor wrote:

> I am attempting to optimize a regression model's parameters to meet a specific
> target for the sum of positive errors over sum of the dependent variable
> (minErr below).

The sum of the dependent variable is a constant so can be ignored.

> I see two courses of action , 1) estimate a linear model then iteratively
> reduce the regressors to achieve the desired positive error threshold
> (naturally the regressors and predicted values are biased - but this is
> acceptable).  Were the problem only a single independent variable problem,
> this approach would be fairly trivial.  But the two variable model proves
> more troublesome from an efficiency point of view (efficiency being the mean
> negative error for a given minErr).
>
> The second method, which seems more efficient, is to optimize the linear
> regressors until the criteria is met.  There are several optimizer packages
> out there, but none seem particularly well suited to this sort of task.
>
> Any suggestions out there?
>
> ===========  Sample Problem = first approach  ================
>
> # if we say...
> B1 <- .8
> B2 <- .2
>
> # then construct some data..
> x1 <- rnorm(100,mean=100,sd=20)
> x2 <- rnorm(100,mean=10,sd=2)
> y <- B1*x1+B2*x2+rnorm(100,mean=5,sd=10)
> # normally of course B1 and B2 are unknown to begin with
>
> # linear model  (presumably yielding B1=.8 and B2=.2)
> m<-lm(y ~ x1+ x2)
> # test on summation of positive errors.
> e <- resid(m)
> minErr <- (sum(ifelse(e<0,0,e))/sum(y))-.03
> while (minErr>.03){
> 	Make some adjustment to B1 and B2
> 	minErr <- (sum(ifelse(e<0,0,e))/sum(y))-.03
> 	}
>
> ======== Second Approach ===========
>
> construct a function to minimize minErr with beginning values from 'm'.

--
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

```