[R] R: nnet questions

[Clark Allan]

hi all

i'm new to the area of neural networks. i've been reading some
references and seem to understand some of the learning algorithms. i am
very familiar with regression and would just like to see how neural nets
handle this problem so i've been using the nnet package.

i simply want to use a 3 layer neural net, ie 1 input, 1 hidden layer
(where the hidden layer is linear, since i want to basically perform
regression analysis by means of neural nets) and 1 output layer.

the x and y vector was simulated as follows:
x<-1:100
x
  [1]   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16 
17  18
 [19]  19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34 
35  36
 [37]  37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52 
53  54
 [55]  55  56  57  58  59  60  61  62  63  64  65  66  67  68  69  70 
71  72
 [73]  73  74  75  76  77  78  79  80  81  82  83  84  85  86  87  88 
89  90
 [91]  91  92  93  94  95  96  97  98  99 100

y<-2+5*x+rnorm(100)*5

y
  [1]   8.789605  11.151109  14.622276  30.381379  19.328647  29.317038 
33.793720  39.557390  51.939294  45.045965
 [11]  58.783991  63.191745  72.882202  79.184778  85.034551  94.446000 
89.243004  88.223547 106.327683 104.424668
 [21] 103.057648 112.855778 111.777823 108.359485 128.956152 127.369102
128.784481 139.760279 151.959887 152.014623
 [31] 158.869586 167.030970 166.711160 177.415680 173.542293 182.484224
179.767128 192.284343 196.173830 202.353030
 [41] 220.449623 213.410307 216.746041 219.812526 230.440402 230.759429
239.311279 244.151390 248.637023 254.648298
 [51] 262.694237 253.619539 276.975714 280.395284 280.173787 286.813617
284.766870 296.705692 295.110064 304.709464
 [61] 305.650793 310.128992 314.035624 314.649213 322.958865 333.640203
342.538307 340.546359 342.433629 344.720633
 [71] 354.115051 363.631246 371.479886 367.066764 377.184512 386.634677
392.310577 386.151325 400.345393 408.831710
 [81] 413.999148 405.009358 418.679828 418.388427 419.282955 432.329471
433.448313 444.166060 447.773185 455.103503
 [91] 448.588598 464.410358 465.565875 478.677403 478.306390 479.565728
487.681689 491.422090 502.468491 500.385458


i then went about to use the nnet function:

> a.net<-nnet(y~x,linout=T,size=1)
# weights:  4
initial  value 8587786.130082 
iter  10 value 2086583.979646
final  value 2079743.529111 
converged

NOTE: the function said that four weights were estimated. This is
correct as shown below. the model can be represented as:

input   ---(w1)--->    hidden   ---(w2)--->   output

x       ---------->    a1+w1*x  ---------->   a2+w2*(a1+w1*x) 


where:
wi	are the weights, i=1,2
x	is an input pattern


further results were:

summary(a.net)
a 1-1-1 network with 4 weights
options were - linear output units 
  b->h1  i1->h1 
-276.48 -295.11 
   b->o   h1->o 
 254.92  764.72 


is the following statement correct? (i think that it is!)
a1=	"b->h1"
w1=	"i1->h1"
a2=	"b->o"
w2=	"h1->o"


If the hidden layer and the output layers are both LINEAR then the
following should be true:
1.	2= a2+a1*w2
2.	5= w1*w2

THIS IS NOT THE CASE, see the results. The only thing that i can think
of thats happening is that the activation function from the hidden layer
is not linear. Is this correct since i used the linout=T arguement? are
we able to change the activation function from the hidden layer?



two other questions:
1.	with regard to the size arguement, how does one know how many nodes
are in the hidden layer? (this might be a silly question.) e.g we 	might
have 2 hidden layers both with 3 nodes.
2.	are we able to plot the drop in the error function as a function of
the epochs?


hope someone can help

thanking you!!!