[Rd] Is it a good choice to increase the NCONNECTION value?

[Martin Maechler]

>>>>> qweytr1--- via R-devel 
>>>>>     on Tue, 24 Aug 2021 00:51:31 +0800 (GMT+08:00) writes:

    > At least in 2015, a github user, tobigithub, submit an
    > [issue](https://github.com/sneumann/xcms/issues/20) about
    > the error "Error in file(con, "w") : all connections are
    > in use" Nowadays, since AMD have really cool CPUs which
    > increases the thread numbers to 128 or even 256 on a
    > single server, we found that the NCONNECTIONS variable
    > could prevent us from utilizing all the 128 threads.  It
    > might be a good choice to increase its value.


    > the variable is defined in
    > `R-4.1.1/src/main/connections.c: 17` I have tested that,
    > increase it to 1024 generates no error and all the
    > clusters (I tried with 256 clusters on my 16 threads
    > Laptop) works fine.

    > Is it possible increase the size of NCONNECTION?

Yes, of course, it is possible.
The question is how much it costs  and to which number it should
be increased.

A quick look at the source connections.c --> src/R_ext/include/Connections.h
reveals that the Rconnection* <--> Rconn is a struct with about
200 chars and ca 30 int-like plus another 20 pointers .. which
would amount to a rough 400 bytes per connection.
Adding 1024-128 = 896 new ones  would then amount to increase
the R executable by about 360 kB .. all the above being rough.
So personally, I guess that's  "about ok" --
are there other things to consider?

Ideally, of course, the number of possible connections could be
increased dynamically only when needed

Martin