[R] Error: C stack usage is too close to the limit when using list.files()

[William Dunlap]

Toss a couple of extra files in there and you will see the output grow exponentially.

% touch dir/IMPORTANT_1 dir/subdir/IMPORTANT_2

and in R those two new files cause 82 more strings to appear in list.file's output:

> nchar(list.files("dir", recursive=TRUE))
 [1]  11  18  33  40  55  62  77  84  99 106 121 128 143 150 165 172 187 194 209
[20] 216 231 238 253 260 275 282 297 304 319 326 341 348 363 370 385 392 407 414
[39] 429 436 451 458 473 480 495 502 517 524 539 546 561 568 583 590 605 612 627
[58] 634 649 656 671 678 693 700 715 722 737 744 759 766 781 788 803 810 825 832
[77] 847 854 869 876 891 898 901

'find', by default, does not following symbolic links.

% find dir
dir
dir/subdir
dir/subdir/IMPORTANT_2
dir/subdir/linkToUpperDir
dir/IMPORTANT_1

The -L option makes it follow them, but it won't follow loops:

% find -L dir
dir
dir/subdir
dir/subdir/IMPORTANT_2
find: File system loop detected; `dir/subdir/linkToUpperDir' is part of the same file system loop as `dir'.
dir/IMPORTANT_1

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com


> -----Original Message-----
> From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf
> Of William Dunlap
> Sent: Friday, September 27, 2013 12:56 PM
> To: Jonathan Greenberg; r-help
> Subject: Re: [R] Error: C stack usage is too close to the limit when using list.files()
> 
> Do you have some symbolic links that make loops in your file system?
> list.files() has problems with such loops and find does not.  E.g.,  on a Linux box:
> 
> % cd /tmp
> % mkdir dir dir/subdir
> % cd dir/subdir
> % ln -s ../../dir linkToUpperDir
> % cd /tmp
> % R --quiet
> > list.files("dir", recursive=TRUE, full=TRUE)
> [1]
> "dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToU
> pperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkT
> oUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/li
> nkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdi
> r/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/su
> bdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir
> /subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpper
> Dir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUp
> perDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkTo
> UpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir/subdir/lin
> kToUpperDir/subdir/linkToUpperDir/subdir/linkToUpperDir"
> > system("find dir")
> dir
> dir/subdir
> dir/subdir/linkToUpperDir
> 
> Bill Dunlap
> Spotfire, TIBCO Software
> wdunlap tibco.com
> 
> 
> > -----Original Message-----
> > From: r-help-bounces at r-project.org [mailto:r-help-bounces at r-project.org] On Behalf
> > Of Jonathan Greenberg
> > Sent: Friday, September 27, 2013 12:13 PM
> > To: r-help
> > Subject: [R] Error: C stack usage is too close to the limit when using list.files()
> >
> > R-helpers:
> >
> > I'm running a file search on my entire drive (Mac OS X) using:
> >
> > files_found <- list.files(dir="/",pattern=somepattern,recursive=TRUE,full.names=TRUE)
> > where somepattern is a search pattern (which I have confirmed via a
> > unix "find / -name somepattern" only returns ~ 3 results).
> >
> > I keep getting an error:
> >
> > Error: C stack usage is too close to the limit
> >
> > when running this command.  Any ideas on 1) how to fix this or 2) if
> > there is an alternative to using list.files() to accomplish this
> > search without resorting to an external package?
> >
> > Cheers!
> >
> > --jonathan
> >
> >
> > --
> > Jonathan A. Greenberg, PhD
> > Assistant Professor
> > Global Environmental Analysis and Remote Sensing (GEARS) Laboratory
> > Department of Geography and Geographic Information Science
> > University of Illinois at Urbana-Champaign
> > 259 Computing Applications Building, MC-150
> > 605 East Springfield Avenue
> > Champaign, IL  61820-6371
> > Phone: 217-300-1924
> > http://www.geog.illinois.edu/~jgrn/
> > AIM: jgrn307, MSN: jgrn307 at hotmail.com, Gchat: jgrn307, Skype: jgrn3007
> >
> > ______________________________________________
> > R-help at r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.