[R-sig-Geo] still struggling with FTP download on MODIS

[Alexandre VILLERS]

Dear all,

I'm still not sure that this list is the right place for that. So if 
those questions bother you, please give me the name of the list I should 
contact.

Here a version of the script I'm using to download MODIS data from NSIDC 
(weekly snow cover)

Tsnow <- 
"ftp://anonymous:test@n4ftl01u.ecs.nasa.gov/SAN/MOST/MOD10A2.005/"  
###server of NSIDC with directories / id and password
myfolder<-paste(tempdir(),"\\" , sep="") #the folder to download data

#####-getting the list of the folders in Tsnow (thanks to Barry Rowlingson)
resp <-getURL(Tsnow)
  items <- strsplit(resp,"\r\n")[[1]]
  folderLines <- items[substr(items,1,1)=='d']
  lastBit <- function(x){x[length(x)]}
  dates <- unlist(lapply(strsplit(folderLines," "),lastBit))
  datesK<-data.frame(dates=dates, keep=rep(NA,length(dates)))
  
##### select months for which I want the data
months<-c("01","02","03","04","05","06","10", "11", "12") #months for 
which I want the data
        for (i in months){ #loop to keep only the dates I'm interested in
              garde<-grep(datesK$dates, pattern=paste(".",i,".", sep=""))
              datesK$keep[garde]<-1
              }
datesK<-subset(datesK, datesK$keep==1) # keep the dates of interest

for (i in 1:length(datesK[,1])){    ## loop to download the tiles of 
interest
        getlist1 <- strsplit(getURL(paste(Tsnow, datesK[i,][[1]], "/", 
sep=""),.opts=curlOptions(ftplistonly=TRUE)), "\r\n")[[1]] #get the 
files in the folder i
        BLOCKtemp <- getlist1[grep(getlist1, 
pattern="MOD10A2.********.h18v04.005.*************.hdf")[1]] #select the 
tile of interest (in my case h18v04.005)
        download.file(paste(Tsnow, datesK[i,][[1]], "/", 
BLOCKtemp,sep=""), destfile=paste(myfolder, BLOCKtemp, sep=""), 
mode='wb', method='wget') #download the tile
        }

So here is my problem. I can't get into some specifc folders and receive 
the error message.
Error in curlPerform(curl = curl, .opts = opts, .encoding = .encoding) :
  server did not report OK, got 550
For example, i=4 (folder "2000.03.13"). And this is always true, 
suggesting it's not a problem of connection. The error appears as soon 
as getURL is called in the loop.
I succeed in downloading for some values of i (manually) but when 
looping, it stops on those "black" folders. I've checked directly from 
filezilla or my web browser: I can get into those folder and download 
the tiles.

The tiles are pretty small (150 to 450 Ko). So if someone could try from 
his place to see if he gets the same error message for the same value of 
i (4, 28,31, etc.).
Since I have 550 tiles to download and process, I would prefer not to do 
that manually (and I can't get to use the tool proposed by the NSIDC for 
the moment).

Thanks for the help

Alex

P.S.: my next question will deal with some projection problem !!!

-- 
Alexandre Villers
PhD Student
Team "Biodiversity"
Centre d'Etudes Biologiques de Chizé-CNRS UPR1934
79360 Beauvoir sur Niort

Phone +33 (0)5 49 09 96 13
Fax   +33 (0)5 49 09 65 26




__________ Information from ESET Mail Security, version of virus signature database 4244 (20090715) __________

The message was checked by ESET Mail Security.
http://www.eset.com