[R-sig-Geo] Request for help about running a loop and reading .NC files

[Ben Tupper]

Hi,

Rolf is correct that you might consider enclosing your nc_open(...) with
try(nc_open(...)), and then testing if the value returned by try() inherits
from the "try-error" class. If there is no problem opening the NCDF file,
then it will return an "ncdf4" class object.

Also, you might isolate the file extraction stuff into a function to make
your life easier. Something like this (obviously not tested)...


### START
# extract data from a NCDF file
# @param filename the path and name of the file
# @return either NULL (for an exception) or a data.frame
extract_from_nc <- function(filename){

  nc <- try(nc_open(filename))
  if (inherits(nc, "try-error")){
    # if it isn't a healthy NCDF file you will be here
    x <- NULL
  } else {
    # otherwise you will be here and you can run your extraction
    mean1 <- ...
     ...
    x <- as.data.frame(mean_chl_df)
    write.csv(x, ... other stuff here ...)
    nc_close(nc)
  }
  return(x)
}

flist <- list.files(path ="NCFiles/", pattern = "^.*\\.(nc|NC|Nc|Nc)$",
full.names = TRUE)

xx <- lapply(flist, extract_from_nc)
ix <- sapply(xx, is.null)                    # find the NULLS
xx <- xx[!ix]                                # remove them (if any)
if (length(xx) > 0) x <- do.call(rbind, xx)  # row bind what remains into
one data frame (in case that is helpful)
### END

I hope that helps.

Cheers,
Ben





On Mon, Mar 23, 2020 at 5:53 PM Bhaskar Mitra <bhaskar.kolkata using gmail.com>
wrote:

> Hello Everyone,
>
> I have written a loop which reads hundreds of .nc files
> and extract information from each .nc file and
> exports that corresponding information as a csv file.
>
> The loop works fine until it encounters a .nc file which it cannot read
> and the loop stops. I would appreciate if anyone can suggest how can I
> modify
> the loop, whereby the loop will continue to run by bypassing those
> particular
> files which it cannot read.
>
> In the end I was also hoping to modify the loop such that it
> will generate a report which will inform me which
> files were not read by the loop.  The codes are given below
>
> Thanks,
> Bhaskar Mitra
> NAU
>
> #_------------------------------------------------------------------
>
>
> library(ncdf4)
> library(reshape2)
> library(dplyr)
> library(stringr)
>
>
> setwd("directory path")
>
> Output <- "directory path"
>
> flist <- list.files(path ="NCFiles/", pattern = "^.*\\.(nc|NC|Nc|Nc)$")
>
> for (i in 1: length(flist))
> {
>
>   nc <- nc_open(paste0("NCFiles/",flist[i]))
>   mean1 <- ncvar_get(nc,attributes(nc$dim)$names[3])
>   nc_close(nc)
>
>   mean_chl_df <- melt(mean1)
>   trial.table.df <-as.data.frame(mean_chl_df)
>
>
> write.csv(trial.table.df,paste0(Output,"/",tools::file_path_sans_ext(flist[i]),".csv"))
>
> }
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> R-sig-Geo mailing list
> R-sig-Geo using r-project.org
> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>


-- 
Ben Tupper
Bigelow Laboratory for Ocean Science
East Boothbay, Maine
http://www.bigelow.org/
https://eco.bigelow.org

	[[alternative HTML version deleted]]