[R-sig-Geo] Request for help about running a loop and reading .NC files

[Bhaskar Mitra]

Hi Ben, Rolf,

Thanks for your help.

Ben : the code works perfectly.

thanks once again,

bhaskar

On Mon, Mar 23, 2020 at 4:17 PM Ben Tupper <btupper using bigelow.org> wrote:

> Hi,
>
> Rolf is correct that you might consider enclosing your nc_open(...) with
> try(nc_open(...)), and then testing if the value returned by try() inherits
> from the "try-error" class. If there is no problem opening the NCDF file,
> then it will return an "ncdf4" class object.
>
> Also, you might isolate the file extraction stuff into a function to make
> your life easier. Something like this (obviously not tested)...
>
>
> ### START
> # extract data from a NCDF file
> # @param filename the path and name of the file
> # @return either NULL (for an exception) or a data.frame
> extract_from_nc <- function(filename){
>
>   nc <- try(nc_open(filename))
>   if (inherits(nc, "try-error")){
>     # if it isn't a healthy NCDF file you will be here
>     x <- NULL
>   } else {
>     # otherwise you will be here and you can run your extraction
>     mean1 <- ...
>      ...
>     x <- as.data.frame(mean_chl_df)
>     write.csv(x, ... other stuff here ...)
>     nc_close(nc)
>   }
>   return(x)
> }
>
> flist <- list.files(path ="NCFiles/", pattern = "^.*\\.(nc|NC|Nc|Nc)$",
> full.names = TRUE)
>
> xx <- lapply(flist, extract_from_nc)
> ix <- sapply(xx, is.null)                    # find the NULLS
> xx <- xx[!ix]                                # remove them (if any)
> if (length(xx) > 0) x <- do.call(rbind, xx)  # row bind what remains into
> one data frame (in case that is helpful)
> ### END
>
> I hope that helps.
>
> Cheers,
> Ben
>
>
>
>
>
> On Mon, Mar 23, 2020 at 5:53 PM Bhaskar Mitra <bhaskar.kolkata using gmail.com>
> wrote:
>
>> Hello Everyone,
>>
>> I have written a loop which reads hundreds of .nc files
>> and extract information from each .nc file and
>> exports that corresponding information as a csv file.
>>
>> The loop works fine until it encounters a .nc file which it cannot read
>> and the loop stops. I would appreciate if anyone can suggest how can I
>> modify
>> the loop, whereby the loop will continue to run by bypassing those
>> particular
>> files which it cannot read.
>>
>> In the end I was also hoping to modify the loop such that it
>> will generate a report which will inform me which
>> files were not read by the loop.  The codes are given below
>>
>> Thanks,
>> Bhaskar Mitra
>> NAU
>>
>> #_------------------------------------------------------------------
>>
>>
>> library(ncdf4)
>> library(reshape2)
>> library(dplyr)
>> library(stringr)
>>
>>
>> setwd("directory path")
>>
>> Output <- "directory path"
>>
>> flist <- list.files(path ="NCFiles/", pattern = "^.*\\.(nc|NC|Nc|Nc)$")
>>
>> for (i in 1: length(flist))
>> {
>>
>>   nc <- nc_open(paste0("NCFiles/",flist[i]))
>>   mean1 <- ncvar_get(nc,attributes(nc$dim)$names[3])
>>   nc_close(nc)
>>
>>   mean_chl_df <- melt(mean1)
>>   trial.table.df <-as.data.frame(mean_chl_df)
>>
>>
>> write.csv(trial.table.df,paste0(Output,"/",tools::file_path_sans_ext(flist[i]),".csv"))
>>
>> }
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> R-sig-Geo mailing list
>> R-sig-Geo using r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-geo
>>
>
>
> --
> Ben Tupper
> Bigelow Laboratory for Ocean Science
> East Boothbay, Maine
> http://www.bigelow.org/
> https://eco.bigelow.org
>
>

	[[alternative HTML version deleted]]