[R] Read fst files

[Jeff Reichman]

Bill

 

So I understand that’s just unzipping the file to a temporary dir which then would allow read_fst to access the file directly .

 

Jeff

 

From: Bill Dunlap <williamwdunlap using gmail.com> 
Sent: Wednesday, June 9, 2021 1:43 PM
To: reichmanj using sbcglobal.net
Cc: Jan van der Laan <rhelp using eoos.dds.nl>; r-help using r-project.org
Subject: Re: [R] Read fst files

 

Try using unzip(zipfile, files="desiredFile", exdir=tf<-tempfile()), not unz(zipfile, "desiredFile"), to copy the desired file from the zip file to a temporary location and use read_fst(tf) to read the desired file.

 

-Bill

 

On Wed, Jun 9, 2021 at 11:27 AM Jeff Reichman <reichmanj using sbcglobal.net <mailto:reichmanj using sbcglobal.net> > wrote:

Jan

Makes sense. Its just that I often receive  large zip files that contain a variety of file types.

Jef

-----Original Message-----
From: R-help <r-help-bounces using r-project.org <mailto:r-help-bounces using r-project.org> > On Behalf Of Jan van der Laan
Sent: Wednesday, June 9, 2021 12:56 PM
To: r-help using r-project.org <mailto:r-help using r-project.org> 
Subject: Re: [R] Read fst files



read_fst is from the package fst. The fileformat fst uses is a binary 
format designed to be fast readable. It is a column  oriented format and 
compressed. So, to be able to work fst needs access to the file itself 
and wont accept a file connection as functions like read.table an 
variants accept.

Also, because it is a binary compressed format using a compression 
method that is fast to read, compressing also to zip seems to defeat the 
purpose of fst.

HTH,
Jan


On 09-06-2021 15:28, Duncan Murdoch wrote:
> On 09/06/2021 9:12 a.m., Jeff Reichman wrote:
>> Duncan
>>
>> Yea that will work. It appears to be related to setting my working 
>> dir, for what ever reason neither seem to work
>> (1) knitr::opts_knit$set(root.dir 
>> ="~/My_Reference_Library/Regression") # from R Notebook or
>> (2) 
>> setwd("C:/Users/reichmaj/Documents/My_Reference_Library/Regression") # 
>> from R chunk
>>
>> So it appears I can either (as you suggested) use two steps or combine 
>> but I need to enter the full path. Why other file types don't seem to 
>> need the full path ....?????
> 
> You need to read the documentation for read_fst() to find what it needs. 
>   If it doesn't explain this, then you should report the issue to its 
> author.
> 
>>
>> myObject <- 
>> read_fst(unz("C:/Users/reichmaj/Documents/My_Reference_Library/Regression/Datasest.zip", 
>> filename = "myFile.fst"))
>>
>> Thank you. I guess just one of those R things
> 
> No, it's a read_fst() thing.
> 
> Duncan Murdoch
> 
> ______________________________________________
> R-help using r-project.org <mailto:R-help using r-project.org>  mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide 
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
R-help using r-project.org <mailto:R-help using r-project.org>  mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

______________________________________________
R-help using r-project.org <mailto:R-help using r-project.org>  mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


	[[alternative HTML version deleted]]