[R] Corrupting files while copying (was Re: saveRDS() and readRDS() Why? [solved, pretty much anyway])

[Patrick Connolly]

Sequence of steps:

Using R-3.5.1 and Windows 7 with the latest Rstudio, shared directory
as working directory on Virtual Box host machine:

> x <- airquality
> saveRDS(x, file = "x.rds")
> saveRDS(x, file = "y.rds")

On guest machine Mint Linux 17.3, KDE desktop, copy x.rds & y.rds to
working directory PWD using file manager Dolphin.

(Don't have the precise version of VirtualBox right now.)

> x <- readRDS(file =  "x.rds")
Error in readRDS(file = "x.rds") : error reading from connection
> x <- readRDS(file =  "y.rds")

>   tools::md5sum(c("x.rds", "y.rds"))
                             x.rds                              y.rds
"5fef054848f39b4be02b7c54f1c71a20" "978a64d1dd342d16a381c9ca728d3665"

Yet, if instead of using Dolphin, use bash commands from the shared
directory 

$ cp *.rds ~/PWD/

no error reading from the connection or other differences between
x.rds and y.rds.

head(x)

> head(datasets::airquality)
  Ozone Solar.R Wind Temp Month Day
1    41     190  7.4   67     5   1
2    36     118  8.0   72     5   2
3    12     149 12.6   74     5   3
4    18     313 11.5   62     5   4
5    NA      NA 14.3   56     5   5
6    28      NA 14.9   66     5   6
> 

On Thu, 15-Nov-2018 at 09:53AM -0500, Ista Zahn wrote:

|> Hi Patrick,
|> 
|> I think it would help to start from the beginning and give complete
|> (but concise!) replication instructions, including telling us what
|> host and gest operating systems you are using (including the
|> versions), the version
|>  of virtualbox you used, and exactly what steps are needed to
|> reproduce the surprising behavior.
|> 
|> Best,
|> Ista
|> On Wed, Nov 14, 2018 at 2:36 AM Patrick Connolly
|> <p_connolly using slingshot.co.nz> wrote:
|> >
|> > Thanks William,
|> >
|> > I've used Dolphin for years and never encountered that phenomenon.
|> > Even so, that description doesn't fit what's going on here.  1.7
|> > kilobytes is hardly a 'large directory'.
|> >
|> > The problem seems to be with the way VirtualBox mounts directories
|> > which isn't an R issue, nor is the fact that copying from Linux to
|> > Windows isn't affected.  But the fact that it happens only with rds
|> > files that use the name of the R object as part of their own names
|> > must be an R issue (that surfaces only when other conditions are
|> > present).
|> >
|> > Theories short of divine intervention appreciated.
|> >
|> >
|> >
|> > On Tue, 13-Nov-2018 at 02:22PM -0800, William Dunlap wrote:
|> >
|> > |> Perhaps you got bitten by Dolphin's non-modal dialogs, as described in
|> > |> https://userbase.kde.org/Dolphin/File_Management:
|> > |>
|> > |> Non Modal Dialogs
|> > |>
|> > |> When Moving, Copying or Deleting files/directories the dialog disappears
|> > |> even when the operation has not yet completed. A progress bar then appears
|> > |> in the bottom right of the screen, this then disappears also, if you want
|> > |> see the progress you need to click a small (i) information icon in the
|> > |> system tray.
|> > |>
|> > |>
|> > |> Warning
|> > |> New users who are not used to this way of working (and even experienced
|> > |> users) can get caught out by this, if you are Moving, Copying or Deleting
|> > |> large directories then you need to use the icon to monitor the progress of
|> > |> your operation. If you don't then any subsequent actions you do, may well
|> > |> use an incomplete file structure resulting in corrupted files. You have
|> > |> been warned!
|> > |>
|> > |> Bill Dunlap
|> > |> TIBCO Software
|> > |> wdunlap tibco.com
|> > |>
|> > |> On Tue, Nov 13, 2018 at 2:10 PM, p_connolly <p_connolly using slingshot.co.nz>
|> > |> wrote:
|> > |>
|> > |> > This is getting more strange.
|> > |> >
|> > |> > I normally copy from the shared folder to the appropriate directory using
|> > |> > Dolphin, the KDE file manager.  If instead I use the standard bash cp
|> > |> > command, no corruption happens -- at least with the limited testing I have
|> > |> > done.  There also seems to be no problem copying from Linux to Windows.  I
|> > |> > installed R-3.5.1 for Windows just to eliminate that possible issue.
|> > |> >
|> > |> > However, R has *something* to do with it because it was used to make the
|> > |> > .rds file.  Just how the relationship between the name of the R object and
|> > |> > the name of the .rds file comes into it, I can't imagine.
|> > |> >
|> > |> > Thanks for the suggestion William.
|> > |> >
|> > |> >
|> > |> > On 2018-11-14 06:26, William Dunlap wrote:
|> > |> >
|> > |> >> It seems like copying the files corrupted them. How did you copy them
|> > |> >> (with R
|> > |> >> or cp or copy or ftp, etc.)?  I don't see how this has anything to do
|> > |> >> with R.
|> > |> >>
|> > |> >> Bill Dunlap
|> > |> >> TIBCO Software
|> > |> >> wdunlap tibco.com [1]
|> > |> >> On Mon, Nov 12, 2018 at 7:10 PM, p_connolly
|> > |> >> <p_connolly using slingshot.co.nz> wrote:
|> > |> >>
|> > |> > [...]
|> > |> >
|> > |> >
|> >
|> > --
|> > ~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.
|> >    ___    Patrick Connolly
|> >  {~._.~}                   Great minds discuss ideas
|> >  _( Y )_                 Average minds discuss events
|> > (:_~*~_:)                  Small minds discuss people
|> >  (_)-(_)                              ..... Eleanor Roosevelt
|> >
|> > ~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.
|> >
|> > ______________________________________________
|> > R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
|> > https://stat.ethz.ch/mailman/listinfo/r-help
|> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
|> > and provide commented, minimal, self-contained, reproducible code.

-- 
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.   
   ___    Patrick Connolly   
 {~._.~}                   Great minds discuss ideas    
 _( Y )_  	         Average minds discuss events 
(:_~*~_:)                  Small minds discuss people  
 (_)-(_)  	                      ..... Eleanor Roosevelt
	  
~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.~.