[R] How to pre-process fwf or csv files to remove unexpected characters in R?

[Lucas Ferreira Mation]

I have some large .txt files about ~100GB containing a dataset in fixed
width file. This contains some errors:
- character characters in column that are supposed to be numeric,
- invalid characters
- rows with too many characters, possibly due to invalid characters or some
missing end of line character (so two rows in the original data become one
row in the .txt file).

The errors are not very frequent, but stop me from importing with readr
::read_fwf()


Is there some package, or workflow, in R to pre-process the files,
separating the valid from the not-valid rows into different files? This can
be done by ETL point-click tools, such as Pentaho PDI. Is there some
equivalent code in R to do this?

I googled it and could not find a solution. I also asked this in
StackOverflow and got no answer (here
<http://stackoverflow.com/questions/39414886/fix-errors-in-csv-and-fwf-files-corrupted-characters-when-importing-to-r>
).

regards
Lucas Mation
IPEA - Brasil

	[[alternative HTML version deleted]]