You can do something like this: count the number of fields in each line of
the file and use the max to determine the number of columns for read.table:

file <- '/tempxx.txt'
maxFields <- max(count.fields(file))  # max
# now setup read.table for max number
input <- read.table(file, colClasses=rep(NA, maxFields), fill=TRUE,
    col.names=paste("V", seq(maxFields), sep=''))


On Sun, May 31, 2009 at 6:06 AM, Martin Tomko <martin.tomko@geo.uzh.ch>wrote:

> Dear Jim,
> with the help of Ted, we diagnosed that the cause is in the extreme
> variability in line length during reading in. As the table column number is
> apparently determined fro mthe first five lines, what exceeds this length
> gets automatically on the next line.
> I am now trying to find a way to read in the data despite this. I have no
> control over the table extent, the only thing that would make sense
> according to my data would be to read in a fixed number of columns and merge
> all remaining columns as a long string in the last one. No idea how to do
> this, though.
>
> Thanks
> Martin
>
>
> jim holtman wrote:
>
>> It is still not clear to me exactly how you want to read the lines in.  If
>> the lines have a variable number of fields, and some of the lines might be
>> wrapped, is there some way to determine where the start of each line is.
>>  If you are reading them in with read.csv, then the system is assuming
>> that each line starts a new row.  If this is not the case, then you will
>> have to state the rules that determine where the lines start.  You can
>> always read the data in with 'scan' to separate each line and then do
>> whatever processing is required to put together the rows in a data frame
>> that you want.
>>  In one of your examples, you indicated that the line was split starting
>> at the word "kempten"; if this is in the middle of the line, then you would
>> have to create the break after reading the line in with 'scan' and then
>> creating the rows in the dataframe.  All of this can be done in R if you can
>> state what the criteria is.
>> On Sat, May 30, 2009 at 4:32 AM, Martin Tomko <martin.tomko@geo.uzh.ch<mailto:
>> martin.tomko@geo.uzh.ch>> wrote:
>>
>>    Jim,
>>    the two lines I put in are the actual problematic input lines.
>>    In these examples, there are no quotes nor # signs, although I
>>    have no means to make sure they do not occur in the inputs (any
>>    hints how I could deal with that?).
>>    I am trying to avoid as much pre-processing outside R as possible,
>>    and I have to process about 500 files with up to 3000 records
>>    each, so I need a more or less automated/batch solution. - so any
>>    string substitution will have to occur in R. But for the moment, I
>>    do not see a reaason for substitution, and the wrapping still occurs.
>>
>>    Cheers
>>    Martin
>>
>>
>>
>>    jim holtman wrote:
>>
>>        You need to supply the actual input line so we can see what is
>>        happening.  Are you sure you do not have unbalanced quotes in
>>        your input (try quote='') or do you have comment characters
>>        ("#") in your input?
>>
>>        On Fri, May 29, 2009 at 3:15 PM, Martin Tomko
>>        <martin.tomko@geo.uzh.ch <mailto:martin.tomko@geo.uzh.ch>
>>        <mailto:martin.tomko@geo.uzh.ch
>>        <mailto:martin.tomko@geo.uzh.ch>>> wrote:
>>
>>           Dear All,
>>           I am observing a strange behavior and searching the
>>        archives and
>>           help pages didn't help much.
>>           I have a csv with a variable number of fields in each line.
>>
>>           I use
>>           dataPoints <- read.csv(inputFile, head=FALSE, sep=";",fill
>>        =TRUE);
>>
>>           to read it in, and it works. But - some lines are long and
>>        'wrap',
>>           or split and continue on the next line. So when I check the
>>        dim of
>>           the frame, they are not correct and I can see when I do a
>>        printout
>>           that the lines is split into two in the frame. I checked
>>        the input
>>           file and all is good.
>>
>>           an example of the input is:
>>                 37;2175168475;13;8.522729;47.19537;16366682@N00
>> ;30;sculpture;bird;tourism;animal;statue;canon;eos;rebel;schweiz;switzerland;eagle;swiss;adler;skulptur;zug;1750;28;tamron;f28;canton;tourismus;vogel;baar;kanton;xti;tamron1750;1750mm;tamron1750mm;400d;rabbitriotnet;
>>
>>           where the last values occurs on the next line in the data
>>        frame.
>>
>>           It does not have to be the last value, as in the follwong
>>        example,
>>           the word "kempten" starts the next line:
>>                 39;167757703;12;10.309295;47.724545;21903142@N00
>> ;36;white;building;tower;clock;clouds;germany;bayern;deutschland;bavaria;europa;europe;eagle;adler;eu;wolke;dome;townhall;rathaus;turm;weiss;allemagne;europeanunion;bundesrepublik;gebaeude;glocke;brd;allgau;kuppel;europ;kempten;niemcy;europo;federalrepublic;europaischeunion;europaeischeunion;germanio;
>>
>>           What could be the reason?
>>
>>           I ws thinking about solving the issue by using a different
>>           separator, that I would use for the first 7 fields and
>>           concatenating all of the remaining values into a single stirng
>>           value, but could not figure out how to do such a
>>        substitution in
>>           R. Unfortunately, on my system I cannot specify a range for
>>        sed...
>>
>>           Thanks for any help/pointers
>>           Martin
>>
>>           ______________________________________________
>>           R-help@r-project.org <mailto:R-help@r-project.org>
>>        <mailto:R-help@r-project.org <mailto:R-help@r-project.org>>
>>        mailing list
>>
>>           https://stat.ethz.ch/mailman/listinfo/r-help
>>           PLEASE do read the posting guide
>>           http://www.R-project.org/posting-guide.html<http://www.r-project.org/posting-guide.html>
>>        <http://www.r-project.org/posting-guide.html>
>>           <http://www.r-project.org/posting-guide.html>
>>
>>           and provide commented, minimal, self-contained,
>>        reproducible code.
>>
>>
>>
>>
>>        --        Jim Holtman
>>        Cincinnati, OH
>>        +1 513 646 9390
>>
>>        What is the problem that you are trying to solve?
>>
>>
>>
>>
>>
>> --
>> Jim Holtman
>> Cincinnati, OH
>> +1 513 646 9390
>>
>> What is the problem that you are trying to solve?
>>
>
>
> --
> Martin Tomko
> Postdoctoral Research Assistant   Geographic Information Systems Division
> Department of Geography
> University of Zurich - Irchel
> Winterthurerstr. 190
> CH-8057 Zurich, Switzerland
>
> email:  martin.tomko@geo.uzh.ch
> site:   http://www.geo.uzh.ch/~mtomko
> mob:    +41-788 629 558
> tel:    +41-44-6355256
> fax:    +41-44-6356848
>
>


-- 
Jim Holtman
Cincinnati, OH
+1 513 646 9390

What is the problem that you are trying to solve?

	[[alternative HTML version deleted]]


