
Hi Prof,

Thank you for your reply. Sorry that I missed out the below information.
>Sys.getlocale()
[1] "LC_COLLATE=English_United States.1252;LC_CTYPE=English_United States.1252;LC_MONETARY=English_United States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252"

I have just noticed that traditional chinese character cause the encoding problem, while chinese simplified works fine.

>library(RCurl)
>theurl <- getURL("http://home.sina.com",encoding='utf8')
#Encoding(theurl)
#[1]"latin1"
>txt <- readLines(con=textConnection(theurl),encoding='utf8')
>write.table(file='D:/fileas.txt',txt)

When I open the fileas.txt, the Chinese traditional character readable in notepad, but when I try to read file to Rgui:-
> smple <- scan('D:/fileas.txt',what='')
Then it comes to unrecognisable character again, I was wondering if Rgui support traditional Chinese character now... 

I think I need to looking for solution of inter-Chinese character's translation.
Thank you.


Best,
Ryusuke

  ===============================================

Hi Ryusuke
 
 I would use the encoding parameter of htmlParse() and 
 download and parse the content in one operation:
 
     htmlParse("http://home.sina.com", encoding = "UTF-8")
 
 If you want to use getURL() in RCurl, use the .encoding parameter
 
  You didn't tell us the output of Sys.getlocale()
  or how your terminal/console is configured, so the above
  may vary under your configuration, but works on various
  machines for me with different settings.
 
    D.
 
 
Ryusuke Kenji wrote:
> 
> Hi All,
> 
> First method:-
> >library(XML)
> 
> >theurl <- "http://home.sina.com"
> >download.file(theurl, "tmp.html")
> 
> >txt <- readLines("tmp.html")
> 
> >txt <- htmlTreeParse(txt, error=function(...){}, useInternalNodes = 
> TRUE)
> 
> >g <- xpathSApply(txt, "//p", function(x) xmlValue(x))
> 
> >head(grep(" ", g, value=T))
> 
> 
> [1] "?????? | ?????? | ENGLISH"                               "??????????????? ???????????????"                        
> [3] "??????? ?????????? ??????????????????(???)"              "?????????????????????????????? ????????????????????????"
> [5] " ???????????????????????????????????????"                "? ??????????! ????? ??????! ????????????????????????!"  
> 
> 
> 
> SecondMethod:-
> >library(RCurl)
> 
> >theurl <- getURL("http://home.sina.com",encoding='GB2312')
> 
> >Encoding(theurl)
> 
> [1]"unknown"
> 
> >txt <- readLines(con=textConnection(theurl),encoding='GB2312')
> >txt[5:10] #show the lines which occurred encoding problem.
> [1] "<meta http-equiv=\"Content-Type\" content=\"text/html; charset=utf-8\" />"
> [2] "<title>SINA.com US ????????? -??????</title>"
> [3] "<meta name=\"Keywords\" content=\"????????????, ???????????????, ???????????????, ??????????????????,????????????, SINA, US, News, Chinese, Asia\" />"
> [4] "<meta name=\"Description\" content=\"???????????????????????????????????????, ???????????????24????????????????????????????????, ????????????????????????, ????????????, ??????????????????, ????????????????????????, ?????????BBS, ???????????????????????????????????.\" />"
> [5]""                                                                                                                                                                                                                                                                                
> [6] "<link rel=\"stylesheet\" type=\"text/css\" href=\"http://ui.sina.com/assets/css/style_home.css\" />"
> 
> i am trying to read data from a Chinese language website, but the Chinese characters always unreadable, may I know if any good idea to cope such encoding problem in RCurl and XML?
> 
> 
> Regards,
> Ryusuke
>                            
> _________________________________________________________________
> 
> 
>     [[alternative HTML version deleted]]
> 
 
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
 
 
-- 
"There are men who can think no deeper than a fact" - Voltaire
 
 
Duncan Temple Lang                duncan@wald.ucdavis.edu
Department of Statistics          work:  (530) 752-4782
4210 Mathematical Sciences Bldg.  fax:   (530) 752-7099
One Shields Ave.
University of California at Davis
Davis, CA 95616, USA
 
 
  		 	   		  
_________________________________________________________________
丟奈伙毛珨嬤民尼永弁ㄐ坻及剠蹋丟奈伙手皿伕田奶母奈丟奈伙手﹝

	[[alternative HTML version deleted]]

