<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Verdana","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Hi Ray,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">The segfault had nothing to do with your data, it was an internal structure used by cwb-scan-corpus for pattern matching. There is a fix in SVN now; recompile
v 3.4.4<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">best<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Andrew.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> cwb-bounces@sslmit.unibo.it [mailto:cwb-bounces@sslmit.unibo.it]
<b>On Behalf Of </b>Ray Wu<br>
<b>Sent:</b> 26 May 2012 13:49<br>
<b>To:</b> Open source development of the Corpus WorkBench<br>
<b>Subject:</b> [CWB] CQPweb 3.0.7 on CWB 3.4.3 cwb-scan-corpus error! Segmentation fault<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">Hi all,<br>
I want to make CQPweb to process Chinese (my native tongue) on my Ubuntu 8.04, so I updated to CWB 3.4.3. The compiling process was successful and I could query a tiny Chinese text via cqp from the terminal.<br>
<br>
I could also load the Chinese text into CQPweb and finished part of the metadata page. But when I wanted to "Manage metadata->Create frequency tables", CQPweb complained and says it encountered an error and could not continue. Here is the error message:<br>
cwb-scan-corpus error! Segmentation fault<br>
... in file /usr/local/apache2/htdocs/cqp/lib/freqtable.inc.php line 100. <br>
<br>
This sounds strange to me as I have browsed the entire archived mailing list and get to know that error message is mostly likely to happen when a token is too long. But my toy corpus is just a few lines long. I tried it on an small English text and the same
situation occurs.<br>
<br>
To make the picture clearer, I will try to illustrate my experiment by listing what I have done.<br>
<br>
My compiling context for CWB 3.4.3: CWB from svn: 3.4.3; PCRE: 7.4; glib-2.0; gcc: Ubuntu 4.2.4-1ubuntu3.
<br>
<br>
The compiling process seemed normal and I could build a tiny Chinese corpus using the following text (See the end of the post). Hopefully it can make it through the wild net to your computer remaining intelligible):<br>
<br>
I then ran the following to index it:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">ray@ray-laptop:~$ cwb-encode -c utf8 -d /home/ray/cqputf8 -f cqpweb_chinese_test_utf8.txt -R /usr/local/share/cwb/registry/test -P pos -S
text -S s -S text_id</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">Annotations of s-attribute <text> not stored (file cqpweb_chinese_test_utf8.txt, line #1, warning issued only once).</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">ray@ray-laptop:~$ cwb-makeall -V TEST (everyting says OK)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">ray@ray-laptop:~$ cwb-huffcode -A TEST (fine, nothing wrong)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">ray@ray-laptop:~$ cwb-compress-rdx -A TEST (fine again)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">I queried the new corpus and nothing broken:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">ray@ray-laptop:~$ cqp -eC</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">[no corpus]> TEST</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="DE" style="color:black">TEST> "</span><span style="font-family:SimSun;color:black">了</span><span lang="DE" style="color:black">";</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="DE" style="color:black"> 7:
</span><span style="font-family:SimSun;color:black">们</span><span style="color:black">
</span><span style="font-family:SimSun;color:black">的</span><span style="color:black">
</span><span style="font-family:SimSun;color:black">行为</span><span style="color:black">
</span><span style="font-family:SimSun;color:black">也</span><span style="color:black">
</span><span style="font-family:SimSun;color:black">引来</span><span lang="DE" style="color:black"> <</span><span style="font-family:SimSun;color:black">了</span><span lang="DE" style="color:black">>
</span><span style="font-family:SimSun;color:black">不少</span><span style="color:black">
</span><span style="font-family:SimSun;color:black">公园</span><span style="color:black">
</span><span style="font-family:SimSun;color:black">游客</span><span style="color:black">
</span><span style="font-family:SimSun;color:black">的</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="DE" style="color:black"> 29:
</span><span style="font-family:SimSun;color:black">，</span><span style="color:black">
</span><span style="font-family:SimSun;color:black">他们</span><span style="color:black">
</span><span style="font-family:SimSun;color:black">早已</span><span style="color:black">
</span><span style="font-family:SimSun;color:black">习惯</span><span lang="DE" style="color:black"> <</span><span style="font-family:SimSun;color:black">了</span><span lang="DE" style="color:black">>
</span><span style="font-family:SimSun;color:black">。</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">TEST> <s> []* "</span><span style="font-family:SimSun;color:black">了</span><span lang="EN-US" style="color:black">" []* </s>; (query is OK)
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black"><br>
Finally, I resorted to run </span><span lang="EN-US" style="color:black"> cwb-scan-corpus manually and did find something usual:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">ray@ray-laptop:~$ cwb-scan-corpus -C TEST pos (fully OK)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">ray@ray-laptop:~$ cwb-scan-corpus TEST pos+0 pos+1 (segmentation fault)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">ray@ray-laptop:~$ cwb-scan-corpus TEST pos+0 pos+1 pos+2</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">Scanning corpus TEST for 3-tuples ...
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">Scan complete. </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">Printing frequency table on stdout ...
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="PL" style="color:black">...</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:SimSun;color:black">段错误</span><span lang="PL" style="color:black"> ("segmentation fault" in English)</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">I have very little knowledge in C so I cannot go further to investigate more.
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">Does anyone know where the problem is? Thanks for any input.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">Best,
<br>
Ray</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">Hunan University of Commerce, China</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">PS: My computer parameters:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">System: Ubuntu 8.04<br>
Apache: 2.0.63<br>
MySQL: 5.0.88<br>
PHP: 5.2.12 (lower than expected 5.3.0)<br>
Perl: 5.8.8<br>
CWB: 3.4.3 (compiled from svn source)<br>
Linux utilites: awk, tar, gzip, iconv</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">LANG=zh_CN.UTF-8</span><span style="color:black"><o:p></o:p></span></p>
<div style="border:none;border-bottom:double windowtext 2.25pt;padding:0cm 0cm 1.0pt 0cm">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">GDM_LANG=zh_CN.UTF-8</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black">Inside cqpweb_chinese_test_utf8.txt:
</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="color:black"><text id="test"><br>
<s><br>
</span><span style="font-family:SimSun;color:black">这些</span><span lang="EN-US" style="color:black"> r<br>
</span><span style="font-family:SimSun;color:black">网友</span><span lang="EN-US" style="color:black"> n<br>
</span><span style="font-family:SimSun;color:black">们</span><span lang="EN-US" style="color:black"> k<br>
</span><span style="font-family:SimSun;color:black">的</span><span lang="EN-US" style="color:black"> u<br>
</span><span style="font-family:SimSun;color:black">行为</span><span lang="EN-US" style="color:black"> n<br>
</span><span style="font-family:SimSun;color:black">也</span><span lang="EN-US" style="color:black"> d<br>
</span><span style="font-family:SimSun;color:black">引来</span><span lang="EN-US" style="color:black"> v<br>
</span><span style="font-family:SimSun;color:black">了</span><span lang="EN-US" style="color:black"> u<br>
</span><span style="font-family:SimSun;color:black">不少</span><span lang="EN-US" style="color:black"> m<br>
</span><span style="font-family:SimSun;color:black">公园</span><span lang="EN-US" style="color:black"> n<br>
</span><span style="font-family:SimSun;color:black">游客</span><span lang="EN-US" style="color:black"> n<br>
</span><span style="font-family:SimSun;color:black">的</span><span lang="EN-US" style="color:black"> u<br>
</span><span style="font-family:SimSun;color:black">围观</span><span lang="EN-US" style="color:black"> v<br>
</span><span style="font-family:SimSun;color:black">。</span><span lang="EN-US" style="color:black"> w<br>
</s><br>
<s><br>
</span><span style="font-family:SimSun;color:black">而</span><span lang="EN-US" style="color:black"> c<br>
</span><span style="font-family:SimSun;color:black">对于</span><span lang="EN-US" style="color:black"> p<br>
</span><span style="font-family:SimSun;color:black">人们</span><span lang="EN-US" style="color:black"> n<br>
</span><span style="font-family:SimSun;color:black">的</span><span lang="EN-US" style="color:black"> u<br>
</span><span style="font-family:SimSun;color:black">议论</span><span lang="EN-US" style="color:black"> v<br>
</span><span style="font-family:SimSun;color:black">，</span><span lang="EN-US" style="color:black"> w<br>
</span><span style="font-family:SimSun;color:black">这些</span><span lang="EN-US" style="color:black"> r<br>
</span><span style="font-family:SimSun;color:black">汉</span><span lang="EN-US" style="color:black"> t<br>
</span><span style="font-family:SimSun;color:black">服</span><span lang="EN-US" style="color:black"> v<br>
</span><span style="font-family:SimSun;color:black">爱好者</span><span lang="EN-US" style="color:black"> n<br>
</span><span style="font-family:SimSun;color:black">表示</span><span lang="EN-US" style="color:black"> v<br>
</span><span style="font-family:SimSun;color:black">，</span><span lang="EN-US" style="color:black"> w<br>
</span><span style="font-family:SimSun;color:black">他们</span><span lang="EN-US" style="color:black"> r<br>
</span><span style="font-family:SimSun;color:black">早已</span><span lang="EN-US" style="color:black"> d<br>
</span><span style="font-family:SimSun;color:black">习惯</span><span lang="EN-US" style="color:black"> v<br>
</span><span style="font-family:SimSun;color:black">了</span><span lang="EN-US" style="color:black"> y<br>
</span><span style="font-family:SimSun;color:black">。</span><span lang="EN-US" style="color:black"> w<br>
</s><br>
</text></span><span style="color:black"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
</div>
</body>
</html>