<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.m1560621951439290829si
{mso-style-name:m_1560621951439290829si;}
span.m1560621951439290829s2
{mso-style-name:m_1560621951439290829s2;}
span.m1560621951439290829nb
{mso-style-name:m_1560621951439290829nb;}
span.m1560621951439290829p
{mso-style-name:m_1560621951439290829p;}
span.m1560621951439290829nv
{mso-style-name:m_1560621951439290829nv;}
span.m1560621951439290829cm
{mso-style-name:m_1560621951439290829cm;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Verdana","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Hi Jorge,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">As suspected, seeing a few of the error messages makes it clear what is happening.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">“is not a valid feature set” is thee error message. You are getting one per POS tag in the corpus (since your POS tags aren’t feature sets). One error message
per POS tag in the corpus exhausts the available RAM. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">The answer is simple: don’t tick the “Feature set?” box when your tags aren’t actually feature sets.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">To read about what feature sets are / aren’t in CWB/CQPweb, have a look at the CWB Encoding Tutorial (search for the word “feature” in the PDF to find the relevant
section.)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">best<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Andrew.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> cwb-bounces@sslmit.unibo.it [mailto:cwb-bounces@sslmit.unibo.it]
<b>On Behalf Of </b>VIVALDI PALATRESI, JORGE<br>
<b>Sent:</b> 26 July 2017 12:23<br>
<b>To:</b> Open source development of the Corpus WorkBench<br>
<b>Subject:</b> Re: [CWB] Unable to index a corpus<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi Andrew,<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">I made the suggested modifications in the file lib/admin-install.inc.php without any positive result the browser always becomes blank with a similar messages in the file error.log.<o:p></o:p></p>
</div>
<p class="MsoNormal">Then, I reduced the size of corpus to a 20% of the original. Then some messages appeared on the browser, but after a while the browser crashes. Anyway I captured the included screenshot.<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">It seems to be a problem related to the cwb-encode and the POS tags (N5-FS, JQ--FS, P, ...).
<br>
As I mention in a previous message the corpus (and its POS tags) is the same used with a previous version of CQP and CQPweb.<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Bests,<o:p></o:p></p>
</div>
<p class="MsoNormal">Jorge<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">2017-07-26 10:22 GMT+02:00 Hardie, Andrew <<a href="mailto:a.hardie@lancaster.ac.uk" target="_blank">a.hardie@lancaster.ac.uk</a>>:<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">As I noted before, the problem is actually error messages. Line 644 simply collects error messages
– so an out-of-memory error here indicates you have generated > 4GB of error messages.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">I suggested increasing the memory previously because it would let you see the problem – but actually,
with 4GB of error messages, I’d suggest that doing that is not likely to help much,</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">So what I would suggest instead is hacking the code to find out the error message.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Open admin-lib.inbc.php</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Go to line 644</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Find the line nearby that says
</span><span class="m1560621951439290829si"><b><span style="font-size:10.0pt;font-family:"Courier New";color:#BB6688;border:none windowtext 1.0pt;padding:0cm">$output_lines_from_cwb</span></b></span><span class="m1560621951439290829s2"><span style="font-size:10.0pt;font-family:"Courier New";color:#BA2121;border:none windowtext 1.0pt;padding:0cm">
= array(</span></span><span class="m1560621951439290829si"><b><span style="font-size:10.0pt;font-family:"Courier New";color:#BB6688;border:none windowtext 1.0pt;padding:0cm">$encode_command</span></b></span><span class="m1560621951439290829s2"><span style="font-size:10.0pt;font-family:"Courier New";color:#BA2121;border:none windowtext 1.0pt;padding:0cm">);</span></span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a name="m_1560621951439290829__MailEndCompose"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span></a><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">AFTER THAT LINE, but before the line that says
</span><span class="m1560621951439290829s2"><span style="font-size:10.0pt;font-family:"Courier New";color:#BA2121;border:none windowtext 1.0pt;padding:0cm">exec(</span></span><span class="m1560621951439290829si"><b><span style="font-size:10.0pt;font-family:"Courier New";color:#BB6688;border:none windowtext 1.0pt;padding:0cm">$encode_command</span></b></span><span class="m1560621951439290829s2"><span style="font-size:10.0pt;font-family:"Courier New";color:#BA2121;border:none windowtext 1.0pt;padding:0cm">,
</span></span><span class="m1560621951439290829si"><b><span style="font-size:10.0pt;font-family:"Courier New";color:#BB6688;border:none windowtext 1.0pt;padding:0cm">$output_lines_from_cwb</span></b></span><span class="m1560621951439290829s2"><span style="font-size:10.0pt;font-family:"Courier New";color:#BA2121;border:none windowtext 1.0pt;padding:0cm">,
</span></span><span class="m1560621951439290829si"><b><span style="font-size:10.0pt;font-family:"Courier New";color:#BB6688;border:none windowtext 1.0pt;padding:0cm">$exit_status_from_cwb</span></b></span><span class="m1560621951439290829s2"><span style="font-size:10.0pt;font-family:"Courier New";color:#BA2121;border:none windowtext 1.0pt;padding:0cm">);
</span></span><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">add the following:</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:36.0pt">
<b><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">if (count($output_lines_from_cwb) > 1000) {show_var($output_lines_from_cwb); exiterror("abort"); }</span></b><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">What this line does is make things abort if it detects too many error messages.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">If you then get a readable error message, that might give you a hint what the real problem is. If
not, try again moving the location o fthe hack line down the file, before the following lines:</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">before
</span><span class="m1560621951439290829nb"><span style="font-size:10.0pt;font-family:"Courier New";color:green;border:none windowtext 1.0pt;padding:0cm">exec</span></span><span class="m1560621951439290829p"><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;border:none windowtext 1.0pt;padding:0cm">(</span></span><span class="m1560621951439290829nv"><span style="font-size:10.0pt;font-family:"Courier New";color:#19177C;border:none windowtext 1.0pt;padding:0cm">$makeall_command</span></span><span class="m1560621951439290829p"><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;border:none windowtext 1.0pt;padding:0cm">,</span></span><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;background:#FFFF99">
</span><span class="m1560621951439290829nv"><span style="font-size:10.0pt;font-family:"Courier New";color:#19177C;border:none windowtext 1.0pt;padding:0cm">$output_lines_from_cwb</span></span><span class="m1560621951439290829p"><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;border:none windowtext 1.0pt;padding:0cm">,</span></span><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;background:#FFFF99">
</span><span class="m1560621951439290829nv"><span style="font-size:10.0pt;font-family:"Courier New";color:#19177C;border:none windowtext 1.0pt;padding:0cm">$exit_status_from_cwb</span></span><span class="m1560621951439290829p"><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;border:none windowtext 1.0pt;padding:0cm">);</span></span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">before
</span><span class="m1560621951439290829nb"><span style="font-size:10.0pt;font-family:"Courier New";color:green;border:none windowtext 1.0pt;padding:0cm">exec</span></span><span class="m1560621951439290829p"><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;border:none windowtext 1.0pt;padding:0cm">(</span></span><span class="m1560621951439290829nv"><span style="font-size:10.0pt;font-family:"Courier New";color:#19177C;border:none windowtext 1.0pt;padding:0cm">$compress_command</span></span><span class="m1560621951439290829p"><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;border:none windowtext 1.0pt;padding:0cm">,</span></span><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;background:#FFFF99">
</span><span class="m1560621951439290829nv"><span style="font-size:10.0pt;font-family:"Courier New";color:#19177C;border:none windowtext 1.0pt;padding:0cm">$compression_output</span></span><span class="m1560621951439290829p"><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;border:none windowtext 1.0pt;padding:0cm">,</span></span><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;background:#FFFF99">
</span><span class="m1560621951439290829nv"><span style="font-size:10.0pt;font-family:"Courier New";color:#19177C;border:none windowtext 1.0pt;padding:0cm">$exit_status_from_cwb</span></span><span class="m1560621951439290829p"><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;border:none windowtext 1.0pt;padding:0cm">);</span></span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">before the second example fo
</span><span class="m1560621951439290829nb"><span style="font-size:10.0pt;font-family:"Courier New";color:green;border:none windowtext 1.0pt;padding:0cm">exec</span></span><span class="m1560621951439290829p"><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;border:none windowtext 1.0pt;padding:0cm">(</span></span><span class="m1560621951439290829nv"><span style="font-size:10.0pt;font-family:"Courier New";color:#19177C;border:none windowtext 1.0pt;padding:0cm">$makeall_command</span></span><span class="m1560621951439290829p"><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;border:none windowtext 1.0pt;padding:0cm">,</span></span><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;background:#FFFF99">
</span><span class="m1560621951439290829nv"><span style="font-size:10.0pt;font-family:"Courier New";color:#19177C;border:none windowtext 1.0pt;padding:0cm">$output_lines_from_cwb</span></span><span class="m1560621951439290829p"><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;border:none windowtext 1.0pt;padding:0cm">,</span></span><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;background:#FFFF99">
</span><span class="m1560621951439290829nv"><span style="font-size:10.0pt;font-family:"Courier New";color:#19177C;border:none windowtext 1.0pt;padding:0cm">$exit_status_from_cwb</span></span><span class="m1560621951439290829p"><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;border:none windowtext 1.0pt;padding:0cm">);</span></span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">before
</span><span class="m1560621951439290829p"><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;border:none windowtext 1.0pt;padding:0cm">}</span></span><span style="font-size:10.0pt;font-family:"Courier New";color:#555555;background:#FFFF99">
</span><span class="m1560621951439290829cm"><i><span style="font-size:10.0pt;font-family:"Courier New";color:#408080;border:none windowtext 1.0pt;padding:0cm">/* end else (from if cwb index already exists) */</span></i></span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Hopefully, as I say, doing this will get you a gimpse of the first 1,000 lines of erro, which may
tell you what the underlying problem is.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Hope this helps</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">best</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Andrew.</span><o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Jorge Vivaldi Palatresi<br>
Institut Universitari de Lingüística Aplicada<br>
Universitat Pompeu Fabra<br>
C/ Roc Boronat, 138<br>
08018 Barcelona<br>
Espanya<br>
<br>
+34 93 542 2332<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>