<div dir="ltr">Hi Andrew,<div><br></div><div>thank you very much for your answer. I've allocated more RAM to PHP modifying the memory limit. But I was still getting some errors with bigger sizes (46,564 Structure ``s'' units and 2,413,480 words). Then I looked into the log file while keeping an eye on the system watching top.</div><div><br></div><div>Now the creation of the subcorpus work. But what it is failing is the compilation of the frequency list.</div><div><br></div><div>It seems that my CQPweb has enough RAM but it is failing due to maximum execution time. I've modified the PHP variable max_execution_time. <span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">I started with 60 seconds, 120, and it still fails with 600.</span></div><div><br></div><div>This is the error in the log</div><div><br></div><div>[pid 1579] PHP Fatal error: Maximum execution time of 600 seconds exceeded in /var/www/html/cqpweb/lib/subcorpus.inc.php on line 4037<br></div><div><br></div><div>This is some additional information on the PID 1579</div><div><br></div><div>
<span></span>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(255,255,255);background-color:rgb(43,102,201)"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">ps -fp 1579</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(255,255,255);background-color:rgb(43,102,201)"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">UID<span class="gmail-Apple-converted-space"> </span>PID<span class="gmail-Apple-converted-space"> </span>PPID<span class="gmail-Apple-converted-space"> </span>C STIME TTY<span class="gmail-Apple-converted-space"> </span>TIME CMD</span></p>
<p class="gmail-p1" style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-weight:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(255,255,255);background-color:rgb(43,102,201)"><span class="gmail-s1" style="font-variant-ligatures:no-common-ligatures">www-data<span class="gmail-Apple-converted-space"> </span>1579<span class="gmail-Apple-converted-space"> </span>1431<span class="gmail-Apple-converted-space"> </span>9 12:03 ?<span class="gmail-Apple-converted-space"> </span>00:11:58 /usr/sbin/apache2 -k start</span></p>
</div><div><br></div><div>When I recreate the frequency lists for the whole corpus, it takes a fairly long time, but it normally does not fail. Could be there something in the way subcorpus compiles the frequency list when compared with the creation of frequency lists for the whole corpus?</div><div><br></div><div>Cheers,</div><div><br></div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div>--</div><div><div>José Manuel Martínez Martínez</div><div><a href="https://chozelinek.github.io" target="_blank">https://chozelinek.github.io</a></div></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Thu, May 31, 2018 at 12:44 PM, Hardie, Andrew <span dir="ltr"><<a href="mailto:a.hardie@lancaster.ac.uk" target="_blank">a.hardie@lancaster.ac.uk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-GB" link="blue" vlink="purple">
<div class="m_-4419003739611821634WordSection1">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1f497d">You’re probably running out of RAM. Wrangling subcorpora that use sub-text regions is very memory-intensive (I have some ideas in the
works to make it less-so). The way to check this is (a) look in php.ini to find out how much RAM each PHP process is allowed (the
</span><span style="font-size:10.0pt;font-family:Consolas;color:#242729;background:#eff0f1">memory_limit</span><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1f497d"> setting) (b) watch in “top” on your server
as it runs, and note that it will probably time out when the CQPweb process hits that amount of allocated memory.
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1f497d">(Your httpd error log may also contain a note of this error, something like “Allowed memory size of BIGNUMBER bytes exhausted (tried
to allocate BIGNUMBER bytes) in php”. Any http 500 error should leave an error message in the log!)
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1f497d">The fix is to let PHP use more RAM. (At least for CQPweb processes). I would not worry about over-allocating RAM as long as you have
an adequate swap disk your server for virtual memory when needed!<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1f497d">best<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1f497d">Andrew.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> <a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a> [mailto:<a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.<wbr>unibo.it</a>]
<b>On Behalf Of </b>José Manuel Martínez Martínez<br>
<b>Sent:</b> 30 May 2018 10:48<br>
<b>To:</b> Open source development of the Corpus WorkBench <<a href="mailto:cwb@sslmit.unibo.it" target="_blank">cwb@sslmit.unibo.it</a>><br>
<b>Subject:</b> [CWB] CQPweb: error with subcorpus creation<u></u><u></u></span></p><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">Dear all,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I'm getting an internal server error when I try to create a subcorpus from a saved query.<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">The saved query has 58000 hits. I try to define the new subcorpus via "partial-text regions found in a saved query". I select the saved query and I use as sub-text region the structural attribute 's' that in my case denotes sentences.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">After a few minutes I get an HTTP 500 ERROR.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Arial",sans-serif;color:#222222">However, if I try it with the same query but on a smaller set of hits (9615) the process is successful (the size of the resulting subcorpus is 402,802 tokens
and 7700 sentences). However, sometimes I get an error when I try to generate the frequency list. I tried with a saved query slightly bigger (11600 hits) and it fails too.<u></u><u></u></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Is there a way to now what's going wrong?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Cheers,<br clear="all">
<u></u><u></u></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">--<u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">José Manuel Martínez Martínez<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="https://chozelinek.github.io" target="_blank">https://chozelinek.github.io</a><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div></div>
</div>
<br>______________________________<wbr>_________________<br>
CWB mailing list<br>
<a href="mailto:CWB@sslmit.unibo.it">CWB@sslmit.unibo.it</a><br>
<a href="http://liste.sslmit.unibo.it/mailman/listinfo/cwb" rel="noreferrer" target="_blank">http://liste.sslmit.unibo.it/<wbr>mailman/listinfo/cwb</a><br>
<br></blockquote></div><br></div>