<div dir="ltr">Hi, Andrew,<div><br></div><div>I think I found the root of the problem. My VRT files contain characters that are valid UTF-8 however, MySQL&#39;s UTF-8 encoding and collation is a subset of the full UTF-8. In order to have the full character set one needs to use utf8mb4.</div><div><br></div><div>After some testing I found the files giving problems, and I think that all of them contained some kind of character out of the subset used by MySQL.</div><div><br></div><div><div>We need to be sure that the database will use utf8mb4 instead of utf8 as character encoding and collation. See &lt;<a href="https://mathiasbynens.be/notes/mysql-utf8mb4">https://mathiasbynens.be/notes/mysql-utf8mb4</a>&gt; or first answer here &lt;<a href="https://stackoverflow.com/questions/22572558/how-to-set-character-set-database-and-collation-database-to-utf8-in-my-ini">https://stackoverflow.com/questions/22572558/how-to-set-character-set-database-and-collation-database-to-utf8-in-my-ini</a>&gt; and the third answer here could be relevant from the Python side &lt;<a href="https://stackoverflow.com/questions/26532722/how-to-encode-utf8mb4-in-python">https://stackoverflow.com/questions/26532722/how-to-encode-utf8mb4-in-python</a>&gt;.</div><div><br></div><div>If we wanted the tables to use the encoding and collation we would need to change the CQPweb&#39;s code. <span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">However, a change from utf8 to utf8mb4 is not trivial because the length of `char`, `varchar` and `handles` are affected (as we use 4 bytes for every character instead of 3 the size in characters of those types of variables is reduced). Having said that, I did not need to mess with the tables, it was enough to change some global configuration and the charset and collation of the database.</span></div><div><br></div><div>However, my issue was solved just by doing the following:</div><div><br></div><div>In mysql configuration file `/etc/mysql/my.cnf` I wrote:</div><div><br></div><div>```sql</div><div>[client]</div><div>default-character-set = utf8mb4</div><div><br></div><div>[mysql]</div><div>default-character-set = utf8mb4</div><div><br></div><div>[mysqld]</div><div>character-set-client-handshake = FALSE</div><div>character-set-server = utf8mb4</div><div>collation-server = utf8mb4_unicode_ci</div><div>```</div><div><br></div></div><div>And then I also modified the character set and the collation for cqpweb_db:</div><div><br></div><div>```sql</div><div>ALTER DATABASE cqpweb_db CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;<br></div><div>```</div><div><br></div><div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial"><div>Check in mysql console with:</div><div><br></div><div>```sql</div><div>SHOW VARIABLES WHERE Variable_name LIKE &#39;character\_set\_%&#39; OR Variable_name LIKE &#39;collation%&#39;;</div><div>```</div></div><br></div><div>If one is creating the database from scratch, one could use:</div><div><br></div><div>```sql</div><div>CREATE DATABASE cqpweb_db2 DEFAULT CHARSET utf8mb4 COLLATE utf8mb4_general_ci;<br></div><div>```</div><div><br></div><div>After modifying the MySQL configuration file and changing the character set and collation for the database (I did not change anything for the tables), CQPweb was able to generate the frequency lists without problems.</div><div><br></div><div>I couldn&#39;t say if this is a critical issue. I never had this problem before, because I used to normalize characters. Now, I&#39;m working with very heterogenous data. I can foresee problems if someone is working with emojis and the like (tweets, etc.).</div><div><br></div><div>Cheers,</div><div><br></div><div>jmm</div><div><br></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div>--</div><div><div>José Manuel Martínez Martínez</div><div><a href="https://chozelinek.github.io" target="_blank">https://chozelinek.github.io</a></div></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Mon, Aug 6, 2018 at 3:01 PM, Hardie, Andrew <span dir="ltr">&lt;<a href="mailto:a.hardie@lancaster.ac.uk" target="_blank">a.hardie@lancaster.ac.uk</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">





<div lang="EN-GB" link="blue" vlink="purple">
<div class="m_-8137660298127512146WordSection1"><span class="">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">&gt;&gt;</span> is it possible to add a new corpus from the command line?<span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"><u></u> <u></u></span></p>
</span><p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">Not yet.<u></u><u></u></span></p><span class="">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">&gt;&gt;</span> I&#39;ve seen a create-corpus.php script but it says //TODO<span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"><u></u> <u></u></span></p>
</span><p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">Precisely!<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif"> <a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a> &lt;<a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a>&gt;
<b>On Behalf Of </b>José Manuel Martínez Martínez<br>
<b>Sent:</b> 06 August 2018 13:30</span></p><div><div class="h5"><br>
<b>To:</b> Open source development of the Corpus WorkBench &lt;<a href="mailto:cwb@sslmit.unibo.it" target="_blank">cwb@sslmit.unibo.it</a>&gt;<br>
<b>Subject:</b> Re: [CWB] Error #1300 generating word frequency lists<u></u><u></u></div></div><p></p><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">Hi again,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">last question, is it possible to add a new corpus from the command line? Not only the generation of the frequency lists? I&#39;ve seen a create-corpus.php script but it says //TODO ;-)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">And just in case it helps, this is what I see regarding my MySQL config<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">mysql&gt; show VARIABLES like &#39;%collation%&#39;;</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">+----------------------+------<wbr>-------------+</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| Variable_name</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">       
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| Value
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">           
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">+----------------------+------<wbr>-------------+</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| collation_connection | utf8_general_ci
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| collation_database
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8_general_ci
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| collation_server
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| latin1_swedish_ci |</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">+----------------------+------<wbr>-------------+</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">3 rows in set (0.00 sec)</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p2" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;min-height:13px">
<span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u> <u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">mysql&gt; show variables like &#39;%character%&#39;;</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">+--------------------------+--<wbr>--------------------------+</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| Variable_name</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">           
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| Value</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">                     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">+--------------------------+--<wbr>--------------------------+</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| character_set_client
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">                     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| character_set_connection | utf8
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">                     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| character_set_database
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">                     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| character_set_filesystem | binary
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">                   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| character_set_results</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">                     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| character_set_server
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| latin1
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">                   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| character_set_system
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">                     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| character_sets_dir
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| /usr/share/mysql/charsets/ |</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">+--------------------------+--<wbr>--------------------------+</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">8 rows in set (0.00 sec)</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">SHOW FULL COLUMNS FROM __tempfreq_spanish;</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">+----------+------------------<wbr>+-----------------+------+----<wbr>-+---------+-------+----------<wbr>-----------------------+------<wbr>---+</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| Field</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| Type
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">           
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| Collation
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| Null | Key | Default | Extra | Privileges</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
                     </span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| Comment |</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">+----------+------------------<wbr>+-----------------+------+----<wbr>-+---------+-------+----------<wbr>-----------------------+------<wbr>---+</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| freq
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| int(11) unsigned | NULL</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
         </span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| YES</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| NULL</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| select,insert,update,<wbr>references |
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">       
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| word
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| varchar(255)
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8_general_ci | NO
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| MUL | NULL</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| select,insert,update,<wbr>references |
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">       
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| dep</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| varchar(255)
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8_general_ci | NO
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| MUL | NULL</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| select,insert,update,<wbr>references |
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">       
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| ent_type | varchar(255)
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8_general_ci | NO
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| MUL | NULL</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| select,insert,update,<wbr>references |
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">       
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| is_alpha | varchar(255)
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8_general_ci | NO
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| MUL | NULL</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| select,insert,update,<wbr>references |
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">       
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| is_digit | varchar(255)
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8_general_ci | NO
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| MUL | NULL</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| select,insert,update,<wbr>references |
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">       
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| is_oov
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| varchar(255)
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8_general_ci | NO
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| MUL | NULL</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| select,insert,update,<wbr>references |
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">       
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| lemma</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| varchar(255)
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8_general_ci | NO
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| MUL | NULL</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| select,insert,update,<wbr>references |
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">       
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| lower</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| varchar(255)
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8_general_ci | NO
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| MUL | NULL</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| select,insert,update,<wbr>references |
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">       
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| pos</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| varchar(255)
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8_general_ci | NO
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| MUL | NULL</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| select,insert,update,<wbr>references |
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">       
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| tag</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| varchar(255)
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| utf8_general_ci | NO
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"> 
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| MUL | NULL</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">   
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">     
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">| select,insert,update,<wbr>references |
</span></span><span class="m_-8137660298127512146gmail-apple-converted-space"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">       
</span></span><span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">|</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">+----------+------------------<wbr>+-----------------+------+----<wbr>-+---------+-------+----------<wbr>-----------------------+------<wbr>---+</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="m_-8137660298127512146gmail-p1" style="margin:0cm;margin-bottom:.0001pt;background:#2b66c9;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal">
<span class="m_-8137660298127512146gmail-s1"><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white">11 rows in set (0.00 sec)</span></span><span style="font-size:8.5pt;font-family:&quot;Menlo&quot;,serif;color:white"><u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><br clear="all">
<u></u><u></u></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">--<u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">José Manuel Martínez Martínez<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="https://chozelinek.github.io" target="_blank">https://chozelinek.github.io</a><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Mon, Aug 6, 2018 at 1:31 PM, José Manuel Martínez Martínez &lt;<a href="mailto:chozelinek@gmail.com" target="_blank">chozelinek@gmail.com</a>&gt; wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal">Hi Andrew,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">thanks for the pointers. I didn&#39;t mention it, but I&#39;m installing the new corpora from already indexed corpora. Just in case this might be relevant.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I&#39;ll check with iconv and also with the generation of the frequency lists.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Cheers,<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><br clear="all">
<u></u><u></u></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">--<u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">José Manuel Martínez Martínez<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="https://chozelinek.github.io" target="_blank">https://chozelinek.github.io</a><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">On Mon, Aug 6, 2018 at 12:03 PM, Hardie, Andrew &lt;<a href="mailto:a.hardie@lancaster.ac.uk" target="_blank">a.hardie@lancaster.ac.uk</a>&gt; wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">A record is kept of the messages retrieved during indexing. Run this MySQL query to see it:</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal" style="text-indent:36.0pt">
<span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">SELECT indexing_notes FROM corpus_info WHERE corpus=&quot;<span style="background:yellow">lowercase corpus handle here</span>&quot;;</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">And you will see all the messages that cwb-encode &amp; friends emitted during indexing.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">&gt;&gt;</span> Would be there a way to run from the command line the command to generate the frequency lists?<u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">Yes, see Admin manual section 5.10 (p 48 in
<a href="http://cwb.sourceforge.net/files/CQPwebAdminManual.pdf" target="_blank">
the version on the website</a>)</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">That’s just the freqlist. To encode offline, use the cwb binaries.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">But actually, it might be easier to run iconv(1) on your files with UTF-8 as input encoding, and see
 whether/where it chokes.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">best</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">Andrew.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">
<a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a> &lt;<a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a>&gt;
<b>On Behalf Of </b>José Manuel Martínez Martínez<br>
<b>Sent:</b> 06 August 2018 10:44<br>
<b>To:</b> Open source development of the Corpus WorkBench &lt;<a href="mailto:cwb@sslmit.unibo.it" target="_blank">cwb@sslmit.unibo.it</a>&gt;<br>
<b>Subject:</b> Re: [CWB] Error #1300 generating word frequency lists</span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal">Hi Andrew,<u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">thank you very much for your quick reply.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">CQPweb v3.2.31<u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">CWB v3.4.14<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">The underlying data should be UTF-8.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">I cannot remember right now if I had encoding error at the encoding stage.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">I&#39;ll re-encode the corpus and let you know if I get any error on that regard.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Would be there a way to run from the command line the command to generate the frequency lists? I think I can leave a script encoding incrementally all texts I have in my corpus,
 to find out at least, which file is producing problems.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Cheers,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><br clear="all">
<u></u><u></u></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">--<u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">José Manuel Martínez Martínez<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="https://chozelinek.github.io" target="_blank">https://chozelinek.github.io</a><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal">On Mon, Aug 6, 2018 at 10:10 AM, Hardie, Andrew &lt;<a href="mailto:a.hardie@lancaster.ac.uk" target="_blank">a.hardie@lancaster.ac.uk</a>&gt; wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">The key bit of the error message is this:</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal">Error # 1300: Invalid utf8 character string: &#39;&#39;<u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">(unfortunate that the actual bad string can’t be identified from this.)</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">This suggests that there is a bad string in the CWB index, and it is caught by the MySql db on freq
 list setup. Recent versions of CWB however should not permit the indexing of badly-encoded strings (recent meaning, last several years). You should have had an error at the encoding stage if there was an encoding error in your data.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">What’s your CWB version? (also your CQPweb version) Also, is the underlying data UTF-8 or Latin-1?</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">best</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d">Andrew.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:&quot;Verdana&quot;,sans-serif;color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:&quot;Calibri&quot;,sans-serif">
<a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a> &lt;<a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a>&gt;
<b>On Behalf Of </b>José Manuel Martínez Martínez<br>
<b>Sent:</b> 06 August 2018 08:18<br>
<b>To:</b> Open source development of the Corpus WorkBench &lt;<a href="mailto:cwb@sslmit.unibo.it" target="_blank">cwb@sslmit.unibo.it</a>&gt;<br>
<b>Subject:</b> [CWB] Error #1300 generating word frequency lists</span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal">Good morning!<u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal">Trying to run collocations on a corpus in Spanish, I&#39;ve got an error.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Somehow, the word frequency list wasn&#39;t generated.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">I tried to generate it again but the process fails and I get the traceback that I copy/paste below.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Is this a CQPweb issue or should I check some settings of the MySQL database?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Cheers,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">jmm<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">--- TRACEBACK ---<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">CQPweb encountered an error and could not continue.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">A MySQL query did not run successfully!<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Original query: LOAD DATA LOCAL INFILE &#39;/data/cqpweb/tmp/______<wbr>tempfreq_spanish.tbl&#39; INTO TABLE `__tempfreq_spanish` FIELDS ESCAPED BY &#39;&#39; /* from User: datamaran | Function: corpus_make_freqtables()
 | 2018-Aug-03 12:41:27 */<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Error # 1300: Invalid utf8 character string: &#39;&#39;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">PHP debugging backtrace<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">array(6) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  [1]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  array(4) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;file&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    string(40) &quot;/var/www/html/cqpweb/lib/<wbr>library.inc.php&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;line&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    int(286)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;function&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    string(20) &quot;exiterror_mysqlquery&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;args&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    array(3) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      [0]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      int(1300)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      [1]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      string(33) &quot;Invalid utf8 character string: &#39;&#39;&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      [2]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      string(210) &quot;LOAD DATA LOCAL INFILE &#39;/data/cqpweb/tmp/______<wbr>tempfreq_spanish.tbl&#39; INTO TABLE `__tempfreq_spanish` FIELDS ESCAPED BY &#39;&#39; <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">            /* from User: datamaran | Function: corpus_make_freqtables() | 2018-Aug-03 12:41:27 */&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    }<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  }<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  [2]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  array(4) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;file&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    string(40) &quot;/var/www/html/cqpweb/lib/<wbr>library.inc.php&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;line&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    int(410)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;function&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    string(14) &quot;do_mysql_query&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;args&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    array(1) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      [0]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      &amp;string(210) &quot;LOAD DATA LOCAL INFILE &#39;/data/cqpweb/tmp/______<wbr>tempfreq_spanish.tbl&#39; INTO TABLE `__tempfreq_spanish` FIELDS ESCAPED BY &#39;&#39; <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">            /* from User: datamaran | Function: corpus_make_freqtables() | 2018-Aug-03 12:41:27 */&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    }<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  }<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  [3]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  array(4) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;file&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    string(42) &quot;/var/www/html/cqpweb/lib/<wbr>freqtable.inc.php&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;line&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    int(124)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;function&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    string(21) &quot;do_mysql_infile_query&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;args&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    array(3) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      [0]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      string(18) &quot;__tempfreq_spanish&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      [1]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      string(43) &quot;/data/cqpweb/tmp/______<wbr>tempfreq_spanish.tbl&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      [2]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      bool(true)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    }<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  }<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  [4]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  array(4) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;file&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    string(42) &quot;/var/www/html/cqpweb/lib/<wbr>admin-lib.inc.php&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;line&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    int(838)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;function&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    string(22) &quot;corpus_make_freqtables&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;args&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    array(1) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      [0]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      string(7) &quot;spanish&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    }<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  }<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  [5]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  array(4) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;file&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    string(47) &quot;/var/www/html/cqpweb/lib/<wbr>metadata-admin.inc.php&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;line&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    int(179)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;function&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    string(40) &quot;create_text_metadata_auto_<wbr>freqlist_calls&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;args&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    array(1) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      [0]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      string(7) &quot;spanish&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    }<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  }<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  [6]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  array(4) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;file&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    string(43) &quot;/var/www/html/cqpweb/exe/<wbr>metadata-admin.php&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;line&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    int(3)<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;args&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    array(1) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      [0]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">      string(47) &quot;/var/www/html/cqpweb/lib/<wbr>metadata-admin.inc.php&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    }<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    [&quot;function&quot;]=&gt;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">    string(7) &quot;require&quot;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">  }<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">}<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">--<u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">José Manuel Martínez Martínez<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><a href="https://chozelinek.github.io" target="_blank">https://chozelinek.github.io</a><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
______________________________<wbr>_________________<br>
CWB mailing list<br>
<a href="mailto:CWB@sslmit.unibo.it" target="_blank">CWB@sslmit.unibo.it</a><br>
<a href="http://liste.sslmit.unibo.it/mailman/listinfo/cwb" target="_blank">http://liste.sslmit.unibo.it/<wbr>mailman/listinfo/cwb</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
______________________________<wbr>_________________<br>
CWB mailing list<br>
<a href="mailto:CWB@sslmit.unibo.it" target="_blank">CWB@sslmit.unibo.it</a><br>
<a href="http://liste.sslmit.unibo.it/mailman/listinfo/cwb" target="_blank">http://liste.sslmit.unibo.it/<wbr>mailman/listinfo/cwb</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div></div></div>
</div>

<br>______________________________<wbr>_________________<br>
CWB mailing list<br>
<a href="mailto:CWB@sslmit.unibo.it">CWB@sslmit.unibo.it</a><br>
<a href="http://liste.sslmit.unibo.it/mailman/listinfo/cwb" rel="noreferrer" target="_blank">http://liste.sslmit.unibo.it/<wbr>mailman/listinfo/cwb</a><br>
<br></blockquote></div><br></div></div>