<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Sat, Sep 24, 2016 at 1:25 PM, Hardie, Andrew <span dir="ltr"><<a href="mailto:a.hardie@lancaster.ac.uk" target="_blank">a.hardie@lancaster.ac.uk</a>></span> wrote:</div><div class="gmail_quote"><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-GB">
<div>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-family:verdana,sans-serif;font-size:10pt">One possibility, given no results for a non-handle character, is that the bad values are empty strings – e.g. if there exist in the original data instances
of <</span><b style="color:rgb(31,73,125);font-family:verdana,sans-serif;font-size:10pt">text</b><span style="color:rgb(31,73,125);font-family:verdana,sans-serif;font-size:10pt">> that did not have a </span><b style="color:rgb(31,73,125);font-family:verdana,sans-serif;font-size:10pt">source</b><span style="color:rgb(31,73,125);font-family:verdana,sans-serif;font-size:10pt">.</span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125);font-size:10pt"><font face="Verdana, sans-serif">[...]</font></span><span style="color:rgb(31,73,125);font-family:verdana,sans-serif;font-size:10pt"> </span><br></p>
<p class="MsoNormal"><span style="font-size:10pt;font-family:verdana,sans-serif;color:rgb(31,73,125)">Incidentally, I’ve just checked in an amendment to the code which fixes a bug (identified while answering your question!) where a too-low maximum handle length
was imposed, and also changes the “because there are non-category-handle values in the CWB index” error message to actually say what the bad value was. So, if you are using the bleeding edge code, you can
<b>svn up</b>, try to change datatype again, and find out what the problem is that way.</span></p></div></div></blockquote><div><br></div><div>Great! As a result of your update, I've figured out that the problem is one or more empty values: </div><div><br></div></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><font face="monospace, monospace">The datatype of text_source cannot be changed to [classification],
because there are non-category-handle values in the CWB index; the first
non-handle value found in the index is [] .</font></div></div></div></blockquote><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div>Also, the following search returns many hits:</div><div><br></div></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><font face="monospace, monospace"><text_source="">[];</font></div></div></div></blockquote><div class="gmail_extra"><div class="gmail_quote"><div><br></div><div>So, for the benefit of anyone else who runs into this, I did the following query:</div><div><br></div></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div class="gmail_extra"><div class="gmail_quote"><div><span style="font-family:monospace,monospace">A = <text_source="">[];</span></div></div></div></blockquote><font face="monospace, monospace"><div><font face="monospace, monospace"><br></font></div></font>And then I performed various and sundry queries like this until I was able to figure out what set of texts caused the problem:<div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><font face="monospace, monospace">tabulate A match text_whatever;</font></div><div><font face="monospace, monospace"><br></font></div></blockquote>The odd thing is that none of the tagged texts had empty values for the field that was causing the problem. Many of them were fairly long, and some had two underscores together as a separator. I wonder if in either of these cases the values would be changed to an empty string.<div><br></div><div>Cheers,</div><div>Scott<br><div><br></div><div><br><div><div class="gmail_extra"><div class="gmail_quote"><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div lang="EN-GB"><div><p class="MsoNormal"><b><span lang="EN-US" style="font-size:10pt;font-family:tahoma,sans-serif">From:</span></b><span lang="EN-US" style="font-size:10pt;font-family:tahoma,sans-serif"> <a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a> [mailto:<a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.<wbr>unibo.it</a>]
<b>On Behalf Of </b>Scott Sadowsky</span><br></p><p class="MsoNormal"><span lang="EN-US" style="font-size:10pt;font-family:tahoma,sans-serif">
<b>Sent:</b> 24 September 2016 16:48<span class="gmail-"><br>
<b>To:</b> Open source development of the Corpus WorkBench<br>
</span><b>Subject:</b> Re: [CWB] Finding bad non-category-handle values<u></u><u></u></span></p><div><div class="gmail-h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">On Sat, Sep 24, 2016 at 3:07 AM, Hardie, Andrew <<a href="mailto:a.hardie@lancaster.ac.uk" target="_blank">a.hardie@lancaster.ac.uk</a>> wrote:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Hi Andrew,<u></u><u></u></p>
</div>
<div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt;font-family:verdana,sans-serif;color:rgb(31,73,125)">Try a CQP query for
</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10pt;font-family:verdana,sans-serif;color:rgb(31,73,125)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10pt;font-family:verdana,sans-serif;color:rgb(31,73,125)"><whichever_att=".*[^a-zA-Z0-9_<wbr>].*">[]</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">The s-attribute in question is <b>text_source</b>, so I ran the following in CQP:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><text_source=".*[^a-zA-Z0-9_].<wbr>*">[]<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">And it produced 0 hits. Same happens with this:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><text_source=".*[^a-z0-9_].*"><wbr>[]<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">This would seem to indicate that all the values of <b>text_source</b> are licit, but CQPweb disagrees. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt;font-family:verdana,sans-serif;color:rgb(31,73,125)">and then tabulate
<b>match whichever_att</b> ?</span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">This just gives me an error:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">tabulate match source_text ?;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">CQP Error:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span> </span>CQP Syntax Error: syntax error, unexpected FIELD, expecting ID or NQRID<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span> </span>tabulate match <--<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Synchronizing to end of line ...<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Cheers,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Scott<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt;font-family:verdana,sans-serif;color:rgb(31,73,125)"> </span><u></u><u></u></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10pt;font-family:tahoma,sans-serif">From:</span></b><span lang="EN-US" style="font-size:10pt;font-family:tahoma,sans-serif">
<a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a> [mailto:<a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.<wbr>unibo.it</a>]
<b>On Behalf Of </b>Scott Sadowsky<br>
<b>Sent:</b> 24 September 2016 04:10<br>
<b>To:</b> Open source development of the Corpus WorkBench<br>
<b>Cc:</b> Open source development of the Corpus WorkBench<br>
<b>Subject:</b> [CWB] Finding bad non-category-handle values</span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">I'm attempting to import a corpus into CQPweb, and when I try to change one of the s-attributes from "free text" to "classification", I get the following error:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><i>The datatype of text_source cannot be changed to [classification], because there are non-category-handle values in the CWB index.</i><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">I understand this to mean that in one or more values of text_source, there's a character that's not a-z or _. My question is simply how do I get a list of these values in order
to figure out which one is causing the problem and then fix it?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Thanks in advance!<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Scott<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12pt"><br>
______________________________<wbr>_________________<br>
CWB mailing list<br>
<a href="mailto:CWB@sslmit.unibo.it" target="_blank">CWB@sslmit.unibo.it</a><br>
<a href="http://liste.sslmit.unibo.it/mailman/listinfo/cwb" target="_blank">http://liste.sslmit.unibo.it/<wbr>mailman/listinfo/cwb</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div></div></div>
</div>
<br>______________________________<wbr>_________________<br>
CWB mailing list<br>
<a href="mailto:CWB@sslmit.unibo.it">CWB@sslmit.unibo.it</a><br>
<a href="http://liste.sslmit.unibo.it/mailman/listinfo/cwb" rel="noreferrer" target="_blank">http://liste.sslmit.unibo.it/<wbr>mailman/listinfo/cwb</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>
</div></div></div></div></div>