<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";
mso-fareast-language:EN-GB;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Verdana","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:988090549;
mso-list-template-ids:803661404;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Daniel’s sample of a datafile exemplifies one of the two methods for more-than-minimal text metadata. This can either be loaded from a tab-delimited file, or
deduced from XML. The latter method is the one Daniel exemplifies.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">For
<i>minimal</i> metadata you only require text with the ID attribute (whose values must be
<i>handles</i>, i.e. just letters, numbers with no space / punctuation).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">It is a rule of CQPweb corpora that the whole corpus needs to occur within <text> elements, each of which must have an id, and there can’t be any words that
are not inside a <text> element. If you don’t care about text boundaries, you can just wrap the whole corpus in one <text id="CORPUS"> … </text>
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">This is explained in my paper:
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<ul style="margin-top:0cm" type="square">
<li class="MsoNormal" style="color:#1F497D;mso-list:l0 level1 lfo1"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif"">Hardie, Andrew (2012). <a href="http://www.ingentaconnect.com/content/jbp/ijcl/2012/00000017/00000003/art00004" target="_blank">CQPweb
– combining power, flexibility and usability in a corpus analysis tool</a>. <i>International Journal of Corpus Linguistics</i> 17 (3): 380-409. <a href="http://www.lancs.ac.uk/staff/hardiea/cqpweb-paper.pdf" target="_blank">[alternative link]</a><o:p></o:p></span></li></ul>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Sorry it’s not written up in the manual yet, only so many hours in a day alas…<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">best<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Andrew.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> cwb-bounces@sslmit.unibo.it [mailto:cwb-bounces@sslmit.unibo.it]
<b>On Behalf Of </b>Daniel Renau<br>
<b>Sent:</b> 14 November 2016 13:46<br>
<b>To:</b> Open source development of the Corpus WorkBench<br>
<b>Subject:</b> Re: [CWB] Can't create metadata<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p>Hi jiayue,<o:p></o:p></p>
<p>My team works with verticalized texts like this:<o:p></o:p></p>
<p><text id="ST1" title="namewithoutspaces" author="name"><br>
<s><br>
word pos lemma<br>
word pos lemma<br>
word pos lemma<br>
word pos lemma<br>
</s><br>
</text><o:p></o:p></p>
<p><text id="ST2" title="anothertextname" author="otherperson"><br>
<s><br>
word pos lemma<br>
word pos lemma<br>
word pos lemma<br>
</s><br>
</text><o:p></o:p></p>
<p>You can add more text tags as: author_sex, language, year, translator...<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">El 14 nov. 2016 2:37 p. m., "Jiayue Wang" <<a href="mailto:arthur0421@gmail.com">arthur0421@gmail.com</a>> escribió:<o:p></o:p></p>
<p class="MsoNormal">Thanks Andrew. I still don't understand where the tags <text id=""> and </text> should be added. Should they enclose a corpus file? I notice that section 7.6 "Metadata template" of the CQPwebAdminManual is empty. Could you show me a template?<br>
<br>
Best,<br>
Jiayue<br>
<br>
On 14/11/16 09:38, Hardie, Andrew wrote:<o:p></o:p></p>
<p class="MsoNormal">Well it looks rather as if you don't have any text tags at all there... which would be part of the problem. Try again with <text id="...">...</text> tags added to the file, as required.<br>
<br>
As for why indexing is taking so long, it's very difficult for me to diagnose at a distance. You should keep an eye on your process list (e.g. via top) to see if anything is actually happening. As long as a cwb-*** process is running, something productive is
happening, and you shouldn't abort.<br>
<br>
best<br>
<br>
Andrew.<br>
<br>
-----Original Message-----<br>
From: <a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a> [mailto:<a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a>] On Behalf Of Jiayue Wang<br>
Sent: 13 November 2016 11:06<br>
To: Open source development of the Corpus WorkBench<br>
Subject: Re: [CWB] Can't create metadata<br>
<br>
Hi Andrew,<br>
<br>
Thanks a lot.<br>
I deleted the us_rhodeisland corpus and tried again to install it. The<br>
corpus file looks like this:<br>
<br>
If IN if<br>
you PP you<br>
have VBP have<br>
any DT any<br>
questions NNS question<br>
or CC or<br>
suggestions NNS suggestion<br>
how WRB how<br>
this DT this<br>
website NN website<br>
might MD might<br>
be VB be<br>
improved VBN improve<br>
, , ,<br>
please VB please<br>
feel VB feel<br>
free JJ free<br>
to TO to<br>
contact VB contact<br>
us PP us<br>
. SENT .<br>
<br>
The corpus contains only this file (44.0 MB). For P-attribute I selected<br>
the POS and lemma (TreeTagger format) option. Then I clicked Install, 31<br>
files were created in the index/us_rhodeisland folder, but the process<br>
goes on endlessly. I interrupted this process and tried again but the<br>
same happened. I'm wondering how long time does this approximately take<br>
on my laptop, which has 8 GB of ram, and a, Intel i5 quadcore CPU?<br>
<br>
Best<br>
Jiayue<br>
<br>
On 13/11/16 06:19, Hardie, Andrew wrote:<o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">This error message suggests that your <text> elements lack valid ID<br>
codes.<br>
<br>
The most likely reason for [UNREADABLE] is that you have declared a<br>
primary annotation, e.g. a part of speech tag, but the annotation in<br>
question does not exist. This can happen if you use a template that<br>
your data does not match, for instance.<br>
<br>
best<br>
<br>
Andrew.<br>
<br>
-----Original Message----- From: <a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">
cwb-bounces@sslmit.unibo.it</a><br>
[mailto:<a href="mailto:cwb-bounces@sslmit.unibo.it" target="_blank">cwb-bounces@sslmit.unibo.it</a>] On Behalf Of Jiayue Wang Sent:<br>
11 November 2016 20:17 To: Open source development of the Corpus<br>
WorkBench Subject: [CWB] Can't create metadata<br>
<br>
Hi,<br>
<br>
After a full installation of CQBweb I installed a corpus called<br>
"us_rhodeisland" (including 2 files, a raw text, and a TreeTagger<br>
tagged text) without metadata. Since I have no idea what a metadata<br>
file looks like, I selected "No thanks, I'll run this myself (safer<br>
for very large corpora)" and clicked "Create minimalist metadata<br>
table" and saw the following error message:<br>
<br>
<br>
A MySQL query did not run successfully!<br>
<br>
<br>
Original query: insert into<br>
___temp_cqp_text_positions_for_us_rhodeisland (text_id, cqp_begin,<br>
cqp_end) VALUES ('', 0, 55858),('', 55859, 3058358) /* from User:<br>
admin | Function: do_append_mysql_comment() | 2016-Nov-11 20:04:20<br>
*/<br>
<br>
<br>
Error # 1062: Duplicate entry '' for key 'PRIMARY'<br>
<br>
<br>
BTW, when I try a standard query, each concordance line begins with<br>
"[UNREADABLE] [UNREADABLE] [UNREADABLE]". What is the most likely<br>
reason?<br>
<br>
Any help is appreciated, thanks!<br>
<br>
Jiayue Wang _______________________________________________ CWB<br>
mailing list <a href="mailto:CWB@sslmit.unibo.it" target="_blank">CWB@sslmit.unibo.it</a><br>
<a href="http://liste.sslmit.unibo.it/mailman/listinfo/cwb" target="_blank">http://liste.sslmit.unibo.it/mailman/listinfo/cwb</a><br>
_______________________________________________ CWB mailing list<br>
<a href="mailto:CWB@sslmit.unibo.it" target="_blank">CWB@sslmit.unibo.it</a><br>
<a href="http://liste.sslmit.unibo.it/mailman/listinfo/cwb" target="_blank">http://liste.sslmit.unibo.it/mailman/listinfo/cwb</a><o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">_______________________________________________<br>
CWB mailing list<br>
<a href="mailto:CWB@sslmit.unibo.it" target="_blank">CWB@sslmit.unibo.it</a><br>
<a href="http://liste.sslmit.unibo.it/mailman/listinfo/cwb" target="_blank">http://liste.sslmit.unibo.it/mailman/listinfo/cwb</a><br>
_______________________________________________<br>
CWB mailing list<br>
<a href="mailto:CWB@sslmit.unibo.it" target="_blank">CWB@sslmit.unibo.it</a><br>
<a href="http://liste.sslmit.unibo.it/mailman/listinfo/cwb" target="_blank">http://liste.sslmit.unibo.it/mailman/listinfo/cwb</a><o:p></o:p></p>
<p class="MsoNormal">_______________________________________________<br>
CWB mailing list<br>
<a href="mailto:CWB@sslmit.unibo.it" target="_blank">CWB@sslmit.unibo.it</a><br>
<a href="http://liste.sslmit.unibo.it/mailman/listinfo/cwb" target="_blank">http://liste.sslmit.unibo.it/mailman/listinfo/cwb</a><o:p></o:p></p>
</div>
</div>
</div>
</body>
</html>