<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Verdana",sans-serif;
color:#1F497D;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Verdana",sans-serif;
color:#1F497D;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
span.EmailStyle21
{mso-style-type:personal;
font-family:"Verdana",sans-serif;
color:#1F497D;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Verdana",sans-serif;
color:#1F497D;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US">>>><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">The need to post-process kwic output in order to remove non-original spaces may not arise at all if I am correct in thinking that David's suggestion, some time ago, of <g/> as
glue is actually implemented in the Sketch Engine as meaning "leave no space between the preceding and following tokens". In cwb, the " <g/> " comes through unchanged into the kwic context, and needs post-processing to remove it. It might be an idea to have
cqp implement <g/> like that too.</span><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><<<<o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US">No, this is not implemented in Manatee (and definitely will never be in CQP). David was pointing out that you could use such an s-attribute
to indicate the presence/absence of orthographic space in the index – but to actually change it into spaces where necessary in the concordance would still be the job of the wrapper script. Basically this is an alternative to doing the same job via a binary
p-attribute. Either way, translation of format is required outside the query program itself.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US">the underlying engine is appropriately neutral about the semantics of any attribute name… so specifying a specific s-attribute as meaning
“glue” would not be something ever to build in at the system level of CQP. Front ends can of course impose whatever requirements about attribute semantics that they like.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US">best<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Andrew.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> cwb-bounces@sslmit.unibo.it [mailto:cwb-bounces@sslmit.unibo.it]
<b>On Behalf Of </b>Ciarán Ó Duibhín<br>
<b>Sent:</b> 27 March 2018 17:32<br>
<b>To:</b> Open source development of the Corpus WorkBench <cwb@sslmit.unibo.it><br>
<b>Subject:</b> Re: [CWB] Suggestion: user intervention in constructing an index<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Thank you, Vlado. That's a really neat feature of NoSketch but for me it is better still to break "seanbhean" and "sean-bhean" each into two tokens in the vertical file:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> (1) word="sean" bzw "sean-"; demut="sean"</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> (2) word="bhean"; demut="bean"</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">The query is then made on the "demut" p-attribute ("demut" is like how people use "lemma", but linguistically this is not a lemma). This results in:<br>
• a search for "sean" and "bean" together will retrieve all of "seanbhean", "sean-bhean" and "sean bhean" (NoSketch "sean--bhean" can do that too)<br>
• a search for "bean" will retrieve all of those, as well as all the other examples of "bean"; and correspondingly for a search for "sean".<br>
That is what will best suit the lexicographical user of the corpus.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Thank you, Andrew, for showing how to display p-attributes in the kwic line; and for clarifying that CWB/Perl has not been made to work under Windows. I have only a couple of
comments.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">>> It will avoid having a permanent multi-column file outside the corpus, but won't the multiple columns still exist internally in some form within the corpus? :-(<br>
<span style="color:blue">> Yes, but it has to. If you want to store more than one item of separately-searchable information about each token – in this case, your word/demut combination – then you have to have multiple attributes.
</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">OK, I see that cwb's architecture requires that.<br>
<br>
<span style="color:blue">> If you want to avoid at all costs multiple attributes being stored under the hood then…. you don’t want to use CWB! (Or Manatee, since that works on precisely the same principle.)</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Yes, I don't need to search on "word", and the program I use in Windows stores only "demut" in the index, and fetches the kwic contexts from a copy of the running text. (Incidentally,
this means that "non-original spaces" never enter the contexts.) Given that cwb doesn't need a copy of the running text, the storage requirements of the two methods should be similar.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">A script to post-process output from cqp before displaying it:<br>
<span style="color:blue">><br>
>- read input line from user standard input<br>
>- pass input line to CQP slave process (either directly, or via a library)<br>
>- if necessary, read output line(s) from CQP slave process<br>
>- modify output line(s) as per whatever requirements you have*<br>
>- print output line(s) to standard output<br>
>- print prompt for next user input.<br>
><br>
>The user then runs your script instead of running CQP.<br>
><br>
>(*) If you use one of the libraries, an easy way to do this is by specifying a "line handler" function when you call "exec/execute()" >or "query()".</span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Many thanks for that explanation. I should also look at existing front-ends to cqp, which must do something like this series of steps, and may allow the user some control over
the fourth step. The most promising of these may be TXM.<br>
<br>
The need to post-process kwic output in order to remove non-original spaces may not arise at all if I am correct in thinking that David's suggestion, some time ago, of <g/> as glue is actually implemented in the Sketch Engine as meaning "leave no space between
the preceding and following tokens". In cwb, the " <g/> " comes through unchanged into the kwic context, and needs post-processing to remove it. It might be an idea to have cqp implement <g/> like that too.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><br>
Regards,<br>
Ciarán.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
</body>
</html>