<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns="http://www.w3.org/TR/REC-html40" xmlns:v =
"urn:schemas-microsoft-com:vml" xmlns:o =
"urn:schemas-microsoft-com:office:office" xmlns:w =
"urn:schemas-microsoft-com:office:word" xmlns:m =
"http://schemas.microsoft.com/office/2004/12/omml"><HEAD>
<META content="text/html; charset=utf-8" http-equiv=Content-Type>
<META name=GENERATOR content="MSHTML 9.00.8112.16872"><!--[if !mso]>
<STYLE>v\:* {
BEHAVIOR: url(#default#VML)
}
o\:* {
BEHAVIOR: url(#default#VML)
}
w\:* {
BEHAVIOR: url(#default#VML)
}
.shape {
BEHAVIOR: url(#default#VML)
}
</STYLE>
<![endif]-->
<STYLE><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Verdana",sans-serif;
color:#1F497D;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Verdana",sans-serif;
color:#1F497D;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:96758683;
mso-list-type:hybrid;
mso-list-template-ids:1228724376 1718550506 134807555 134807557 134807553 134807555 134807557 134807553 134807555 134807557;}
@list l0:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></STYLE>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></HEAD>
<BODY lang=EN-GB bgColor=white vLink=purple link=blue>
<DIV><FONT size=2 face=Arial>Thanks again Andrew.</FONT></DIV>
<BLOCKQUOTE
style="BORDER-LEFT: #000000 2px solid; PADDING-LEFT: 5px; PADDING-RIGHT: 0px; MARGIN-LEFT: 5px; MARGIN-RIGHT: 0px">
<DIV class=WordSection1>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US">>></SPAN><SPAN
style="FONT-FAMILY: 'Courier New'; FONT-SIZE: 10pt"> I am not comfortable with
the idea of storing two columns to hold things which (unlike with normal
lemmatisation) can be automatically generated from one column
— during the indexing process, if access by a user-supplied script
were usable there, acting on the text shown in column 1 to produce what
is shown in column 2.</SPAN><o:p></o:p></P>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US"><o:p> </o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US">But
as I’ve explained, there is already a way to do that if you don’t want a
permanent multi-column file – just put your user script into a pipeline with
cwb-encode on the end. IE:<o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US"><o:p> </o:p></SPAN></P>
<UL style="MARGIN-TOP: 0cm" type=disc>
<LI style="COLOR: #1f497d; MARGIN-LEFT: 0cm; mso-list: l0 level1 lfo1"
class=MsoListParagraph><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; FONT-SIZE: 10pt; mso-fareast-language: EN-US">cat
one-col-file | column-transform-script | cwb-encode
[options]</SPAN></LI></UL></DIV></BLOCKQUOTE>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US"><o:p><FONT
color=#000000 face=Arial>OK, I had thought your pipeline suggestion applied
only to your first answer (transforming "word"), but I see now that it can
apply to the second answer too (transform "word" and add "lemma").
Pipelining is not something I have worked with in Windows/DOS, but I
assume it will be feasible.</FONT></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US"><o:p><FONT
color=#000000 face=Arial></FONT></o:p></SPAN> </P>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US"><o:p><FONT
color=#000000 face=Arial>It will avoid having a permanent multi-column file
outside the corpus, but won't the multiple columns still exist internally
in some form within the corpus? :-(</FONT></o:p></SPAN></P>
<BLOCKQUOTE
style="BORDER-LEFT: #000000 2px solid; PADDING-LEFT: 5px; PADDING-RIGHT: 0px; MARGIN-LEFT: 5px; MARGIN-RIGHT: 0px">
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US"><o:p></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US">Some
display systems like BNCweb remove non-original orthographic spaces from the
CQP concordance. (BNCweb does this by having an additional binary p-attribute
storing the “orthographic-space-after” data) </SPAN><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US">...</SPAN><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US"><o:p> </o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US"><o:p></o:p></SPAN> </P>
<P class=MsoNormal>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US"><o:p></o:p></SPAN></P><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US">...
you can address the second point (of rendering) by writing a display
program which lays things out to your liking using one of the interface
libraries i.e. the CWB-Perl modules or the cqp.inc.php module from
CQPweb. </SPAN><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US">Or,
if you prefer, just write your rendering script to pipe text in and out of a
cqp slave instance (which is what the Perl and PHP libraries do behind the
scenes). <o:p></o:p></SPAN></P></BLOCKQUOTE><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US"><o:p><FONT
color=#000000>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US"><o:p><FONT
color=#000000 face=Arial>I'm not sure whether these two things — the
additional binary attribute, and CWB-Perl — are two independent
suggestions, or two aspects of the same suggestion.</FONT></o:p></SPAN></P>
<P class=MsoNormal><FONT size=2 face=Arial></FONT> </P>
<P class=MsoNormal><FONT size=2 face=Arial>I'm definitely interested in copying
the BNCweb idea. Where can I get info about binary p-attributes?
Where should I look to find out about reading this attribute from a script
or program?</FONT></P>
<P class=MsoNormal><FONT size=2 face=Arial></FONT> </P>
<P class=MsoNormal><FONT size=2 face=Arial>If I need to use CWB-Perl, or if
using it would make things easier, I notice that the README in CWB-Perl 2.2.102
mentions "cwb-config", but <A
href="https://github.com/cran/rcqp/blob/master/src/cwb/man/cwb-config.pod">https://github.com/cran/rcqp/blob/master/src/cwb/man/cwb-config.pod</A> says
that cwb-config is not yet available for Windows.</FONT></FONT></o:p></SPAN></P>
<P class=MsoNormal><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US"><o:p><FONT
color=#000000></FONT></o:p></SPAN> </P><SPAN
style="FONT-FAMILY: 'Verdana',sans-serif; COLOR: #1f497d; FONT-SIZE: 10pt; mso-fareast-language: EN-US"><o:p><FONT
color=#000000>
<P class=MsoNormal><FONT size=2 face=Arial>Regards,</FONT></P>
<P class=MsoNormal><FONT size=2
face=Arial>Ciarán.</FONT></FONT></o:p></SPAN></P></BODY></HTML>