<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Fri, Mar 31, 2017 at 5:48 AM, Stefan Evert <span dir="ltr"><<a href="mailto:stefanML@collocations.de" target="_blank">stefanML@collocations.de</a>></span> wrote:</div><div class="gmail_quote"><br></div><div class="gmail_quote">Hi Stefan,</div><div class="gmail_quote"><br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">As Andrew pointed out, the root cause of the problem is that your corpus seems to contain a sentence of several hundred million tokens (so it formats to over 2 GiB). This easily happens if there's a missing </s> tag somewhere in the middle and you encode with "-S s:0" (because the following sentences are nested in the one that hasn't been closed). You probably got warnings about missing </s> tags when you encoded the corpus, didn't you?<br></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">If you can't be sure that the structural annotation in a corpus is well-formed XML, it's often better to do a flat encode with "-S s". </blockquote><div><br></div><div>I encoded this corpus some years ago, so I have no recollection of what warnings I received. But I can say this was the set of options I used:</div><div><br></div><div><font face="monospace, monospace">-xsB -P lemma -P pos -P spos -P tag -P subtag -S s:0 -S p:0 -S text:0+id+corpus+tagger+label+channel+audience+purpose+type+medium+field+area+location+source</font><br></div><br>And I do indeed have <font face="monospace, monospace">-S s:0</font>, as well as <font face="monospace, monospace">-S p:0</font> and even <font face="monospace, monospace">-S text:0+</font>... From reading the encoding tutorial, the :0 option seems to prevent nested elements, which sounded like a good idea... at the time. Would it be advisable to drop the <font face="monospace, monospace">:0</font> from all three elements above, or only from s:0?</div><div class="gmail_quote"><br></div><div class="gmail_quote"><div>Cheers,</div><div>Scott</div><div><br></div></div>-- <br><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div style="font-size:12.7273px">Dr. Scott Sadowsky<br>Profesor Asistente de Lingüística</div><div dir="ltr" style="font-size:12.7273px">Pontificia Universidad Católica de Chile<br></div><div dir="ltr" style="font-size:12.7273px"><br></div><div dir="ltr" style="font-size:12.7273px">ssadowsky gmail com</div><div dir="ltr" style="font-size:12.7273px">scsadowsky uc cl<br><a href="http://sadowsky.cl/" target="_blank">http://sadowsky.cl/</a></div><div dir="ltr" style="font-size:12.7273px"> </div></div></div></div></div></div></div></div></div></div></div>
</div></div>