[CWB] Question re: parallel corpora set up

[Stephanie Evert]

Hi Graham!

> I have a quick question regarding parallel corpora set up. Does the cwb-align command actually do the alignment, as I understand it does, from the terminal messages, and, if so, is it possible to override the cwb alignment to impose one's own alignment?

Correct. cwb-align ist the actual sentence aligner and does indeed use a rather simplistic and old-fashioned algorithm. It outputs the alignment in a text format (basically listing start/end corpus positions of corresponding regions in source and target corpus), which is then indexed with cwb-align-encode for use with CWB.

Which means that you can "simply" generate your own alignment in this text format and then run cwb-align-encode. You can also preview your alignment with cwb-align-show first to check that you've got the format right. 

> I'm asking because I remember reading that the cwb-align command was not a very efficient aligner, and so far I prefer to do the alignment myself in advance, and to give cwb-align files with exactly the same number of alignment fields (s tags). Despite this, I have the impression that the command can knock this initial, hand-checked alignment out of kilter, which is a shame, and so an override flag, if it existed, would be nice.

If you have a pre-alignment (which might be at a different level than sentences), you can indicate this to cwb-align, which will force the aligner to only consider sentence correspondences that are consistent with the pre-alignment. A trick I've sometimes used is to specify the same s-attribute that encodes the pre-alignment (keep in mind that there must be a single region for each pre-alignment block) as alignment grid, which keeps cwb-align from making any changes.

It is usually much easier, though, to use cwb-align-import from the CWB/Perl modules. This script is intended to import an external alignment given in terms of sentence IDs (which must be annotated in the corpus via an s-attribute). From its documentation:

>     As an example, the line
> 
>         id_1 id_2 id_3  <TAB>  id_2 id_3  [ <TAB> ... ]
> 
>     indicates that the first three sentences of the source corpus are
>     aligned to the second and third sentence of the target corpus. Further
>     TAB-delimited fields (e.g. specifying the confidence in an alignment
>     bead) are silently ignored.



> A related question: what does the "no alignments found" message most often mean? I'm setting up a three-way corpus English-French-Polish. The source text is English, the French and Polish texts are translations but, since the alignments are based on the English, I would like to be able to provide the option for searching in one translated corpus, and looking for the equivalent in the other translated corpus. So far, things work perfectly Eng-Fre and Eng-Pol but I get the "no alignments found" message when searching in "Fre-Pol" and yet I think I've set it all up identically...

"no alignment found" is the message you get if a query match in your source corpus isn't aligned to the target corpus. Since alignments don't have to cover the corpus completely, this may well happen.

However, if you always get the message, there's something wrong with the Fre-Pol alignment. Perhaps a first step would be to use cwb-align-decode on the French corpus to see which alignment beads you actually have.

Hope this helps,
Stephanie
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://liste.sslmit.unibo.it/pipermail/cwb/attachments/20260710/8441a654/attachment-0001.html>