[CWB] cwb testing

[lars.nygaard at iln.uio.no]

>
> I attach the Perl source code of a CQP test suite that a student
> developed at the IMS several years ago.  Although we weren't
> completely happy with it (that's why we didn't build a more
> comprehensive testing environment around this test suite), it might
> be a useful starting point - or it might at least give you some ideas
> how to tackle the problem.

Well, a lot of it was in German, and my mastery of German is rather weak,
so that will limit the usefulness. I think I'll just start over (in fact,
I just did).

What would be useful for me would be a list of functions/features that
need to be supported.

>
> The test suite also includes a number of test queries and expected
> output (if you have the right corpus ...).

I could not find any ...

> I think that in most cases we could just run CQP through the Perl
> modules.  We should discover errors regardless of whether they're
> caused by CQP or by the Perl interface, and I don't think it's
> essential for the test suite to make that distinction.  On the other
> hand, an option to run the entire test suite with shell commands
> instead of the Perl interface would definitely be neat, if that is
> feasible.

It is indeed feasible, and I think we need it for queries that are not
supported by the Perl interface (unless I can figure out a way to send
several commands), for example dump/undump.

> I suppose this would be an entirely different test suite, which needs
> much less flexibility in terms of the features that it can test.
> Bascially, all you have to do is encode an enclosed or automatically
> generated test corpus, then decode everything to compare with the
> original text, and decode the complete index to ensure that it's
> correct.

Even simpler: If the encoding/indexing is incorrect, the test queries will
reveal it, so we can give the encode/decode procedure lower priority.

cheers,
lars