[CWB] low-level python interface to CWB?

[Rob Malouf]

Hi,

I'm using cqp in a subprocess as a backend for a turbogears  
application and it works quite well.  One of the advantages to running  
cqp in a subprocess is that you can have multiple python threads  
interacting with multiple cqp subprocesses simultaneously (for the  
operations that the command-line cqp doesn't give me, I use small C  
programs running in subprocesses).  This is especially useful for when  
several students are trying expensive queries at the same time -- our  
server slows down, but the web application doesn't get stuck and can  
still respond to new requests.  If you access cqp directly through  
something like a SWIG wrapper, you'll run into problems with the  
global interpreter lock.  Foreign function calls can't normally be  
interrupted by another python thread (unless you specifically allow  
it), so long-running cqp queries will potentially prevent your  
application from responding to new requests. One careless or greedy  
user can prevent anyone else from using the application.

Just something to watch out for...

---
Rob Malouf <rmalouf at mail.sdsu.edu>
Department of Linguistics and Asian/Middle Eastern Languages
San Diego State University



On Nov 6, 2009, at 6:18 AM, Jørg Asmussen wrote:

> Hi,
>
> good initiative! I once intended to write that wrapper but never got  
> started due to lack of time, so I think it is a great idea if you  
> could do that!
>
> Best,
>
> Joerg
>
> --
>
> Jørg Asmussen
>
> DET DANSKE SPROG- OG LITTERATURSELSKAB, DSL
> Society for Danish Language and Literature
>
> AFD. FOR DIGITALE ORDBØGER OG TEKSTKORPORA
> Dept. for Digital Dictionaries and Text Corpora
>
> Christians Brygge 1 Tel. +45 33 13 84 20 - 9632
> DK-1219 København K Fax  +45 33 14 06 08
>
> http://www.dsl.dk
> http://dsl.dk/ja/
>
>
> On 5 Nov 2009, at 22:46, Yannick Versley wrote:
>
>> Hello,
>>
>> I've begun writing a Cython-based wrapper around CQP's C API to
>> complement the subprocess-based wrapper that Joerg Asmussen
>> sent on the mailing list a while ago -- the issue would be to grab
>> POS/word/lemmas on the fly when doing linguistic processing,
>> but also be able to retrieve documents and sentences given
>> a corpus position (i.e., similar things that CQPweb does with
>> the Perl interface, but in Python and - probably - Django).
>>
>> Just in case, did anyone already do something like this (and
>> would be willing to share)? I'd hate to write the N+1th wrapper
>> when there actually is some usable alternative.
>>
>> Best wishes,
>> Yannick
>> _______________________________________________
>> CWB mailing list
>> CWB at sslmit.unibo.it
>> http://devel.sslmit.unibo.it/mailman/listinfo/cwb
>
>
> _______________________________________________
> CWB mailing list
> CWB at sslmit.unibo.it
> http://devel.sslmit.unibo.it/mailman/listinfo/cwb
>