[CWB] Incorrect total words count in a Traditional Chinese corpus on CQPweb

Hardie, Andrew a.hardie at lancaster.ac.uk
Mon Jul 16 08:11:42 CEST 2018 

OK, this one has taken a while to work out.

There are two distinct errors here (repeated in the log). Each affects a stage in frequency list setup such that all subsequent phases fail. Here’s the first:

[Mon Jun 25 04:56:37.677126 2018] [core:notice] [pid 1397] AH00094: Command line: '/usr/sbin/apache2'
Problem: No output generated -- no items?
/var/cqpweb/index/canton1__freq/__freq.hcd: No such file or directory
ERROR: reading /var/cqpweb/index/canton1__freq/__freq.hcd failed. Aborted.

This means that the encoding of the CWB frequency data has failed. Possible cause: www-data doesn’t have write permission for your data directory. OR, the encoding / indexing of the original corpus failed. Check the contents of your “/var/cqpweb/index” folder to find out which!

(You can also look at the “indexing_notes” column in the corpus_info table in the MySQL database for possible errors.)

And here’s the other error:


::1 - - [25/Jun/2018:05:03:20 +0100] "GET /cqpweb/canton1/execute.php?function=populate_corpus_cqp_positions&args=canton1&locationAfter=index.php%3FthisQ%3DmanageFreqLists%26uT%3Dy&uT=y HTTP/1.1" 500 185 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
[Mon Jun 25 05:03:20.068066 2018] [:error] [pid 2262] [client ::1:43456] PHP Fatal error:  Call to undefined method CQP::get_corpus_tokens() in /var/www/html/cqpweb/lib/metadata.inc.php on line 418, referer: http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y

This suggests that some of your code files are out of sync. The function get_corpus_tokens() IS in the CQP object if it is at the same version as the other code files.  try updating just cqp.inc.php to the latest version.

Generally – to  be clearer about what stage the error occurs at, try running the commandline script for frequency list setup. This will print the error messages (plus progress info) direct to the terminal. This ought to be easier to diagnose than the Aapche error log.

Alternatively, edit your PHP config file to contain the line

display_errors = On

which will send error messages  to the browser, instead of the apache log.

hope this helps

best

Andrew.

From: cwb-bounces at sslmit.unibo.it [mailto:cwb-bounces at sslmit.unibo.it] On Behalf Of Hermann Lai
Sent: 04 July 2018 19:57
To: Open source development of the Corpus WorkBench <cwb at sslmit.unibo.it>
Subject: Re: [CWB] Incorrect total words count in a Traditional Chinese corpus on CQPweb

Sorry for late reply.

These logs are in /var/log/apache2

error.log

[Mon Jun 25 04:56:37.677092 2018] [mpm_prefork:notice] [pid 1397] AH00163: Apache/2.4.12 (Ubuntu) configured -- resuming normal operations
[Mon Jun 25 04:56:37.677126 2018] [core:notice] [pid 1397] AH00094: Command line: '/usr/sbin/apache2'
Problem: No output generated -- no items?
/var/cqpweb/index/canton1__freq/__freq.hcd: No such file or directory
ERROR: reading /var/cqpweb/index/canton1__freq/__freq.hcd failed. Aborted.
[Mon Jun 25 05:02:35.306755 2018] [:error] [pid 2258] [client ::1:43448] PHP Notice:  Undefined variable: visible_options in /var/www/html/cqpweb/lib/indexforms-adminhome.inc.php on line 102, referer: http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y
[Mon Jun 25 05:02:35.306961 2018] [:error] [pid 2258] [client ::1:43448] PHP Notice:  Undefined variable: visible_options in /var/www/html/cqpweb/lib/indexforms-adminhome.inc.php on line 102, referer: http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y
[Mon Jun 25 05:02:35.306975 2018] [:error] [pid 2258] [client ::1:43448] PHP Notice:  Undefined variable: visible_options in /var/www/html/cqpweb/lib/indexforms-adminhome.inc.php on line 102, referer: http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y
[Mon Jun 25 05:02:35.307085 2018] [:error] [pid 2258] [client ::1:43448] PHP Notice:  Undefined variable: visible_options in /var/www/html/cqpweb/lib/indexforms-adminhome.inc.php on line 102, referer: http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y
[Mon Jun 25 05:02:35.307196 2018] [:error] [pid 2258] [client ::1:43448] PHP Notice:  Undefined variable: visible_options in /var/www/html/cqpweb/lib/indexforms-adminhome.inc.php on line 102, referer: http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y
[Mon Jun 25 05:02:39.142641 2018] [:error] [pid 2258] [client ::1:43448] PHP Notice:  Undefined variable: visible_options in /var/www/html/cqpweb/lib/indexforms-adminhome.inc.php on line 102, referer: http://localhost/cqpweb/adm/index.php?thisF=deleteCorpus&corpus=canton1&uT=y
[Mon Jun 25 05:02:39.142845 2018] [:error] [pid 2258] [client ::1:43448] PHP Notice:  Undefined variable: visible_options in /var/www/html/cqpweb/lib/indexforms-adminhome.inc.php on line 102, referer: http://localhost/cqpweb/adm/index.php?thisF=deleteCorpus&corpus=canton1&uT=y
[Mon Jun 25 05:02:39.142860 2018] [:error] [pid 2258] [client ::1:43448] PHP Notice:  Undefined variable: visible_options in /var/www/html/cqpweb/lib/indexforms-adminhome.inc.php on line 102, referer: http://localhost/cqpweb/adm/index.php?thisF=deleteCorpus&corpus=canton1&uT=y
[Mon Jun 25 05:02:39.143014 2018] [:error] [pid 2258] [client ::1:43448] PHP Notice:  Undefined variable: visible_options in /var/www/html/cqpweb/lib/indexforms-adminhome.inc.php on line 102, referer: http://localhost/cqpweb/adm/index.php?thisF=deleteCorpus&corpus=canton1&uT=y
[Mon Jun 25 05:03:05.137906 2018] [:error] [pid 2261] [client ::1:43454] PHP Notice:  Undefined variable: primary_classification in /var/www/html/cqpweb/lib/metadata-admin.inc.php on line 134, referer: http://localhost/cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y
[Mon Jun 25 05:03:11.973121 2018] [:error] [pid 2261] [client ::1:43454] PHP Fatal error:  Call to undefined method CQP::get_corpus_tokens() in /var/www/html/cqpweb/lib/metadata.inc.php on line 418, referer: http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y
Problem: No output generated -- no items?
/var/cqpweb/index/canton1__freq/__freq.hcd: No such file or directory
ERROR: reading /var/cqpweb/index/canton1__freq/__freq.hcd failed. Aborted.
[Mon Jun 25 05:03:20.068066 2018] [:error] [pid 2262] [client ::1:43456] PHP Fatal error:  Call to undefined method CQP::get_corpus_tokens() in /var/www/html/cqpweb/lib/metadata.inc.php on line 418, referer: http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y
[Mon Jun 25 05:03:37.731910 2018] [:error] [pid 2259] [client ::1:43460] PHP Fatal error:  Call to undefined method CQP::get_corpus_tokens() in /var/www/html/cqpweb/lib/metadata.inc.php on line 418, referer: http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y
[Wed Jul 04 19:46:02.460824 2018] [mpm_prefork:notice] [pid 1413] AH00163: Apache/2.4.12 (Ubuntu) configured -- resuming normal operations
[Wed Jul 04 19:46:02.476985 2018] [core:notice] [pid 1413] AH00094: Command line: '/usr/sbin/apache2'

error.log.1

[Sun Jun 24 10:59:49.540267 2018] [mpm_prefork:notice] [pid 1415] AH00163: Apache/2.4.12 (Ubuntu) configured -- resuming normal operations
[Sun Jun 24 10:59:49.540300 2018] [core:notice] [pid 1415] AH00094: Command line: '/usr/sbin/apache2'
[Sun Jun 24 19:20:47.348163 2018] [mpm_prefork:notice] [pid 1442] AH00163: Apache/2.4.12 (Ubuntu) configured -- resuming normal operations
[Sun Jun 24 19:20:47.383451 2018] [core:notice] [pid 1442] AH00094: Command line: '/usr/sbin/apache2'
[Sun Jun 24 19:29:43.586913 2018] [:error] [pid 1470] [client ::1:45934] PHP Notice:  Undefined variable: primary_classification in /var/www/html/cqpweb/lib/metadata-admin.inc.php on line 134, referer: http://localhost/cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y
Problem: No output generated -- no items?
/var/cqpweb/index/canton1__freq/__freq.hcd: No such file or directory
ERROR: reading /var/cqpweb/index/canton1__freq/__freq.hcd failed. Aborted.
[Sun Jun 24 20:01:54.009133 2018] [mpm_prefork:notice] [pid 1671] AH00163: Apache/2.4.12 (Ubuntu) configured -- resuming normal operations
[Sun Jun 24 20:01:54.026868 2018] [core:notice] [pid 1671] AH00094: Command line: '/usr/sbin/apache2'
[Mon Jun 25 04:37:37.219278 2018] [mpm_prefork:notice] [pid 1397] AH00163: Apache/2.4.12 (Ubuntu) configured -- resuming normal operations
[Mon Jun 25 04:37:37.312244 2018] [core:notice] [pid 1397] AH00094: Command line: '/usr/sbin/apache2'
[Mon Jun 25 04:56:09.637419 2018] [:error] [pid 1426] [client ::1:43414] PHP Notice:  Undefined property: CQPwebEnvConfig::$rss_feed_available in /var/www/html/cqpweb/lib/library.inc.php on line 1307, referer: http://localhost/
[Mon Jun 25 04:56:13.821643 2018] [:error] [pid 1426] [client ::1:43414] PHP Notice:  Undefined variable: visible_options in /var/www/html/cqpweb/lib/indexforms-adminhome.inc.php on line 102, referer: http://localhost/cqpweb/
[Mon Jun 25 04:56:13.821844 2018] [:error] [pid 1426] [client ::1:43414] PHP Notice:  Undefined variable: visible_options in /var/www/html/cqpweb/lib/indexforms-adminhome.inc.php on line 102, referer: http://localhost/cqpweb/
[Mon Jun 25 04:56:13.821860 2018] [:error] [pid 1426] [client ::1:43414] PHP Notice:  Undefined variable: visible_options in /var/www/html/cqpweb/lib/indexforms-adminhome.inc.php on line 102, referer: http://localhost/cqpweb/
[Mon Jun 25 04:56:13.821972 2018] [:error] [pid 1426] [client ::1:43414] PHP Notice:  Undefined variable: visible_options in /var/www/html/cqpweb/lib/indexforms-adminhome.inc.php on line 102, referer: http://localhost/cqpweb/
[Mon Jun 25 04:56:13.822084 2018] [:error] [pid 1426] [client ::1:43414] PHP Notice:  Undefined variable: visible_options in /var/www/html/cqpweb/lib/indexforms-adminhome.inc.php on line 102, referer: http://localhost/cqpweb/
[Mon Jun 25 04:56:16.759475 2018] [:error] [pid 1426] [client ::1:43414] PHP Notice:  Undefined property: CQPwebEnvConfig::$rss_feed_available in /var/www/html/cqpweb/lib/library.inc.php on line 1307, referer: http://localhost/cqpweb/adm/
[Mon Jun 25 04:56:27.389770 2018] [:error] [pid 1426] [client ::1:43414] PHP Fatal error:  Call to undefined method CQP::get_corpus_tokens() in /var/www/html/cqpweb/lib/metadata.inc.php on line 418, referer: http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y
[Mon Jun 25 04:56:37.643089 2018] [mpm_prefork:notice] [pid 1397] AH00171: Graceful restart requested, doing restart
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message

access.log

::1 - - [25/Jun/2018:04:58:16 +0100] "GET /cqpweb/canton1/execute.php?function=make_cwb_freq_index&args=canton1&locationAfter=index.php%3FthisQ%3DmanageFreqLists%26uT%3Dy&uT=y HTTP/1.1" 302 286 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:58:17 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y HTTP/1.1" 200 2636 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:58:18 +0100] "GET /cqpweb/canton1/execute.php?function=corpus_make_freqtables&args=canton1&locationAfter=index.php%3FthisQ%3DmanageFreqLists%26uT%3Dy&uT=y HTTP/1.1" 302 285 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:58:18 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y HTTP/1.1" 200 2636 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:58:22 +0100] "GET /cqpweb/canton1/index.php?thisQ=corpusMetadata&uT=y HTTP/1.1" 200 2144 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:58:52 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y HTTP/1.1" 200 2522 "http://localhost/cqpweb/canton1/index.php?thisQ=corpusMetadata&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:58:59 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y HTTP/1.1" 200 2637 "http://localhost/cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:02:35 +0100] "GET /cqpweb/adm/ HTTP/1.1" 200 2665 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:02:36 +0100] "GET /cqpweb/adm/index.php?thisF=deleteCorpus&corpus=canton1&uT=y HTTP/1.1" 200 1848 "http://localhost/cqpweb/adm/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:02:38 +0100] "GET /cqpweb/adm/index.php?sureyouwantto=yes&admFunction=deleteCorpus&corpus=canton1&uT=y HTTP/1.1" 302 254 "http://localhost/cqpweb/adm/index.php?thisF=deleteCorpus&corpus=canton1&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:02:39 +0100] "GET /cqpweb/adm/index.php HTTP/1.1" 200 2574 "http://localhost/cqpweb/adm/index.php?thisF=deleteCorpus&corpus=canton1&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:02:41 +0100] "GET /cqpweb/adm/index.php?thisF=installCorpus&uT=y HTTP/1.1" 200 4878 "http://localhost/cqpweb/adm/index.php" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:02:42 +0100] "GET /cqpweb/adm/index.php?thisF=installCorpusIndexed&uT=y HTTP/1.1" 200 2646 "http://localhost/cqpweb/adm/index.php?thisF=installCorpus&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:02:49 +0100] "GET /cqpweb/adm/index.php?corpus_cwb_name=canton1&corpus_description=Cantonese1&corpus_useDefaultRegistry=1&corpus_cwb_registry_folder=&cssCustom=0&cssBuiltIn=CQPweb-aqua.css&cssCustomUrl=&admFunction=installCorpusIndexed&uT=y HTTP/1.1" 302 313 "http://localhost/cqpweb/adm/index.php?thisF=installCorpusIndexed&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:02:49 +0100] "GET /cqpweb/adm/index.php?thisF=installCorpusDone&newlyInstalledCorpus=canton1&uT=y HTTP/1.1" 200 1853 "http://localhost/cqpweb/adm/index.php?thisF=installCorpusIndexed&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:02:53 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y HTTP/1.1" 200 4161 "http://localhost/cqpweb/adm/index.php?thisF=installCorpusDone&newlyInstalledCorpus=canton1&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:02:56 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y HTTP/1.1" 200 4161 "http://localhost/cqpweb/adm/index.php?thisF=installCorpusDone&newlyInstalledCorpus=canton1&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:05 +0100] "GET /cqpweb/canton1/metadata-admin.php?dataFile=canton1.meta&useMetadataTemplate=%7E%7EcustomMetadata&fieldHandle1=texttype&fieldDescription1=Text+type&fieldType1=1&fieldHandle2=&fieldDescription2=&fieldType2=1&fieldHandle3=&fieldDescription3=&fieldType3=1&fieldHandle4=&fieldDescription4=&fieldType4=1&fieldHandle5=&fieldDescription5=&fieldType5=1&createMetadataRunFullSetupAfter=0&mdAction=createMetadataFromFile&fieldCount=5&corpus=canton1&uT=y HTTP/1.1" 302 285 "http://localhost/cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:05 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y HTTP/1.1" 200 2521 "http://localhost/cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:09 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y HTTP/1.1" 200 2659 "http://localhost/cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:11 +0100] "GET /cqpweb/canton1/execute.php?function=populate_corpus_cqp_positions&args=canton1&locationAfter=index.php%3FthisQ%3DmanageFreqLists%26uT%3Dy&uT=y HTTP/1.1" 500 185 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:14 +0100] "GET /cqpweb/canton1/execute.php?function=metadata_calculate_category_sizes&args=canton1&locationAfter=index.php%3FthisQ%3DmanageFreqLists%26uT%3Dy&uT=y HTTP/1.1" 302 286 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:14 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y HTTP/1.1" 200 2673 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:15 +0100] "GET /cqpweb/canton1/execute.php?function=make_cwb_freq_index&args=canton1&locationAfter=index.php%3FthisQ%3DmanageFreqLists%26uT%3Dy&uT=y HTTP/1.1" 302 285 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:16 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y HTTP/1.1" 200 2683 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:18 +0100] "GET /cqpweb/canton1/execute.php?function=corpus_make_freqtables&args=canton1&locationAfter=index.php%3FthisQ%3DmanageFreqLists%26uT%3Dy&uT=y HTTP/1.1" 302 285 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:18 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y HTTP/1.1" 200 2676 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:20 +0100] "GET /cqpweb/canton1/execute.php?function=populate_corpus_cqp_positions&args=canton1&locationAfter=index.php%3FthisQ%3DmanageFreqLists%26uT%3Dy&uT=y HTTP/1.1" 500 185 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:24 +0100] "GET /cqpweb/canton1/index.php?thisQ=corpusMetadata&uT=y HTTP/1.1" 200 2157 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:32 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y HTTP/1.1" 200 2522 "http://localhost/cqpweb/canton1/index.php?thisQ=corpusMetadata&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:34 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y HTTP/1.1" 200 2676 "http://localhost/cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:05:03:37 +0100] "GET /cqpweb/canton1/execute.php?function=populate_corpus_cqp_positions&args=canton1&locationAfter=index.php%3FthisQ%3DmanageFreqLists%26uT%3Dy&uT=y HTTP/1.1" 500 185 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"

access.log.1

::1 - - [24/Jun/2018:19:29:19 +0100] "GET /favicon.ico HTTP/1.1" 404 501 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [24/Jun/2018:19:29:19 +0100] "GET /favicon.ico HTTP/1.1" 404 501 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [24/Jun/2018:19:29:24 +0100] "GET /cqpweb/canton1/index.php?thisQ=corpusMetadata&uT=y HTTP/1.1" 200 2365 "http://localhost/cqpweb/canton1/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [24/Jun/2018:19:29:29 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y HTTP/1.1" 200 2521 "http://localhost/cqpweb/canton1/index.php?thisQ=corpusMetadata&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [24/Jun/2018:19:29:36 +0100] "GET /cqpweb/canton1/metadata-admin.php?clearMetadataAreYouReallySure=yesYesYes&mdAction=clearMetadataTable&corpus=canton1&uT=y HTTP/1.1" 302 285 "http://localhost/cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [24/Jun/2018:19:29:37 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y HTTP/1.1" 200 4161 "http://localhost/cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [24/Jun/2018:19:29:43 +0100] "GET /cqpweb/canton1/metadata-admin.php?dataFile=canton1.meta&useMetadataTemplate=%7E%7EcustomMetadata&fieldHandle1=&fieldDescription1=&fieldType1=1&fieldHandle2=&fieldDescription2=&fieldType2=1&fieldHandle3=&fieldDescription3=&fieldType3=1&fieldHandle4=&fieldDescription4=&fieldType4=1&fieldHandle5=&fieldDescription5=&fieldType5=1&createMetadataRunFullSetupAfter=0&mdAction=createMetadataFromFile&fieldCount=5&corpus=canton1&uT=y HTTP/1.1" 302 285 "http://localhost/cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [24/Jun/2018:19:29:43 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y HTTP/1.1" 200 2521 "http://localhost/cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [24/Jun/2018:19:29:47 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y HTTP/1.1" 200 2617 "http://localhost/cqpweb/canton1/index.php?thisQ=manageMetadata&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [24/Jun/2018:19:29:49 +0100] "GET /cqpweb/canton1/execute.php?function=populate_corpus_cqp_positions&args=canton1&locationAfter=index.php%3FthisQ%3DmanageFreqLists%26uT%3Dy&uT=y HTTP/1.1" 302 285 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [24/Jun/2018:19:29:49 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y HTTP/1.1" 200 2633 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [24/Jun/2018:19:29:50 +0100] "GET /cqpweb/canton1/execute.php?function=make_cwb_freq_index&args=canton1&locationAfter=index.php%3FthisQ%3DmanageFreqLists%26uT%3Dy&uT=y HTTP/1.1" 302 285 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [24/Jun/2018:19:29:51 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y HTTP/1.1" 200 2645 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [24/Jun/2018:19:29:52 +0100] "GET /cqpweb/canton1/execute.php?function=corpus_make_freqtables&args=canton1&locationAfter=index.php%3FthisQ%3DmanageFreqLists%26uT%3Dy&uT=y HTTP/1.1" 302 285 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [24/Jun/2018:19:29:52 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y HTTP/1.1" 200 2636 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:54:49 +0100] "GET /favicon.ico HTTP/1.1" 404 501 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:56:07 +0100] "GET / HTTP/1.1" 200 705 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:56:09 +0100] "GET /cqpweb/ HTTP/1.1" 200 1791 "http://localhost/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:56:09 +0100] "GET /jsc/wz_tooltip.js HTTP/1.1" 404 506 "http://localhost/cqpweb/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:56:09 +0100] "GET /jsc/wz_tooltip.js HTTP/1.1" 404 506 "http://localhost/cqpweb/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:56:13 +0100] "GET /cqpweb/adm/ HTTP/1.1" 200 2664 "http://localhost/cqpweb/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:56:16 +0100] "GET /cqpweb/canton1/ HTTP/1.1" 200 3027 "http://localhost/cqpweb/adm/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:56:20 +0100] "GET /cqpweb/canton1/index.php?thisQ=corpusMetadata&uT=y HTTP/1.1" 200 2144 "http://localhost/cqpweb/canton1/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:56:25 +0100] "GET /cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y HTTP/1.1" 200 2636 "http://localhost/cqpweb/canton1/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"
::1 - - [25/Jun/2018:04:56:27 +0100] "GET /cqpweb/canton1/execute.php?function=populate_corpus_cqp_positions&args=canton1&locationAfter=index.php%3FthisQ%3DmanageFreqLists%26uT%3Dy&uT=y HTTP/1.1" 500 185 "http://localhost/cqpweb/canton1/index.php?thisQ=manageFreqLists&uT=y" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0"


2018-06-25 16:57 GMT+08:00 Hardie, Andrew <a.hardie at lancaster.ac.uk<mailto:a.hardie at lancaster.ac.uk>>:
OK, so the problem is other than what I thought it was.

When you get that blank page, is there a PHP error in the httpd log? IF so, can you copy-paste it in a reply? Thanks.

best

Andrew.

From: cwb-bounces at sslmit.unibo.it<mailto:cwb-bounces at sslmit.unibo.it> [mailto:cwb-bounces at sslmit.unibo.it<mailto:cwb-bounces at sslmit.unibo.it>] On Behalf Of Hermann Lai
Sent: 25 June 2018 05:05

To: Open source development of the Corpus WorkBench <cwb at sslmit.unibo.it<mailto:cwb at sslmit.unibo.it>>
Subject: Re: [CWB] Incorrect total words count in a Traditional Chinese corpus on CQPweb

I am sorry. The new version does not help.

After I replace the old version with the new version. I go to "Corpus frequency list controls" and try to "Update CWB text-position records". It redirect me to

http://localhost/cqpweb/canton1/execute.php?function=populate_corpus_cqp_positions&args=canton1&locationAfter=index.php%3FthisQ%3DmanageFreqLists%26uT%3Dy&uT=y

with a blank page.

However, "Recreate CWB frequency table" and "Recreate frequency tables" redirect me to "Corpus frequency list controls".

I try to reinstall the corpus again but the blank page problem is still there.

Regards,
Lai

2018-06-21 21:10 GMT+08:00 Hardie, Andrew <a.hardie at lancaster.ac.uk<mailto:a.hardie at lancaster.ac.uk>>:
Hmm. I think this is a bug already fixed after 3.2.11.

The culprit is, if I recall correctly the function “update_corpus_sizes” – to be found in the file metadata.inc.php.

Old version:


function update_corpus_size($corpus = NULL)
{

        $corpus = safe_specified_or_global_corpus($corpus);

        $result = do_mysql_query("select sum(words), count(*) from text_metadata_for_$corpus");

        list($ntok, $ntext) = mysql_fetch_row($result);

        do_mysql_query("update corpus_info set size_tokens = $ntok, size_texts = $ntext where corpus = '$corpus'");

}


New version:


function update_corpus_size($corpus = NULL)
{
        $corpus = safe_specified_or_global_corpus($corpus);
        $result = do_mysql_query("select count(*) from text_metadata_for_$corpus");
        list($ntext) = mysql_fetch_row($result);

        $info = get_corpus_info($corpus);
        global $cqp;
        if (empty($cqp))
               connect_global_cqp();
        $cqp->set_corpus($info->cqp_name);
        $ntok = $cqp->get_corpus_tokens();
        do_mysql_query("update corpus_info set size_tokens = $ntok, size_texts = $ntext where corpus = '$corpus'");
}

Can I suggest that you replace the “old version” in your code with the “new version”, and redo frequency list setup?

That may well fix the issue. But if not, let me know…

best

Andrew.



From: cwb-bounces at sslmit.unibo.it<mailto:cwb-bounces at sslmit.unibo.it> [mailto:cwb-bounces at sslmit.unibo.it<mailto:cwb-bounces at sslmit.unibo.it>] On Behalf Of Hermann Lai
Sent: 19 June 2018 20:10

To: Open source development of the Corpus WorkBench <cwb at sslmit.unibo.it<mailto:cwb at sslmit.unibo.it>>
Subject: Re: [CWB] Incorrect total words count in a Traditional Chinese corpus on CQPweb

No, I didn't get any messages when I use the frequency list controls.

I am using CQPwebinabox Esmeralda (CQPweb 3.2.11) and CWB 3.4.8(checked by using "cqb -v").

Regards,
Lai

2018-06-19 23:06 GMT+08:00 Hardie, Andrew <a.hardie at lancaster.ac.uk<mailto:a.hardie at lancaster.ac.uk>>:
Did you get any odd messages when you ran the frequency-list setup on CQPweb?

If not – what version of the code do you have?

best

Andrew.

From: cwb-bounces at sslmit.unibo.it<mailto:cwb-bounces at sslmit.unibo.it> [mailto:cwb-bounces at sslmit.unibo.it<mailto:cwb-bounces at sslmit.unibo.it>] On Behalf Of Hermann Lai
Sent: 19 June 2018 11:32
To: Open source development of the Corpus WorkBench <cwb at sslmit.unibo.it<mailto:cwb at sslmit.unibo.it>>
Subject: Re: [CWB] Incorrect total words count in a Traditional Chinese corpus on CQPweb

part of the output of "cwb-decode -C CANTON1 -ALL | less"

<s>
<text>
<text_id T01>
中環    N       中環
保育    V       保育
奇觀    N       奇觀
：      PU      ：
孫中山  N       孫中山
史蹟    N       史蹟
徑      N       徑
至      CONJ    至
大館    N       大館
</text_id>
</text>
</s>


part of the output of "cwb-described-corpus -s CANTON1"

============================================================
Corpus: CANTON1
============================================================

description:
registry file:  /usr/local/share/cwb/registry/canton1
home directory: /usr/local/corpora/data/canton1/
info file:      /usr/local/corpora/data/canton1/.info
size (tokens):  23

  3 positional attributes
  3 structural attributes
  0 alignment  attributes

p-ATT word                     23 tokens,       22 types
p-ATT pos                      23 tokens,        8 types
p-ATT lemma                    23 tokens,       22 types
s-ATT s                         2 regions
s-ATT text                      2 regions
s-ATT text_id                   2 regions (with annotations)


It seems that CWB can recognize the number of words but CQPweb doesn't.

Regards,
Lai

2018-06-19 15:43 GMT+08:00 Stefan Evert <stefanML at collocations.de<mailto:stefanML at collocations.de>>:
What does the corpus look like if you decode it from the CWB index with the following command?

        cwb-decode -C CANTON1 -ALL | less

Can you show us part of the output?  It would also be useful to see the output of

        cwb-described-corpus -s CANTON1


One possibility I can think of is that your linebreaks are messed up so that CWB treats everything within the text region as a single long line.

Best,
Stefan


> On 19 Jun 2018, at 09:26, Hermann Lai <halflifelai at gmail.com<mailto:halflifelai at gmail.com>> wrote:
>
> I am using CQPwebinabox and I have indexed a Traditonal Chinese corpus called "canton1" by using two commands:
>
> sudo cwb-encode -d /usr/local/corpora/data/canton1 -f /home/user/Desktop/corpora/canton1/canton1.vrt -R /usr/local/share/cwb/registry/canton1 -c utf8 -xsB -P pos -P lemma -S s:0 -S text:0+id
>
> sudo cwb-make -V CANTON1
>
> After that, I install the corpus onto CQPweb. Most of the thing are correct. However, the total number of corpus texts is as same as the total words in all corpus texts.

_______________________________________________
CWB mailing list
CWB at sslmit.unibo.it<mailto:CWB at sslmit.unibo.it>
http://liste.sslmit.unibo.it/mailman/listinfo/cwb



--
Gaspard Germannson

_______________________________________________
CWB mailing list
CWB at sslmit.unibo.it<mailto:CWB at sslmit.unibo.it>
http://liste.sslmit.unibo.it/mailman/listinfo/cwb



--
Gaspard Germannson

_______________________________________________
CWB mailing list
CWB at sslmit.unibo.it<mailto:CWB at sslmit.unibo.it>
http://liste.sslmit.unibo.it/mailman/listinfo/cwb



--
Gaspard Germannson

_______________________________________________
CWB mailing list
CWB at sslmit.unibo.it<mailto:CWB at sslmit.unibo.it>
http://liste.sslmit.unibo.it/mailman/listinfo/cwb



--
Gaspard Germannson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://liste.sslmit.unibo.it/pipermail/cwb/attachments/20180716/5c7ec4f4/attachment-0001.html>

More information about the CWB mailing list