[Sigwac] CALL FOR PAPERS: EMNLP 2020 (Dominican Republic, November 8-12, 2020)

[Anna Rogers]

[Apologies for cross-posting]

------------------------------------------------------

CALL FOR PAPERS: EMNLP 2020

November 8-12, 2020
Barceló Bávaro Convention Centre, Punta Cana, Dominican Republic
https://2020.emnlp.org/
Long and short paper submission deadline: May 11, 2020

------------------------------------------------------

The 2020 Conference on Empirical Methods in Natural Language Processing 
(EMNLP 2020) invites the submission of long and short papers on 
substantial, original, and unpublished research in empirical methods for 
Natural Language Processing. As in recent years, some of the 
presentations at the conference will be for papers accepted by the 
Transactions of the ACL (TACL) and Computational Linguistics (CL) journals.


IMPORTANT DATES

- Anonymity period begins: April 11, 2020
- Submission deadline (long & short papers): May 11, 2020
- Author response period: July 8-14, 2020
- Notification of acceptance (long & short papers): August 8, 2020
- Camera-ready papers due (long & short papers): August 28, 2020
- Main conference: November 8-10, 2020
- Workshops and tutorials: November 11-12, 2020

* All deadlines are 11.59 pm UTC -12h ("anywhere on Earth").


SUBMISSIONS

EMNLP 2020 has the goal of a broad technical program. Relevant topics 
for the conference include, but are not limited to, the following areas 
(in alphabetical order):

- Computational Social Science and Social Media
- Dialogue and Interactive Systems
- Discourse and Pragmatics
- Generation
- Information Extraction
- Information Retrieval and Text Mining
- Interpretability and Analysis of Models for NLP
- Language Grounding to Vision, Robotics and Beyond
- Linguistic Theories, Cognitive Modeling and Psycholinguistics
- Machine Learning for NLP
- Machine Translation and Multilinguality
- NLP Applications
- Phonology, Morphology and Word Segmentation
- Question Answering
- Resources and Evaluation
- Semantics: Lexical, Sentence level, Textual Inference and Other areas
- Sentiment Analysis, Stylistic Analysis, and Argument Mining
- Speech and Multimodality
- Summarization
- Syntax: Tagging, Chunking and Parsing


PAPER SUBMISSION INFORMATION

Long Papers

Long paper submissions must describe substantial, original, completed 
and unpublished work. Wherever appropriate, concrete evaluation and 
analysis should be included. Review forms will be made available prior 
to the deadlines.

Long papers may consist of up to 8 pages of content, plus unlimited 
pages for references; final versions of long papers will be given one 
additional page of content (up to 9 pages) so that reviewers’ comments 
can be taken into account.

Long papers will be presented orally or as posters as determined by the 
program committee. The decisions as to which papers will be presented 
orally and which as poster presentations will be based on the nature 
rather than the quality of the work. There will be no distinction in the 
proceedings between long papers presented orally and as posters.

Short Papers

Short paper submissions must describe original and unpublished work. 
Please note that a short paper is not a shortened long paper. Instead 
short papers should have a point that can be made in a few pages. Some 
kinds of short papers are:

- A small, focused contribution
- A negative result
- An opinion piece
- An interesting application nugget

Short papers may consist of up to 4 pages of content, plus unlimited 
references. Upon acceptance, short papers will be given 5 content pages 
in the proceedings. Authors are encouraged to use this additional page 
to address reviewers’ comments in their final versions.

Short papers will be presented orally or as posters as determined by the 
program committee. While short papers will be distinguished from long 
papers in the proceedings, there will be no distinction in the 
proceedings between short papers presented orally and as posters.


Authorship

The author list for submissions should include all (and only) 
individuals who made substantial contributions to the work presented. 
Each author listed on a submission to EMNLP 2020 will be notified of 
submissions, revisions and the final decision. No changes to the order 
or composition of authorship may be made to submissions to EMNLP 2020 
after the paper submission deadline.


Citation and Comparison

You are expected to cite all refereed publications relevant to your 
submission, but you may be excused for not knowing about all unpublished 
work (especially work that has been recently posted and/or is not widely 
cited).

In cases where a preprint has been superseded by a refereed publication, 
the refereed publication should be cited instead of the preprint version.

Papers (whether refereed or not) appearing less than 3 months before the 
submission deadline are considered contemporaneous to your submission, 
and you are therefore not obliged to make detailed comparisons that 
require additional experimentation and/or in-depth analysis.

For more information, see the ACL Policies for Submission, Review, and 
Citation 
[https://www.aclweb.org/portal/content/new-policies-submission-review-and-citation]


Multiple Submission Policy

EMNLP 2020 will not consider any paper that is under review in a journal 
or another conference at the time of submission, and submitted papers 
must not be submitted elsewhere during the EMNLP 2020 review period. 
This policy covers all refereed and archival conferences and workshops 
(e.g., COLING, NeurIPS, ACL workshops). For example, a paper under 
review at an ACL workshop cannot be dual-submitted to EMNLP 2020. In 
addition, we will not consider any paper that overlaps significantly in 
content or results with papers that will be (or have been) published 
elsewhere. Authors submitting more than one paper to EMNLP 2020 must 
ensure that their submissions do not overlap significantly (>25%) with 
each other in content or results.


Paper Submission and Templates

Submission is electronic, using the Softconf START conference management 
system. Paper template files will be provided soon on the conference 
website.


Optional Supplementary Materials: Appendices, Software and Data

Each EMNLP 2020 submission can be accompanied by one PDF appendix for 
the paper, one PDF for prior reviews and author response, one .tgz or 
.zip archive containing software, and one.tgz or .zip archive containing 
data. EMNLP 2020 encourages the submission of these supplementary 
materials to improve the reproducibility of results, and to enable 
authors to provide additional information that does not fit in the 
paper. For example, anonymised related work (see above), preprocessing 
decisions, model parameters, feature templates, lengthy proofs or 
derivations, pseudocode, sample system inputs/outputs, and other details 
that are necessary for the exact replication of the work described in 
the paper can be put into the appendix. However, the paper submissions 
need to remain fully self-contained, as these supplementary materials 
are completely optional, and reviewers are not even asked to review or 
download them. If the pseudo-code or derivations or model specifications 
are an important part of the contribution, or if they are important for 
the reviewers to assess the technical correctness of the work, they 
should be a part of the main paper, and not appear in the appendix. 
Supplementary materials need to be fully anonymized to preserve the 
double-blind reviewing policy.


ANOMINITY PERIOD

The following rules and guidelines are meant to protect the integrity of 
double-blind review and ensure that submissions are reviewed fairly. The 
rules make reference to the anonymity period, which runs from 1 month 
before the submission deadline (starting April 8th 2020) up to the date 
when your paper is accepted or rejected (Aug 8th 2020). Papers that are 
withdrawn during this period will no longer be subject to these rules.

- You may not make a non-anonymized version of your paper available 
online to the general community (for example, via a preprint server) 
during the anonymity period. Versions of the paper include papers having 
essentially the same scientific content but possibly differing in minor 
details (including title and structure) and/or in length.

- If you have posted a non-anonymized version of your paper online 
before the start of the anonymity period, you may submit an anonymized 
version to the conference. The submitted version must not refer to the 
non-anonymized version, and you must inform the programme chairs that a 
non-anonymized version exists.

- You may not update the non-anonymized version during the anonymity 
period, and we ask you not to advertise it on social media or take other 
actions that would further compromise double-blind reviewing during the 
anonymity period.

- You may make an anonymized version of your paper available (for 
example, on OpenReview), even during the anonymity period.

- Note that, while you are not prohibited from making a non-anonymous 
version available online before the start of the anonymity period, this 
does make double-blind reviewing more difficult to maintain, and we 
therefore encourage you to wait until the end of the anonymity period. 
Alternatively, you may consider submitting your work to the 
Computational Linguistics journal, which does not require anonymization 
and has a track for “short” (i.e., conference-length) papers.


INSTRUCTIONS FOR DOUBLE-BLIND REVIEW

As reviewing will be double blind, papers must not include authors’ 
names and affiliations. Furthermore, self-references or links (such as 
github) that reveal the author’s identity, e.g., “We previously showed 
(Smith, 1991) …” must be avoided. Instead, use citations such as “Smith 
previously showed (Smith, 1991) …” Papers that do not conform to these 
requirements will be rejected without review.

Papers should not refer, for further detail, to documents that are not 
available to the reviewers. For example, do not omit or redact important 
citation information to preserve anonymity. Instead, use third person or 
named reference to this work, as described above (“Smith showed” rather 
than “we showed”). If important citations are not available to reviewers 
(e.g., awaiting publication), these paper/s should be anonymised and 
included in the appendix. They can then be referenced from the 
submission without compromising anonymity.

Papers may be accompanied by a resource (software and/or data) described 
in the paper, but these resources should also be anonymized.


NEW: STICKY REVIEWS (optional)

Authors resubmitting a paper that has been rejected from another venue 
are invited to submit alongside their paper the previous version of the 
paper, the reviews and an author response. This is strictly optional. It 
is designed to mimic the revise-and-resubmit procedure underlying 
journals like TACL, and this trial for EMNLP will help to inform 
potential changes to the review process under consideration for future 
EMNLP and ACL conferences. We expect that the fact that a paper was 
rejected from another venue will not necessarily affect the paper’s 
decision in a negative way, but is likely to be beneficial to authors 
who believe they have addressed the problems identified, and can argue 
strongly for how the paper has been improved. The prior reviews will not 
be seen by reviewers, but be used as part of the EMNLP decision process, 
primarily by area chairs and program chairs in review quality control, 
resolving disagreements between reviewers, and in deciding borderline 
papers.


NEW: REPRODUCIBILITY CRITERIA

To foster reproducibility, authors will be asked to answer all questions 
from the following Reproducibility Checklist during the submission 
process. Authors are not required to meet all criteria on the checklist, 
but rather check off the criteria relevant to their submission. The 
answers will be made available to the reviewers to help them evaluate 
the submission. Reviewers will be expressly asked to assess the 
reproducibility of the work as part of their reviews.

The following list is a preliminary checklist we will use.

For all reported experimental results:

[ ] A clear description of the mathematical setting, algorithm, and/or 
model.
[ ] A link to a downloadable source code, with specification of all 
dependencies, including external libraries
[ ] Description of computing infrastructure used
[ ] Average runtime for each approach
[ ] Number of parameters in each model
[ ] Corresponding validation performance for each reported test result
[ ] Explanation of evaluation metrics used, with links to code

For all experiments with hyperparameter search:

[ ] Bounds for each hyperparameter
[ ] Hyperparameter configurations for best-performing models
[ ] Number of hyperparameter search trials
[ ] The method of choosing hyperparameter values (e.g., uniform 
sampling, manual tuning, etc.) and the criterion used to select among 
them (e.g., accuracy)
[ ] Expected validation performance, as introduced in Section 3.1 in 
Paper (https://arxiv.org/pdf/1909.03004.pdf), or another measure of the 
mean and variance as a function of the number of hyperparameter trials.

For all datasets used:

[ ] Relevant statistics such as number of examples
[ ] Details of train/validation/test splits
[ ] Explanation of any data that were excluded, and all pre-processing steps
[ ] A link to a downloadable version of the data
[ ] For new data collected, a complete description of the data 
collection process, such as instructions to annotators and methods for 
quality control.

Thanks to Jesse Dodge for helping with the above checklist. It is based 
on https://arxiv.org/pdf/1909.03004.pdf and 
https://www.cs.mcgill.ca/~jpineau/ReproducibilityChecklist.pdf


PRESENTATION REQUIREMENT

All accepted papers must be presented at the conference to appear in the 
proceedings. Authors of papers accepted for presentation at EMNLP 2020 
must notify the program chairs by the camera-ready deadline if they wish 
to withdraw the paper.

Previous presentations of the work (e.g. preprints on arXiv.org) should 
be indicated in a footnote in the final version of papers appearing in 
the EMNLP 2020 proceedings. Please note that this footnote should not be 
in the submission version of the paper.

At least one author of each accepted paper must register for EMNLP 2020 
by the early registration deadline.


ORGANIZERS

General Chair: Bonnie Webber (University of Edinburgh, UK)

Program Co-Chairs:

* Trevor Cohn (University of Melbourne, Australia)
* Yulan He (University of Warwick, UK)
* Yang Liu (Amazon, Alexa AI, USA)


CONTACT INFORMATION

e-mail: EMNLP2020ProgrammeChairs at gmail.com


FURTHER INFORMATION

The conference webpage (https://2020.emnlp.org/) will be continually 
updated with information on workshops, tutorials, etc.