On 8/16/2010 11:48 AM, Henning Schulzrinne wrote:
>> But the authors would be able to rebut bad reviews. The ability of
>> refute bad reviews is a major advantage of the proposed system. The TPC
>> mentor can ultimately judge the worth of a review.
>
> A number of conferences have had rebuttals for years; the general
> experience is that they greatly increase the work for the authors (and
> the reviewers), but rarely change the result, among other problems.
> Based on this experience, some conferences have abandoned them.
(again, speaking as an individual)
FWIW, I addressed this in detail back on 8/4/10 on this list.
I'll add that
- "most" conferences that have abandoned them, AFAICT
- they add not only work, but time lag (increasing the review cycle)
- the rebuttals were abandoned for many reasons, but AFAICT mostly
because the vast majority of papers simply aren't even eligible for a
rebuttal
a paper would need to have been rejected for a key single
technical reason that is factually incorrect
E.g., "the reader didn't understand me", or "I can fix all the minor
negative points noted" do not qualify as rebuttable reasons for a reject.
Very few papers are rejected for a single technical reason; the reviews
give reasons to reject, but, as noted below, it is the lack of positive
reviews, rather than the specifics of the negative ones, that are more
frequently the cause.
Further, as noted below, chairs already have the ability to overturn
decisions where such errors occur and are pointed out.
Joe
-----
This keeps being raised in various contexts, but it's a red herring IMO.
Most negative reviews are negative because they don't say something
positive, not because of the flaws listed.
However, authors largely use rebuttals to argue the merit or logic of
the listed flaws.
A truly erroneous decision can always be overturned - in a conference or
in a journal. But the error needs to be the one entire core of the
review, i.e., "this paper was published before" when it wasn't, or "this
equation is wrong so the paper is wrong" when it isn't.
It would be more useful as a community to expect chairs and editors to
treat this as the exception that it is, and override process where truly
needed. Treating this as a (missing) part of the process is a waste of
everyone's time, and a red herring to the broader difference between
conferences and journals, IMO.
----
_______________________________________________
Tccc mailing list
Tccc@lists.cs.columbia.edu
https://lists.cs.columbia.edu/cucslists/listinfo/tccc
Posts
No comments:
Post a Comment