It is not a bad idea at all. If we could do an experimental conference
based on such a review process that will be great. In fact, past few
ACM sigcomms did allow public comments on the accepted papers. To my
observation, that did add value in finding some of the issues with
papers accepted by the TPC. Further, it also helped authors in
improving their idea. Public comments seemed valuable there.
I think the challenge is in coordinating the public review process in
a decent and less chaotic manner.
However, I have one doubt: What is the significance of a paper
rejection in such a review process?
That is, rejection is not a valid decision. Every paper is accepted
with certain popularity, rating, or voting. Which is not an entirely
bad idea. Better ideas will get elevated to the top over a period of
time. In other words, music rating, movie rating, and popularity
rating of other creative products will come very close to what you
suggested.
If we have answers to such questions, this seems an alternative idea.
Thanks
bsmanoj
On Sat, Aug 14, 2010 at 3:07 PM, Mukul Goyal <mukul@uwm.edu> wrote:
> Forget about the term "IETF". What's wrong with the following review process for a conference/journal:
>
> 1. The review process starts with the posting of the submitted paper on a public "wall" for a certain time window.
> 2. Any one can post their review of the paper on the "wall" before a certain deadline.
> 3. The paper has a TPC mentor that has power to accept/reject the paper.
> 4. TPC mentor assigns a certain number of official reviewers for the paper. However, their reviews do not necessarily carry more weight than those by voluntary reviewers.
> 5. The authors and reviewers communicate with each other, possibly using pseudonyms, on the "wall" during the time window for the review process. However, the authors are not allowed to submit a new version of the paper during the review process.
> 6. At the conclusion of the time window, the TPC mentor makes the accept/reject decision about the paper based on posted reviews and author/reviewer communication.
>
> Thanks
> Mukul
>
> ----- Original Message -----
> From: "L Wood" <L.Wood@surrey.ac.uk>
> To: mukul@uwm.edu, hgs@cs.columbia.edu
> Cc: tccc@lists.cs.columbia.edu
> Sent: Saturday, August 14, 2010 3:58:41 PM
> Subject: RE: [Tccc] IETF model? Re: Different community, similar problems? (Henning Schulzrinne)
>
> Before advocating the IETF process, I suggest looking at the IRTF and seeing how the IETF process fails in research.
>
> -----Original Message-----
> From: tccc-bounces@lists.cs.columbia.edu [mailto:tccc-bounces@lists.cs.columbia.edu] On Behalf Of Mukul Goyal
> Sent: 14 August 2010 20:36
> To: Henning Schulzrinne
> Cc: tccc
> Subject: Re: [Tccc] IETF model? Re: Different community, similar problems? (Henning Schulzrinne)
>
> Henning
>
>>>
>>> A related concern with IETF-style iterative reviewing is that the
>>> reviewer may end up contributing more than some of the authors. It
>>> is easy to imagine a conscientious reviewer going through many
>>> iterations with a student whose (co-author) advisor is preoccupied.
>
>>And if people are complaining about conference
>>submission-to-publication delays of 5 months today, they will be thrilled when they see the 5-year delays from -00 I-D to RFC...
>
> There is no suggestion that the review process would extend for 5 years. It would still be same as before. Just that it would be open.
>
>>In general, without stating what you're trying to optimize and which problem you are trying to solve (quality? pick future faculty? timeliness? perception of fairness), the discussion of mechanisms seems a bit besides the point.
>
> In my mind, the problem is fairness of the review process. Another problem, that such a model may possibly solve, is dearth of reviewers and submission of sub-quality papers.
>
>> To once again cite the IETF process: first, you need a requirements draft.
>
> By IETF model, I was basically referring to its open review process.
>
> Thanks
> Mukul
> _______________________________________________
> Tccc mailing list
> Tccc@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/cucslists/listinfo/tccc
> _______________________________________________
> Tccc mailing list
> Tccc@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/cucslists/listinfo/tccc
> _______________________________________________
> Tccc mailing list
> Tccc@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/cucslists/listinfo/tccc
>
--
B. S. Manoj, Ph.D
Electrical and Computer Engineering
University of California San Diego,
CA 92093-0436, USA
Ph:+1-858-822-2564 (office)
+1-858-777-9769 (mobile)
Fax:+1-858-822-4633
_______________________________________________
Tccc mailing list
Tccc@lists.cs.columbia.edu
https://lists.cs.columbia.edu/cucslists/listinfo/tccc
Posts
No comments:
Post a Comment