Re: [Tccc] Different community, similar problems? (Henning Schulzrinne)

Wearing the hat of a customer of research results, I like the fact that certain conferences are highly selective. I appreciate that the selected papers have gone through some good filters, because frankly I am simply overwhelmed by the amount of "junk" out there. I don't have the time or the inclination to look through 1000s of papers every year to determine which ones I should read, so I align myself to the "culture" of the conference that I like most (the one closest to my philosophy and one that has a good track record) and then I trust the collective judgment of PC members to do the right thing. I expect them do their job fairly and honestly before putting their stamp of approval on the papers.

Now wearing the PC hat - I too have participated in PCs of some of the major conferences for many years and I like the process - I like the fact that not only do we provide reviews (sometimes as many as 6 per paper) but we also meet face to face and debate the merits of the paper (almost everyone on the PC of good conferences gets together for the meeting) . If the paper is selected we provide shepherding -- all of this is done in a timely manner. Then we follow this up by providing a forum where we ensure that a large audience listens to the presentation and is able to ask the tough questions.

Do not put journals on a pedestal because if you do you will miss out on a lot of great research and the authors will miss out on some great audience & feedback. Not everyone publishes in and reads journals. Journals in themselves do not have the magic bullet. I have served on a number of editorial boards as well and they are only as good as their reviewers. The top conferences tend to have some of the most active researchers in the field with stellar reputation. They are put to test in the face-to-face meetings where knowingly or unknowing their colleagues judge them based on their review. It hurts them to not do a good job, so there is fire under their feet. If after they have taken a hard look at the paper and decided to accept it, you should consider this very seriously as you evaluate the younger researchers. Placing faith in the judgment of the PCs is not a good thing.

Only time is the true measure of research success and neither the journals nor the conferences can predict that.

Victor

-----Original Message-----
From: tccc-bounces@lists.cs.columbia.edu [mailto:tccc-bounces@lists.cs.columbia.edu] On Behalf Of Mani Srivastava
Sent: Wednesday, July 21, 2010 8:38 AM
To: tccc@lists.cs.columbia.edu
Subject: Re: [Tccc] Different community, similar problems? (Henning Schulzrinne)

I agree completely. Moreover, another pernicious effect is that many
(particularly in CS) have started treating the mere fact of acceptance
in one of these hyper-selective conferences as a seal of approval that
the research is unimpeachable. When combined with research driven
by conference deadlines, and a review process that is constrained by
not only time but also by expertise of the TPC (many hyperselective
conferences also have small TPC and insistence that all papers be
personally reviews by TPC members), all that this hyper-selectivity is
encouraging is an emphasis on research that would sell well at the
expense of rigor, correctness, and high risk ideas.

In my AE/EiC role for various journals, I have often had authors of papers
which run into trouble during the more deliberative review process of
a journal complain how the reviewers or AE can give a negative review
in light of prior acceptance at Mobicom, Sensys, Infocom etc.

Another pernicious effect is all these tenure review letter requests I get
with young faculty painstakingly listing accept percentage against each
conference and workshop paper, and trying to make the case of
"journal equivalence" for those papers. Again, we are encouraging a
false equation between quality and selectivity.

Mani

_______________________________________________
Tccc mailing list
Tccc@lists.cs.columbia.edu
https://lists.cs.columbia.edu/cucslists/listinfo/tccc