Good summary, Joe. I just want to put a couple more points to the TCCC
discussion table:
1. Improving the quality of papers.
2. GOLD and WOMEN reservations to the TPCs and Editorial boards.
3. Division of ComSoc to handle the growth of the networking area.
(For example, ComSoc-Wireless, ComSoc-Optical, etc)
Let us hear from members to see if these are relevant. We can discard
them if they don't improve the current state.
See the detailed discussion below.
The very issue of quality of papers and the contribution of IEEE
conferences in the degradation of quality came under discussion in
2007 in TCCC mailing list. It is good to see that again, thanks to
Tony, because that issue is very challenging and complex to find an
easy solution. Ideally, there is no single answer to the quality
issue. Many factors contribute to the poor quality that we observe
today. Some of which I think of importance are below:
1. Large volume of submissions: This issue is beyond anyone's control.
Internet has grown, so are the number of people doing research on
networking. Naturally, the number of submissions will be huge. Many of
the traditionally non-research universities around the world are now
doing wonderful research in this area. That is not under IEEE's
control. Actually IEEE encourages and works hard for that! In fact, in
2007 discussions, Nitin used the term "Democratization of Research" to
describe this effect. Since we cannot control that, we have to learn
to live with it. We could, however, try to scale up IEEE or ComSoc to
handle such huge growth. IEEE conferences and strategies need radical
solutions to take the fast growing opportunities. Any potential
solutions are welcome on this topic. Point (3) down this email will
discuss about scaling ComSoc towards this.
2. Review process: This is where IEEE can contribute better for
improving the quality of publications. The current review process need
more scrutiny. Again, this topic came up in 2007 and subsequent
changes in the organization of Infocom resulted in the formation of
Infocom Miniconferences. However, the complexity and scalability of
review process, in general, are still relevant in both conferences as
well as journals as rightly pointed to by Tony. That means, we need
more reviewers and better (faster) process to ensure quality. Unlike
in the past, quality and punctuality of reviewers have degraded as
well. We are collectively responsible for that. In the past, there
were only 100 nodes in the Internet, half a dozen networking
conferences, and a few dozen researchers in the networking area. So,
maintaining quality was easy and straight forward. Today, Internet
became the lifeline of the world with millions of nodes and and
unknown number of researchers work for improving it. I think we are
mistaken if we think or even compare today's research with networking
research in the 90s. And more importantly, many of us drop jaws when
we come to know new areas of research emerging in networking.
My point is that we need radical changes in the review process. Some
ideas are the following:
(i) Group reviews: This can be considered where a collection of
papers will be reviewed by a collection of people and discuss among
them before recommending a set of papers. That is a paper assigned to
a group led by a Professor/Engineer will be read collectively by a
group of junior researchers/students and discuss among themselves
before suggesting a decision.
(ii) Assigning a Spectrum of reviewers: Every paper must be reviewed
by a spectrum, interms of experience, of people that includes a senior
professor level individual, an associate/assistant professor level
individual, a Post doc/Recent graduate level individual, and probably
a current student. Each individual will review it based on his/her
level or expertise and such a wide spectrum of feedback can enrich the
resulting reviews. Such a spectrum (experience-wise) can give a
collection of different feedback which will eventually help improve
the quality of published papers. A young researcher may not catch a
research idea that appeared thirty years ago that a senior professor
may easily recognize. On the other hand, a senior professor with lot
of administrative load may not know (no offense intended) certain
recent papers published where the younger researchers can better help
with. To be more clear, the paper assignment must have a set of
reviewers spanning the experience-spectrum. Tools such as EDAS can
help lessen the burden of TPC chairs/Editorial boards to assign the
papers to such a wide spectrum of reviewers in an automated and
resource efficient manner.
(iii) Increasing the number of reviews per paper: To help quality of
papers, we can increase the number of reviewers per paper instead of
the current average that stands between 2 and 3 reviews per paper. One
popular counterargument against this strategy is the limited pool of
available reviewers. That is where we significantly lack ideas. My
estimate is that the large set of potential reviewers out there are
never invited to review any papers, either as part of TPCs or as part
of editorial boards. That is mainly because the TPC selection model is
very OLD and is typically based on personal familiarity of the TPC
chair or TPC chair's friends. This strategy worked well in the 80s and
90s where there were only a hundred researchers in the entire
networking area. However, this strategy is failing the entire
networking research area. It has the main disadvantage of "Rich (in
terms of invitations) gets richer and poor becomes poorer." To be
precise, an existing TPC member gets several more invitations and
thereby being super busy (and sometimes inefficient) for providing a
quality review besides taking significant amount of effort from
his/her time for creative research. On the other hand, the vast
majority of Graduates of the Last Decade (GOLD) is never invited to be
in any of the TPCs or Editorial boards. Much worse is the case of
Women representation. I had spoken to many women professors (mostly
GOLD) in the Public University System in the US and many complained
that they have never been invited to be on any TPC or Editorial board.
Further, in the past, the three reviewers per paper had plenty of time
to give a scrutiny of the paper under review. Today, the three
reviewers are too tightly scheduled to read through the paper a second
time.
We clearly need a larger review force for the next decade of
networking research. In order to solve this problem, let me suggest a
20:20 solution. We MUST have atleast 20% of GOLD or Half-GOLD
(Graduates of the Last Half Decade) members in every ComSoc TPC and
every Journal editorial board and collectively there should be atleast
20% (or if not possible even 10%) women representation. Sometimes we
will have difficulty in getting absolute numbers in the next few
years, however, we MUST make an serious effort in developing future
review force. I am not sure where the current percentages stand. Let
every one of us know if any one of us have information on these.
That is where EDAS can help. If, as a TPC member/chair, one decides to
suggest say 20% Women, GOLD, or Half-GOLD members, then EDAS could
suggest potential members matching that class. Ten years down the
line, we will have enough people so that we can have close to SIX
REVIEWS PER PAPER which I believe can help SIGNIFICANTLY improve the
quality of presentations of future research literature.
3) Scaling up ComSoc: Given the huge volume of submissions in
networking research, we have to look for ways that can scale up
ComSoc. Computer communication has grown significantly in many
directions and therefore, we need to make ComSoc fit for handling the
growth. The main question now is DO WE NEED DIVISION OF COMSOC to
multiple smaller focus groups to meet the scalability challenge. For
example should we need ComSoc-Wireless, ComSoc-Optical, ComSoc-P2P, or
similar focus groups to handle the large growth of each of the areas
within the communication society. This way, we can keep the costs down
for ComSoc and better reorganize along the side of ACM SIGs. It might
help as well.
Comments are welcome.
Thanks
Sincerely
bsmanoj
On Thu, Dec 10, 2009 at 3:10 PM, Joe Touch <touch@isi.edu> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi, all,
>
> The following is an attempt to distill the recent threads on conference
> concerns down to short list of focus points. Please respond either
> on-list or to me directly if I have missed anything or if you feel this
> summary should be revised:
>
> 1- reducing non-author presenters
> ACTION: it seems like the most direct solution would be to
> change the Comsoc requirements for a "full-time registration per
> paper" to state that this must be a named author, and to
> allow the existing rules for exceptions (at the
> discretion of the chairs) to address unusual cases.
>
> 2- support for resource-challenged authors
> ACTION: TCCC should identify resources for such authors
> to be able to attend meetings in person, or to present
> at meetings where attendance isn't possible.
>
> 3- meeting costs
> - cost trends
> - costs associated with executive committee members
> - costs associated with other free registrations
> - other ways to reduce costs
> - university locations
> - LEAN meetings (bag lunches, etc.)
> - pointers to conference planning information where
> available
>
> ACTION: TCCC will collect the relevant information and
> report back.
>
> I'd like to deal with these three issues separately. All information
> will be posted on the TCCC website for future reference and
> augmentation, in addition to being posted to the list.
>
> Other issues raised:
>
> - - ways to increase meeting attendance on the last day
>
> - - ways to reward good presenters
>
> These are useful to discuss on the list, but in most cases they fall
> under the discretion of the program chair. Discussing them as a
> community is useful, but I didn't see a need for a TCCC position on
> these (nor did I see a particular consensus forming).
>
> Joe
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (MingW32)
>
> iEYEARECAAYFAkshf+8ACgkQE5f5cImnZruBvwCggViZKO0QElUwsbs51UZHJG5z
> bKYAoIGmnMdM6QWE9OKPfreCCJuh6Nyb
> =YlAv
> -----END PGP SIGNATURE-----
> _______________________________________________
> Tccc mailing list
> Tccc@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/cucslists/listinfo/tccc
>
--
B. S. Manoj, Ph.D
Electrical and Computer Engineering
University of California San Diego,
CA 92093-0436, USA
Ph:+1-858-822-2564 (office)
+1-858-429-8804 (mobile)
Fax:+1-858-822-4633
_______________________________________________
Tccc mailing list
Tccc@lists.cs.columbia.edu
https://lists.cs.columbia.edu/cucslists/listinfo/tccc
Posts
No comments:
Post a Comment