Jonathan Grudin recently published an article on conference/journal culture in computer science entitled Technology, Conferences, and Community in Communications of the ACM. Unfortunately it’s behind a paywall, but I’m sure there’s fulltext out there via Google or Google Scholar. I’ll give a (hopefully short) summary and then some commentary.
The article starts off nicely, saying how the past two years have produced many critiques of the academic publishing process. Grudin proposes that the prevalence of conference publishing in CS (compared to other fields) has led to the focus on short-term research and the low acceptance rates favor “cautious incremental results over innovative work”.
The problem of conference focus has affected the US differently than Europe and Asia. In the US, conference publications were widely disseminated and (typically) permanently available. In contrast, archival was less common in European/Asian computer science conferences, so journals were the mainstay. The archival aspect of conference proceeding led to their prominence: ACM and IEEE decided in the early 90s that conference results couldn’t be republished as journal articles, which elevated conference publications to the level of peer-reviewed research.
Grudin provides an excellent history of the change of acceptance rates over time, leading to conservative research, leading to senior researchers ceasing to review, but I don’t know that I’ve seen it myself. He goes on to discuss conference attendance and SIG membership, which peaked or plateaued a while ago. He suggests that the reason for reduced participation (also in other venues like mailing lists) is due to low acceptance rates – highly polished papers leave little to discuss. High rejection rates have also undermined our sense of community in conferences.
Grudin suggests a few possible ways forward. With lower publication costs due to exclusive online publishing, we could accept more papers and distinguish the top 25% or so as best paper nominations. A more drastic change would be to have the evolution of a project as part of the community process – developing Wiki-like descriptions of our work that change and improve over time in response to comments.
I can’t comment on the historical aspect of CS conferences due to my age. I’d guess that decreasing conference membership is due to many factors, like the current economic (and funding) climate, easy access to papers online, and effectively a limited number of accepted authors. It’s possible that the commentary is overly focused on SIGCHI; SIGACCESS seems to be still growing.
I’d still like to see conferences have a “rigorous research” track and an “interesting idea” track, but maybe that’s too progressive? I’d also be interested in seeing journals move more in the direction of Computational Linguistics’ squibs. Vastly higher acceptance rates with distinction for high quality papers might also be interesting, but I don’t think I’d enjoy such a large venue. Instead, it might be nice to have a larger focus on workshops for acceptable papers below the 25% line. Maybe it’d work for ACL if there were joint submission for conference and workshop proceedings and some workshops were formed by clustering.
The idea of publication as a Wiki is interesting, but the problem is that it would undermine the hiring process. I can’t imagine applying for jobs without a significant peer-reviewed publication list. That said, this blog is intended to be an experiment in that vein. Part of my intention is to “publish” some of my new directions and respond to feedback.
As I mentioned in the previous post, I get the feeling that we’re trying to fit a circle block into a square or triangle hole. It seems like some sort of major shift might be suitable, but we’re so entrenched in the current system.