Download
1 / 12
120 likes | 226 Vues
Two Issues Concerning Research Conferences. Dave Patterson October 2004. 1) Big Idea/New Directions Papers. Problem Trying to Solve : Given avalanche of submissions to conferences, Big Idea / New Directions papers rarely accepted at many conferences
E N D
Two Issues Concerning Research Conferences Dave Patterson October 2004
1) Big Idea/New Directions Papers • Problem Trying to Solve: Given avalanche of submissions to conferences, Big Idea / New Directions papers rarely accepted at many conferences • Program Committee (PC) must reject 100-170 papers, easy to find bullet holes in such papers, they get a few low scores, and PC rejects them • Occasionally “hero” PC member gets 1 thru, but not sustainable (not just call for papers, PC chair) • Numbers making many conferences conservative • Given important stature of conferences today, Big Ideas / New Directions papers even more important now
1) Big Idea/New Directions Papers • Proposal:Try a 3 – 4 year experiment to attract such papers at some conferences • Mechanism:Separate Big Idea / New Directions PC • Former PC chairs + people with reputation for Big Idea / New Directions select ~ 3 papers • Dedicate 1 session to such papers • Evaluate value of Big Idea papers to decide if discard/keep/expand this approach: attendance at session, good discussions at conference, lead to subsequent papers, …
1) Big Idea/New Directions Papers • Our conference already accepts them • Great! Please share your experience, but no need to try this idea. • But we lose slots for 3 “good” papers? • For 3 years, let’s drop 1 panel session or 1 keynote (often nil in my experience) • Discuss! • Suggestions for conferences and/or people to contact as likely match?
2) Is large scale a threat to conference quality? • Background – Number of paper submissions increasing over time while number of papers presented roughly constant
2) Is large scale a threat to conference quality? Papers Submitted
2) Is large scale a threat to conference quality? Acceptance Rate
2) Is large scale a threat to conference quality? • Impact on Program Committee – Either more papers per member or larger PC • Either affects quality of reviews and of decision? • Impact on Authors of Rejected Papers – Inaccurate reviews decision feels arbitrary, unfair • Resubmit them to related conferences or submit more papers per conference more papers per conference • Impact on Conference,Field – Losing good ideas? • Are papers ranked 31-60 really that bad? (Its research) • Researchers don’t try riskier ideas since less chance?
2) Is large scale a threat to conference quality? • Proposal – Create an ad hoc committee to write report (& talk in Feb) on whether / when a problem including: • Why are numbers increasing? (More people in field, resubmiting, multiple papers/conf) • What size is a problem? (150? 250? 350?) • What have some conferences tried, and did it work or not? (best/worst practices) • New good ideas to help to cope with issue
2) Is large scale a threat to conference quality? • ExamplesofBest/Worst Practice • STOC – Went to dual tracks • PODC – 5 minute Brief Announcements (1 page in proceedings) + 25-min papers • 75 talks in 3 days, 50% - 50%, single track • SIGMOD – lower bound to accept rate, so more papers submittedmore presented • SIGGRAPH – add 5-minute recent result papers with later deadline, quick review • DAC? – Hierarchical Program Committees (divide by topic, allocate by track)
2) Is large scale a threat to conference quality? • Examples of New Ideas • No downside to submitting, so create downside by publishing authors’ batting averages to • Limit number of papers one can submit to a single conference (as often a limit for PC members) • Since now many related conferences, if some submit papers to each, go to a common paper deadline to force authors to decide target • Journals cope with surge by having editors reject papers outright without further review, for otherwise surge would break review system: give PC chair same authority smaller PC
2) Is large scale a threat to conference quality? • Request – If you think it’s a good idea, suggest names of people who might be willing to volunteer for ad hoc committee, possibly a chair? • Important to get people from diverse communities to collect practices, ideas
