Reviewing for ICIS should be taken seriously as a reviewer has significant inputs into paper acceptance decisions for ICIS and may potentially shape papers that eventually progress to publications (Urquhart, Carte & Heinzl, 2017). Therefore, this guide should provide guidance for associate editors and reviewers on how submissions to ICIS 2019 should be assessed and what are the essential steps in the review process.
Key points include:
- Authors need to be aware of the review process (please also see the Guide for a Successful Submission to ICIS 2019)
- Associate editors and reviewers need to be aware of the essential steps and criteria related to assessing submissions for ICIS 2019
The review process of completed research and short papers follows a double-blind process with several involved participants, i.e., Program Chairs (Program Chairs), Review Coordinators (RCs), Track Chairs (Track Chairs), Associate Editors (AEs) as well as Reviewers. Therefore, authors need to anonymize their submissions (also delete self-citations where necessary). Thus, all parties related to review process should immediately declare a conflict of interest as soon as they become aware of this fact (see Urquhart, Carte & Heinzl, 2017 concerning “amigo papers”). This especially includes personal relationships, teacher/student/advisor or employment relationships, same institution, co-authors or current research collaborators. In addition, at any point of time, all participants involved in the reviewing process are expected to handle the submissions, reviewers, and the review process in a confidential manner.
Key Steps in the Review Process
Overall, the ICIS 2019 review process contains several crucial steps to ensure high quality proceedings and presentations in Munich. Due to the high number of submissions, it is essential that all involved parties strictly adhere to the communicated timelines to ensure a timely review process.
- Desk Reject
After the submission deadline expires, Review Coordinators assess papers concerning mainly technical issues that warrant a desk rejection (e.g., exceeding the maximum paper length, non-compliance with submission template, etc.). These papers will not be assigned to the Track Chairs. Authors will be informed by the Program Chairs concerning desk rejections.
- Track Chair Assignment
Papers that comply with the ICIS 2019 submission requirements are assigned to Track Chairs or moved to other tracks due to better fit.
- Early and Constructive Rejects
Track Chairs assign an AE for every paper. Before submitted papers are sent out for review, AEs have to assess each paper according to its contribution and potential for ICIS 2019. If the AE feels that a paper has very little likelihood of being accepted at ICIS 2019, (s)he can propose early rejection of the paper without sending the paper out for review. The goal is to provide authors of such papers early feedback and offer them the opportunity to submit their work elsewhere if the review criteria will clearly not be met. If an AE selects a paper for early and constructive reject, (s)he should provide a detailed statement to the Track Chairs, Program Chairs and authors by pointing out why the paper in its current form does not have the potential required for acceptance at a premier outlet such as ICIS. All other papers will be transferred into the regular review process.
- Review Phase
Associate Editors assign papers to three reviewers, ensuring a diverse and also experienced review team. Each paper should be reviewed by at least one faculty; the set of reviewers for each paper should not be completely made up of PhD students. Reviewers should provide detailed and constructive reviews for each paper reviewed. Please ensure that comments to the authors and to the editors reflect the scores reviewers make in the review system.
- Associate Editor Reports
AEs write a report (not review) for every paper based on the review team’s comments. The report should be detailed and should highlight the AE’s views about the fundamental reasons leading to the rejection/acceptance of papers. In the unfortunate case of short or non-constructive reviews by one or more reviewers, AEs should ensure a more detailed assessment to ensure a high-quality review. If AEs recommend a conditional accept of the paper, the AE should provide a comprehensive list of issues that need to be fixed by the authors to have the paper accepted at ICIS 2019, keeping in mind the revision cycle of ICIS 2019.
- Track Chair Decisions
Track chairs will review all the AE reports and reviewer feedback to prepare a final shortlist of papers recommended for acceptance from their track. As the standards of reviewers and AEs differ, Track Chairs should not depend purely on the quantitative scores provided by the review team, but should examine the qualitative comments of the review team, and read the papers where needed.
- First Decision
On the basis of the AE recommendations, TC decisions, the Program Chairs will assess all submissions and make the final decisions for all papers. Decisions will be informed to the authors by the Program Chairs.
In the revision process, authors have to make adjustments to their papers in accordance to the reviews and submit their paper as a camera-ready version with author information as well as acknowledgements. The program chairs will provide a revision template for this purpose. Afterwards, all revised papers will be screened to verify that all major points of criticism raised by the review team have been addressed. If the authors failed to do so, a paper may still be rejected at this point. To accelerate the process, authors are required to provide a point-by-point response to the major review comments highlighted by the AE. Final acceptance notifications will be issued by the Program Chairs.
How to Assess Submissions to ICIS 2019
Key aspect for assessing submissions to ICIS 2019 were communicated to the authors (please see the “Submissions Types, Instructions and Requirements”) prior to submission. Central to assessing the submissions is the contribution of the paper. Reviewers should therefore take three very important aspects into account:
First, paper submissions to ICIS 2019 can make a contribution to IS research by drawing on diverse methodological and theoretical approaches. Thus, please assess papers with respect to its methodological and theoretical approach and review the paper appropriately. Please see also the Program Chairs guide for a successful submission. Thus, please adopt an open-minded attitude when reviewing papers.
Second, there are different submission types for ICIS papers. Mainly, we distinguish between completed research and short papers. Thus, please take a look at the “Submissions Types, Instructions and Requirements” for the review criteria for different submission types.
Third, a high-quality review is typically at least about one page of written text that provides constructive recommendations for criticism being raised, especially in the case of rejection. Authors will be thankful for guidance, even though they might have been rejected. In either case, it is important to set the tone of the review as constructive, respectful and open-minded. Rai (2016) provides excellent guidance for the IS community that also applies for reviewing at ICIS 2019.
In conclusion, we would encourage you to review for ICIS. It will provide more practice and training in reviewing and will help you to increase your visibility to senior members of the IS community. Also, it is a good way to give back to the research community, especially if you also submitted a paper to ICIS 2019.
Rai, Arun. 2016. “Editor’s Comments: Writing a Virtuous Review,” MIS Quarterly, (40: 3) pp.iii-x.
Urquhart, C., Carte, T., & Heinzl, A. (2017). Time for Some Changes to ICIS? Reflections on our Highest-quality Conference. (41) Communications of the Association for Information Systems.