Reviewing Guidelines: What Makes A Good Review And Formal Rules

The role of a reviewer is to identify papers that the MICCAI community must hear about. It is not to reward authors for their hard work and dedication. As such, the review should tell program committee which papers are exciting and could have a great impact on the field. Therefore, it is very important to approach the preparation of the review, as well as to turn to special services and order papers online to make the correct emphasis on the main points.

A good review expresses an opinion about the paper and backs it up with details on strengths and weaknesses of the paper. It is not sufficient to simply summarize the paper and add a couple of questions about low-level details in the paper. Nor it is acceptable to express an opinion without backing it up with specifics.

A good review is polite. Just like in a conversation, being rude is typically ineffective if one wants to be heard.

Given the page limit, it is unfair to ask the authors to substantially expand their paper. Similarly, it is not useful to recommend acceptance conditional on substantial revisions of the paper. The paper should be evaluated as submitted since the conference has no mechanism to ensure proposed changes will be carried out, and the authors have no room to add derivations, plots, or text.

While the format might vary, a good review typically includes the following components:

A quick summary of the paper, which can be as short as a couple of sentences. This part tells the program committee what the major contributions are, what the authors did, how they did it, and what were the results. This part is also helpful for the authors to verify that the reviewer understood their approach and interpretation of the results.
The opinion of the reviewer about the paper overall. Is it an interesting contribution? Should this paper be presented at the conference? Should it be known to a larger group of people? Is it a significant advance for the field?
Important: Please remember that a novel algorithm is only one of many ways to contribute meaningfully. A novel interventional system, a validation study with surprising or insightful results, an application of existing analysis methods to a novel problem with interesting results are just a few more examples. A paper would make a good contribution if you think others in the MICCAI community would want to know about what authors have done and can learn from their experience.
The opinion of the reviewer about the clarity of presentation, paper organization and other stylistic aspects of the paper. It is important to know whether the paper is very clear and a pleasure to read, or whether it is hard to understand.
The opinion of the reviewer about the major strengths of the paper. However tempting it is to immediately point out the problems, a reviewer should also write about a novel formulation, a principled derivation, an original way to use data, a novel application, or anything else that is a strong aspect of this work.
The major criticisms that a reviewer has about the paper. An effective way to deliver this critique is to summarize it briefly (at most two-three sentences), and then provide detailed arguments so that the program committee and the authors can understand the reviewer's concerns about particular aspects of the paper.
A list of minor problems, such as grammatical errors, typos, and other problems that can be easily fixed by carefully editing the text of the paper.
If the reviewer's expertise is limited to a particular aspect of the paper, a confidential note to the program committee that describes his/her relevant expertise. The review is more likely to be taken seriously if the limitations of the reviewer's understanding are clearly acknowledged.

Before submitting a finished report, a wise referee asks, "Would I be embarrassed if this were to appear in print with my name on it?"

Confidentiality

As a reviewer for MICCAI, you have the responsibility to protect the confidentiality of the ideas represented in the papers you review. MICCAI submissions are by their very nature not published documents. The work is considered new or proprietary by the authors; otherwise they would not have submitted it. Sometimes the submitted material is still considered confidential by the author's employers. These organizations do not consider sending a paper to MICCAI for review to constitute a public disclosure. Therefore, it is required that you strictly follow the following recommendations:

Do not show the paper to anyone else, including colleagues or students, unless you have asked them to write a review, or to help with your review.
Do not show any results or videos/images or any of the supplementary material to non-reviewers.
Do not use ideas from a paper you review to develop new ones of your own before its publication.
After the review process, destroy all copies of papers and supplementary material associated with the submission.

Conflict of Interest

The blind reviewing process will help hide the authorship of many papers, and primary Program Committee (PC) members will try hard to avoid conflicts of interest. But if you recognize the work or the author and feel it could present a conflict of interest, send the paper back to the primary PC Member and inform the Program Chairs. You have a conflict of interest if any of the following is true:

you belong to the same institution,
you co-authored together in the past five years,
you hold or have applied for a grant together,
you currently collaborate or plan to collaborate,
you have a business partnership,
you are relatives have a close personal relationship.

Reviewer Anonymity

A reviewer's identity should not be revealed to the authors at any point in times, both during and the after submission phase. Requesting citations primarily to one's own work may break anonymity so it should be considered carefully.