Dual Anonymous Guidance

Updated: June 2023 

EMSL is committed to fostering an inclusive culture that welcomes and values people with diverse identities, perspectives, and backgrounds and that provides researchers equitable access to EMSL resources. For that reason, EMSL has adopted a dual anonymous peer-review process. Dual anonymous reviews have been shown to reduce bias in the evaluation of scientific research proposals, keeping the focus of peer review on the scientific content of a proposal rather than the individual team members. 

Applicants prepare anonymous research proposals, which peer reviewers will evaluate for scientific merit, mission alignment, and relevance to EMSL Science Areas. Detailed guidance for both submitters and reviewers is provided below. 

EMSL’s dual anonymous review process has been modeled after the process developed by NASA. 

Guidance for Submitters 

Proposers are required to write their Title, Abstract, Project Description, and References in an anonymized format that does not explicitly identify the names of the team members or their institutions. Some specific points follow: 

• Do not claim ownership of past work (e.g., "My previously funded work..." or "Our prior analysis demonstrates that…"). 

• Use neutral, third-person wording when citing your own work. For example, replace phrases like "In previous work, we have demonstrated… [12]." with "It has previously been demonstrated… [12]." 

• Write in-text citations in the form of a number in a square bracket (e.g., [1]), and provide the full citation in the reference list. 

• Do not include the names of personnel associated with the proposal or their organizational affiliations. In the Approach/Work Plan, refer to individual team members with anonymous wording, such as “the PI,” “Co-Investigator 1,” or “Grad Student 2.” 

• If you need to cite unpublished data, exclusive-access datasets, or findings that have been presented in public but are not citable, use language such as, “obtained in private communication” or “from private consultation.” 

Examples of Anonymized Proposal Text 

Example 1 

Previously: Our previous MALDI-FTICR imaging publication revealed that… 

Anonymized: A publication on MALDI-FTICR imaging revealed that… 

Example 2 

Previously: We have recently published an open-source GPU-accelerated simulation code that is scalable to large proteins (Simons et al.). However, we have not yet implemented the complex logic required to calculate titration curves of proteins. 

Anonymized: An open-source GPU-accelerated simulation code has recently been published that is scalable to large proteins [1]. However, the complex logic required to calculate titration curves of proteins has not yet been implemented. 

Example 3 

Previously: As shown in Figure 3, our preliminary results indicate that this is a viable method. 

Anonymized: As shown in Figure 3, data received in a private communication indicate that this is a viable method. 

NOTE: In this example, the data shown in the figure had been publicly shared in a conference but not yet published. 

Guidance for Reviewers 

The Peer Review Panel (PRP) will evaluate proposals for scientific merit, mission alignment, impact on EMSL Science Areas, and resources requested without access to identifying details about the team members submitting the proposal. See the call announcement for detailed evaluation criteria. 

EMSL will avoid assigning reviewers to proposals for which they might have a conflict of interest (COI) due to recent collaborations or shared affiliation. If a reviewer suspects they have a strong COI with a proposal assigned to them, they must disclose it and will be excluded from reviewing that proposal. 

Reviewers will assign preliminary scores and comments to proposals before the PRP meeting. At the meeting, reviewers will discuss and determine a final score and comment for each proposal based on the anonymous proposal content. Guidance for reviewers will include the following instructions: 

• Evaluate proposals based solely on the anonymous proposal content, following the evaluation criteria. 

• Do not try to guess the identities of proposers. 

• During the meeting, do not speculate or make insinuations even if you suspect you know the identity of proposers. 

• If you suspect you know the identity of team members on a given proposal and you have a COI, you must disclose that immediately to the meeting chair. 

• When writing comments for proposals, use neutral pronouns “they” and “them.” 

As a final check, after all scores and comments are complete, the non-anonymous portion of a subset of proposals (the top 30–50%, based on projected acceptance rates) will be released to reviewers, who will then evaluate the team qualifications for those proposals. The results of these assessments will not raise or lower the score of a given proposal, but if reviewers identify clear, compelling deficiencies in the expertise of team members to execute the proposed work, the proposal may be removed from ranking at EMSL’s discretion. 

Dual Anonymous FAQs 

Question: Does my letter of intent also have to be anonymous? 

Answer: No. The letter of intent will not be shared with peer reviewers so does not need to be anonymous. 

Question: Can I still talk to EMSL staff about my proposal? 

Answer: Yes! EMSL staff will not review your proposal. You are encouraged to discuss your research plans with them. 

Question: Which portions of my proposal form will be visible to reviewers? 

Answer: Reviewers will receive the following sections of your proposal: 

• Title 
• Abstract 
• Requested Resources 
• Samples and Equipment 
• Project Description, Computing Approach, Appendix 1 (List of References) extracted from the uploaded Proposed Research Document. 

Question: Why am I uploading a single PDF, including both anonymous and non-anonymous sections? 

Answer: The EMSL User Portal is not set up to accept multiple PDFs at this time. User Program Services staff will extract the anonymous portions of your uploaded PDF to share with reviewers, along with the anonymous sections of the proposal form completed in the EMSL User Portal. 

Question: Aren’t the team qualifications an important part of predicting the success of a proposed project? How can reviewers evaluate the likelihood of high-impact science without that information? 

Answer: Proposers may discuss relevant qualifications in the body of the proposal, as long as team member names and institutions or other obviously identifying information are not shared.  

Question: What will happen if I accidently fail to completely anonymize my proposal? 

Answer: EMSL will screen proposals for compliance with the anonymization requirements. In the case that a proposal contains minor errors, the proposer will be allowed to submit a correction. Proposals with extensive errors, where the proposer does not appear to have made a good-faith effort to anonymize their proposal, will be rejected without review. 

Question: Even if I don’t claim ownership for my past work, won’t the frequency of self-citations make my identity obvious to reviewers? 

Answer: It’s true that a reviewer might be able to guess a proposer’s identity by examining the frequency of citations or other details. However, reviewers will be asked to try not to deduce proposer identities, and an observer will be present during the panel discussion to interrupt any speculation along those lines. Reviewers will inevitably have some clues about proposer identities, but the intent of the dual anonymous review is to de-emphasize those identities and focus evaluations on the scientific content instead. 

Townhalls and Webinars

The double-blind peer-review process has been renamed dual anonymous peer review.