Last update: June 25, 2020
We strongly encourage users to request a Startup Allocation prior to requesting a Research Allocation, in order to obtain benchmark results before you prepare your Research request. While having a Startup project is not absolutely required, the lack of benchmark results on the resources requested will greatly increase the level of scrutiny from the panel, especially for requests on highly oversubscribed resources.
For projects that have progressed beyond the Startup phase, either in purpose or scale of computational activities, a Research request is appropriate. Research requests are accepted and reviewed quarterly by the XSEDE Resource Allocations Committee (XRAC). Research requests are highly competitive. To give your request the best chance of success, please follow the instructions for successful requests and required components carefully. Omitting required components from your request may result in rejection of the request.
XSEDE provides twice-quarterly webinars as well as example successful allocation requests to help you prepare a successful Research request.
NEW Ken Hackworth from PSC and Lars Koesterke from TACC discuss the technical aspects of research allocation proposals including how best to gather and present scaling and code performance statistics and estimating SU requests in your allocation proposals.
|Code Performance and Scaling |
Recorded: April 1, 2020
Run time: 1hr 7mins
This recently recorded presentation details all the steps and requirements needed for successful research allocation requests. Topics covered include Computation Methodologies, Justification for SUs requested and many others.
|Writing and Submitting a Successful XSEDE Proposal |
Recorded: October 9, 2019
Run time: 1 hour 27 minutes
The quarterly submission deadlines for Research requests are the same every year. XRAC meetings usually happen at the start of the months noted below. PIs are notified of the results of their Research requests via email prior to the start of the allocation period.
|Submission Period||Meeting Date||Users Notified||Allocation Begin Date|
|Dec 15 thru Jan 15||Early March||March 15||April 1|
|Mar 15 thru Apr 15||Early June||June 15||Jul 1|
|Jun 15 thru Jul 15||Late August/Early September||September 15||Oct 1|
|Sep 15 thru Oct 15||Early December||December 15||Jan 1|
Research allocations are available to faculty members and researchers, including postdoctoral researchers, at a U.S.-based institution. Eligible institutions include federal research labs or commercial organizations; however, special rules may apply if your institution is not a university or a two- or four-year college. Investigators with support from any funding source, not just NSF, are encouraged to apply.
In general, an individual investigator may have only one active Research project at a time. After receiving an allocation, PIs can request that other users be given accounts to use the allocation.
Examples of well-written Research Requests from a variety of domains are provided to help you prepare successful documents. You are encouraged to refer to these past requests when writing your own proposals.
This table displays the estimated available Services Units per resource available for the upcoming quarterly XRAC meeting.
|Resource||Service Units available|
|Jetstream (IU/TACC)||16,000,000 vCPU hours|
|Open Science Grid||2,000,000 CPU hours|
|Bridges-2 Regular Memory (PSC)||127,000,000|
|Bridges-2 Extreme Memory (PSC)||757,000|
|Bridges-2 GPU (PSC)||440,000|
|Bridges-2 Storage (PSC) (Bridges-2 Ocean)||TBD|
|Bridges GPU-AI (PSC)||173,000|
|Comet (SDSC) |
Dell Cluster with Intel Haswell Processors
|40,000,000 core hours|
|Comet GPU (SDSC) |
Dell Cluster with Intel Haswell Processors
|400,000 GPU hours|
|Data Oasis (SDSC) |
Medium-term disk storage
|Expanse (SDSC) |
Dell Cluster with AMD Rome HDR IB
|Expanse GPU (SDSC) |
Dell Cluster with NVIDIA V100 GPUs NVLINK and HDR IB
|SDSC Expanse Projects Storage||TBD|
|Stampede2 - (TACC) |
TACC Dell/Intel Skylake/Knight's Landing System
|10,000,000 node hours|
|Ranch (TACC) |
Long-term tape Archival Storage
No other documents will be reviewed, and documents that exceed specified page limits will not be accepted by the XRAS system.
When writing your Research request, be clear and concise. We strive to have domain experts review every request, but they may not have deep expertise in your specific subdomain. Someone outside of your area should be able to understand the scientific objectives and understand why the chosen technique is preferred over another.
The documents required of a Research request have been requested by XRAC reviewers to ensure they can effectively determine how each request satisfies the XSEDE allocation Review Criteria. Requests need not use the full page limit in all documents; concise submissions are appreciated. No other documents will be reviewed.
- Main Document
- Progress Report
- Code Performance & Resource Costs
- Curriculum Vitae (CV)
- Special Requirements
The Main Document is the primary and most critical component of a successful request. Research requests must include a well-documented resource-use plan that describes how the requested allocations are necessary and sufficient to accomplish the project's research objectives. An effective resource-use plan must address the XSEDE allocation Review Criteria and, in particular, must include the following elements, with the bulk of the document focused on the Research Objectives and the Resource Usage Plan:
- Scientific Background-with more detail required of requests that do not have a merit-reviewed funding award
- Research Objectives and specific questions to be pursued
- Resource Usage Plan to achieve the research objectives
- Justification of the allocation amounts for all resources and resource types
- Resource Appropriateness
- Access to other CI resources and why those resources are not available or sufficient for the work proposed in this request
Page limits: 10 pages for less than 10 million SUs total; 15 pages for 10 million SUs or more
The Progress Report for Renewal requests should describe how the PI's current or prior allocation was used and summarize the findings or results; the results can be discussed with respect to the publications and other communications that the PI's team has entered in their XSEDE User Profiles and linked to this request. The Progress Report should also explain any significant digressions from the originally proposed resource usage plan for the prior period.
If a PI's allocation was significantly underutilized, the Progress Report should briefly describe the reasons for the underutilization and any mitigation by the PI to avoid the situation with the current allocation.
Page limit: 3 pages.
This REQUIRED document should contain code performance timings, resource usage details, and scaling information (for HPC resources) to support the calculation of the resource request.
For highly competitive HPC resources, the code performance data should be provided from benchmark runs on the resource requested, using the model configuration(s) needed for the computational plan proposed and demonstrating the scaling efficiency to the job size(s) planned. For smaller size requests, the panel may accept well-justified performance data from architecturally equivalent systems.
For less competitive systems or for non-traditional resources, this document should contain sufficient justification to convince the panel that you have a solid basis for calculating your allocation request.
Page limit: 5 pages.
Two-page (maximum) CVs for each PI and co-PI are required. The two-page NSF or NIH formats are highly recommended.
Page limit: 2 pages per individual
A PI may use this OPTIONAL document to separate a lengthy bibliography of cited work to take full advantage of the page limits in the Main Document. This bibliography is not the publications resulting from prior XSEDE support, which should be entered into the XSEDE publications database, but rather other citations referenced in describing or supporting the intellectual merit of the proposed work or the appropriateness of the proposed approach for addressing the research objectives.
Page limit: No limit.
This OPTIONAL document should be included if the completion of the Resource Usage Plan requires capabilities that fall outside a Service Provider's (SP's) standard user and usage environment. For example, jobs longer than the standard queue lengths, specific scheduling demands, or dedicated file system space. The XRAC is not required to read this document; the intended audience is the staff at the relevant SP.
Page limit: 1 page.
In addition to confirming user eligibility, reviewers evaluate the merits of each request according to three XSEDE allocation review criteria. PIs must provide justification, suitable to the nature of the request, that addresses these criteria; reviewers apply the criteria in the context of the request.
Appropriateness of Methodology: Does the request describe appropriate tools, methods, and approaches for addressing the research objectives? These methodologies may be community codes or models, data analysis methods, or algorithmic formulations expressed in user-developed scripts or tools.
Appropriateness of Research Plan: Does the request describe necessary and sufficient computational experiments to answer the research questions posed? In some cases, the research plan may be more reasonably expressed as estimates of resource use, supported by past or early experience. Serious concerns about the research plan will be documented in reviews and may lead to reduced allocation awards.
Efficient Use of Resources: Has the request identified appropriate resources for undertaking the research plan with the proposed methodology? And will those resources be used as efficiently as is reasonably possible and in accordance with the recommended use guidelines of those resources?
In addition to the computational review criteria, the panel also considers whether the work is aligned appropriately with any supporting grants for which the science has been merit-reviewed. If no such grants are identified, the panel will assess the intellectual merit of the work as well.
Intellectual Merit: Is the proposed work supported by a grant or grants that have undergone review for intellectual merit and/or broader impact (if relevant to the allocation)? If so, is the allocation request consistent with the objectives of the supporting grants and is the scale of resource use commensurate with the level and purpose of support? If a request is not supported by a merit-reviewed award, reviewers will assess the intellectual merit of the proposed work and factor that into their overall recommendation. Reviewers will also consider whether the identified support provides necessary and sufficient human resources to complete the proposed work.
To ensure all required and relevant aspects of each request are considered, the XRAC reviews and rates each request according to the rubric below. Submissions should address each of these elements within the required documents.
To ensure all required and relevant aspects of each request are considered, the XRAC reviews and rates each request according to the following rubric. Submissions should address each of these elements within the required documents.
Grounds for Rejection
Failure to address either of the following two items is grounds for rejection.
- Proposal describes access to other compute resources
- Code performance and scaling data are provided
Assessment and Summary
- Research objectives described
- Peer-reviewed supporting grant(s) - OR - Science review
- Progress report, publications, and prior usage (if applicable)
- Proposal adequately describes access to other compute resources
- Right tools, codes, algorithms, etc., for the research objectives
- Appropriate parameterizations, model configurations, etc., for the research objectives
Appropriate Research Plan
- Necessary & sufficient experiments or work plans to answer the research objectives?
- Request totals calculated correctly
- Justification provided for number of replicates, problems sizes, duration of calculations, etc.
Efficient Use of Resources
- Appropriate resources chosen
- Resources to be efficiently used
- Code performance and scaling data are provided and appropriate
The Main Document should include the following components:
- Scientific Background and Support
- Research Questions
- Resource Usage Plan
- Justifying Allocation Amounts
- Resource Appropriateness
- Disclosure of Access to Other Compute Resources
The Main Document will succinctly state the scientific objectives that will be facilitated by the allocation(s). (The existing merit-reviewed supporting grants should be listed in the submission forms.) Requests with merit-reviewed supporting grants will not be subject to further scientific review by the XRAC; however, the stated scientific objectives must match or be sub-goals of those described in the listed funding award(s). The description of the objectives should be sufficient to allow the reviewers to assess the resource usage plan.
If a request does not have merit-reviewed support, the Main Document should describe the science objectives in sufficient detail for the reviewers to evaluate the intellectual merit of the work covered by the allocation request. The XSEDE Allocations Coordinator will highlight such requests and the XRAC will review the scientific merit of the proposed work.
Next, an effective Main Document will identify the specific research questions that are covered by the allocation request. Within the context of the scientific background and supporting grants, these objectives and questions should be stated so that the reviewers can understand how elements of the resource plan will contribute to relevant answers. If you consider the allocation request in terms of computational experiments, the research objectives and questions define the experiments to be conducted.
The bulk of the Main Document should focus on the resource usage plan and allocation request. Inadequate justification for requested resources is the primary reason for most reduced or denied allocations. Once again, the PI should keep in mind the XSEDE allocation Review Criteria when describing and justifying the choice of resources and allocation amounts, and the Main Document should include appropriate justifications for all resources being requested.
The justification of allocation amounts takes the quantitative parameters from the resource usage plan and combines them with basic resource usage cost information (detailed in the Code Performance and Resource Cost document) to calculate the allocation needed. In terms of computational experiments, the justification should tabulate and calculate the costs of conducting each experiment for each resource and resource type.
Allocations for most resources are made in "Service Units" (SUs); however, the definition of an SU varies from resource to resource. Refer to the User Guide for each resource to determine how to calculate SUs (or other allocable units, if appropriate) for the resources in your request.
For compute resources, the request should address the appropriateness of the computational methodology, the architectural features or capabilities that make this problem a good fit for the requested systems, and the rationale for the number of runs being proposed. In terms of computational experiments, the Resource Usage Plan details the methodology, set-up, and conduct of each experiment.
For other types of resources, the request should address the appropriateness of the resource for the project's needs, the planned use and usage patterns, and the rationale for the allocation amount requested.
When faced with high demand for certain resources, the XRAC reviewers may make recommendations for allocations that could be migrated to less busy resources. This Main Document section should briefly detail the rationale for selecting the requested resources and highlight any resource features or capabilities that would constrain how the allocation might be moved.
For example, the rationale section could mention the need for specific visualization capabilities, the need for tightly integrated or specialized storage needs, the limited availability or compatibility of specific software, a code's ability to use specific resource features (such as GPUs or Xeon Phis), dedicated support for gateway access and usage, or the need to run at very large scale.
Conversely, this section can be used to highlight the PI's flexibility in using alternate resources, which may be advantageous for the PI if the selected resources are highly oversubscribed. (If your work permits, it is generally to your advantage to request less utilized resources directly, rather than simply noting your flexibility.)
Because of the high demand for XSEDE-allocated resources, XSEDE and the XRAC closely consider whether the PI has access to other, less constrained resources on which the proposed work could be conducted. All PIs must describe their local resources and other non-XSEDE resources available to them.
Local resource environment: Requests should describe briefly the local (e.g., campus or lab) computing environment available to a research team and, most significantly, how XSEDE resources will provide capabilities beyond those of local resources or why the requested XSEDE resources are required in addition to those resources.
Other non-XSEDE resources: If a PI has access to other resources (e.g., Blue Waters, NCAR systems, INCITE or other Department of Energy resource allocations, or resources supported by other agencies) or could be logically expected to have such access-as with researchers affiliated with or collaborating with other agencies that may have their own resources or facilities, the Main Document should describe why those other resources are not available to the group or why the requested XSEDE resources are required in addition to those resources.
Research requests associated with Gateway projects (sometimes called "Community" projects when a formal gateway is not the primary access mechanism) are submitted and reviewed in the same quarterly process as single-investigator requests, and most of the guidance for single investigator requests also applies to gateway requests. However, some Main Document sections and supporting documents should be modified to address gateway-specific issues.
Main Document-Research questions: In this section, gateway projects would describe the general codes or other services provided and the broad classes of research questions that could be pursued via the gateway.
Main Document-Resource usage plan: Requests for Gateways or Community Services projects would typically use this section describe the methods or applications provided, the expected or projected consumption of resources, and mechanisms for monitoring the users and usage of the service.
Progress Report: For Gateway projects, the Progress Report should include statistics and reports of community usage to help the reviewers understand how the prior allocation was effectively used and managed.
Code Performance and Scaling: For community or gateway requests, this document can be used to explain the methods used for extrapolating from prior gateway usage, to provide "typical" model performance on the requested or architecturally similar systems, and to explain expected user needs were projected to arrive at the allocation amounts requested.
All documents must satisfy the following minimum requirements. Documents that conform to NSF proposal format guidelines will satisfy these requirements.
- Margins: Documents must have 2.5-cm (1-inch) margins at the top, bottom, and sides.
- Fonts: Use one of the following typefaces identified below:
- Arial, Courier New, or Palatino Linotype at a size of 10 points or larger;
- Times New Roman at a size of 11 points or larger; or
- Computer Modern family of fonts at a size of 11 points or larger.
- A font size of less than 10 points may be used for mathematical formulas or equations; figures, table or diagram captions; or when using a Symbol font to insert Greek letters or special characters. PIs are cautioned, however, that the text must still be readable.
- Character spacing: No more than 15 characters per 2.5 cm (1 inch) horizontal space.
- Line spacing: No more than 6 lines within a vertical space of 2.5 cm (1 inch).
- Page numbering: Page numbers should be included in each file by the submitter. Page numbering is not provided by the submission system.
- File format: Only PDF file formats are accepted.
Documents that exceed the page limits will not be accepted by the XRAS system.
|Request Type||NEW||RENEWAL||Page Limits|
|Main Document||Required||Required||10 or 15 pp.*|
|Progress Report||Optional||Required||3 pp.|
|Code Perf / Scaling||Required||Required||5 pp.|
|CVs||Required||Required||2 pp. per CV|
|Special Requirements||Optional||Optional||1 p.|
*10-page limit for SU requests less than 10 million SUs; 15-page limit for requests greater than 10 million SUs.