Arm Performance Report submission for large applications on Fram

On this page you will find: A step-by-step guide for generating and submitting an Arm Performance Report (APR) for your application, common questions and answers (FAQ), and other useful information related to the subject.

Projects spending more than a certain amount of CPU hours on a single application will from allocation period 2020.2, be asked to submit an Arm Performance Report (APR) alongside their quota application. We ask that APRs are submitted for software-applications (e.g., VASP, Gromacs, ...) where:

a) The expected use of the software-application for the next period is 1M CPU hours or more on Fram, or

b) The software-application has been run 1M CPU hours or more during the current allocation period on Fram.

At most 3 APRs per project must be submitted. We understand the exact amount of CPU hours spent on one single software-application alone can be difficult to estimate, so we will not be very strict on the accuracy of this.

These limits imply that projects with software-applications which are not estimated to use more than 1M CPU hours do not need to submit an APR, even if the project is granted more than 1M CPU hours in total. Projects with quotas below 1M CPU hours are not required to submit an APR.


What is APR?

APR provides an overview of how well a software-application performs on highly parallel systems. The generated reports are usually in the form of a one-page HTML document or text file. You can learn more about APR in our documentation, or by reading the Arm Performance Reports User Guide.

Project leaders of projects that are suggested to submit an APR will receive a reminder by email alongside the announcement of the allocation period 2020.2.

Step-by-step guide

Generating the APR

Quick start:

  1. Check which APR modules are available on your current machine, and select one of them.
    • ml avail Arm-PerfReports

  2. Add the module to your job script
    • module load Arm-PerfReports/<selected-version>

  3. Edit the launch command in your job script.
    • If your job script contains the line mpirun -n <num> <application-name>, APR can be run by simply typing perf-report instead of mpirun:
      • perf-report -n <number-of-cores> <your-application>

Once the job is finished, an HTML document will be in the job folder. The HTML file name will be <your-application-name>_<core-information>_<date-stamp>.html

 

More information on running APR can be found in our documentation.

Submit the APR

Eventually, the APRs can be uploaded in MAS simultaneously as you send your quota application. For now, we request that you send us the APRs by email.

Please use the following format for your subject field: <Allocation period> APR <Project number>.

This implies that when project nnXXXXk is applying for 2020.2, the subject should be: 2020.2 APR nnXXXXk

Send the email to: apr@uninett.no

Please send all your APRs in one email.

If you have questions about this process, please check the FAQ section below before contacting support.

FAQ

Does my project have to submit an APR?

We at Sigma2 would like an APR to be submitted if any of the following criteria are met:

                a) The expected use of the software-application for the next period is 1M CPU hours or more on Fram, or

                b) The software-application has been run 1M CPU hours or more during the current allocation period on Fram.

This means that if you estimate that any of the software you are using in your project consumes 1M CPU-hours or more, we would indeed like an APR to be generated of that application for a typical job in your project. If you say that no application you use has an accumulated usage (throughout the allocation period) of 1M CPUh or more, you do not have to submit an APR and Sigma2 will trust your judgement. As of now we have no easy way of measuring the spent CPU hours for each individual application within a project.

Consider the following example. Some projects may be running several applications, spending for example 0.5M CPUh on application1, 0.5M CPUh on application2 and 0.5M CPUh on application3 throughout 2020.1. In this case not one of the applications spend more than or equal to 1M CPUh, therefore they do not have to submit any APR. Another project may be spending 1M CPUh on application4 and 0.3M CPUh on application5. In this case the usage of application4 is more than or equal to 1M CPUh. We would therefore like an APR of that application.

My project uses several different applications. Do I have to submit an APR for all of them?

No. We are mostly interested in APRs for applications (jobs) that consume the majority of your CPUh quota. Only submit APRs for your 3 largest (in CPUh consumption) applications.

My project has many members, and a large variety of applications. How do I know which application to submit an APR for?

We recommend that you (the project leader) contacts the top quota spenders in your project, and ask them to generate an APR for a job that they deem representative for their work.

How can I find out who are the top spenders of CPU hours in my project?

A machine independent overview of usage statistics can be found by logging-in to your metacenter user, navigating to your project page and clicking "Detailed usage statistics". That page contains a table with project members and their corresponding CPU hour usage.

Alternatively, you can login to a specific machine and run the following command to see details about the quota usage on that machine:

cost -d

My project is running applications on several machines, which machine should the APR be generated on?

In general, we would like you to generate the APR on the machine that will be normally running that application. If you are running the same application on several systems, use the machine with the largest CPU hour consumption.

We are as of now only interested in applications running on Fram.

Why are you requiring projects to submit APRs? Will the APR results have any negative consequences for my project?

The vision of Sigma2 is "To maximize the impact and return of scientific research by providing a sustainable, predictable and cost-efficient e-infrastructure". By requiring that APRs are submitted, we hope to put some extra motivation into the cost-efficiency part of the vision.

Our motivation for asking for such a report is to improve the resource usage, and we want to work together with our users to achieve that goal. These reports also provide an opportunity to identify bottlenecks early, and to offer advanced user support to overcome these.

There are no planned consequences for projects submitting APRs. As mentioned, in the future, users of poorly performing code may be contacted and offered advanced user support to improve their code.

How can I get support in generating an APR?

Create a support request, making sure to follow our guide on writing good support requests, to ensure that we can efficiently help you. Send your request to support@metacenter.no.