Sigma2: HPC and NIRD User Course, Part 1

Dates: March 16, 17, 18 and 19, 2021
Time: Each day 08:50-12:15/13:15 with 3-4 short breaks
Location: Video meeting (information will be send to registered participants). Note, sessions will be recorded and made publicly available after the course.

Participants may choose the lectures which they are interested in.

  • About

    HPC and NIRD User Course, Part 1 - About

    In March 2021, the Metacenter partners Sigma2, UiB, UiO, UiT & NTNU are jointly offering a hands-on course for current and future users of national HPC systems (Saga, Fram, Betzy) and the NIRD Toolkit. The course is open to all users of these systems and free of charge.

    Practical information

    Each course day starts at 08:50 and conclude at around 12:15 (3 sessions) or 13:15 (4 sessions).

    Registration will close minimum 14 days before the first lecture day to have sufficient time in providing access to HPC systems or the NIRD Toolkit if necessary.

    Please be aware that in order to use the NIRD Toolkit during the course you will need either a Feide account or a Feide OpenIDP guest account. You can create a Feide OpenIDP guest account at openidp.feide.no. If you're unsure about this, please contact us well in advance - minimum two weeks before the NIRD Toolkit sessions.

    IMPORTANT: Limited resources are available for hands-on exercises for the Toolkit session. Everyone can join the session, but only first 30 registered may be able to do hands-on exercises during the course.

    More information on the course page.

     

  • Program
    08:50
    -
    09:00
    Connect to zoom meeting
    09:00
    -
    09:50
    Welcome, Infrastructure news & HPC system details
    • Prerequisites: None
    • Learning outcomes: get to know what infrastructure is available/coming soon; understand details & differences of available HPC systems
    10:00
    -
    10:50
    Software installation - basics & sharing: pip/conda & R
    • Prerequisites: Account on HPC system, know how to use UNIX shell
    • Learning outcomes: know how to find, install & use a Python or R package under a user-writeable location, know how to share a self-installed Python or R package with other users
    11:00
    -
    11:50
    Software installation - basics & sharing: EasyBuild
    • Prerequisites: Account on HPC system, know how to use UNIX shell
    • Learning outcomes: know how to find, install & use an EasyBuild package under a user-writeable location, know how to share a self-installed EasyBuild package with other users
    12:00
    -
    12:40
    Asking for help (ca 20 min) & Q&A (ca 40 min)
    • Prerequisites: None
    • Learning outcomes: know how to get help, particularly how to send requests to the helpdesk/support and know about the existing documentation
    08:50
    -
    09:00
    Connect to zoom meeting
    09:00
    -
    09:50
    Jobs: layouts, dependencies, array jobs, parameter sweep runs, access to internet, job priority, accounting
    • Prerequisites: Know what a job is and have used them already on an HPC system; know what Slurm is; know how to use UNIX shell and an editor
    • Learning outcomes: understand more details about jobs; know when to use nodes, tasks, threads and how to specify them; know when and how to use array jobs and dependencies between jobs and how these are scheduled; be able to run simple parameter sweep runs; understand job priorities; be able to determine the cost of a job (accounting); be able to access the internet within a job, e.g., to download data from a web server or clone some GitHub repository
    10:00
    -
    10:50
    How to improve job scripts for better resource usage (cores and memory)
    • Prerequisites: Know what a job is; know a scripting language (e.g. bash) which is used for job scripts; know how to use UNIX shell and an editor
    • Learning outcomes: assess how many resources a job has used; be able to adjust resource requirements of a job as accurate as possible
    11:00
    -
    11:50
    Storage 1 - Data sharing, Interoperability HPC-NIRD, Striping, How to use file systems well?
    • Prerequisites: Account on HPC system and/or NIRD; know how to use UNIX shell; know how to do remote data transfers
    • Learning outcomes: be able to share data with external collaborators; be able to transfer data between HPC systems and NIRD; understand and be able to use striping; know about limitations of file systems and means to improve file system use (for yourself and other users)
    12:00
    -
    12:50
    Storage 2 - Disk quota and usage, data archiving
    • Prerequisites: Account on HPC system and/or NIRD; know how to use UNIX shell; know how to do remote data transfers
    • Learning outcomes: understand characteristics of different storage areas; figure out how much disk you use and how to solve problems (e.g., "disk quota exceeded"); be able to transfer data to/from NIRD archive
    13:00
    -
    13:20
    Questions & Answers
    08:50
    -
    09:00
    Connect to zoom meeting
    09:00
    -
    09:50
    NIRD Toolkit: Introduction & basics
    • Prerequisites: account to access the NIRD Toolkit; web browser
    • Learning outcomes: understand what the NIRD Toolkit is, what it can be used for
    10:00
    -
    10:50
    NIRD Toolkit: demos & setting up student group exercises
    • Prerequisites: account to access the NIRD Toolkit; web browser
    • Learning outcomes: how to launch an application/a service with the NIRD Toolkit; how to setup a group (e.g., of students) and give them access to an application/a service
    11:00
    -
    11:50
    User/project management
    • Prerequisites: web browser; not strictly a requirement, but this session is likely most useful for leaders or executive officers of a project
    • Learning outcomes: understand project accounts, user accounts, user accesses and how to manage all these via the web interface, e.g., how to give a specific user access to a project account on a certain HPC machine
    12:00
    -
    12:20
    Questions & Answers
    08:50
    -
    09:00
    Connect to zoom meeting
    09:00
    -
    09:50
    Containers: Essentials
    • Prerequisites: none
    • Learning outcomes: understand what a container is and how to use existing containers
    10:00
    -
    10:50
    Containers: How to use them on our HPC machines or the NIRD SP/Toolkit
    • Prerequisites: account on an HPC system or the NIRD Toolkit; web browser; understand basics of containers
    • Learning outcomes: how to launch a container on an HPC system or the NIRD Toolkit; how to access data on the file system from inside a container
    11:00
    -
    11:50
    Containers: Advanced topics
    • Prerequisites: account on an HPC system or the NIRD Toolkit; web browser; have used containers before
    • Learning outcomes: use containers more efficiently; be able to build your own containers or modify/extend existing containers
    12:00
    -
    12:20
    Questions & Answers
  • Registration

    HPC and NIRD Users Course, Part 1 - Registration

    The course is free of charge and is organised by the Metacenter partners Sigma2, UiB, UiO, UiT & NTNU

    Register here

    Registration closes on March 2nd

HPC and NIRD User Course, Part 1 - About

In March 2021, the Metacenter partners Sigma2, UiB, UiO, UiT & NTNU are jointly offering a hands-on course for current and future users of national HPC systems (Saga, Fram, Betzy) and the NIRD Toolkit. The course is open to all users of these systems and free of charge.

Practical information

Each course day starts at 08:50 and conclude at around 12:15 (3 sessions) or 13:15 (4 sessions).

Registration will close minimum 14 days before the first lecture day to have sufficient time in providing access to HPC systems or the NIRD Toolkit if necessary.

Please be aware that in order to use the NIRD Toolkit during the course you will need either a Feide account or a Feide OpenIDP guest account. You can create a Feide OpenIDP guest account at openidp.feide.no. If you're unsure about this, please contact us well in advance - minimum two weeks before the NIRD Toolkit sessions.

IMPORTANT: Limited resources are available for hands-on exercises for the Toolkit session. Everyone can join the session, but only first 30 registered may be able to do hands-on exercises during the course.

More information on the course page.

 

08:50
-
09:00
Connect to zoom meeting
09:00
-
09:50
Welcome, Infrastructure news & HPC system details
  • Prerequisites: None
  • Learning outcomes: get to know what infrastructure is available/coming soon; understand details & differences of available HPC systems
10:00
-
10:50
Software installation - basics & sharing: pip/conda & R
  • Prerequisites: Account on HPC system, know how to use UNIX shell
  • Learning outcomes: know how to find, install & use a Python or R package under a user-writeable location, know how to share a self-installed Python or R package with other users
11:00
-
11:50
Software installation - basics & sharing: EasyBuild
  • Prerequisites: Account on HPC system, know how to use UNIX shell
  • Learning outcomes: know how to find, install & use an EasyBuild package under a user-writeable location, know how to share a self-installed EasyBuild package with other users
12:00
-
12:40
Asking for help (ca 20 min) & Q&A (ca 40 min)
  • Prerequisites: None
  • Learning outcomes: know how to get help, particularly how to send requests to the helpdesk/support and know about the existing documentation
08:50
-
09:00
Connect to zoom meeting
09:00
-
09:50
Jobs: layouts, dependencies, array jobs, parameter sweep runs, access to internet, job priority, accounting
  • Prerequisites: Know what a job is and have used them already on an HPC system; know what Slurm is; know how to use UNIX shell and an editor
  • Learning outcomes: understand more details about jobs; know when to use nodes, tasks, threads and how to specify them; know when and how to use array jobs and dependencies between jobs and how these are scheduled; be able to run simple parameter sweep runs; understand job priorities; be able to determine the cost of a job (accounting); be able to access the internet within a job, e.g., to download data from a web server or clone some GitHub repository
10:00
-
10:50
How to improve job scripts for better resource usage (cores and memory)
  • Prerequisites: Know what a job is; know a scripting language (e.g. bash) which is used for job scripts; know how to use UNIX shell and an editor
  • Learning outcomes: assess how many resources a job has used; be able to adjust resource requirements of a job as accurate as possible
11:00
-
11:50
Storage 1 - Data sharing, Interoperability HPC-NIRD, Striping, How to use file systems well?
  • Prerequisites: Account on HPC system and/or NIRD; know how to use UNIX shell; know how to do remote data transfers
  • Learning outcomes: be able to share data with external collaborators; be able to transfer data between HPC systems and NIRD; understand and be able to use striping; know about limitations of file systems and means to improve file system use (for yourself and other users)
12:00
-
12:50
Storage 2 - Disk quota and usage, data archiving
  • Prerequisites: Account on HPC system and/or NIRD; know how to use UNIX shell; know how to do remote data transfers
  • Learning outcomes: understand characteristics of different storage areas; figure out how much disk you use and how to solve problems (e.g., "disk quota exceeded"); be able to transfer data to/from NIRD archive
13:00
-
13:20
Questions & Answers
08:50
-
09:00
Connect to zoom meeting
09:00
-
09:50
NIRD Toolkit: Introduction & basics
  • Prerequisites: account to access the NIRD Toolkit; web browser
  • Learning outcomes: understand what the NIRD Toolkit is, what it can be used for
10:00
-
10:50
NIRD Toolkit: demos & setting up student group exercises
  • Prerequisites: account to access the NIRD Toolkit; web browser
  • Learning outcomes: how to launch an application/a service with the NIRD Toolkit; how to setup a group (e.g., of students) and give them access to an application/a service
11:00
-
11:50
User/project management
  • Prerequisites: web browser; not strictly a requirement, but this session is likely most useful for leaders or executive officers of a project
  • Learning outcomes: understand project accounts, user accounts, user accesses and how to manage all these via the web interface, e.g., how to give a specific user access to a project account on a certain HPC machine
12:00
-
12:20
Questions & Answers
08:50
-
09:00
Connect to zoom meeting
09:00
-
09:50
Containers: Essentials
  • Prerequisites: none
  • Learning outcomes: understand what a container is and how to use existing containers
10:00
-
10:50
Containers: How to use them on our HPC machines or the NIRD SP/Toolkit
  • Prerequisites: account on an HPC system or the NIRD Toolkit; web browser; understand basics of containers
  • Learning outcomes: how to launch a container on an HPC system or the NIRD Toolkit; how to access data on the file system from inside a container
11:00
-
11:50
Containers: Advanced topics
  • Prerequisites: account on an HPC system or the NIRD Toolkit; web browser; have used containers before
  • Learning outcomes: use containers more efficiently; be able to build your own containers or modify/extend existing containers
12:00
-
12:20
Questions & Answers

HPC and NIRD Users Course, Part 1 - Registration

The course is free of charge and is organised by the Metacenter partners Sigma2, UiB, UiO, UiT & NTNU

Register here

Registration closes on March 2nd