Data decommissioning policies

Starting at the 2020.1 resource allocation period, storage decommissioning procedures have been established for project storage on the Sigma2’s national e-infrastructure resources. The data decommissioning policy is intended to help research projects define data management plans and contribute to more sustainable provisioning and capacity planning.

Projects

Projects without an allocation in both the previous and the current period will be decommissioned.

The decommissioning procedure consists of deleting data after a grace period of 6 weeks, where project leaders will get support to download the data.

Particularities for HPC projects: If a project remains without computation quota on a machine (e.g. machine A) for an entire allocation period, that project’s datasets on that machine will be scheduled for deletion. More specifically, projects’ datasets on machine A will still be deleted even though the project has computational quota on another HPC machine (e.g. machine B).

1. First period without an allocation

If you don't have an allocation at the period start, project leaders and executive officers will be notified by e-mail that the project may be decommissioned.

At this stage, the following actions are taken:

  • Access to the project will be revoked for all users.

In addition, if applicable:

  • Backups are stopped.
  • Snapshots are stopped.

2. Second period without an allocation

If you still haven´t got an allocation, project leaders and executive officers will receive another notification by e-mail that the grace period of 6 weeks has started.

After this, notifications will be sent to project leaders and executive officers 30 and 15 days before the actual decommissioning.

3. Grace period — 6 weeks after second period start

At this point, the following actions are taken:

  • The project area is permanently deleted. This action is irreversible.
  • The project leader and executive officer(s) are notified.

Home catalogue

Every user given access to Sigma2’s national e-infrastructure (be it HPC or NIRD resource) has at least one home ($HOME) catalogue. There is one home catalogue per resource. 

In either of the following cases a home catalogue will be decommissioned: 

  • Users with expired accounts: Home catalogues for users with expired accounts will be decommissioned.  

  • Users without an allocation: Home catalogues for users which are not tied to a project with an active allocation on a system in both the previous and the current period will be decommissioned (on that system). 

The decommissioning procedure consists of deleting users’ home catalogue after a grace period. 

Regaining access to the resource will automatically interrupt the decommissioning process of the home catalogue. 

Timeline for users with expired accounts

1. 6 weeks before user account is going to expire

The user is notified that the account is going to expire

2. On the expiry date

At this sage, the following actions are taken:

  • Access to the users` home catalogue on all SIgma2 resources is locked
  • The user is notified that the account has expired and their $HOME catalogue(s) will be deleted from Sigma2 resource(s) after a grace period. 
3. 90 days after user expiry date
  • The users` home catalogues(s) are permanently deleted. This action is irreversible.
  • The user is notified by e-mail that their home catalogue(s) have been permanently deleted

Timeline for users without an allocation

1. First period without access to resource

If you don't have access to a project with a quota on a specific resource, your home catalogue on that resource is scheduled for decommissioning.  

At this stage, the following actions are taken:  

  • You will receive a notification by e-mail that your home catalogue on the particular resource is scheduled for decommissioning.  
2. Second period without access to resource

If you still don’t have access to a project with a quota on that particular resource, you will receive another notification by email informing you that your home catalogue is scheduled for decommissioning, and that the grace period has started.  

3. 90 day after period start

If you still don’t have access to a project with a quota on that resource, your home catalogue is deleted.   

At this stage, the following actions are taken:

  • The home catalogue on the resource(s) where the user is not a member of a project with an active quota is permanently deleted. This action is irreversible.  

  • The user is notified by e-mail that the home catalogue on the particular resource has been deleted.  

Examples

Projects

  • HPC project NN0000K last had an allocation in 2019.1 on the system Fram. They did not apply for 2019.2 and didn't have an allocation for the entire period. Access to the project on Fram is closed at the start of 2019.2 period, and the project will be decommissioned and deleted on Fram 6 weeks after the period start of 2020.1. 

  • HPC project NN0000K last had an allocation in 2019.1 on the systems Saga and Fram. They did apply for quota in 2019.2, but only got an allocation on Saga. Access to the project on Fram is closed at the start of the 2019.2 period, and the project will be decommissioned and deleted on Fram 6 weeks after the 2020.1 period starts. 

  • NIRD project NS0000K last had an allocation in 2019.1. They did not apply for 2019.2 and didn't have an allocation for the entire period. Access to the project on NIRD is closed at the start of the 2019.2 period, and the project will be decommissioned and deleted 6 weeks after the 2020.1 period starts. 

Home catalogues

  • Anders has data stored in his $HOME catalogue on Fram in period 2019.1, while he was a member of HPC project nn0000k. The project of which Anders is a member did not apply for a new compute quota on Fram for the allocation period 2019.2, and therefore Anders lost access to the machine. Anders was not a member of a project with an active quota on Fram for the entirety of the period 2019.2. In this case, Anders’ $HOME catalogue would be deleted 90 days after the 2019.2 period starts.  
  • Bella has data stored in her $HOME catalogue on Fram and Saga in period 2019.1, while she was a member of HPC project nn0000k. The project did not apply for a new compute quota on Fram for allocation period 2019.2, and so Bella lost access to Fram from the start of that period. Bella was not a member of a project with an active quota on Fram for the entirety of the period 2019.2. The HPC project was granted a quota on Saga for the same period, and thus she had access to Saga. Bellas’ $HOME catalogue on Fram would be deleted 90 days after the 2019.2 period starts.