A journey through the history of the Norwegian e-infrastructure

Groundbreaking research and scientific discoveries, particularly in areas such as technology, natural sciences, and medicine, depend on rapid simulations and analysis of large datasets. Data simulations are i.e. used to study climate change, solve problems related to renewable energy, or develop vaccines faster than before. Digitalisation leads to new ways of conducting research, and the volume of research data is constantly increasing.

What is e-infrastructure?

To process and store data, researchers need powerful computers for high-capacity calculations and specialised storage systems. Computers with a lot of computing power are often referred to as HPC systems (High-Performance Computing), or supercomputers. HPC, including high-capacity data analysis, artificial intelligence, and machine learning, involves thousands of processors working in parallel to analyse vast amounts of data in real-time. Such extensive operations would take thousands of years to perform on a regular PC.

E-infrastructure is the collective term for the data systems used for high-capacity calculations, big data processing and storage, and the high-capacity networks that connect the data systems and researchers. In addition, e-infrastructure also includes software and specialised operational services necessary for researchers to use the systems.

Norway was early to invest in high-performance computing, and investments were made by the Research Council, Statoil, Hydro, SINTEF, and NTNU as early as the mid-1980s (ref: National strategy for eInfrastructure, The Research Council of Norway).

Bilde
Supercomputer Betzy.
Norway`s most powerful supercomputer is Betzy. She is located at NTNU in Trondheim.

Norway´s leap into high-performance computing (HPC)

From the mid-1980s, the need for increased computing power emerges in Norwegian research environments, and NTH (now NTNU) acquires Norway's first high-performance computing system, a Cray XMP, in 1986 at an estimated cost of 85 million kroner. At the Christian Michelsen Institute in Bergen, the first "hypercube" is installed, known as INTEL PSC, the first to be exported outside the USA.

From weather forecasts to research

The Meteorological Institute is a major consumer of computing power for weather forecasting and a significant financial contributor to the acquisition of high-performance computing systems. In addition to the Meteorological Institute, it is mainly academic environments at the Universities in Bergen, Oslo, Tromsø, and NTNU (collectively called the BOTT universities) that need computing power for research projects.

Competing for funds sparks national coordination

Each university purchases data systems to meet local demand from its own researchers and must compete with each other for funding. The first joint national coordination began in 1985 when the Ministry of Education established a joint grant of 22 million kroner per year to be managed by the Research Council of Norway.

Building Norway´s collaborative e-infrastructure

The need to coordinate the universities' high-performance computing offerings becomes more prominent towards the end of the 90s. Over a 10-15 year period, a distributed center with IT experts at the universities — a "Metacenter" — is gradually established, providing user support to researchers, regardless of the institution they belong to.

Securing a robust university network connection

There is also a gradually increasing need for long-term data storage solutions. Early in the 2000s, the four BOTT universities begin to collaborate to provide e-infrastructure to the university and college sector. First within high-performance computing, and later also for the storage of scientific data. The research network UNINETT ensures a powerful and secure network connection for data transfer between the universities' computers.

Collaboration through projects

The purpose of the university collaboration is to make the HPC and storage systems available to all research institutions in Norway, regardless of the institution the researcher belongs to. The university collaboration is organised through various projects, financed by the universities themselves and the Research Council of Norway. However, the universities still have to compete for the same funding, and a common strategy for further development of the e-infrastructure is lacking.


Milestones from the early 2000s

2000: The NOTUR project is established for HPC

The NOTUR (Norsk tungregning) project is established by the BOTT universities and the Meteorological Institute and coordinated from NTNU. The project is financed by consortium partners and the Research Council, which provides 100 million kroner for the implementation of a national infrastructure for high-capacity calculations with several distributed computing resources.

The mission of the project is to provide a modern, national HPC infrastructure in an international and competitive setting, and stimulate computational science as the third scientific path.

The project concludes in 2004.

2004: UNINETT Sigma is established

UNINETT Sigma is established as a subsidiary of UNINETT AS to coordinate computational services in collaboration with the BOTT universities.

2005: The NOTUR II project begins

The NOTUR II project receives 290 million kroner from the Research Council to build a national infrastructure for HPC. Participants are UNINETT Sigma, NTNU, UiB, UiO, UiT, and the Meteorological Institute.

The project is financed by the Research Council of Norway and will further develop elements from the first NOTUR project and other HPC projects. The project concludes in 2014.

2007: The NorStore project for storage begins

The NorStore project (Norwegian Data Storage Infrastructure) is initiated by UNINETT Sigma in collaboration with the BOTT universities to ensure that data used in high-performance computing contexts are collected in common services. It is planned and operated separately from HPC, and the storage systems are located at UiO and UiT.

2015: UNINETT Sigma2 AS is established

UNINETT Sigma2 AS is established as a non-profit company under UNINETT AS, based on an agreement between the partner universities and the Research Council.

The company becomes the formal owner of the national computing and storage facilities, and will also coordinate Norway's participation in Nordic and European collaborations on e-infrastructure.

The company is jointly financed by university partners and the Research Council, and collaborates closely with the university partners. Sigma2 operates on a 6+6 years model which entails evaulation every six years.

2019: UNINETT Sigma2 is evaluated

A committee consisting of members appointed by the partner universities and the Research Council evaluates UNINETT Sigma2, which receives very good feedback and secures financing for another five years.

2020: National Competence Centre for HPC is established to assist industry

The National Competence Centre for HPC is established in a collaboration between Sigma2, NORCE, and SINTEF, to contribute to increased technology competence in industry and public administration.

The establishment of the centre is a European initiative, and similar centers are being established all over Europe. The funding comes from the EU and from the Ministry of Education via the Research Council.

2021: External data center provider and name change for the Metacenter

Lefdal Mine Datacenter (LMD) is chosen as the external data center provider for the co-location of future national computing and storage facilities.

The Metacenter changes its name to NRIS and becomes Norwegian research infrastructure services.

2022: Sigma2 becomes a subsidiary under Sikt

UNINETT Sigma2 changes its name to Sigma2 and transitions from being a subsidiary under UNINETT to becoming a subsidiary under the newly established Sikt — The Knowledge Sector's Service Provider, which consists of the organisations that previously made up NSD, UNIT, and UNINETT.

The national e-infrastructure is available to all research institutions in Norway, and computing time and storage capacity are allocated based on scientific quality.

The supercomputers

Supercomputers procured after the establishment of Sigma and the NOTUR-II project.

2006: Njord

Njord, an IBM p575+ system, is installed at NTNU in Trondheim and made available to researchers in Norway.

Njord is operational until 2012.

2007: Titan

Titan, a Dell 1425 / Sun X2200 system, is installed at the University of Oslo.

Titan is operational until 2012.

2008: Stallo and Hexagon

Stallo, a HP BL 460c system, is installed at the University of Tromsø.

Hexagon, a Cray XT4 system, is installed at the University of Bergen.

Hexagon is operational until 2017.

2012: Abel, Gardar and Vilje

Abel, a MEGWARE MiriQuid 2600 system is installed at the University of Oslo.

Gardar, an HP BL280cG6 system installed at the University of Iceland, is included in Notur.

Vilje, a SGI Altix 8600 system, is installed at NTNU in Trondheim, and reaches the 44th position on the list of the world's top 500 supercomputers.

Vilje is opeartional until 2021.

2016: Fram and NIRD

Fram, a Lenovo NeXtScale nx360 system, is installed at the University of Tromsø. Fram is still in use by researchers in Norway.

The same year, the storage facility NIRD (Norwegian Infrastructure for Research Data) is acquired and replaces NorStore. NIRD consists of two facilities with a total storage capacity of 2x11 PB, and is located at NTNU in Trondheim and the University of Tromsø.

NIRD is turned off in 2023, replaced by a new generation NIRD storage system.

2018: Saga

Saga, an Apollo 2000/6500 Gen10 system, is installed at NTNU in Trondheim and is still in use by researchers in Norway.

2020: Betzy

Betzy, a BullSequana XH2000 system, is installed at NTNU in Trondheim.

Betzy enters at the 56th position on the top 500 list, Betzy is Norway's most powerful supercomputer to date.

2022: LUMI and new NIRD

The new pan-European supercomputer LUMI, an HPE Cray EX supercomputer, is opened tor researchers and immediately takes the 3rd place on the Top500. LUMI is installed at a data centre in Kajaani, Finland.

Norway is a co-owner of LUMI through Sigma2.

The next generation of the NIRD storage system is installed at Lefdal Mine Datacentre and is opened for Norwegian researchers. At the time of opening, the new NIRD has a storage capacity of 32 petabytes (PB) which can be expanded up to a total of 70 PB if needed.