Page tree
Skip to end of metadata
Go to start of metadata

Status:

 COMPLETED

Points of Contact:

 help@pawsey.org.au 

Start Date/Time (AWST):

08:00

Responsible:

 Mark O'Shea (SCOps Team Lead)

)Estimated End Date/Time (AWST):

17:00

Accountable:

 Mark Gray (Haead of Platforms)

End Date/Time (AWST):

17:35

Informed:

 pawsey-users Mailchimp list

Summary:

Pawsey Scheduled Maintenance

Systems/Services Affected:

Setonix, Magnus, Garrawarla, Galaxy, Zeus, Topaz, 

Description:

Scheduled maintenance to Supercomputing Related Infrastructure to get ready for Setonix.

Changes:

  • Upgrade of Slurm Accounting backend (DBD) to support new version of Slurm on Setonix.
  • Upgrade of Controller firmware on filesystems.
  • Updating LDAP SSL Certs.

Updates:

  • 08:00 SSL LDAP Certs update
  • 09:05 Access to login nodes removed ahead of Slurm database updates
  • 09:07 Slurm Database being backup
    • Affects All clusters, ie Magnuz, Galaxy, Zeus, Topaz, Garrawarla & Askap Ingest is offline as they use the Pawsey Slurm Database
  • 09:35 Slurm Database completed
  • 09:40 Upgrade the Slurm Database components from version 21.08.8 to 22.05.5
  • 10:00 Slurm database controller is now version 22.05.5
  • 10:20 Upgrade Slurm Database Schema to be compliant with 22.05.5
    • This can take awhile just because of the sheer size of accounting data of all clusters
  • 10:30 Update SSL certs for remove visualisation servers ie (remotevis.pawsey.org.au)
  • 14:10 Slurm Database Schema / Data is now compliant with Slurm 22.05.x
    • Restart respective slurm controllers from the clusters mentioned above
  • 14:15 Update SSL certs for askap-ingest service controllers
  • 17:05 Askapbuffer Lustre Filesystem restored
    • Firmware update to backend physical storage ME4084 Completed
  • 17:35 All Clusters Released for general use
    • Askap Ingest
    • Magnus
    • Galaxy
    • Zeus
    • Garrawarla
    • Topaz

Further information:

This page documents the work of the SCOps team on this Maintenance day: other work may be visible via the page for the whole day