Mana Upgrade

Beginning June 7, 2023, Mana will undergo a six month, progressive upgrade to its physical and software infrastructure. This upgrade schedule is designed to provide minimal impact on cluster usage so no extended downtime is anticipated and research can continue as normal.

This upgrade will involve the following activities and is projected to be completed by December 2023.

We expect the changes to Mana to provide improved performance, stability, and security features for our users. However, we understand that this may cause some inconvenience and we are committed to working closely with our users to minimize any disruptions during this transition period.

A complete timeline and a general description of what will be done in each phase can be found below.


Phase 1:  Completion date June 7, 2023

    • Allow access to Rocky 9 login nodes
    • New home and scratch folders for active users
    • Migrate 40 older CPU nodes that will only allow for High throughput computation
    • Migrate 12 newer CPU nodes that allow for MPI workloads
    • Migrate 3 GPU nodes – 2x A30 nodes, 1x A4000 node

Phase 2: Completion date August 2, 2023

    • Allow access to Rocky 9 Open OnDemand nodes
    • Migrate all remaining older CPU nodes
    • Migrate up to 50% of the total newer CPU nodes
    • Migrate 4 GPU nodes, 3x A30 nodes, 1x A4000 node
    • Start taking/fulfilling requests from Condo owners to migrate

Phase 3: Completion date October 4, 2023

    • Migrate all remaining newer community CPU nodes
    • Migrate all remaining community GPU nodes
    • Continue taking/fulfilling requests from Condo owners to migrate

Phase 4: Completion date December 6, 2023

    • Migrate all remaining nodes running CentOS 7

Upgrade Notices

Date: 5/24/2023

Notification: Rocky 9 System Configuration – Deprecation of QDR Infiniband support

Impacts: Multi-node MPI jobs on nodes that are QDR connected

On June 7, 2023, the previously announced 6-month Mana update to a new hardware and software configuration utilizing Rocky 9 will begin.  If you run multi-node MPI jobs on nodes that are QDR connected you will be impacted by the absence of Quad Data Rate (QDR) InfiniBand support in Rocky 9. 

We understand that this may require some adjustments on your part and we apologize for any inconvenience this may cause. To help with the transition, we strongly recommend that you migrate your applications and workflows to High Data Rate (HDR) nodes or consider procuring nodes during our Fall purchasing period that support HDR Infiniband.

HDR InfiniBand is the recommended replacement for QDR InfiniBand, providing higher bandwidth, lower latency, and improved scalability. By upgrading to HDR, you can take advantage of the latest technology and ensure optimal performance for your applications.

If you have any questions or concerns about this update or the migration to HDR, please do not hesitate to contact us at Our team is available to provide guidance and support throughout the transition process.

Thank you for your attention and cooperation. We look forward to continuing to support your research and computational needs on the updated system.