Integrated Award Environment - Transitioning from RDS to Aurora Postgres Cluster

Introduction

The goal of the Integrated Award Environment (IAE) is to enhance transparency, efficiency, and accountability in the management of federal grants, contracts, and other financial assistance programs. The System for Award Management (SAM.gov) is an official website of the U.S. Government required by statute. There are 2.8Million+ entity registrants in SAM.gov. Entities from the Acquisition Community and Financial Assistance Community use SAM.gov to:

• Register to do business with the U.S. Government
• Update, renew, or check the status of your entity registration
• Search for entity registration and exclusion records
• Search for assistance listings (formerly CFDA.gov), wage determinations (formerly WDOL.gov), contract opportunities (formerly FBO.gov), and contract data reports (formerly part of FPDS.gov).
• View and submit BioPreferred and Service Contract Reports
• Access publicly available award data via data extracts and system accounts

This case study focuses on the methodology used to improve overall efficiency, scalability, and reduce program costs, by leveraging efficient technology tools to transition from RDS to Aurora Postgres Cluster.

Business Challenge

With the exponential growth of data and users accessing the system/database, and with the addition of new services, SAM.gov is facing challenges with:

• Storage Threshold Prod issues
• Database Performance Issues
• Database creation request latency
• Upfront provisioning of database configurations

SAM.gov needs to explore other solutions to keep the database performance high/consistent, optimize storage and maintain the system by leveraging efficient technology tools.

IAE IT’s Vision to Address the Business Challenge

• Conduct Proof of Concept
• Assess current SAM Architecture
• Conduct research on the current Databases/sizing
• Propose options
• Pick the best option that meets set criteria
• Conduct thorough testing in the lower environments
• Conduct performance testing
• Deploy into production

SAM Application Architecture

Within the modernized application architecture, SAM.gov is decomposed into independent micro-frontend Angular applications and Spring Boot powered microservices.

• Each one of these components is responsible for a specific SAM.gov functionality and can be updated, tested, and deployed quickly since they are limited in scope and contain few dependencies on other services or packages.
• The individual components are connected through a custom reverse implementation based on Spring Cloud Gateway. This gateway receives and routes all requests for the application to the correct micro-frontend or API Docker cluster.
• User authentication is provided by an external provider, Login.gov.

IAE IT Solution Approach

The IAE IT team identified an opportunity to conduct a Proof of Concept (PoC) to transition from RDS to Aurora Postgres Cluster to address the Business challenge.

Solution Methodology

Step 1: Conduct analysis on the size/storage of RDS instances in the lower non-prod environment. Assess the scope of transition and propose options for transition:

Step 2: Conduct detailed research on the 3 proposed options below, collaborate with Joint Product Team and Site Reliability Engineering (SRE) team on estimated sizing of the cluster. Conduct thorough tests in the lower environments prior to deploying into production.

Technical Approach of Rollout Following the Successful Proof of Concept (POC)

No impact to business
Phased rollout

1. Establish a timeline to create all new databases under the Aurora Cluster in the Comp (development) environment.
2. A new cluster will be created in the Minc(test) environment and new databases will be created/promoted to this cluster.
3. Establish a timeline that starting from MM/DD/YY, identified services will be migrated to the Aurora cluster in Comp(development) and Minc (Test).
4. This established process will continue and spread out through multiple Program Increments (PI’s) for smooth transition from RDS to Aurora.

Business Benefits of Transitioning from RDS to Aurora Postgres Cluster

Performance

• Amazon Aurora gives two times the throughput provided by PostgreSQL running on similar hardware. This helps with the Read and Write operations and could enhance the performance of long running queries.
• Currently, we have provisioned individual databases with very high capacity. That is why we have not encountered major performance issues.
• Moving to Aurora will enhance the performance and bring in cost efficiencies

Storage Auto-Scaling

• Amazon Aurora can grow its storage from its minimum of 10 GB to a maximum of 128 TB. depending on your database usage.
• This is done in increments of 10 GB without any impact on the database performance. It can scale down automatically if data is removed.
• The IAE program doesn’t have to predict and provision the storage needed for uninterrupted business operations.
• This will save some cost/effort compared to RDS’s where a fixed storage needs to be provisioned and Auto-scaling is bounded by a threshold.

Scalability

• Aurora can scale the memory and compute resources down or up, up to a maximum of 244 GiB of RAM and 32 vCPUs.
• As we will provision at least 2 nodes (writer/reader) in each environment, the scaling operation will have no downtime on the application side.

Replication / Backup / High Availability / Disaster Recovery / Database Cloning

• Aurora supports replication with the replicas sharing a similar underlying volume with the primary instance.
• Aurora allows provisioning up to 15 replicas, and the replication is done in milliseconds.
• In Aurora, failover is done automatically to prevent data loss. In RDS, failover is done manually which can lead to last-minute data loss and the process of replication is slower compared to Amazon Aurora.
• Aurora PostgreSQL backs up cluster volume automatically and retains backups for the length of 35 days defined as per retention period parameter.

Serverless

• In Aurora, DB cluster deployment automatically starts, scales, and shuts down an Aurora database based on application needs. It’s a suitable option for infrequent, intermittent, or unpredictable workloads.
• The environment like Comp (development), Minc (Test) and Charlie (Performance testing) can use this feature to save cost while they are not being used.