SMX will provide big data cloud architecture design and service support for the NAWCAD NTA cloud environments. This engagement delivers the resources and services with support for big data analytics, structure, storage, and lifecycle management in the current cloud environment (AWS).
SMX is using Amazon Redshift to query data in the same way as the current data warehouse to test if Redshift can reproduce a set of queries and workloads that the current data warehouse conducts. NDC14 will be utilized as a standalone AWS Government Cloud account to ensure sensitive organizational data is handled securely (IL4/5). SMX will upload data into an S3 Bucket, then import the data into Redshift for testing.
Redshift with S3 becomes the main data warehouse system with other cloud native services such as:
- Glue for data cataloging and indexing.
- Quicksight for data and analytics visualizations
- S3 Storage lifecycle
- Machine learning and advanced analytics using Redshift ML and potentially Sagemaker.
The components listed above will be integrated into the IaC of the overall cloud environment and a data engineering management plane for ingestion, transformation, storage, and analysis.