Deploy Staging to SMCE

[smohiudd]

Description

1. Deploy the following to SMCE Staging:- Backend

• Auth
• SM2A
• MWAA
• Feature API
• Routes?

Options for CloudFront:

• Create manual distribution in UAH instead, add DNS records in Route53
• use smce/api/* root path and use existing staging CF distribution

2. Cutover existing Staging urls to new staging deployment

• remove smce/api/ root path
• add uah/api root path to original staging deployment

Acceptance Criteria

• Staging environment deployed for SMCE using veda-deploy

[anayeaye]

Progress

• new secrets added to smce
  • veda-deploy-smce-staging Environment configuration used to deploy the smce-staging environment from veda-deploy
  • 
  • veda-shared-base Configuration used to CDK to provision a VPC for all veda-deploy instances. https://github.com/NASA-IMPACT/veda-backend/tree/develop/standalone_base_infrastructure
• standalone base infrastructure deployed
• confirmed that the DATA_ACCESS_ROLE used by the titiler, ingest, and airflow can be assumed by services running in SMCE
• created bucket for browser veda-smce-staging-browser
• created and shared a snapshot of the the staging veda-backend database

Questions + TODOs

• arn:aws:iam::444055461661:policy/smce-tenantOperator appears to be the permissions boundary policy we want to use but we should sort this out (and identify what variables we are still using--there are 3 flavors of permissions boundary variable in our uah-staging secret
• need to configure git refs for the components we deploy
• need to generate a SM2A secret in smce-veda for the staging deploy
• need to look into using a snapshot of features-api db for our deployment (could do this after getting the backend and sm2a deployed though)

Notes

• we expect to use the existing staging.openveda.cloud cloudfront and will manually add new behaviors for our services with custom root paths
• let's try an alternate root path for the stac api--something like /smce/api/stac_catalog instead of just stac because the radiant earth browser has behaviors configured for /stac that are incompatible with a root path prefix /stac

[anayeaye]

Status update

We now have Auth, Backend, Features, and SM2A deployed to veda-smce. Remaining work includes manual cloudfront and route53 configuration and testing all services and pipelines. The features API database configuration appears to need more work.

Note: the RDS instances for backend and features were created from encrypted shapshots of the uah-staging databases. The steps were (see this re:Post:

1. in uah: create a custom KMS key and share with smce account
2. in uah: create a copy of a backup and encrypt w/the custom kms key
3. in uah: share the copied snapshot with smce account
4. in smce: find the shared database (in shared with me section of rds console) and make a copy to smce
5. in smce: finally, use the arn of the copied encrypted snapshot in the deployment env vars

Routes to configure

• features: https://uolm4md0j0.execute-api.us-west-2.amazonaws.com/ note: the links do not seem to be working, the configuration the features database was deployed with should be revisited, as should the vpc endpoints in smce shared vpc
• ingest-api: https://u3pizeieb1.execute-api.us-west-2.amazonaws.com/
• raster-api: https://w6fsgo99j9.execute-api.us-west-2.amazonaws.com/
• stac-api: https://hp26w25smk.execute-api.us-west-2.amazonaws.com/
• browser: http://veda-smce-staging-browser.s3-website-us-west-2.amazonaws.com/ note: this needs a subdomain as well because we already have a browser at the base staging.openveda.cloud

[anayeaye]

Still sorting out some routing quirks for features api and the browser as well as an issue with loading tables from a restored features database but we do now have staging.openveda origins and behaviors added to the existing uah-staging cloudfront distribution.

smce staging

• stac-api: https://staging.openveda.cloud/smce/api/stac
• raster-api: https://staging.openveda.cloud/smce/api/raster/docs
• ingest-api: https://staging.openveda.cloud/smce/api/raster/docs
• features-api https://staging.openveda.cloud/smce/api/features/ (bug: needs trailing slash right now)
• browser: http://veda-smce-staging-browser.s3-website-us-west-2.amazonaws.com/ (expected at https://staging.openveda.cloud/smce/browser)

[anayeaye]

Features API url bug fixed NASA-IMPACT/veda-features-api-cdk#11