/gcp_cloud_function_sql_export/README.md

https://github.com/HariSekhon/pytools · Markdown · 52 lines · 35 code · 17 blank · 0 comment · 0 complexity · 1c1ade163f5756a22f0ed50047c39802 MD5 · raw file

  1. Google Cloud Function - SQL Backup Exporter to GCS
  2. =====================
  3. Triggers GCP [Cloud SQL](https://cloud.google.com/sql) export backups to [GCS](https://cloud.google.com/storage).
  4. Solution documentation:
  5. https://cloud.google.com/solutions/scheduling-cloud-sql-database-exports-using-cloud-scheduler
  6. - `main.py` - the code
  7. - `requirements.txt` - the pip modules to bootstrap
  8. - `deploy.sh` - upload the code and deps
  9. Upload the function to GCF in the current GCP project - this script will call `gcloud functions deploy` with the required switches:
  10. ```
  11. ./deploy.sh
  12. ```
  13. ### Solution Dependencies
  14. - a [Cloud PubSub](https://cloud.google.com/pubsub) topic
  15. - [Cloud Scheduler](https://cloud.google.com/scheduler) jobs to trigger backups
  16. - see `gcp_cloud_schedule_sql_exports.sh` in [DevOps Bash tools](https://github.com/HariSekhon/DevOps-Bash-tools/) repo
  17. - a service account with permissions to access [Cloud SQL](https://cloud.google.com/sql)
  18. - see `gcp_sql_create_readonly_service_account.sh` in [DevOps Bash tools](https://github.com/HariSekhon/DevOps-Bash-tools/) repo
  19. - each [Cloud SQL](https://cloud.google.com/sql) instance to be backed up requires objectCreator permissions to the [GCS](https://cloud.google.com/storage) bucket
  20. - see `gcp_sql_grant_instances_gcs_object_creator.sh` in [DevOps Bash tools](https://github.com/HariSekhon/DevOps-Bash-tools/) repo
  21. ### Serverless Framework
  22. Instead of `deploy.sh` you can alternatively use the [Serverless](https://www.serverless.com/) framework for which a `serverless.yml` config is provided:
  23. ```
  24. serverless deploy
  25. ```
  26. If this is your first time using Serverless then you'll need to install the GCP plugin:
  27. ```
  28. serverless plugin install --name serverless-google-cloudfunctions
  29. ```
  30. The `serverless.yml` config expects to find `$GOOGLE_PROJECT_ID` and `$GOOGLE_REGION` environment variables.
  31. Serverless requires additional permissions for the service account: Deployment Manager Editor and Storage Admin to create deployments and staging buckets.
  32. You can also build a serverless artifact to `.serverless/` without deploying it (generates Google [Deployment Manager](https://cloud.google.com/deployment-manager) templates and a zip file - useful to check what would be uploaded / ignored):
  33. ```
  34. serverless package
  35. ```