Backing Up Google Search Console Data with Google Cloud, Docker, and Python

Backing up data from Google Search Console (GSC) is crucial for digital marketers and SEO specialists, as GSC provides key insights into website performance. However, Google only retains this data for 16 months. To ensure access to historical data, setting up an automated backup process using Google Cloud, Docker, and Python is a powerful solution.

tl;dr:

By using Docker, Google Cloud, and Python, you can create an automated backup solution for your Google Search Console data. This setup eliminates traditional server management with Cloud Run and ensures secure, accessible backups in Google Cloud Storage.

Requirements

To start, you need:

• Google Cloud Account and Google Cloud SDK
• Google Search Console Access
• Docker

Step-by-Step Guide

1. Setting Up the Python Script

The core of this backup process is a Python script that fetches data from GSC using the Google API client. Here’s how it works:

• API Authentication: The script uses IAM credentials for authentication, eliminating the need for manual service account credentials management.
• Fetching GSC Data: It retrieves data such as clicks, impressions, average position, and CTR from Google Search Console for the last 30 days.
• Data Export: The data is converted to a CSV file for storage.
• Cloud Storage Upload: The CSV file is uploaded to Google Cloud Storage for later access.

2. Dockerfile Setup

To deploy the Python script on Google Cloud, you need a Dockerfile that defines the application environment. Docker allows you to “containerize” the application, making it easy to deploy.

• Requirements: Include necessary libraries like google-api-python-client, google-auth, and pandas in your requirements.txt file.

3. Setting Up Google Cloud and IAM

• Enable APIs: In the Google Cloud console, enable the Google Search Console API and Google Cloud Storage API.
• Create a Service Account: Set up a service account with the necessary permissions to access GSC data and upload to Cloud Storage. Assign roles like Viewer for Search Console and Storage Admin for the cloud storage bucket.
• Cloud Storage Bucket: Create a private Google Cloud Storage bucket to store backup files.

4. Deploying to Google Cloud Run

• Build and Push Docker Image: Build the Docker image and push it to Google Container Registry.
• Deploy to Cloud Run: Deploy the containerized application to Cloud Run, which allows you to run applications without managing servers.

Once deployed, Cloud Run will run your Python script on a schedule, automatically pulling data from GSC and uploading it to your private Google Cloud Storage bucket.

5. Automating the Process

To ensure periodic backups, use Google Cloud Scheduler to trigger the Cloud Run service. Set up a cron job to invoke the service daily or weekly to keep your data backed up automatically.

Benefits of This Solution

• Data Security: Google Cloud Storage ensures that your backup data is secure and easily accessible.
• Automation: The process is fully automated, saving you time and preventing data loss.
• Flexibility: You can focus on growing your business while maintaining access to historical GSC data.

Additional Considerations

Compliance: Implementing a retention policy helps enforce compliance with organizational data recovery strategies.

Data Retention: Ensure that your Google Cloud Storage bucket has a suitable retention policy to prevent accidental deletion of backup files. You can set a retention period of up to 100 years.

Alex Bieth

Alex is an experienced SEO consultant with over 14 years of working with global brands like Montblanc, Ricoh, Rogue, Gropius Bau and Spartoo. With a focus on data-driven strategies, Alex helps businesses grow their online presence and optimise SEO efforts.

After working in-house as Head of SEO at Spreadshirt, he now works independently, supporting clients globally with a focus on digital transformation through SEO.

He holds an MBA and has completed a Data Science certification, bringing strong analytical skills to SEO. With experience in web development and Scrum methodologies, they excel at collaborating with cross-functional teams to implement scalable digital strategies.

Outside work, he loves sport: running, tennis and swimming in particular!