Is Google Storage Transfer supposed to update files too?

[GustavoG]

Hello there,

I setup a job to sync my files from AWS S3 to a Google Cloud Storage bucket. I followed the official guide provided here:

https://cloud.google.com/blog/topics/developers-practitioners/transfer-data-aws-gcp-using-storage-tr...

Everything works fine when I upload new files to my AWS S3 bucket, but when I update them, I don't see the changes on my GC Storage bucket.

I can see the SQS messages being sent, with different eTAGs, but nothing happens on the GC bucket. All the files have the same "Creation" and "Last Update" date.

This bucket contains CSV files with certain structure and I've set up a BigQuery table reading these files.

Thanks in advance!

[juliadeanne]

Hello @GustavoG,

Welcome to the Google Cloud Community!

Please make sure that the timestamps of your files in AWS S3 are preserved during the sync process. If the "Creation" and "Last Update" dates stay the same, it could mean there's a problem with how the sync process deals with file details. Double-check your sync settings to ensure it's set to recognize and sync changes to files. It might be set up to only sync new files and not update existing ones.