Passing parameters between Cloud Run Jobs in Cloud Workflows

[Karsten]

Hi all,

I'm migrating ETL pipelines from Airflow to Cloud Workflows with (Python) Cloud Run Jobs. In Airflow, one job handed back a result that was provided to the next job via X-Com. However, in Workflows the result I get back is only a description of the job, not the return value of the Python function. So, how can I do this with Cloud Run Jobs?

Currently I don't want to switch to Cloud Run Services or use other GCP products (Pub/Sub) to workaround this.

Thank you

[robertcarlos]

Hi @Karsten,

Welcome to Google Cloud Community!

You may check out this documentation on passing runtime arguments in an execution request so that you can return values of your Python functions through Cloud Workflows from your Airflow.

You may check this link on using client libraries to execute a workflow for the sample code.

Hope this helps.

[ipeluffo]

Hey, were you able to find an answer or solution to this? Thanks