Suppose you added two jobs to step 1 as follows:
In order for the pipeline to run, it is necessary to only have one of the job steps have a non-zero “Runs per day”
It is not necessary to have both jobs with a non-zero “Runs per day”
This pipeline will run each job one after the other starting with the job which has the non-zero “Runs per day”
If both jobs have the same “Run Time” and “Number of Runs per day”, the first (top) job will be run. Thereafter subsequent jobs will run in the order they appear.
In order to have both job execute simultaneously, create a separate pipeline and ensure that both jobs are set at step 1 and have the same “Run time” and “Number of Runs per day”:
Consider this example below:
If you inspect the logs to the right of “INFO” you’ll see “[rockstarETL-3]” and “[rockstarETL-2]”
These are separate threads running at the same time.
You can also filter the logs in the Google Cloud console:
Select the ellipsis next to your vm:
And select “View logs”:
Then you can inspect the logs:
Select inside the “Query preview” block and add in any text you’d like to filter on:
Then select “Run Query”
You can then inspect the result of your filter in the log entries: