But what is orchestration?
rockstarETL’s pipelines allow you to define processing steps. Multiple jobs can execute simultaneously in the same step.
Jobs in subsequent steps will only begin once the previous steps have executed.
rockstarETL also allows you to schedule your pipeline to run at a certain time and up to every 5 minutes:
Task aware and Data aware Architecture
rockstarETL offers both a task aware and data aware architecture for your pipelines!
By default, all job types are task aware. The next step will not begin until the tasks in the currently executing step are finished.
But the conditionalSQL job type is data aware!