Loading

What is the best way to load data into BigQuery?

rockstarETL (Rest API) vs DataFlow vs Client libraries? DataFlow I would not recommend it for batch loads. It’s just too slow. DataFlow has it’s place if you want to stream.¬† Just bear in mind, streaming has a cost associated with it! And using rockstarETL is so performant, you have to seriously wonder if the additional …

What is the best way to load data into BigQuery? Read More »

Limitations with Cloud Storage Transfer

While Cloud Storage Transfer does offer support for loading data on a schedule, there are some important limitations. Let’s take a look: rockstarETL does not share these limitations! Waiting an hour before being able to use your data may be a deal-breaker! By contrast, rockstarETL will detect your source files immediately: As soon as they are copied to …

Limitations with Cloud Storage Transfer Read More »