hub.compute
for parallelizing workflowshub.compute
can be used to rapidly upload datasets. This tutorial expands further and highlights the power of parallel computing for dataset processing.hub.like
is used to create an empty dataset with the same tensor structure.ds_in
, and the result is automatically stored in ds_out
.hub.ingest
to automatically convert this image classification dataset into hub format and save it in ./animals_hub
.flip_vertical()
transformation function from the previous example. If a second dataset is not specified as an input to .eval()
, the transformation is applied to the input dataset. flip_vertical
function from the first example and the resize
function below.hub.compute
can be easily combined into pipelines using hub.compose
. Required arguments for the functions must be passed into the pipeline in this step:ds_in
. The result is automatically stored in ds_out
.