Step 7: Connecting Hub Datasets to ML Frameworks

You can connect Hub Datasets to popular ML frameworks such as PyTorch and TensorFlow using minimal boilerplate code.

PyTorch

You can train a model by creating a PyTorch DataLoader from a Hub Dataset using ds.pytorch() .

from Hub import Dataset
from torch.utils.data import DataLoader
​
ds = Dataset('./dataset_path') # Hub Dataset
dataloader= ds.pytorch() #PyTorch Dataloader
​
for data in dataloader:
    print(data)
    # Training Loop

TensorFlow - Coming Soon

Similarly, you can convert a Hub Dataset to a TensorFlow Dataset via the tf.Data API.

ds # Hub Dataset object, to be used for training
ds = ds.tensorflow() # A TensorFlow Dataset