Compute Engine
C++ part of the Deep Lake codebase optimized for faster data fetching and computations
Last updated
Was this helpful?
C++ part of the Deep Lake codebase optimized for faster data fetching and computations
Last updated
Was this helpful?
In order to maximize performance of compute-heavy Deep Lake's features, such as distributed dataloading or large queries, it is necessary to move beyond their pure-Python OSS implementations and use the C++ codebase in the Deep Lake Compute Engine.
The interface for the Compute Engine is still in Python, and it is installed using pip install "deeplake[enterprise]", which delivers the compiled C++ code from binaries.
In order to use the optimized Compute Engine, datasets can be stored in the user's cloud, but they must be connected to Deep Lake using Managed Credentials.