The Dataset API is intended to act as an abstraction layer between user-uploaded files and data that is ingested internally for downstream applications. It will act as the single interface for data uploading, downloading, format conversion, and validation, and it also persistently tracks all relevant metadata.

You can read more about the Dataset API on its official page.