Access Azure Datalake Gen1 with fsspec and dask

Filesystem interface to Azure-Datalake Gen1 and Gen2 Storage To use the Gen1 filesystem: import dask.dataframe as dd storage_options={'tenant_id': TENANT_ID, 'client_id': CLIENT_ID, 'client_secret': CLIENT_SECRET} dd.read_csv('adl:/{STORE_NAME}/{FOLDER}/*.csv', storage_options=storage_options) To use the Gen2 filesystem you can use the protocol abfs or az: import dask.dataframe as dd storage_options={'account_name': ACCOUNT_NAME, 'account_key': ACCOUNT_KEY} ddf = dd.read_csv('abfs:/{CONTAINER}/{FOLDER}/*.csv', storage_options=storage_options) ddf = dd.read_parquet('az:/{CONTAINER}/folder.parquet', storage_options=storage_options)