Isolation Provider
Runtime Operator Isolation in Airflow.
Created with ❤️ by the CSE Team @ Astronomer
Summary
Quickstart
Pre-requisites:
Steps
Installation
CLI
Usage
How
What it is
What it isn't
Requirements
CLI
Host Airflow
Target Isolated Environment
Summary
The Isolation Provider provides the IsolatedOperator and the isolationctl CLI.
It provides the capacity to run any Airflow Operator in an isolated fashion.
Why use IsolatedOperator?
Run a different version of an underlying library,
between separate teams on the same Airflow instance,
even between separate tasks in the same DAG
Run entirely separate versions of Python for a task
Keep "heavy" dependencies separate from Airflow
Run an Airflow task in a completely separate environment - or on a server all the way across the world.
Run a task "safely" - separate from the Airflow instance
Run a task with dependencies that conflict more easily
Do all of the above while having unmodified access to (almost) all of 'no
|