DagsterDocs
Quick search

Executing Pipelines#

Here you should include one short sentence about the concept you are describing on the page.

Relevant APIs#

NameDescription
@solidDescription of the API method
@pipelineDescription of the API method
@resourceDescription of the API method
@scheduleDescription of the API method

Overview#

https://docs.dagster.io/overview/execution

Dagit#

TODO

Python API#

TODO

Dagster CLI#

TODO

Run Config#

Dagster includes a system for defining the schema that configuration values must abide by.

Several dimensions of pipeline execution can be determined at execution time through configuration. We call this set of chosen values run config. The run config is passed as a dictionary in the python API or as a yaml document when using dagit or the CLI. The following top-level keys in the run config allow you to configure different aspects:

  • Solids: Configure solids that belong to the pipeline. In addition to providing values for solid specific configuration, inputs may also be configured here, when dependencies on upstream solids outputs have not been set in the pipeline.

  • Resources: Configure resources that belong to the pipeline that have defined configuration schema.

  • Execution: Determine and configure the Executor to be used to control execution of the pipeline.

  • Loggers : Determine and configure the LoggerDefinition to be used when logging.

You can find detailed information in Run Config Schema.

Configuring Solids#

TODO

Configuring Resources#

TODO

Configuring Executor#

TODO

Configuring Logger#

TODO

Solid Selection#

https://docs.dagster.io/overview/solid-selection

TODO