diff --git a/docs/proposals/workflow.md b/docs/proposals/workflow.md deleted file mode 100644 index 11f0b59e38..0000000000 --- a/docs/proposals/workflow.md +++ /dev/null @@ -1,300 +0,0 @@ - - - - -WARNING -WARNING -WARNING -WARNING -WARNING - -

PLEASE NOTE: This document applies to the HEAD of the source tree

- -If you are using a released version of Kubernetes, you should -refer to the docs that go with that version. - - - -The latest release of this document can be found -[here](http://releases.k8s.io/release-1.3/docs/proposals/workflow.md). - -Documentation for other releases can be found at -[releases.k8s.io](http://releases.k8s.io). - --- - - - - - - -## Abstract - -This proposal introduces [workflow](https://en.wikipedia.org/wiki/Workflow_management_system) -functionality in kubernetes. -Workflows (aka [DAG](https://en.wikipedia.org/wiki/Directed_acyclic_graph) workflows -since _tasks_ are organized in a Directed Acyclic Graph) are ubiquitous -in modern [job schedulers](https://en.wikipedia.org/wiki/Job_scheduler), see for example: - -* [luigi](https://github.com/spotify/luigi) -* [ozie](http://oozie.apache.org/) -* [azkaban](https://azkaban.github.io/) - -Most of the [job schedulers](https://en.wikipedia.org/wiki/List_of_job_scheduler_software) offer -workflow functionality to some extent. - - -## Use Cases - -* As a user, I want to create a _JobB_ which depends upon _JobA_ running to completion. -* As a user, I want workflow composability. I want to create a _JobA_ which will be triggered -as soon as an already running workflow runs to completion. -* As a user, I want to delete a workflow (eventually cascading to running _tasks_). -* As a user, I want to debug a workflow (ability to track failure): in case a _task_ -didn't run user should have a way to backtrack the reason of the failure, understanding which -dependency has not been satisified. - - -## Implementation - -In this proposal a new REST resource `Workflow` is introduced. A `Workflow` is represented as a -[graph](https://en.wikipedia.org/wiki/Graph_(mathematics)), more specifically as a DAG. -Vertices of the graph represent steps of the workflow. The workflow steps are represented via a -`WorkflowStep`1 resource. -The edges of the graph represent _dependecies_. To represent edges there is no explicit resource -- rather they are stored as predecessors in each `WorkflowStep` (i.e. each node). -The basic idea of this proposal consists in creation of each step postponing execution -until all predecessors' steps run to completion. - -### Workflow - -A new resource will be introduced in the API. A `Workflow` is a graph. -In the simplest case it's a graph of `Job`s but it can also -be a graph of other entities (for example cross-cluster objects or other `Workflow`s). - -```go - -// Workflow is a DAG workflow -type Workflow struct { - unversioned.TypeMeta `json:",inline"` - - // Standard object's metadata. - // More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#metadata. - api.ObjectMeta `json:"metadata,omitempty"` - - // Spec defines the expected behavior of a Workflow. More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#spec-and-status. - Spec WorkflowSpec `json:"spec,omitempty"` - - // Status represents the current status of the Workflow. More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#spec-and-status. - Status WorkflowStatus `json:"status,omitempty"` -} - -// WorkflowList implements list of Workflow. -type WorkflowList struct { - unversioned.TypeMeta `json:",inline"` - - // Standard list metadata - // More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#metadata - unversioned.ListMeta `json:"metadata,omitempty"` - - // Items is the list of Workflow - Items []Workflow `json:"items"` -} - -// WorkflowSpec contains Workflow specification -type WorkflowSpec struct { - // Standard object's metadata. - // More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#metadata - api.ObjectMeta `json:"metadata,omitempty"` - - //ActiveDealineSeconds contains - ActiveDeadlineSeconds *int64 `json:"activeDeadlineSeconds,omitempty"` - - // Steps is a map containing the workflow steps. Key of the - // map is a string which uniquely identifies the workflow step - Steps map[string]WorkflowStep `json:"steps,omitempty"` - - // Selector for created jobs (if any) - Selector *LabelSelector `json:"selector,omitempty"` -} - -// WorkflowStep contains necessary information to identifiy the node of the workflow graph -type WorkflowStep struct { - // JobTemplate contains the job specificaton that should be run in this Workflow. - // Only one between externalRef and jobTemplate can be set. - JobTemplate *JobTemplateSpec `json:"jobTemplate,omitempty"` - - // External contains a reference to another schedulable resource. - // Only one between ExternalRef and JobTemplate can be set. - ExternalRef *api.ObjectReference `json:"externalRef,omitempty"` - - // Dependecies represent dependecies of the current workflow step. - // Irrilevant for ExteranlRef step - Dependencies []string `json:"dependencies,omitempty"` -} - -type WorkflowConditionType string - -// These are valid conditions of a workflow. -const ( - // WorkflowComplete means the workflow has completed its execution. - WorkflowComplete WorkflowConditionType = "Complete" -) - -type WorkflowCondition struct { - // Type of workflow condition, currently only Complete. - Type WorkflowConditionType `json:"type"` - - // Status of the condition, one of True, False, Unknown. - Status api.ConditionStatus `json:"status"` - - // Last time the condition was checked. - LastProbeTime unversioned.Time `json:"lastProbeTime,omitempty"` - - // Last time the condition transited from one status to another. - LastTransitionTime unversioned.Time `json:"lastTransitionTime,omitempty"` - - // (brief) reason for the condition's last transition. - Reason string `json:"reason,omitempty"` - - // Human readable message indicating details about last transition. - Message string `json:"message,omitempty"` -} - -// WorkflowStatus represents the -type WorkflowStatus struct { - // Conditions represent the latest available observations of an object's current state. - Conditions []WorkflowCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"` - - // StartTime represents time when the job was acknowledged by the Workflow controller - // It is not guaranteed to be set in happens-before order across separate operations. - // It is represented in RFC3339 form and is in UTC. - // StartTime doesn't consider startime of `ExternalReference` - StartTime *unversioned.Time `json:"startTime,omitempty"` - - // CompletionTime represents time when the workflow was completed. It is not guaranteed to - // be set in happens-before order across separate operations. - // It is represented in RFC3339 form and is in UTC. - CompletionTime *unversioned.Time `json:"completionTime,omitempty"` - - // Statuses represent status of different steps - Statuses map[string]WorkflowStepStatus `json:statuses` -} - -// WorkflowStepStatus contains necessary information for the step status -type WorkflowStepStatus struct { - //Complete is set to true when resource run to complete - Complete bool `json:"complete"` - - // Reference of the step resource - Reference api.ObjectReference `json:"reference"` -} -``` - -`JobTemplateSpec` is already introduced by -ScheduledJob controller proposal](https://github.com/kubernetes/kubernetes/pull/11980). -Reported for readability: - -```go -type JobTemplateSpec struct { - // Standard object's metadata. - // More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#metadata - api.ObjectMeta - - // Spec is a structure defining the expected behavior of a job. - // More info: http://releases.k8s.io/HEAD/docs/devel/api-conventions.md#spec-and-status - Spec JobSpec -} -``` - -## Controller - -Workflow controller will watch `Workflow` objects and any `Job` objects created by the workflow. -the `Job`s objects created in each step. -Each step can contain either another `Workflow` referenced via `workflowStep.ExternalRef` -or a `Job` created via `workflowStep.jobTemplate`. -For each non finished workflow (similarly to Job, Workflow completion is detected iterating -over all the `status.conditions` condition) we check if deadline is not expired. -If deadline expired the workfow is terminated. -If deadline didn't expires the workflow controller iterates over all workflow steps: - - If step has a status (retrieved via step name (map key) in the `status.statuses` - map check whether the step is already completed. - - If step is completed nothing is done. - - If step is not completed two sub-cases should be analyzed: - * Step containing workflow: check wether workflow terminated and eventually update - the `status.statuses[name].complete` entry if applicable - * Step containing job: check whether job needs to be started or is already started. - - A step/job is started if it has no dependecies or all its dependencies are already - terminated. Workflow controller adds some labels to the Job. - This will permit to obtain the workflow each job belongs to (via `spec.Selector`). - The step name is equally inserted as a label in each job. - - If the job is already running, a completion check is performed. If the job terminated - (checked via conditions `job.status`) the field `status.statusues[name].complete` is updated. - - When all steps are complete: `complete` condition is added to `status.condition` and the - `status.completionTime` is updated. At this point, workflow is finished. - - -## Changing a Workflow - -### Updating - -User can modify a workflow only if the `step`s under modification are not already running. - - -### Deleting - -Users can cancel a workflow by deleting it before it's completed. All running jobs will be deleted. -Other workflows referenced in steps will not be deleted as they are not owned by the parent workflow. - - -## Events - -The events associated to `Workflow`s will be: - -* WorkflowCreated -* WorkflowStarted -* WorkflowEnded -* WorkflowDeleted - -## Kubectl - -Kubectl will be modified to display workflows. More particularly the `describe` command -will display all the steps with their status. Steps will be topologically sorted and -each dependency will be decorated with its status (wether or not step is waitin for -dependency). - -## Future evolution - -In the future we may want to extend _Workflow_ with other kinds of resources, modifying `WorkflowStep` to -support a more general template to create other resources. -One of the major functionalities missing here is the ability to set a recurring `Workflow` (cron-like), -similar to the `ScheduledJob` [#11980](https://github.com/kubernetes/kubernetes/pull/11980) for `Job`. -If the scheduled job is able to support -[various resources](https://github.com/kubernetes/kubernetes/pull/11980#discussion_r46729699) -`Workflow` will benefit from the _schedule_ functionality of `Job`. - -### Relevant use cases out of scope of this proposal - -* As an admin I want to set quota on workflow resources -[#13567](https://github.com/kubernetes/kubernetes/issues/13567). -* As an admin I want to re-assign a workflow resource to another namespace/user2. -* As a user I want to set an action when a workflow ends/start -[#3585](https://github.com/kubernetes/kubernetes/issues/3585) - - -1Something about naming: literature is full of different names, a commonly used -name is _task_, but since we plan to compose `Workflow`s (i.e. a task can execute -another whole `Workflow`) we have chosen the more generic word `Step`. - - -2A very common feature in industrial strength workflow tools. - - - -[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/proposals/workflow.md?pixel)]() -