You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
portainer/api/stacks/deploy.go

163 lines
5.1 KiB

package stacks
import (
feat(k8s): support git automated sync for k8s applications [EE-577] (#5548) * feat(stack): backport changes to CE EE-1189 * feat(stack): front end backport changes to CE EE-1199 (#5455) * feat(stack): front end backport changes to CE EE-1199 * fix k8s deploy logic * fixed web editor confirmation message typo. EE-1501 * fix(stack): fixed issue auth detail not remembered EE-1502 (#5459) * show status in buttons * removed onChangeRef function. * moved buttons in git form to its own component * removed unused variable. Co-authored-by: ArrisLee <arris_li@hotmail.com> * moved formvalue to kube app component * fix(stack): failed to pull and redeploy compose format k8s stack * fixed form value * fix(k8s): file content overridden when deployment failed with compose format EE-1548 * updated API response to get IsComposeFormat and show appropriate text. * feat(k8s): front end backport to CE * feat(kube): kube app auto update backend (#5547) * error message updates for different file type * not display creation source for external application * added confirmation modal to advanced app created by web editor * stop showing confirmation modal when updating application * disable rollback button when application type is not applicatiom form * only update file after deployment succeded * Revert "only update file after deployment succeded" This reverts commit b94bd2e96f27ca337e9d2fa698da2ab264d9ac86. * fix(k8s): file content overridden when deployment failed with compose format EE-1556 * added analytics-on directive to pull and redeploy button * fix(kube): don't valide resource control access for kube (#5568) * added missing question mark to k8s confirmation modal * fixed webhook format issue * added question marks to k8s app confirmation modal * added space in additional file list. * ignoring error on deletion * fix(k8s): Git authentication info not persisted * added RepositoryMechanismTypes constant * updated analytics functions * covert RepositoryMechanism to constant * fixed typo * removed unused function. * post tech review updates * fixed save settings n redeploy button * refact kub deploy logic * Revert "refact kub deploy logic" This reverts commit cbfdd58eceed438b148c836ed09004dd270d0aa6. * feat(k8s): utilize user token for k8s auto update EE-1594 * feat(k8s): persist kub stack name EE-1630 * feat(k8s): support delete kub stack * fix(app): updated logic to delete stack for different kind apps. (#5648) * fix(app): updated logic to delete stack for different kind apps. * renamed variable * fix import * added StackName field. * fixed stack id not found issue. * fix(k8s): fixed qusetion mark alignment issue in PAT field. (#5611) * fix(k8s): fixed qusetion mark alignment issue in PAT field. * moved inline css to file. * fix(git-form: made auth input text full width * add ignore deleted arg * tech review updates * typo fix * fix(k8s): added console error when deleting k8s service. * fix(console): added no-console config * fix(deploy): added missing service. * fix: use stack editor as an owner when exists (#5678) * fix: tempalte/content based stacks edit/delete * fix(stack): remove stack when no app. (#5769) * fix(stack): remove stack when no app. * support compose format in delete Co-authored-by: ArrisLee <arris_li@hotmail.com> Co-authored-by: Hui <arris_li@hotmail.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com> Co-authored-by: Felix Han <felix.han@portainer.io>
3 years ago
"fmt"
"strings"
"time"
"github.com/pkg/errors"
portainer "github.com/portainer/portainer/api"
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173) * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix app templates symbol (#6221) * feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220) * feat(webhook) EE-2125 add some helpers to registry utils * feat(webhook) EE-2125 persist registryID when creating a webhook * feat(webhook) EE-2125 send registry auth header when executing a webhook * feat(webhook) EE-2125 send registryID to backend when creating a service with webhook * feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen * feat(webhook) EE-2125 update webhook when update registry * feat(webhook) EE-2125 add endpoint of update webhook * feat(webhook) EE-2125 code cleanup * feat(webhook) EE-2125 fix a typo * feat(webhook) EE-2125 fix circle import issue with unit test Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204) Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix data-cy for k8s cluster menu (#6226) LGTM * feat(stack): make stack created from app template editable EE-1941 (#6104) feat(stack): make stack from app template editable * fix(container):disable Duplicate/Edit button when the container is portainer (#6223) * fix/ee-1909/show-pull-image-error (#6195) Co-authored-by: sunportainer <ericsun@SG1.local> * feat(cy): add data-cy to helm install button (#6241) * feat(cy): add data-cy to add registry button (#6242) * refactor(app): convert root folder files to es6 (#4159) * refactor(app): duplicate constants as es6 exports (#4158) * fix(docker): provide workaround to save network name variable (#6080) * fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file * fix/EE-1862/unable-to-stop-or-remove-stack check yaml file * fixed func and var names * wrapper error and used bool for stringset * UT case for createNetworkEnvFile * UT case for %s=%s * powerful StringSet * wrapper error for extract network name * wrapper all the return err * store more env * put to env file * make default value None * feat: gzip static resources (#6258) * fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203) * fix/ee-2106/handle-sslcert-sslkey Co-authored-by: sunportainer <ericsun@SG1.local> * fix(server):support disable https only ee-2068 (#6232) * fix/ee-2068/disable-forcely-https * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * Update missing helm user repo methods * remove redundant comments * add webhook export * update webhooks * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Update missing helm user repo methods * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * remove redundant comments * add webhook export * update webhooks * fix build issues after rebasing * move migratorparams * remove unneeded integer type conversions * disable the db import/export for now Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com> Co-authored-by: cong meng <mcpacino@gmail.com> Co-authored-by: Simon Meng <simon.meng@portainer.io> Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com> Co-authored-by: Hao Zhang <hao.zhang@portainer.io> Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com> Co-authored-by: sunportainer <ericsun@SG1.local> Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com> Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com> Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com> Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: Matt Hook <hookenz@gmail.com>
3 years ago
"github.com/portainer/portainer/api/dataservices"
"github.com/portainer/portainer/api/http/security"
feat(k8s): support git automated sync for k8s applications [EE-577] (#5548) * feat(stack): backport changes to CE EE-1189 * feat(stack): front end backport changes to CE EE-1199 (#5455) * feat(stack): front end backport changes to CE EE-1199 * fix k8s deploy logic * fixed web editor confirmation message typo. EE-1501 * fix(stack): fixed issue auth detail not remembered EE-1502 (#5459) * show status in buttons * removed onChangeRef function. * moved buttons in git form to its own component * removed unused variable. Co-authored-by: ArrisLee <arris_li@hotmail.com> * moved formvalue to kube app component * fix(stack): failed to pull and redeploy compose format k8s stack * fixed form value * fix(k8s): file content overridden when deployment failed with compose format EE-1548 * updated API response to get IsComposeFormat and show appropriate text. * feat(k8s): front end backport to CE * feat(kube): kube app auto update backend (#5547) * error message updates for different file type * not display creation source for external application * added confirmation modal to advanced app created by web editor * stop showing confirmation modal when updating application * disable rollback button when application type is not applicatiom form * only update file after deployment succeded * Revert "only update file after deployment succeded" This reverts commit b94bd2e96f27ca337e9d2fa698da2ab264d9ac86. * fix(k8s): file content overridden when deployment failed with compose format EE-1556 * added analytics-on directive to pull and redeploy button * fix(kube): don't valide resource control access for kube (#5568) * added missing question mark to k8s confirmation modal * fixed webhook format issue * added question marks to k8s app confirmation modal * added space in additional file list. * ignoring error on deletion * fix(k8s): Git authentication info not persisted * added RepositoryMechanismTypes constant * updated analytics functions * covert RepositoryMechanism to constant * fixed typo * removed unused function. * post tech review updates * fixed save settings n redeploy button * refact kub deploy logic * Revert "refact kub deploy logic" This reverts commit cbfdd58eceed438b148c836ed09004dd270d0aa6. * feat(k8s): utilize user token for k8s auto update EE-1594 * feat(k8s): persist kub stack name EE-1630 * feat(k8s): support delete kub stack * fix(app): updated logic to delete stack for different kind apps. (#5648) * fix(app): updated logic to delete stack for different kind apps. * renamed variable * fix import * added StackName field. * fixed stack id not found issue. * fix(k8s): fixed qusetion mark alignment issue in PAT field. (#5611) * fix(k8s): fixed qusetion mark alignment issue in PAT field. * moved inline css to file. * fix(git-form: made auth input text full width * add ignore deleted arg * tech review updates * typo fix * fix(k8s): added console error when deleting k8s service. * fix(console): added no-console config * fix(deploy): added missing service. * fix: use stack editor as an owner when exists (#5678) * fix: tempalte/content based stacks edit/delete * fix(stack): remove stack when no app. (#5769) * fix(stack): remove stack when no app. * support compose format in delete Co-authored-by: ArrisLee <arris_li@hotmail.com> Co-authored-by: Hui <arris_li@hotmail.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com> Co-authored-by: Felix Han <felix.han@portainer.io>
3 years ago
log "github.com/sirupsen/logrus"
)
feat(k8s): support git automated sync for k8s applications [EE-577] (#5548) * feat(stack): backport changes to CE EE-1189 * feat(stack): front end backport changes to CE EE-1199 (#5455) * feat(stack): front end backport changes to CE EE-1199 * fix k8s deploy logic * fixed web editor confirmation message typo. EE-1501 * fix(stack): fixed issue auth detail not remembered EE-1502 (#5459) * show status in buttons * removed onChangeRef function. * moved buttons in git form to its own component * removed unused variable. Co-authored-by: ArrisLee <arris_li@hotmail.com> * moved formvalue to kube app component * fix(stack): failed to pull and redeploy compose format k8s stack * fixed form value * fix(k8s): file content overridden when deployment failed with compose format EE-1548 * updated API response to get IsComposeFormat and show appropriate text. * feat(k8s): front end backport to CE * feat(kube): kube app auto update backend (#5547) * error message updates for different file type * not display creation source for external application * added confirmation modal to advanced app created by web editor * stop showing confirmation modal when updating application * disable rollback button when application type is not applicatiom form * only update file after deployment succeded * Revert "only update file after deployment succeded" This reverts commit b94bd2e96f27ca337e9d2fa698da2ab264d9ac86. * fix(k8s): file content overridden when deployment failed with compose format EE-1556 * added analytics-on directive to pull and redeploy button * fix(kube): don't valide resource control access for kube (#5568) * added missing question mark to k8s confirmation modal * fixed webhook format issue * added question marks to k8s app confirmation modal * added space in additional file list. * ignoring error on deletion * fix(k8s): Git authentication info not persisted * added RepositoryMechanismTypes constant * updated analytics functions * covert RepositoryMechanism to constant * fixed typo * removed unused function. * post tech review updates * fixed save settings n redeploy button * refact kub deploy logic * Revert "refact kub deploy logic" This reverts commit cbfdd58eceed438b148c836ed09004dd270d0aa6. * feat(k8s): utilize user token for k8s auto update EE-1594 * feat(k8s): persist kub stack name EE-1630 * feat(k8s): support delete kub stack * fix(app): updated logic to delete stack for different kind apps. (#5648) * fix(app): updated logic to delete stack for different kind apps. * renamed variable * fix import * added StackName field. * fixed stack id not found issue. * fix(k8s): fixed qusetion mark alignment issue in PAT field. (#5611) * fix(k8s): fixed qusetion mark alignment issue in PAT field. * moved inline css to file. * fix(git-form: made auth input text full width * add ignore deleted arg * tech review updates * typo fix * fix(k8s): added console error when deleting k8s service. * fix(console): added no-console config * fix(deploy): added missing service. * fix: use stack editor as an owner when exists (#5678) * fix: tempalte/content based stacks edit/delete * fix(stack): remove stack when no app. (#5769) * fix(stack): remove stack when no app. * support compose format in delete Co-authored-by: ArrisLee <arris_li@hotmail.com> Co-authored-by: Hui <arris_li@hotmail.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com> Co-authored-by: Felix Han <felix.han@portainer.io>
3 years ago
type StackAuthorMissingErr struct {
stackID int
authorName string
}
func (e *StackAuthorMissingErr) Error() string {
return fmt.Sprintf("stack's %v author %s is missing", e.stackID, e.authorName)
}
// RedeployWhenChanged pull and redeploy the stack when git repo changed
// Stack will always be redeployed if force deployment is set to true
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173) * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix app templates symbol (#6221) * feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220) * feat(webhook) EE-2125 add some helpers to registry utils * feat(webhook) EE-2125 persist registryID when creating a webhook * feat(webhook) EE-2125 send registry auth header when executing a webhook * feat(webhook) EE-2125 send registryID to backend when creating a service with webhook * feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen * feat(webhook) EE-2125 update webhook when update registry * feat(webhook) EE-2125 add endpoint of update webhook * feat(webhook) EE-2125 code cleanup * feat(webhook) EE-2125 fix a typo * feat(webhook) EE-2125 fix circle import issue with unit test Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204) Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix data-cy for k8s cluster menu (#6226) LGTM * feat(stack): make stack created from app template editable EE-1941 (#6104) feat(stack): make stack from app template editable * fix(container):disable Duplicate/Edit button when the container is portainer (#6223) * fix/ee-1909/show-pull-image-error (#6195) Co-authored-by: sunportainer <ericsun@SG1.local> * feat(cy): add data-cy to helm install button (#6241) * feat(cy): add data-cy to add registry button (#6242) * refactor(app): convert root folder files to es6 (#4159) * refactor(app): duplicate constants as es6 exports (#4158) * fix(docker): provide workaround to save network name variable (#6080) * fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file * fix/EE-1862/unable-to-stop-or-remove-stack check yaml file * fixed func and var names * wrapper error and used bool for stringset * UT case for createNetworkEnvFile * UT case for %s=%s * powerful StringSet * wrapper error for extract network name * wrapper all the return err * store more env * put to env file * make default value None * feat: gzip static resources (#6258) * fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203) * fix/ee-2106/handle-sslcert-sslkey Co-authored-by: sunportainer <ericsun@SG1.local> * fix(server):support disable https only ee-2068 (#6232) * fix/ee-2068/disable-forcely-https * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * Update missing helm user repo methods * remove redundant comments * add webhook export * update webhooks * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Update missing helm user repo methods * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * remove redundant comments * add webhook export * update webhooks * fix build issues after rebasing * move migratorparams * remove unneeded integer type conversions * disable the db import/export for now Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com> Co-authored-by: cong meng <mcpacino@gmail.com> Co-authored-by: Simon Meng <simon.meng@portainer.io> Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com> Co-authored-by: Hao Zhang <hao.zhang@portainer.io> Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com> Co-authored-by: sunportainer <ericsun@SG1.local> Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com> Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com> Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com> Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: Matt Hook <hookenz@gmail.com>
3 years ago
func RedeployWhenChanged(stackID portainer.StackID, deployer StackDeployer, datastore dataservices.DataStore, gitService portainer.GitService) error {
feat(k8s): support git automated sync for k8s applications [EE-577] (#5548) * feat(stack): backport changes to CE EE-1189 * feat(stack): front end backport changes to CE EE-1199 (#5455) * feat(stack): front end backport changes to CE EE-1199 * fix k8s deploy logic * fixed web editor confirmation message typo. EE-1501 * fix(stack): fixed issue auth detail not remembered EE-1502 (#5459) * show status in buttons * removed onChangeRef function. * moved buttons in git form to its own component * removed unused variable. Co-authored-by: ArrisLee <arris_li@hotmail.com> * moved formvalue to kube app component * fix(stack): failed to pull and redeploy compose format k8s stack * fixed form value * fix(k8s): file content overridden when deployment failed with compose format EE-1548 * updated API response to get IsComposeFormat and show appropriate text. * feat(k8s): front end backport to CE * feat(kube): kube app auto update backend (#5547) * error message updates for different file type * not display creation source for external application * added confirmation modal to advanced app created by web editor * stop showing confirmation modal when updating application * disable rollback button when application type is not applicatiom form * only update file after deployment succeded * Revert "only update file after deployment succeded" This reverts commit b94bd2e96f27ca337e9d2fa698da2ab264d9ac86. * fix(k8s): file content overridden when deployment failed with compose format EE-1556 * added analytics-on directive to pull and redeploy button * fix(kube): don't valide resource control access for kube (#5568) * added missing question mark to k8s confirmation modal * fixed webhook format issue * added question marks to k8s app confirmation modal * added space in additional file list. * ignoring error on deletion * fix(k8s): Git authentication info not persisted * added RepositoryMechanismTypes constant * updated analytics functions * covert RepositoryMechanism to constant * fixed typo * removed unused function. * post tech review updates * fixed save settings n redeploy button * refact kub deploy logic * Revert "refact kub deploy logic" This reverts commit cbfdd58eceed438b148c836ed09004dd270d0aa6. * feat(k8s): utilize user token for k8s auto update EE-1594 * feat(k8s): persist kub stack name EE-1630 * feat(k8s): support delete kub stack * fix(app): updated logic to delete stack for different kind apps. (#5648) * fix(app): updated logic to delete stack for different kind apps. * renamed variable * fix import * added StackName field. * fixed stack id not found issue. * fix(k8s): fixed qusetion mark alignment issue in PAT field. (#5611) * fix(k8s): fixed qusetion mark alignment issue in PAT field. * moved inline css to file. * fix(git-form: made auth input text full width * add ignore deleted arg * tech review updates * typo fix * fix(k8s): added console error when deleting k8s service. * fix(console): added no-console config * fix(deploy): added missing service. * fix: use stack editor as an owner when exists (#5678) * fix: tempalte/content based stacks edit/delete * fix(stack): remove stack when no app. (#5769) * fix(stack): remove stack when no app. * support compose format in delete Co-authored-by: ArrisLee <arris_li@hotmail.com> Co-authored-by: Hui <arris_li@hotmail.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com> Co-authored-by: Felix Han <felix.han@portainer.io>
3 years ago
logger := log.WithFields(log.Fields{"stackID": stackID})
logger.Debug("redeploying stack")
stack, err := datastore.Stack().Stack(stackID)
if err != nil {
return errors.WithMessagef(err, "failed to get the stack %v", stackID)
}
if stack.GitConfig == nil {
return nil // do nothing if it isn't a git-based stack
}
feat(k8s): support git automated sync for k8s applications [EE-577] (#5548) * feat(stack): backport changes to CE EE-1189 * feat(stack): front end backport changes to CE EE-1199 (#5455) * feat(stack): front end backport changes to CE EE-1199 * fix k8s deploy logic * fixed web editor confirmation message typo. EE-1501 * fix(stack): fixed issue auth detail not remembered EE-1502 (#5459) * show status in buttons * removed onChangeRef function. * moved buttons in git form to its own component * removed unused variable. Co-authored-by: ArrisLee <arris_li@hotmail.com> * moved formvalue to kube app component * fix(stack): failed to pull and redeploy compose format k8s stack * fixed form value * fix(k8s): file content overridden when deployment failed with compose format EE-1548 * updated API response to get IsComposeFormat and show appropriate text. * feat(k8s): front end backport to CE * feat(kube): kube app auto update backend (#5547) * error message updates for different file type * not display creation source for external application * added confirmation modal to advanced app created by web editor * stop showing confirmation modal when updating application * disable rollback button when application type is not applicatiom form * only update file after deployment succeded * Revert "only update file after deployment succeded" This reverts commit b94bd2e96f27ca337e9d2fa698da2ab264d9ac86. * fix(k8s): file content overridden when deployment failed with compose format EE-1556 * added analytics-on directive to pull and redeploy button * fix(kube): don't valide resource control access for kube (#5568) * added missing question mark to k8s confirmation modal * fixed webhook format issue * added question marks to k8s app confirmation modal * added space in additional file list. * ignoring error on deletion * fix(k8s): Git authentication info not persisted * added RepositoryMechanismTypes constant * updated analytics functions * covert RepositoryMechanism to constant * fixed typo * removed unused function. * post tech review updates * fixed save settings n redeploy button * refact kub deploy logic * Revert "refact kub deploy logic" This reverts commit cbfdd58eceed438b148c836ed09004dd270d0aa6. * feat(k8s): utilize user token for k8s auto update EE-1594 * feat(k8s): persist kub stack name EE-1630 * feat(k8s): support delete kub stack * fix(app): updated logic to delete stack for different kind apps. (#5648) * fix(app): updated logic to delete stack for different kind apps. * renamed variable * fix import * added StackName field. * fixed stack id not found issue. * fix(k8s): fixed qusetion mark alignment issue in PAT field. (#5611) * fix(k8s): fixed qusetion mark alignment issue in PAT field. * moved inline css to file. * fix(git-form: made auth input text full width * add ignore deleted arg * tech review updates * typo fix * fix(k8s): added console error when deleting k8s service. * fix(console): added no-console config * fix(deploy): added missing service. * fix: use stack editor as an owner when exists (#5678) * fix: tempalte/content based stacks edit/delete * fix(stack): remove stack when no app. (#5769) * fix(stack): remove stack when no app. * support compose format in delete Co-authored-by: ArrisLee <arris_li@hotmail.com> Co-authored-by: Hui <arris_li@hotmail.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com> Co-authored-by: Felix Han <felix.han@portainer.io>
3 years ago
author := stack.UpdatedBy
if author == "" {
author = stack.CreatedBy
}
user, err := datastore.User().UserByUsername(author)
if err != nil {
logger.WithFields(log.Fields{"author": author, "stack": stack.Name, "endpointID": stack.EndpointID}).Warn("cannot autoupdate a stack, stack author user is missing")
return &StackAuthorMissingErr{int(stack.ID), author}
}
username, password := "", ""
if stack.GitConfig.Authentication != nil {
username, password = stack.GitConfig.Authentication.Username, stack.GitConfig.Authentication.Password
}
newHash, err := gitService.LatestCommitID(stack.GitConfig.URL, stack.GitConfig.ReferenceName, username, password)
if err != nil {
return errors.WithMessagef(err, "failed to fetch latest commit id of the stack %v", stack.ID)
}
if strings.EqualFold(newHash, string(stack.GitConfig.ConfigHash)) {
return nil
}
cloneParams := &cloneRepositoryParameters{
url: stack.GitConfig.URL,
ref: stack.GitConfig.ReferenceName,
toDir: stack.ProjectPath,
}
if stack.GitConfig.Authentication != nil {
cloneParams.auth = &gitAuth{
username: username,
password: password,
}
}
if err := cloneGitRepository(gitService, cloneParams); err != nil {
return errors.WithMessagef(err, "failed to do a fresh clone of the stack %v", stack.ID)
}
endpoint, err := datastore.Endpoint().Endpoint(stack.EndpointID)
if err != nil {
return errors.WithMessagef(err, "failed to find the environment %v associated to the stack %v", stack.EndpointID, stack.ID)
}
feat(k8s): support git automated sync for k8s applications [EE-577] (#5548) * feat(stack): backport changes to CE EE-1189 * feat(stack): front end backport changes to CE EE-1199 (#5455) * feat(stack): front end backport changes to CE EE-1199 * fix k8s deploy logic * fixed web editor confirmation message typo. EE-1501 * fix(stack): fixed issue auth detail not remembered EE-1502 (#5459) * show status in buttons * removed onChangeRef function. * moved buttons in git form to its own component * removed unused variable. Co-authored-by: ArrisLee <arris_li@hotmail.com> * moved formvalue to kube app component * fix(stack): failed to pull and redeploy compose format k8s stack * fixed form value * fix(k8s): file content overridden when deployment failed with compose format EE-1548 * updated API response to get IsComposeFormat and show appropriate text. * feat(k8s): front end backport to CE * feat(kube): kube app auto update backend (#5547) * error message updates for different file type * not display creation source for external application * added confirmation modal to advanced app created by web editor * stop showing confirmation modal when updating application * disable rollback button when application type is not applicatiom form * only update file after deployment succeded * Revert "only update file after deployment succeded" This reverts commit b94bd2e96f27ca337e9d2fa698da2ab264d9ac86. * fix(k8s): file content overridden when deployment failed with compose format EE-1556 * added analytics-on directive to pull and redeploy button * fix(kube): don't valide resource control access for kube (#5568) * added missing question mark to k8s confirmation modal * fixed webhook format issue * added question marks to k8s app confirmation modal * added space in additional file list. * ignoring error on deletion * fix(k8s): Git authentication info not persisted * added RepositoryMechanismTypes constant * updated analytics functions * covert RepositoryMechanism to constant * fixed typo * removed unused function. * post tech review updates * fixed save settings n redeploy button * refact kub deploy logic * Revert "refact kub deploy logic" This reverts commit cbfdd58eceed438b148c836ed09004dd270d0aa6. * feat(k8s): utilize user token for k8s auto update EE-1594 * feat(k8s): persist kub stack name EE-1630 * feat(k8s): support delete kub stack * fix(app): updated logic to delete stack for different kind apps. (#5648) * fix(app): updated logic to delete stack for different kind apps. * renamed variable * fix import * added StackName field. * fixed stack id not found issue. * fix(k8s): fixed qusetion mark alignment issue in PAT field. (#5611) * fix(k8s): fixed qusetion mark alignment issue in PAT field. * moved inline css to file. * fix(git-form: made auth input text full width * add ignore deleted arg * tech review updates * typo fix * fix(k8s): added console error when deleting k8s service. * fix(console): added no-console config * fix(deploy): added missing service. * fix: use stack editor as an owner when exists (#5678) * fix: tempalte/content based stacks edit/delete * fix(stack): remove stack when no app. (#5769) * fix(stack): remove stack when no app. * support compose format in delete Co-authored-by: ArrisLee <arris_li@hotmail.com> Co-authored-by: Hui <arris_li@hotmail.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com> Co-authored-by: Felix Han <felix.han@portainer.io>
3 years ago
registries, err := getUserRegistries(datastore, user, endpoint.ID)
if err != nil {
return err
}
switch stack.Type {
case portainer.DockerComposeStack:
err := deployer.DeployComposeStack(stack, endpoint, registries, false)
if err != nil {
return errors.WithMessagef(err, "failed to deploy a docker compose stack %v", stackID)
}
case portainer.DockerSwarmStack:
err := deployer.DeploySwarmStack(stack, endpoint, registries, true)
if err != nil {
return errors.WithMessagef(err, "failed to deploy a docker compose stack %v", stackID)
}
feat(k8s): support git automated sync for k8s applications [EE-577] (#5548) * feat(stack): backport changes to CE EE-1189 * feat(stack): front end backport changes to CE EE-1199 (#5455) * feat(stack): front end backport changes to CE EE-1199 * fix k8s deploy logic * fixed web editor confirmation message typo. EE-1501 * fix(stack): fixed issue auth detail not remembered EE-1502 (#5459) * show status in buttons * removed onChangeRef function. * moved buttons in git form to its own component * removed unused variable. Co-authored-by: ArrisLee <arris_li@hotmail.com> * moved formvalue to kube app component * fix(stack): failed to pull and redeploy compose format k8s stack * fixed form value * fix(k8s): file content overridden when deployment failed with compose format EE-1548 * updated API response to get IsComposeFormat and show appropriate text. * feat(k8s): front end backport to CE * feat(kube): kube app auto update backend (#5547) * error message updates for different file type * not display creation source for external application * added confirmation modal to advanced app created by web editor * stop showing confirmation modal when updating application * disable rollback button when application type is not applicatiom form * only update file after deployment succeded * Revert "only update file after deployment succeded" This reverts commit b94bd2e96f27ca337e9d2fa698da2ab264d9ac86. * fix(k8s): file content overridden when deployment failed with compose format EE-1556 * added analytics-on directive to pull and redeploy button * fix(kube): don't valide resource control access for kube (#5568) * added missing question mark to k8s confirmation modal * fixed webhook format issue * added question marks to k8s app confirmation modal * added space in additional file list. * ignoring error on deletion * fix(k8s): Git authentication info not persisted * added RepositoryMechanismTypes constant * updated analytics functions * covert RepositoryMechanism to constant * fixed typo * removed unused function. * post tech review updates * fixed save settings n redeploy button * refact kub deploy logic * Revert "refact kub deploy logic" This reverts commit cbfdd58eceed438b148c836ed09004dd270d0aa6. * feat(k8s): utilize user token for k8s auto update EE-1594 * feat(k8s): persist kub stack name EE-1630 * feat(k8s): support delete kub stack * fix(app): updated logic to delete stack for different kind apps. (#5648) * fix(app): updated logic to delete stack for different kind apps. * renamed variable * fix import * added StackName field. * fixed stack id not found issue. * fix(k8s): fixed qusetion mark alignment issue in PAT field. (#5611) * fix(k8s): fixed qusetion mark alignment issue in PAT field. * moved inline css to file. * fix(git-form: made auth input text full width * add ignore deleted arg * tech review updates * typo fix * fix(k8s): added console error when deleting k8s service. * fix(console): added no-console config * fix(deploy): added missing service. * fix: use stack editor as an owner when exists (#5678) * fix: tempalte/content based stacks edit/delete * fix(stack): remove stack when no app. (#5769) * fix(stack): remove stack when no app. * support compose format in delete Co-authored-by: ArrisLee <arris_li@hotmail.com> Co-authored-by: Hui <arris_li@hotmail.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com> Co-authored-by: Felix Han <felix.han@portainer.io>
3 years ago
case portainer.KubernetesStack:
logger.Debugf("deploying a kube app")
err := deployer.DeployKubernetesStack(stack, endpoint, user)
if err != nil {
return errors.WithMessagef(err, "failed to deploy a kubternetes app stack %v", stackID)
}
default:
return errors.Errorf("cannot update stack, type %v is unsupported", stack.Type)
}
stack.UpdateDate = time.Now().Unix()
stack.GitConfig.ConfigHash = newHash
if err := datastore.Stack().UpdateStack(stack.ID, stack); err != nil {
return errors.WithMessagef(err, "failed to update the stack %v", stack.ID)
}
return nil
}
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173) * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix app templates symbol (#6221) * feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220) * feat(webhook) EE-2125 add some helpers to registry utils * feat(webhook) EE-2125 persist registryID when creating a webhook * feat(webhook) EE-2125 send registry auth header when executing a webhook * feat(webhook) EE-2125 send registryID to backend when creating a service with webhook * feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen * feat(webhook) EE-2125 update webhook when update registry * feat(webhook) EE-2125 add endpoint of update webhook * feat(webhook) EE-2125 code cleanup * feat(webhook) EE-2125 fix a typo * feat(webhook) EE-2125 fix circle import issue with unit test Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204) Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix data-cy for k8s cluster menu (#6226) LGTM * feat(stack): make stack created from app template editable EE-1941 (#6104) feat(stack): make stack from app template editable * fix(container):disable Duplicate/Edit button when the container is portainer (#6223) * fix/ee-1909/show-pull-image-error (#6195) Co-authored-by: sunportainer <ericsun@SG1.local> * feat(cy): add data-cy to helm install button (#6241) * feat(cy): add data-cy to add registry button (#6242) * refactor(app): convert root folder files to es6 (#4159) * refactor(app): duplicate constants as es6 exports (#4158) * fix(docker): provide workaround to save network name variable (#6080) * fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file * fix/EE-1862/unable-to-stop-or-remove-stack check yaml file * fixed func and var names * wrapper error and used bool for stringset * UT case for createNetworkEnvFile * UT case for %s=%s * powerful StringSet * wrapper error for extract network name * wrapper all the return err * store more env * put to env file * make default value None * feat: gzip static resources (#6258) * fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203) * fix/ee-2106/handle-sslcert-sslkey Co-authored-by: sunportainer <ericsun@SG1.local> * fix(server):support disable https only ee-2068 (#6232) * fix/ee-2068/disable-forcely-https * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * Update missing helm user repo methods * remove redundant comments * add webhook export * update webhooks * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Update missing helm user repo methods * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * remove redundant comments * add webhook export * update webhooks * fix build issues after rebasing * move migratorparams * remove unneeded integer type conversions * disable the db import/export for now Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com> Co-authored-by: cong meng <mcpacino@gmail.com> Co-authored-by: Simon Meng <simon.meng@portainer.io> Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com> Co-authored-by: Hao Zhang <hao.zhang@portainer.io> Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com> Co-authored-by: sunportainer <ericsun@SG1.local> Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com> Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com> Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com> Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: Matt Hook <hookenz@gmail.com>
3 years ago
func getUserRegistries(datastore dataservices.DataStore, user *portainer.User, endpointID portainer.EndpointID) ([]portainer.Registry, error) {
registries, err := datastore.Registry().Registries()
if err != nil {
return nil, errors.WithMessage(err, "unable to retrieve registries from the database")
}
if user.Role == portainer.AdministratorRole {
return registries, nil
}
userMemberships, err := datastore.TeamMembership().TeamMembershipsByUserID(user.ID)
if err != nil {
feat(k8s): support git automated sync for k8s applications [EE-577] (#5548) * feat(stack): backport changes to CE EE-1189 * feat(stack): front end backport changes to CE EE-1199 (#5455) * feat(stack): front end backport changes to CE EE-1199 * fix k8s deploy logic * fixed web editor confirmation message typo. EE-1501 * fix(stack): fixed issue auth detail not remembered EE-1502 (#5459) * show status in buttons * removed onChangeRef function. * moved buttons in git form to its own component * removed unused variable. Co-authored-by: ArrisLee <arris_li@hotmail.com> * moved formvalue to kube app component * fix(stack): failed to pull and redeploy compose format k8s stack * fixed form value * fix(k8s): file content overridden when deployment failed with compose format EE-1548 * updated API response to get IsComposeFormat and show appropriate text. * feat(k8s): front end backport to CE * feat(kube): kube app auto update backend (#5547) * error message updates for different file type * not display creation source for external application * added confirmation modal to advanced app created by web editor * stop showing confirmation modal when updating application * disable rollback button when application type is not applicatiom form * only update file after deployment succeded * Revert "only update file after deployment succeded" This reverts commit b94bd2e96f27ca337e9d2fa698da2ab264d9ac86. * fix(k8s): file content overridden when deployment failed with compose format EE-1556 * added analytics-on directive to pull and redeploy button * fix(kube): don't valide resource control access for kube (#5568) * added missing question mark to k8s confirmation modal * fixed webhook format issue * added question marks to k8s app confirmation modal * added space in additional file list. * ignoring error on deletion * fix(k8s): Git authentication info not persisted * added RepositoryMechanismTypes constant * updated analytics functions * covert RepositoryMechanism to constant * fixed typo * removed unused function. * post tech review updates * fixed save settings n redeploy button * refact kub deploy logic * Revert "refact kub deploy logic" This reverts commit cbfdd58eceed438b148c836ed09004dd270d0aa6. * feat(k8s): utilize user token for k8s auto update EE-1594 * feat(k8s): persist kub stack name EE-1630 * feat(k8s): support delete kub stack * fix(app): updated logic to delete stack for different kind apps. (#5648) * fix(app): updated logic to delete stack for different kind apps. * renamed variable * fix import * added StackName field. * fixed stack id not found issue. * fix(k8s): fixed qusetion mark alignment issue in PAT field. (#5611) * fix(k8s): fixed qusetion mark alignment issue in PAT field. * moved inline css to file. * fix(git-form: made auth input text full width * add ignore deleted arg * tech review updates * typo fix * fix(k8s): added console error when deleting k8s service. * fix(console): added no-console config * fix(deploy): added missing service. * fix: use stack editor as an owner when exists (#5678) * fix: tempalte/content based stacks edit/delete * fix(stack): remove stack when no app. (#5769) * fix(stack): remove stack when no app. * support compose format in delete Co-authored-by: ArrisLee <arris_li@hotmail.com> Co-authored-by: Hui <arris_li@hotmail.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com> Co-authored-by: Felix Han <felix.han@portainer.io>
3 years ago
return nil, errors.WithMessagef(err, "failed to fetch memberships of the stack author [%s]", user.Username)
}
filteredRegistries := make([]portainer.Registry, 0, len(registries))
for _, registry := range registries {
if security.AuthorizedRegistryAccess(&registry, user, userMemberships, endpointID) {
filteredRegistries = append(filteredRegistries, registry)
}
}
return filteredRegistries, nil
}
type cloneRepositoryParameters struct {
url string
ref string
toDir string
auth *gitAuth
}
type gitAuth struct {
username string
password string
}
func cloneGitRepository(gitService portainer.GitService, cloneParams *cloneRepositoryParameters) error {
if cloneParams.auth != nil {
return gitService.CloneRepository(cloneParams.toDir, cloneParams.url, cloneParams.ref, cloneParams.auth.username, cloneParams.auth.password)
}
return gitService.CloneRepository(cloneParams.toDir, cloneParams.url, cloneParams.ref, "", "")
}