Commit Graph

4 Commits (f4ac6f832053c0c555345d33cd2b7b4e50360e0a)

Author SHA1 Message Date
Sven Dowideit f99329eb7e
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173)
* use the Store interface IsErrObjectNotFound() to avoid revealing internal errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* what happens when you extract the datastore interfaces into their own package

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Start renaming Storage methods

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the boltdb specific code from the Portainer storage code (example, the others need the same)

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extract bolt.Tx from datastore code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* minimise imports by putting moving the struct definition into the file that needs the Service imports

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extraction of boltdb.Tx

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the use of bucket.SetSequence

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* almost done - just endpoint.Synchonise :/

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring'

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix test compile errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* test compile fixes after rebase

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* set us up to make the connection an interface

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make the db connection a datastore interface, and separate out our datastore services from the bolt ones

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* rename methods to something less oltdb internals specific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* these errors are not boltdb secific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* start using the db-backend factory method too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* export boltdb raw in case we can't export from the service layer

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add the version info by hand for now

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* actually, the export from services can be fully typed - its the import that needs to do more work

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* redo raw export, and make import capable of using it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add DockerHub

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix go test ./... again

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* my goland wasn't setup to gofmt

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the two extremely dubious migration tests down into store, so they can use the test store code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* the migrator is now free of boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* reverse goland overzealous replcement of internal with boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more undo over-zealous goland internal->boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* yay, now bolt is only mentioned inside the api/database/ dir

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* and this might be the last of the boltdb references?

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add todo

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the store code into a separate module too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* don't need the fileService in boltdb anymore

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use IsErrObjectNotFound()

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use a string to select what database backend we use

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make isNew store an ephemeral bool that doesn't stay true after we've initialised it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* undo vscode formatting html

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix app templates symbol (#6221)

* feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220)

* feat(webhook) EE-2125 add some helpers to registry utils

* feat(webhook) EE-2125 persist registryID when creating a webhook

* feat(webhook) EE-2125 send registry auth header when executing a webhook

* feat(webhook) EE-2125 send registryID to backend when creating a service with webhook

* feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen

* feat(webhook) EE-2125 update webhook when update registry

* feat(webhook) EE-2125 add endpoint of update webhook

* feat(webhook) EE-2125 code cleanup

* feat(webhook) EE-2125 fix a typo

* feat(webhook) EE-2125 fix circle import issue with unit test

Co-authored-by: Simon Meng <simon.meng@portainer.io>

* fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204)

Co-authored-by: Simon Meng <simon.meng@portainer.io>

* fix data-cy for k8s cluster menu (#6226)

LGTM

* feat(stack): make stack created from app template editable EE-1941 (#6104)

feat(stack): make stack from app template editable

* fix(container):disable Duplicate/Edit button when the container is portainer (#6223)

* fix/ee-1909/show-pull-image-error (#6195)

Co-authored-by: sunportainer <ericsun@SG1.local>

* feat(cy): add data-cy to helm install button (#6241)

* feat(cy): add data-cy to add registry button (#6242)

* refactor(app): convert root folder files to es6 (#4159)

* refactor(app): duplicate constants as es6 exports (#4158)

* fix(docker): provide workaround to save network name variable  (#6080)

* fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file

* fix/EE-1862/unable-to-stop-or-remove-stack check yaml file

* fixed func and var names

* wrapper error and used bool for stringset

* UT case for createNetworkEnvFile

* UT case for %s=%s

* powerful StringSet

* wrapper error for extract network name

* wrapper all the return err

* store more env

* put to env file

* make default value None

* feat: gzip static resources (#6258)

* fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203)

* fix/ee-2106/handle-sslcert-sslkey

Co-authored-by: sunportainer <ericsun@SG1.local>

* fix(server):support disable https only ee-2068 (#6232)

* fix/ee-2068/disable-forcely-https

* feat(store): implement store tests EE-2112 (#6224)

* add store tests

* add some more tests

* Update missing helm user repo methods

* remove redundant comments

* add webhook export

* update webhooks

* use the Store interface IsErrObjectNotFound() to avoid revealing internal errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* what happens when you extract the datastore interfaces into their own package

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Start renaming Storage methods

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the boltdb specific code from the Portainer storage code (example, the others need the same)

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extract bolt.Tx from datastore code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* minimise imports by putting moving the struct definition into the file that needs the Service imports

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extraction of boltdb.Tx

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the use of bucket.SetSequence

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* almost done - just endpoint.Synchonise :/

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring'

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix test compile errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* test compile fixes after rebase

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* set us up to make the connection an interface

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make the db connection a datastore interface, and separate out our datastore services from the bolt ones

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* rename methods to something less oltdb internals specific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* these errors are not boltdb secific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* start using the db-backend factory method too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* export boltdb raw in case we can't export from the service layer

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add the version info by hand for now

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* actually, the export from services can be fully typed - its the import that needs to do more work

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* redo raw export, and make import capable of using it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add DockerHub

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix go test ./... again

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* my goland wasn't setup to gofmt

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the two extremely dubious migration tests down into store, so they can use the test store code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* the migrator is now free of boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* reverse goland overzealous replcement of internal with boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more undo over-zealous goland internal->boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* yay, now bolt is only mentioned inside the api/database/ dir

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* and this might be the last of the boltdb references?

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add todo

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the store code into a separate module too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* don't need the fileService in boltdb anymore

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use IsErrObjectNotFound()

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use a string to select what database backend we use

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make isNew store an ephemeral bool that doesn't stay true after we've initialised it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* undo vscode formatting html

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Update missing helm user repo methods

* feat(store): implement store tests EE-2112 (#6224)

* add store tests

* add some more tests

* remove redundant comments

* add webhook export

* update webhooks

* fix build issues after rebasing

* move migratorparams

* remove unneeded integer type conversions

* disable the db import/export for now

Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: cong meng <mcpacino@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com>
Co-authored-by: Hao Zhang <hao.zhang@portainer.io>
Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com>
Co-authored-by: sunportainer <ericsun@SG1.local>
Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com>
Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com>
Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com>
Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>
Co-authored-by: Matt Hook <hookenz@gmail.com>
2021-12-15 15:26:09 +13:00
Marcelo Rydel 048613a0c5
feature(kubeconfig): Do not invalidate kubeconfig upon Portainer restarting [EE-1854] (#5905) 2021-11-15 18:45:20 -03:00
zees-dev 2a60b8fcdf
feat(helm/templates): helm app templates EE-943 (#5449)
* feat(helm): add helm chart backport to ce EE-1409 (#5425)

* EE-1311 Helm Chart Backport from EE

* backport to ce

Co-authored-by: Matt Hook <hookenz@gmail.com>

* feat(helm): list and configure helm chart (#5431)

* backport and tidyup code

* --amend

* using rocket icon for charts

* helm chart bugfix - clear category button

* added matomo analytics for helm chart install

* fix web editor exit warning without changes

* editor modified exit bugfix

* fixed notifications typo

* updated helm template text

* helper text to convey slow helm templates load

Co-authored-by: zees-dev <dev.786zshan@gmail.com>

* removing redundant time-consuming api call by using prop attribute

* feat(helm) helm chart backport from ee EE-1311 (#5436)

* Add missing defaultHelmRepoUrl and mock testing

* Backport EE-1477

* Backport updates to helm tests from EE

* add https by default changes and ssl to tls renaming from EE

* Port install integration test. Disabled by default to pass CI checks

* merged changes from EE for the integration test

* kube proxy whitelist updated to support internal helm install command

Co-authored-by: zees-dev <dev.786zshan@gmail.com>

* Pull in all changes from tech review in EE-943

* added helm to sidebar after rebase, sync CE with EE

* bugfix: kubectl shell not opening - bearer token bug

* tidy go modules & remove yarn-error.log

* removed redundant handler (not used) - to match EE

* resolved merge conflicts, updated code

* feat(helm/views): helm release and application views EE-1236  (#5529)

* feat(helm): add helm chart backport to ce EE-1409 (#5425)

* EE-1311 Helm Chart Backport from EE

* backport to ce

Co-authored-by: Matt Hook <hookenz@gmail.com>

* Pull in all changes from tech review in EE-943

* added helm to sidebar after rebase, sync CE with EE

* removed redundant handler (not used) - to match EE

* feat(helm) display helm charts - backend EE-1236

* copy over components for new applications view EE-1236

* Add new applications datatable component

* Add more migrated files

* removed test not applicable to CE

* baclkported EE app data table code to CE

* removed redundant helm repo url

* resolved conflicts, updated code

* using endpoint middleware

* PR review fixes

* using constants, openapi updated

Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: zees-dev <dev.786zshan@gmail.com>

* fixed test conflicts, go linted

* feat(helm/templates-add): helm templates add repo for user support EE-1278 (#5514)

* feat(helm): add helm chart backport to ce EE-1409 (#5425)

* EE-1311 Helm Chart Backport from EE

* backport to ce

Co-authored-by: Matt Hook <hookenz@gmail.com>

* feat(helm) helm chart backport from ee EE-1311 (#5436)

* Add missing defaultHelmRepoUrl and mock testing

* Backport EE-1477

* Backport updates to helm tests from EE

* add https by default changes and ssl to tls renaming from EE

* Port install integration test. Disabled by default to pass CI checks

* merged changes from EE for the integration test

* kube proxy whitelist updated to support internal helm install command

Co-authored-by: zees-dev <dev.786zshan@gmail.com>

* Pull in all changes from tech review in EE-943

* feat(helm): add helm chart backport to ce EE-1409 (#5425)

* EE-1311 Helm Chart Backport from EE

* backport to ce

Co-authored-by: Matt Hook <hookenz@gmail.com>

* Pull in all changes from tech review in EE-943

* added helm to sidebar after rebase, sync CE with EE

* backport EE-1278, squashed, diffed, updated

* helm install openapi spec update

* resolved conflicts, updated code

* - matching ee codebase at 0afe57034449ee0e9f333d92c252a13995a93019
- helm install using endpoint middleware
- remove trailing slash from added/persisted helm repo urls

* feat(helm) use libhelm url validator and improved path assembly EE-1554 (#5561)

* feat(helm/userrepos) fix getting global repo for ordinary users EE-1562 (#5567)

* feat(helm/userrepos) fix getting global repo for ordinary users EE-1562

* post review changes and further backported changes from EE

* resolved conflicts, updated code

* fixed helm_install handler unit test

* user cannot add existing repo if suffix is '/' (#5571)

* feat(helm/docs) fix broken swagger docs EE-1278 (#5572)

* Fix swagger docs

* minor correction

* fix(helm): migrating code from user handler to helm handler (#5573)

* - migrated user_helm_repos to helm endpoint handler
- migrated api operations from user factory/service to helm factory/service
- passing endpointId into helm service/factory as endpoint provider is deprecated

* upgrade libhelm to hide secrets

Co-authored-by: Matt Hook <hookenz@gmail.com>

* removed duplicate file - due to merge conflict

* dependency injection in helm factory

Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: Matt Hook <hookenz@gmail.com>

* kubernetes.templates -> kubernetes.templates.helm name conflict fix

* Validate the URL added as a public helm repo (#5579)

* fix(helm): helm app deletion fix EE-1581 (#5582)

* updated helm lib to show correct error on uninstall failure

* passing down helm app namespace on deletion

* fix(k8s): EE-1591 non-admin users cannot deploy charts containing secrets (#5590)

Co-authored-by: Simon Meng <simon.meng@portainer.io>

* fix(helm): helm epic bugfixes EE-1582 EE-1593 (#5585)

* - trim trailing slash and lowercase before persisting helm repo
- browser helm templates url /kubernetes/templates/templates -> /kubernetes/templates/helm
- fix publish url
- fix helm repo add refresh
- semi-fix k8s app expansion

* Tidy up swagger documentation related to helm. Make json consistent

* fixed helm release page for non-default namespaces

* k8s app view table expansion bugfix

* EE-1593: publish url load balancer fallback

Co-authored-by: Matt Hook <hookenz@gmail.com>

* k8s app list fix for charts with deployments containing multiple pods - which use the same label (#5599)

* fix(kubernetes): app list view fix for secrets with long keys or values EE-1600 (#5600)

* k8s app secrets key value text overflow ellipses

* wrapping key value pairs instead of ellipses

* fix(helm): helm apps bundling issue across different namespaces EE-1619 (#5602)

* helm apps bundling issue across different namespaces

* - code comments and indentation to ease reading
- moved namespace calc out of loop

* feat(helm/test) disable slow helm search test by default EE-1599 (#5598)

* skip helm_repo_search as it's an integration test

* switch to portainer built in integration test checker

* make module order match EE

* don't print test struct out when skipping integration test

Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: Matt Hook <hookenz@gmail.com>
Co-authored-by: cong meng <mcpacino@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
2021-09-10 14:06:57 +12:00
cong meng 35013e7b6a
feat(kubeconfig): Introduce the ability to change the expiry of a kubeconfig EE-1153 (#5421)
* feat(kubeconfig) EE-1153 Introduce the ability to change the expiry of a kubeconfig

* feat(kubeconfig) EE-1153 pr feedback update

* feat(kubeconfig) EE-1153 code cleanup

Co-authored-by: Simon Meng <simon.meng@portainer.io>
2021-09-01 09:23:21 +12:00