* feat(performance): add settings to tune the performance of the database EE-2363
* Change panics to log.Fatals.
Co-authored-by: andres-portainer <andres-portainer@users.noreply.github.com>
* bootstrap encryption key
* secret key message change in cli and secret key file content trimmed
* Migrate encryption code to latest version
* pull in newer code
* tidying up
* working data encryption layer
* fix tests
* remove stray comment
* fix a few minor issues and improve the comments
* split out databasefilename with param to two methods to be more obvious
* DB encryption integration (#6374)
* json methods moved under DBConnection
* store encryption fixed
* cleaned
* review comments addressed
* newstore value fixed
* backup test updated
* logrus format config updated
* Fix for newStore
Co-authored-by: Matt Hook <hookenz@gmail.com>
* Minor improvements
* Improve the export code. Add missing webhook for import
* rename HelmUserRepositorys to HelmUserRepositories
* fix logging messages
* when starting portainer with a key (first use) http is disabled by default. But when starting fresh without a key, http is enabled?
* Fix bug for default settings on new installs
Co-authored-by: Prabhat Khera <prabhat.khera@portainer.io>
Co-authored-by: Prabhat Khera <91852476+prabhat-org@users.noreply.github.com>
* use the Store interface IsErrObjectNotFound() to avoid revealing internal errors
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* what happens when you extract the datastore interfaces into their own package
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* Start renaming Storage methods
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* extract the boltdb specific code from the Portainer storage code (example, the others need the same)
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* more extract bolt.Tx from datastore code
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* minimise imports by putting moving the struct definition into the file that needs the Service imports
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* more extraction of boltdb.Tx
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* extract the use of bucket.SetSequence
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* almost done - just endpoint.Synchonise :/
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring'
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* fix test compile errors
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* test compile fixes after rebase
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* set us up to make the connection an interface
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* make the db connection a datastore interface, and separate out our datastore services from the bolt ones
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* rename methods to something less oltdb internals specific
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* these errors are not boltdb secific
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* start using the db-backend factory method too
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* export boltdb raw in case we can't export from the service layer
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* add the version info by hand for now
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* actually, the export from services can be fully typed - its the import that needs to do more work
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* redo raw export, and make import capable of using it
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* add DockerHub
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* fix go test ./... again
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* my goland wasn't setup to gofmt
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* move the two extremely dubious migration tests down into store, so they can use the test store code
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* the migrator is now free of boltdb
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* reverse goland overzealous replcement of internal with boltdb
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* more undo over-zealous goland internal->boltdb
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* yay, now bolt is only mentioned inside the api/database/ dir
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* and this might be the last of the boltdb references?
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* add todo
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* extract the store code into a separate module too
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* don't need the fileService in boltdb anymore
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* use IsErrObjectNotFound()
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* use a string to select what database backend we use
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* make isNew store an ephemeral bool that doesn't stay true after we've initialised it
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* undo vscode formatting html
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* fix app templates symbol (#6221)
* feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220)
* feat(webhook) EE-2125 add some helpers to registry utils
* feat(webhook) EE-2125 persist registryID when creating a webhook
* feat(webhook) EE-2125 send registry auth header when executing a webhook
* feat(webhook) EE-2125 send registryID to backend when creating a service with webhook
* feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen
* feat(webhook) EE-2125 update webhook when update registry
* feat(webhook) EE-2125 add endpoint of update webhook
* feat(webhook) EE-2125 code cleanup
* feat(webhook) EE-2125 fix a typo
* feat(webhook) EE-2125 fix circle import issue with unit test
Co-authored-by: Simon Meng <simon.meng@portainer.io>
* fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204)
Co-authored-by: Simon Meng <simon.meng@portainer.io>
* fix data-cy for k8s cluster menu (#6226)
LGTM
* feat(stack): make stack created from app template editable EE-1941 (#6104)
feat(stack): make stack from app template editable
* fix(container):disable Duplicate/Edit button when the container is portainer (#6223)
* fix/ee-1909/show-pull-image-error (#6195)
Co-authored-by: sunportainer <ericsun@SG1.local>
* feat(cy): add data-cy to helm install button (#6241)
* feat(cy): add data-cy to add registry button (#6242)
* refactor(app): convert root folder files to es6 (#4159)
* refactor(app): duplicate constants as es6 exports (#4158)
* fix(docker): provide workaround to save network name variable (#6080)
* fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file
* fix/EE-1862/unable-to-stop-or-remove-stack check yaml file
* fixed func and var names
* wrapper error and used bool for stringset
* UT case for createNetworkEnvFile
* UT case for %s=%s
* powerful StringSet
* wrapper error for extract network name
* wrapper all the return err
* store more env
* put to env file
* make default value None
* feat: gzip static resources (#6258)
* fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203)
* fix/ee-2106/handle-sslcert-sslkey
Co-authored-by: sunportainer <ericsun@SG1.local>
* fix(server):support disable https only ee-2068 (#6232)
* fix/ee-2068/disable-forcely-https
* feat(store): implement store tests EE-2112 (#6224)
* add store tests
* add some more tests
* Update missing helm user repo methods
* remove redundant comments
* add webhook export
* update webhooks
* use the Store interface IsErrObjectNotFound() to avoid revealing internal errors
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* what happens when you extract the datastore interfaces into their own package
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* Start renaming Storage methods
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* extract the boltdb specific code from the Portainer storage code (example, the others need the same)
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* more extract bolt.Tx from datastore code
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* minimise imports by putting moving the struct definition into the file that needs the Service imports
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* more extraction of boltdb.Tx
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* extract the use of bucket.SetSequence
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* almost done - just endpoint.Synchonise :/
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring'
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* fix test compile errors
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* test compile fixes after rebase
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* set us up to make the connection an interface
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* make the db connection a datastore interface, and separate out our datastore services from the bolt ones
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* rename methods to something less oltdb internals specific
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* these errors are not boltdb secific
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* start using the db-backend factory method too
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* export boltdb raw in case we can't export from the service layer
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* add the version info by hand for now
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* actually, the export from services can be fully typed - its the import that needs to do more work
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* redo raw export, and make import capable of using it
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* add DockerHub
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* fix go test ./... again
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* my goland wasn't setup to gofmt
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* move the two extremely dubious migration tests down into store, so they can use the test store code
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* the migrator is now free of boltdb
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* reverse goland overzealous replcement of internal with boltdb
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* more undo over-zealous goland internal->boltdb
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* yay, now bolt is only mentioned inside the api/database/ dir
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* and this might be the last of the boltdb references?
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* add todo
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* extract the store code into a separate module too
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* don't need the fileService in boltdb anymore
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* use IsErrObjectNotFound()
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* use a string to select what database backend we use
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* make isNew store an ephemeral bool that doesn't stay true after we've initialised it
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* undo vscode formatting html
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
* Update missing helm user repo methods
* feat(store): implement store tests EE-2112 (#6224)
* add store tests
* add some more tests
* remove redundant comments
* add webhook export
* update webhooks
* fix build issues after rebasing
* move migratorparams
* remove unneeded integer type conversions
* disable the db import/export for now
Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: cong meng <mcpacino@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com>
Co-authored-by: Hao Zhang <hao.zhang@portainer.io>
Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com>
Co-authored-by: sunportainer <ericsun@SG1.local>
Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com>
Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com>
Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com>
Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>
Co-authored-by: Matt Hook <hookenz@gmail.com>
* feat(webhook) EE-2125 add some helpers to registry utils
* feat(webhook) EE-2125 persist registryID when creating a webhook
* feat(webhook) EE-2125 send registry auth header when executing a webhook
* feat(webhook) EE-2125 send registryID to backend when creating a service with webhook
* feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen
* feat(webhook) EE-2125 update webhook when update registry
* feat(webhook) EE-2125 add endpoint of update webhook
* feat(webhook) EE-2125 code cleanup
* feat(webhook) EE-2125 fix a typo
* feat(webhook) EE-2125 fix circle import issue with unit test
Co-authored-by: Simon Meng <simon.meng@portainer.io>
* feat(access-token): Multi-auth middleware support EE-1891 (#5936)
* AnyAuth middleware initial implementation with tests
* using mux.MiddlewareFunc instead of custom definition
* removed redundant comments
* - ExtractBearerToken bouncer func made private
- changed helm token handling functionality to use jwt service to convert token to jwt string
- updated tests
- fixed helm list broken test due to missing token in request context
* rename mwCheckAuthentication -> mwCheckJWTAuthentication
* - introduce initial api-key auth support using X-API-KEY header
- added tests to validate x-api-key request header presence
* updated core mwAuthenticatedUser middleware to support multiple auth paradigms
* - simplified anyAuth middleware
- enforcing authmiddleware to implement verificationFunc interface
- created tests for middleware
* simplify bouncer
Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>
* feat(api-key): user-access-token generation endpoint EE-1889 EE-1888 EE-1895 (#6012)
* user-access-token generation endpoint
* fix comment
* - introduction of apikey service
- seperation of repository from service logic - called in handler
* fixed tests
* - fixed api key prefix
- added tests
* added another test for digest matching
* updated swagger spec for access token creation
* api key response returns raw key and struct - easing testability
* test for api key prefix length
* added another TODO to middleware
* - api-key prefix rune -> string (rune does not auto-encode when response sent back to client)
- digest -> pointer as we want to allow nil values and omit digest in responses (when nil)
* - updated apikey struct
- updated apikey service to support all common operations
- updated apikey repo
- integration of apikey service into bouncer
- added test for all apikey service functions
- boilerplate code for apikey service integration
* - user access token generation tests
- apiKeyLookup updated to support query params
- added api-key tests for query params
- added api-key tests for apiKeyLookup
* get and remove access token handlers
* get and remove access token handler tests
* - delete user deletes all associated api keys
- tests for this functionality
* removed redundant []byte cast
* automatic api-key eviction set within cache for 1 hour
* fixed bug with loop var using final value
* fixed service comment
* ignore bolt error responses
* case-insensitive query param check
* simplified query var assignment
* - added GetAPIKey func to get by unique id
- updated DeleteAPIKey func to not require user ID
- updated tests
* GenerateRandomKey helper func from github.com/gorilla/securecookie moved to codebase
* json response casing for api-keys fixed
* updating api-key will update the cache
* updated golang LRU cache
* using hashicorps golang-LRU cache for api keys
* simplified jwt check in create user access token
* fixed api-key update logic on cache miss
* Prefix generated api-keys with `ptr_` (#6067)
* prefix api-keys with 'ptr_'
* updated apikey description
* refactor
Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>
* helm list test refactor
* fixed user delete test
* reduce test nil pointer errors
* using correct http 201 created status code for token creation; updated tests
* fixed swagger doc user id path param for user access token based endpoints
* added api-key security openapi spec to existing jwt secured endpoints (#6091)
* fixed flaky test
* apikey datecreated and lastused attrs converted to unix timestamp
* feat(user): added access token datatable. (#6124)
* feat(user): added access token datatable.
* feat(tokens): only display lastUsed time when it is not the default date
* Update app/portainer/views/account/accountController.js
Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>
* Update app/portainer/views/account/accountController.js
Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>
* Update app/portainer/views/account/accountController.js
Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>
* Update app/portainer/components/datatables/access-tokens-datatable/accessTokensDatatableController.js
Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>
* Update app/portainer/services/api/userService.js
Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>
* feat(improvements): proposed datatable improvements to speed up dev time (#6138)
* modal code update
* updated datatable filenames, updated controller to be default class export
* fix(access-token): code improvement.
Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>
* feat(apikeys): create access token view initial implementation EE-1886 (#6129)
* CopyButton implementation
* Code component implementation
* ToolTip component migration to another folder
* TextTip component implementation - continued
* form Heading component
* Button component updated to be more dynamic
* copybutton - small size
* form control pass tip error
* texttip small text
* CreateAccessToken react feature initial implementation
* create user access token angularjs view implementation
* registration of CreateAccessToken component in AngularJS
* user token generation API request moved to angular service, method passed down instead
* consistent naming of access token operations; clustered similar code together
* any user can add access token
* create access token page routing
* moved code component to the correct location
* removed isadmin check as all functionality applicable to all users
* create access token angular view moved up a level
* fixed PR issues, updated PR
* addressed PR issues/improvements
* explicit hr for horizontal line
* fixed merge conflict storybook build breaking
* - apikey test
- cache test
* addressed testing issues:
- description validations
- remove token description link on table
* fix(api-keys): user role change evicts user keys in cache EE-2113 (#6168)
* user role change evicts user api keys in cache
* EvictUserKeyCache -> InvalidateUserKeyCache
* godoc for InvalidateUserKeyCache func
* additional test line
* disable add access token button after adding token to prevent spam
Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>
Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
* fix(migration) make dockerhub registry migration idempotent EE-2042
* add missing changes to make updateDockerhubToDB32 idempotent
* add tests for bad migrations
* feat(stack): backport changes to CE EE-1189
* feat(stack): front end backport changes to CE EE-1199 (#5455)
* feat(stack): front end backport changes to CE EE-1199
* fix k8s deploy logic
* fixed web editor confirmation message typo. EE-1501
* fix(stack): fixed issue auth detail not remembered EE-1502 (#5459)
* show status in buttons
* removed onChangeRef function.
* moved buttons in git form to its own component
* removed unused variable.
Co-authored-by: ArrisLee <arris_li@hotmail.com>
* moved formvalue to kube app component
* fix(stack): failed to pull and redeploy compose format k8s stack
* fixed form value
* fix(k8s): file content overridden when deployment failed with compose format EE-1548
* updated API response to get IsComposeFormat and show appropriate text.
* feat(k8s): front end backport to CE
* feat(kube): kube app auto update backend (#5547)
* error message updates for different file type
* not display creation source for external application
* added confirmation modal to advanced app created by web editor
* stop showing confirmation modal when updating application
* disable rollback button when application type is not applicatiom form
* only update file after deployment succeded
* Revert "only update file after deployment succeded"
This reverts commit b94bd2e96f.
* fix(k8s): file content overridden when deployment failed with compose format EE-1556
* added analytics-on directive to pull and redeploy button
* fix(kube): don't valide resource control access for kube (#5568)
* added missing question mark to k8s confirmation modal
* fixed webhook format issue
* added question marks to k8s app confirmation modal
* added space in additional file list.
* ignoring error on deletion
* fix(k8s): Git authentication info not persisted
* added RepositoryMechanismTypes constant
* updated analytics functions
* covert RepositoryMechanism to constant
* fixed typo
* removed unused function.
* post tech review updates
* fixed save settings n redeploy button
* refact kub deploy logic
* Revert "refact kub deploy logic"
This reverts commit cbfdd58ece.
* feat(k8s): utilize user token for k8s auto update EE-1594
* feat(k8s): persist kub stack name EE-1630
* feat(k8s): support delete kub stack
* fix(app): updated logic to delete stack for different kind apps. (#5648)
* fix(app): updated logic to delete stack for different kind apps.
* renamed variable
* fix import
* added StackName field.
* fixed stack id not found issue.
* fix(k8s): fixed qusetion mark alignment issue in PAT field. (#5611)
* fix(k8s): fixed qusetion mark alignment issue in PAT field.
* moved inline css to file.
* fix(git-form: made auth input text full width
* add ignore deleted arg
* tech review updates
* typo fix
* fix(k8s): added console error when deleting k8s service.
* fix(console): added no-console config
* fix(deploy): added missing service.
* fix: use stack editor as an owner when exists (#5678)
* fix: tempalte/content based stacks edit/delete
* fix(stack): remove stack when no app. (#5769)
* fix(stack): remove stack when no app.
* support compose format in delete
Co-authored-by: ArrisLee <arris_li@hotmail.com>
Co-authored-by: Hui <arris_li@hotmail.com>
Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
Co-authored-by: Felix Han <felix.han@portainer.io>
* feat(kubeshell) allow overriding default kubeshell
* Add missing error check and struct tag
* Add migrator for kube shell image and add it as a default in the db
* Fix file name to match migrator pattern
* remove default as it's now coming from the db
* remove blank line
* - conflict resolution code update
- logging migration error on migration failures
* - migrateDBVersionTo34 -> migrateDBVersionToDB34 (naming consistency)
Co-authored-by: zees-dev <dev.786zshan@gmail.com>
* backport migration EE code structure
* filesystem copy function
* set db status to updating before migration - reset on completion
* support for auto-backup on version upgrade
* - rollback cli flag support (with confirmation)
- rollback implementation backport from EE
* removed edition as it is not required in CE
* migrated test datastore from bolttest to bolt package to make it usable for testing
* backported failsafe migration
* - backported tests from EE
- refactored tests to use test datastore
* test store implementing datastore interface
* addressed PR issues/improvements
* refactor test
* added backup file removal error logging
* resolved conflicts, updated code
* fixed missing bolttest package - migrated to bolt
* feat(migration): wrap migration errors to provide context for failure EE-1742 (#5711)
* feat(migrator): wrap errors to provide more context to failures EE-1742
* add overall failure back in. diff log file
* updated helm tests pointing to correct teststore
Co-authored-by: Matt Hook <hookenz@gmail.com>
* fix(edge) EE-1733 cant edit edge groups
* fix(edge) EE-1733 correct json names of a few edge objects
Co-authored-by: Simon Meng <simon.meng@portainer.io>
* feat(helm): add helm chart backport to ce EE-1409 (#5425)
* EE-1311 Helm Chart Backport from EE
* backport to ce
Co-authored-by: Matt Hook <hookenz@gmail.com>
* feat(helm): list and configure helm chart (#5431)
* backport and tidyup code
* --amend
* using rocket icon for charts
* helm chart bugfix - clear category button
* added matomo analytics for helm chart install
* fix web editor exit warning without changes
* editor modified exit bugfix
* fixed notifications typo
* updated helm template text
* helper text to convey slow helm templates load
Co-authored-by: zees-dev <dev.786zshan@gmail.com>
* removing redundant time-consuming api call by using prop attribute
* feat(helm) helm chart backport from ee EE-1311 (#5436)
* Add missing defaultHelmRepoUrl and mock testing
* Backport EE-1477
* Backport updates to helm tests from EE
* add https by default changes and ssl to tls renaming from EE
* Port install integration test. Disabled by default to pass CI checks
* merged changes from EE for the integration test
* kube proxy whitelist updated to support internal helm install command
Co-authored-by: zees-dev <dev.786zshan@gmail.com>
* Pull in all changes from tech review in EE-943
* added helm to sidebar after rebase, sync CE with EE
* bugfix: kubectl shell not opening - bearer token bug
* tidy go modules & remove yarn-error.log
* removed redundant handler (not used) - to match EE
* resolved merge conflicts, updated code
* feat(helm/views): helm release and application views EE-1236 (#5529)
* feat(helm): add helm chart backport to ce EE-1409 (#5425)
* EE-1311 Helm Chart Backport from EE
* backport to ce
Co-authored-by: Matt Hook <hookenz@gmail.com>
* Pull in all changes from tech review in EE-943
* added helm to sidebar after rebase, sync CE with EE
* removed redundant handler (not used) - to match EE
* feat(helm) display helm charts - backend EE-1236
* copy over components for new applications view EE-1236
* Add new applications datatable component
* Add more migrated files
* removed test not applicable to CE
* baclkported EE app data table code to CE
* removed redundant helm repo url
* resolved conflicts, updated code
* using endpoint middleware
* PR review fixes
* using constants, openapi updated
Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: zees-dev <dev.786zshan@gmail.com>
* fixed test conflicts, go linted
* feat(helm/templates-add): helm templates add repo for user support EE-1278 (#5514)
* feat(helm): add helm chart backport to ce EE-1409 (#5425)
* EE-1311 Helm Chart Backport from EE
* backport to ce
Co-authored-by: Matt Hook <hookenz@gmail.com>
* feat(helm) helm chart backport from ee EE-1311 (#5436)
* Add missing defaultHelmRepoUrl and mock testing
* Backport EE-1477
* Backport updates to helm tests from EE
* add https by default changes and ssl to tls renaming from EE
* Port install integration test. Disabled by default to pass CI checks
* merged changes from EE for the integration test
* kube proxy whitelist updated to support internal helm install command
Co-authored-by: zees-dev <dev.786zshan@gmail.com>
* Pull in all changes from tech review in EE-943
* feat(helm): add helm chart backport to ce EE-1409 (#5425)
* EE-1311 Helm Chart Backport from EE
* backport to ce
Co-authored-by: Matt Hook <hookenz@gmail.com>
* Pull in all changes from tech review in EE-943
* added helm to sidebar after rebase, sync CE with EE
* backport EE-1278, squashed, diffed, updated
* helm install openapi spec update
* resolved conflicts, updated code
* - matching ee codebase at 0afe57034449ee0e9f333d92c252a13995a93019
- helm install using endpoint middleware
- remove trailing slash from added/persisted helm repo urls
* feat(helm) use libhelm url validator and improved path assembly EE-1554 (#5561)
* feat(helm/userrepos) fix getting global repo for ordinary users EE-1562 (#5567)
* feat(helm/userrepos) fix getting global repo for ordinary users EE-1562
* post review changes and further backported changes from EE
* resolved conflicts, updated code
* fixed helm_install handler unit test
* user cannot add existing repo if suffix is '/' (#5571)
* feat(helm/docs) fix broken swagger docs EE-1278 (#5572)
* Fix swagger docs
* minor correction
* fix(helm): migrating code from user handler to helm handler (#5573)
* - migrated user_helm_repos to helm endpoint handler
- migrated api operations from user factory/service to helm factory/service
- passing endpointId into helm service/factory as endpoint provider is deprecated
* upgrade libhelm to hide secrets
Co-authored-by: Matt Hook <hookenz@gmail.com>
* removed duplicate file - due to merge conflict
* dependency injection in helm factory
Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: Matt Hook <hookenz@gmail.com>
* kubernetes.templates -> kubernetes.templates.helm name conflict fix
* Validate the URL added as a public helm repo (#5579)
* fix(helm): helm app deletion fix EE-1581 (#5582)
* updated helm lib to show correct error on uninstall failure
* passing down helm app namespace on deletion
* fix(k8s): EE-1591 non-admin users cannot deploy charts containing secrets (#5590)
Co-authored-by: Simon Meng <simon.meng@portainer.io>
* fix(helm): helm epic bugfixes EE-1582 EE-1593 (#5585)
* - trim trailing slash and lowercase before persisting helm repo
- browser helm templates url /kubernetes/templates/templates -> /kubernetes/templates/helm
- fix publish url
- fix helm repo add refresh
- semi-fix k8s app expansion
* Tidy up swagger documentation related to helm. Make json consistent
* fixed helm release page for non-default namespaces
* k8s app view table expansion bugfix
* EE-1593: publish url load balancer fallback
Co-authored-by: Matt Hook <hookenz@gmail.com>
* k8s app list fix for charts with deployments containing multiple pods - which use the same label (#5599)
* fix(kubernetes): app list view fix for secrets with long keys or values EE-1600 (#5600)
* k8s app secrets key value text overflow ellipses
* wrapping key value pairs instead of ellipses
* fix(helm): helm apps bundling issue across different namespaces EE-1619 (#5602)
* helm apps bundling issue across different namespaces
* - code comments and indentation to ease reading
- moved namespace calc out of loop
* feat(helm/test) disable slow helm search test by default EE-1599 (#5598)
* skip helm_repo_search as it's an integration test
* switch to portainer built in integration test checker
* make module order match EE
* don't print test struct out when skipping integration test
Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: Matt Hook <hookenz@gmail.com>
Co-authored-by: cong meng <mcpacino@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
* feat(stack): backport changes to CE EE-1189
* feat(stack): front end backport changes to CE EE-1199 (#5455)
* feat(stack): front end backport changes to CE EE-1199
* fix k8s deploy logic
* fixed web editor confirmation message typo. EE-1501
* fix(stack): fixed issue auth detail not remembered EE-1502 (#5459)
* show status in buttons
* removed onChangeRef function.
* moved buttons in git form to its own component
* removed unused variable.
Co-authored-by: ArrisLee <arris_li@hotmail.com>
* moved formvalue to kube app component
* fix(stack): failed to pull and redeploy compose format k8s stack
* fixed form value
* fix(k8s): file content overridden when deployment failed with compose format EE-1548
* updated API response to get IsComposeFormat and show appropriate text.
* error message updates for different file type
* not display creation source for external application
* added confirmation modal to advanced app created by web editor
* stop showing confirmation modal when updating application
* disable rollback button when application type is not applicatiom form
* added analytics-on directive to pull and redeploy button
* fix(kube): don't valide resource control access for kube (#5568)
* added question marks to k8s app confirmation modal
* fix(k8s): Git authentication info not persisted
* removed unused function.
Co-authored-by: Hui <arris_li@hotmail.com>
Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
Co-authored-by: Felix Han <felix.han@portainer.io>
* - fixed zombie kubectl shell pod issue
- fixed bug with websocket connection error
* exec error if not websocket disconnect
* StartExecProcess updated to write error to channel
* feat(kubeconfig) EE-1153 Introduce the ability to change the expiry of a kubeconfig
* feat(kubeconfig) EE-1153 pr feedback update
* feat(kubeconfig) EE-1153 code cleanup
Co-authored-by: Simon Meng <simon.meng@portainer.io>
* feat(nodes limits)Review the resource assignement when creating a Kubernetes application EE-437
* feat(nodes limits) review feedback EE-437
* feat(nodes limits) workaround for lodash cloneDeep not working in production mode EE-437
* feat(nodes limits) calculate max cpu of slide bar with floor function instead of round function EE-437
* feat(nodes limits) another review feedback EE-437
* feat(nodes limits) cleanup code EE-437
* feat(nodes limits) EE-437 pr feedback update
* feat(nodes limits) EE-437 rebase onto develop branch
* feat(nodes limits) EE-437 another pr feedback update
Co-authored-by: Simon Meng <simon.meng@portainer.io>
* feat(k8s/resource-pool): add the ability to mark/unmark resource pool as system
fix(kube/ns): check label to see if namespace is system
refactor(k8s/namespaces): rename variables
feat(kubernetes): toggle system state in the server (#5361)
fix(app/resource-pool): UI fixes
feat(app/resource-pool): add confirmation modal when unamrking system namespace
* refactor(app): review changes
* feat(app/namespaces): introduce store to retrieve namespace system status without changing all the kubernetes models
refactor(app/namespaces): remove unused code first introduced for system tagging
fix(app/namespaces): cache namespaces to retrieve system status regardless of namespace reference format
refactor(app): migrate namespace store from helper to a separate singleton
refactor(app): remove KubernetesNamespaceHelper from DI cycle
* refactor(app): normalize usage of KubernetesNamespaceHelper functions
* refactor(app/k8s): change namespace store to functions instead of class
Co-authored-by: LP B <xAt0mZ@users.noreply.github.com>
* feat(analytics): add apis for event tracking
feat(api): fetch instanceID
feat(state): set instance id and version on matomo
refactor(state): export validation of app state
feat(analytics): update dimensions
refactor(analytics): move matomo to module
feat(analytics): disable analytics on non production
feat(analytics): track event metadata
refactor(analytics): clean push function
refactor(analytics): rename init function
feat(analytics): track user role
feat(analytics): track user global role
fix(stacks): remove event tracking for stack create
* style(analytics): remove TODO
* feat(build): add testing env
* fix(advance deploy): EE-1141 A standard user can escalate to cluster administrator privileges on Kubernetes
* fix(advance deploy): EE-1141 reuse existing token cache when do deployment
* fix: EE-1141 use user's SA token to exec pod command
* fix: EE-1141 stop advanced-deploy or pod-exec if user's SA token is empty
* fix: EE-1141 resolve merge conflicts
Co-authored-by: Simon Meng <simon.meng@portainer.io>
* intermediate commit
* feat(registry): backport ProGet registry to CE (#954)
* backport EE changes
* label updates and remove auth-toggle
Co-authored-by: Dennis Buduev <dennis.buduev@portainer.io>
* fix(k8s) cleaning up namespace access policies when removing users or teams from endpoint or endpoint group EE-718
* fix(k8s) minor code cleanup EE-718
Co-authored-by: Simon Meng <simon.meng@portainer.io>
* feat(git): save git config when creating stack (#5048)
* feat(git): save git config when creating stack
* chore(fs): test fileExists
* fix(git): fix tests to use CloneRepository
* refactor(git): move options to new object
* feat(stacks): redeploy git stack api (#5112)
* feat(stacks): redeploy git stacks form
[EE-666]
* feat(stack): show loading after confirmation
* fix(stacks): show same size description
* fix(stacks): reload state when deployed
* feat(stacks): set stopped stacks status to activate when updating
* feat(stacks): backup stack folder before cloning
* feat(stacks): don't accept prune and env on update git
Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com>
Co-authored-by: Chaim Lev-Ari <chiptus@gmail.com>
* add updateSettingsToDB28 func and test
* update DBversion const
* migration func naming modification
* feat(oauth): add sso, hide internal auth teaser and logout options. (#5039)
* cleanup and make helper func for unit testing
* dbversion update
* feat(publicSettings): public settings response modification for OAuth SSO EE-608 (#5062)
* feat(oauth): updated logout logic with logoutUrl. (#5064)
* add exclusive token generation for OAuth
* swagger annotation revision
* add unit test
* updates based on tech review feedback
* feat(oauth): updated oauth settings model
* feat(oauth): added oauth logout url
* feat(oauth): fixed SSO toggle and logout issue.
* set SSO to ON by default
* update migrator unit test
* set SSO to true by default for new instance
* prevent applying the SSO logout url to the initial admin user
Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
Co-authored-by: Felix Han <felix.han@portainer.io>
* refactor(http/kube): convert compose format
* feat(kube/deploy): deploy to agent
* feat(kube/deploy): show more details about error
* refactor(kube): return string from deploy
* feat(kube/deploy): revert to use local kubectl
* Revert "feat(kube/deploy): revert to use local kubectl"
This reverts commit 7c4a1c70
* feat(kube/deploy): GH#4321 use the v2 version of agent api instead of v3
Co-authored-by: Chaim Lev-Ari <chiptus@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
* fix windows build
* fix(endpoints): show correct values of security settings (#4889)
* fix(app): EndpointProvider fallback on URL EndpointID when no endpoint is selected (#4892)
* fix(templates): App templates not loading with error in browser console (#4895)
* fix(kube/config): show used key warning when needed (#4890)
fix [CE-469]
- recalculate duplcate keys when they are changed
- show used warning on duplicate keys
* fix(k8s): CE-471 variables from configuration showing on environment variables section on application edit screen (#4896)
* fix(k8s): CE-471 variables from configuration showing on environment variables section on application edit screen
* fix(k8s): CE-471 avoid to remove value path of env when patch k8s deployment, as the value path does not exist if env variable has empty value.
Co-authored-by: Simon Meng <simon.meng@portainer.io>
Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>
Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com>
Co-authored-by: LP B <xAt0mZ@users.noreply.github.com>
Co-authored-by: Maxime Bajeux <max.bajeux@gmail.com>
Co-authored-by: cong meng <mcpacino@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
* feat(k8s): better form validation for configuration keys (#4728) (#4733)
Co-authored-by: Simon Meng <simon.meng@portainer.io>
* feat(home): show edge valid tag
* fix(endpoint): show right heartbeat
* style(endpoints): add some comments
Co-authored-by: cong meng <mcpacino@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
* refactor(stack): create unique name function
* refactor(stack): change stack resource control id
* feat(stacks): validate stack unique name in endpoint
* feat(stacks): prevent name collision with external stacks
* refactor(stacks): move resource id util
* refactor(stacks): supply resource id util with name and endpoint
* fix(docker): calculate swarm resource id
* feat(stack): prevent migration if stack name already exist
* feat(authorization): use stackutils
* feat(compose): add docker-compose wrapper
ce-187
* fix(compose): pick compose implementation upon startup
* Add static compose build for linux
* Fix wget
* Fix platofrm specific docker-compose download
* Keep amd64 architecture as download parameter
* Add tmp folder for docker-compose
* fix: line endings
* add proxy server
* logs
* Proxy
* Add lite transport for compose
* Fix local deployment
* refactor: pass proxyManager by ref
* fix: string conversion
* refactor: compose wrapper remove unused code
* fix: tests
* Add edge
* Fix merge issue
* refactor: remove unused code
* Move server to proxy implementation
* Cleanup wrapper and manager
* feat: pass max supported compose syntax version with each endpoint
* fix: pick compose syntax version
* fix: store wrapper version in portainer
* Get and show composeSyntaxMaxVersion at stack creation screen
* Get and show composeSyntaxMaxVersion at stack editor screen
* refactor: proxy server
* Fix used tmp
* Bump docker-compose to 1.28.0
* remove message for docker compose limitation
* fix: markup typo
* Rollback docker compose to 1.27.4
* * attempt to fix the windows build issue
* * attempt to debug grunt issue
* * use console log in grunt file
* fix: try to fix windows build by removing indirect deps from go.mod
* Remove tmp folder
* Remove builder stage
* feat(build/windows): add git for Docker Compose
* feat(build/windows): add git for Docker Compose
* feat(build/windows): add git for Docker Compose
* feat(build/windows): add git for Docker Compose
* feat(build/windows): add git for Docker Compose
* feat(build/windows): add git for Docker Compose - fixed verbose output
* refactor: renames
* fix(stack): get endpoint by EndpointProvider
* fix(stack): use margin to add space between line instead of using br tag
Co-authored-by: Stéphane Busso <stephane.busso@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
Co-authored-by: yi-portainer <yi.chen@portainer.io>
Co-authored-by: Steven Kang <skan070@gmail.com>
* feat(docker/stacks): add creation and update dates
* feat(docker/stacks): put ownership column as the last column
* feat(docker/stacks): fix the no stacks message
* refactor(docker/stacks): make external stacks helpers more readable
* feat(docker/stacks): add updated and created by
* feat(docker/stacks): toggle updated column
* refactor(datatable): create column visibility component
Co-authored-by: alice groux <alice.grx@gmail.com>
* feat(docker/stacks): add creation and update dates
* feat(docker/stacks): put ownership column as the last column
* feat(docker/stacks): fix the no stacks message
Marcelo Rydel