Commit Graph

86 Commits (c47e840b37cb20f28e97ad87a6e3e6b81677bc4f)

Author SHA1 Message Date
Hao Zhang 9ff8f42a66
feat(stack): detach git based stacks from git EE-2143 (#6307)
* feat(stack): detach git based stacks from git
2022-01-14 11:47:47 +08:00
Sven Dowideit f99329eb7e
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173)
* use the Store interface IsErrObjectNotFound() to avoid revealing internal errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* what happens when you extract the datastore interfaces into their own package

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Start renaming Storage methods

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the boltdb specific code from the Portainer storage code (example, the others need the same)

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extract bolt.Tx from datastore code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* minimise imports by putting moving the struct definition into the file that needs the Service imports

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extraction of boltdb.Tx

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the use of bucket.SetSequence

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* almost done - just endpoint.Synchonise :/

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring'

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix test compile errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* test compile fixes after rebase

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* set us up to make the connection an interface

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make the db connection a datastore interface, and separate out our datastore services from the bolt ones

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* rename methods to something less oltdb internals specific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* these errors are not boltdb secific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* start using the db-backend factory method too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* export boltdb raw in case we can't export from the service layer

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add the version info by hand for now

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* actually, the export from services can be fully typed - its the import that needs to do more work

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* redo raw export, and make import capable of using it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add DockerHub

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix go test ./... again

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* my goland wasn't setup to gofmt

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the two extremely dubious migration tests down into store, so they can use the test store code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* the migrator is now free of boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* reverse goland overzealous replcement of internal with boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more undo over-zealous goland internal->boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* yay, now bolt is only mentioned inside the api/database/ dir

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* and this might be the last of the boltdb references?

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add todo

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the store code into a separate module too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* don't need the fileService in boltdb anymore

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use IsErrObjectNotFound()

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use a string to select what database backend we use

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make isNew store an ephemeral bool that doesn't stay true after we've initialised it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* undo vscode formatting html

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix app templates symbol (#6221)

* feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220)

* feat(webhook) EE-2125 add some helpers to registry utils

* feat(webhook) EE-2125 persist registryID when creating a webhook

* feat(webhook) EE-2125 send registry auth header when executing a webhook

* feat(webhook) EE-2125 send registryID to backend when creating a service with webhook

* feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen

* feat(webhook) EE-2125 update webhook when update registry

* feat(webhook) EE-2125 add endpoint of update webhook

* feat(webhook) EE-2125 code cleanup

* feat(webhook) EE-2125 fix a typo

* feat(webhook) EE-2125 fix circle import issue with unit test

Co-authored-by: Simon Meng <simon.meng@portainer.io>

* fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204)

Co-authored-by: Simon Meng <simon.meng@portainer.io>

* fix data-cy for k8s cluster menu (#6226)

LGTM

* feat(stack): make stack created from app template editable EE-1941 (#6104)

feat(stack): make stack from app template editable

* fix(container):disable Duplicate/Edit button when the container is portainer (#6223)

* fix/ee-1909/show-pull-image-error (#6195)

Co-authored-by: sunportainer <ericsun@SG1.local>

* feat(cy): add data-cy to helm install button (#6241)

* feat(cy): add data-cy to add registry button (#6242)

* refactor(app): convert root folder files to es6 (#4159)

* refactor(app): duplicate constants as es6 exports (#4158)

* fix(docker): provide workaround to save network name variable  (#6080)

* fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file

* fix/EE-1862/unable-to-stop-or-remove-stack check yaml file

* fixed func and var names

* wrapper error and used bool for stringset

* UT case for createNetworkEnvFile

* UT case for %s=%s

* powerful StringSet

* wrapper error for extract network name

* wrapper all the return err

* store more env

* put to env file

* make default value None

* feat: gzip static resources (#6258)

* fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203)

* fix/ee-2106/handle-sslcert-sslkey

Co-authored-by: sunportainer <ericsun@SG1.local>

* fix(server):support disable https only ee-2068 (#6232)

* fix/ee-2068/disable-forcely-https

* feat(store): implement store tests EE-2112 (#6224)

* add store tests

* add some more tests

* Update missing helm user repo methods

* remove redundant comments

* add webhook export

* update webhooks

* use the Store interface IsErrObjectNotFound() to avoid revealing internal errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* what happens when you extract the datastore interfaces into their own package

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Start renaming Storage methods

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the boltdb specific code from the Portainer storage code (example, the others need the same)

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extract bolt.Tx from datastore code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* minimise imports by putting moving the struct definition into the file that needs the Service imports

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extraction of boltdb.Tx

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the use of bucket.SetSequence

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* almost done - just endpoint.Synchonise :/

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring'

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix test compile errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* test compile fixes after rebase

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* set us up to make the connection an interface

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make the db connection a datastore interface, and separate out our datastore services from the bolt ones

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* rename methods to something less oltdb internals specific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* these errors are not boltdb secific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* start using the db-backend factory method too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* export boltdb raw in case we can't export from the service layer

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add the version info by hand for now

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* actually, the export from services can be fully typed - its the import that needs to do more work

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* redo raw export, and make import capable of using it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add DockerHub

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix go test ./... again

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* my goland wasn't setup to gofmt

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the two extremely dubious migration tests down into store, so they can use the test store code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* the migrator is now free of boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* reverse goland overzealous replcement of internal with boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more undo over-zealous goland internal->boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* yay, now bolt is only mentioned inside the api/database/ dir

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* and this might be the last of the boltdb references?

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add todo

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the store code into a separate module too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* don't need the fileService in boltdb anymore

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use IsErrObjectNotFound()

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use a string to select what database backend we use

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make isNew store an ephemeral bool that doesn't stay true after we've initialised it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* undo vscode formatting html

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Update missing helm user repo methods

* feat(store): implement store tests EE-2112 (#6224)

* add store tests

* add some more tests

* remove redundant comments

* add webhook export

* update webhooks

* fix build issues after rebasing

* move migratorparams

* remove unneeded integer type conversions

* disable the db import/export for now

Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: cong meng <mcpacino@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com>
Co-authored-by: Hao Zhang <hao.zhang@portainer.io>
Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com>
Co-authored-by: sunportainer <ericsun@SG1.local>
Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com>
Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com>
Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com>
Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>
Co-authored-by: Matt Hook <hookenz@gmail.com>
2021-12-15 15:26:09 +13:00
Hao Zhang 9f5ac154aa
feat(stack): make stack created from app template editable EE-1941 (#6104)
feat(stack): make stack from app template editable
2021-12-07 19:46:58 +08:00
sunportainer 335f951e6b
Fix(stack)/update StackUpdateGit swagger info to POST EE-2019 (#6176)
* fix/EE-2019/Fix-stackgitupdate-swagger

Co-authored-by: sunportainer <ericsun@SG1.local>
2021-12-02 09:54:38 +08:00
zees-dev 69c17986d9
feat(api-key/backend): introducing support for api-key based auth EE-978 (#6079)
* feat(access-token): Multi-auth middleware support EE-1891 (#5936)

* AnyAuth middleware initial implementation with tests

* using mux.MiddlewareFunc instead of custom definition

* removed redundant comments

* - ExtractBearerToken bouncer func made private
- changed helm token handling functionality to use jwt service to convert token to jwt string
- updated tests
- fixed helm list broken test due to missing token in request context

* rename mwCheckAuthentication -> mwCheckJWTAuthentication

* - introduce initial api-key auth support using X-API-KEY header
- added tests to validate x-api-key request header presence

* updated core mwAuthenticatedUser middleware to support multiple auth paradigms

* - simplified anyAuth middleware
- enforcing authmiddleware to implement verificationFunc interface
- created tests for middleware

* simplify bouncer

Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>

* feat(api-key): user-access-token generation endpoint EE-1889 EE-1888 EE-1895 (#6012)

* user-access-token generation endpoint

* fix comment

* - introduction of apikey service
- seperation of repository from service logic - called in handler

* fixed tests

* - fixed api key prefix
- added tests

* added another test for digest matching

* updated swagger spec for access token creation

* api key response returns raw key and struct - easing testability

* test for api key prefix length

* added another TODO to middleware

* - api-key prefix rune -> string (rune does not auto-encode when response sent back to client)
- digest -> pointer as we want to allow nil values and omit digest in responses (when nil)

* - updated apikey struct
- updated apikey service to support all common operations
- updated apikey repo
- integration of apikey service into bouncer
- added test for all apikey service functions
- boilerplate code for apikey service integration

* - user access token generation tests
- apiKeyLookup updated to support query params
- added api-key tests for query params
- added api-key tests for apiKeyLookup

* get and remove access token handlers

* get and remove access token handler tests

* - delete user deletes all associated api keys
- tests for this functionality

* removed redundant []byte cast

* automatic api-key eviction set within cache for 1 hour

* fixed bug with loop var using final value

* fixed service comment

* ignore bolt error responses

* case-insensitive query param check

* simplified query var assignment

* - added GetAPIKey func to get by unique id
- updated DeleteAPIKey func to not require user ID
- updated tests

* GenerateRandomKey helper func from github.com/gorilla/securecookie moved to codebase

* json response casing for api-keys fixed

* updating api-key will update the cache

* updated golang LRU cache

* using hashicorps golang-LRU cache for api keys

* simplified jwt check in create user access token

* fixed api-key update logic on cache miss

* Prefix generated api-keys with `ptr_` (#6067)

* prefix api-keys with 'ptr_'

* updated apikey description

* refactor

Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>

* helm list test refactor

* fixed user delete test

* reduce test nil pointer errors

* using correct http 201 created status code for token creation; updated tests

* fixed swagger doc user id path param for user access token based endpoints

* added api-key security openapi spec to existing jwt secured endpoints (#6091)

* fixed flaky test

* apikey datecreated and lastused attrs converted to unix timestamp

* feat(user): added access token datatable. (#6124)

* feat(user): added access token datatable.

* feat(tokens): only display lastUsed time when it is not the default date

* Update app/portainer/views/account/accountController.js

Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>

* Update app/portainer/views/account/accountController.js

Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>

* Update app/portainer/views/account/accountController.js

Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>

* Update app/portainer/components/datatables/access-tokens-datatable/accessTokensDatatableController.js

Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>

* Update app/portainer/services/api/userService.js

Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>

* feat(improvements): proposed datatable improvements to speed up dev time (#6138)

* modal code update

* updated datatable filenames, updated controller to be default class export

* fix(access-token): code improvement.

Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>

* feat(apikeys): create access token view initial implementation EE-1886 (#6129)

* CopyButton implementation

* Code component implementation

* ToolTip component migration to another folder

* TextTip component implementation - continued

* form Heading component

* Button component updated to be more dynamic

* copybutton - small size

* form control pass tip error

* texttip small text

* CreateAccessToken react feature initial implementation

* create user access token angularjs view implementation

* registration of CreateAccessToken component in AngularJS

* user token generation API request moved to angular service, method passed down instead

* consistent naming of access token operations; clustered similar code together

* any user can add access token

* create access token page routing

* moved code component to the correct location

* removed isadmin check as all functionality applicable to all users

* create access token angular view moved up a level

* fixed PR issues, updated PR

* addressed PR issues/improvements

* explicit hr for horizontal line

* fixed merge conflict storybook build breaking

* - apikey test
- cache test

* addressed testing issues:
- description validations
- remove token description link on table

* fix(api-keys): user role change evicts user keys in cache EE-2113 (#6168)

* user role change evicts user api keys in cache

* EvictUserKeyCache -> InvalidateUserKeyCache

* godoc for InvalidateUserKeyCache func

* additional test line

* disable add access token button after adding token to prevent spam

Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>
Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
2021-11-30 15:31:16 +13:00
Richard Wei a598b2d72d
change the namespace selector behavior (#5768) 2021-11-23 09:51:02 +13:00
sunportainer cea634a7aa
fix(stack): support removing duplicated stacks EE-1962 (#6068)
* fix/EE-1962/cannot-same-stack-name handle multiple names duplicate case

Co-authored-by: Eric Sun <ericsun@SG1.local>
2021-11-22 12:23:56 +08:00
Hui 98b8d6d0b2
fix(stack): git credential got reset when updating docker stack EE-1967 2021-11-12 11:52:09 +13:00
andres-portainer 28f71e486a
fix(filesystem): harden the filesystem service to avoid path traversal attacks EE-1922 (#5957)
fix(filesystem): harden the filesystem service to avoid path traversal attacks EE-1922
2021-11-01 08:01:03 -03:00
Marcelo Rydel 32756f9e1b
fix(git-stacks): UI bugs when using a PAT when deploying from Git [EE-1731] (#5882) 2021-10-25 18:19:05 -03:00
Sven Dowideit 623079442f
fix(swagger): double quotes in swagger param breaks parser (#5806)
Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>
2021-10-19 10:25:53 +10:00
Marcelo Rydel 1ee363f8c9
overrite stack name for update (#5743) 2021-10-12 18:48:28 -03:00
Hui e6d690e31e
fix(swagger) swagger annotations fixes and improvements EE-1205 2021-10-12 12:12:08 +13:00
cong meng 328abfd74e
fix(stack) normalize stack name EE-1701 (#5776)
* fix(stack) normalize stack name EE-1701

* fix(stack) normalize swarm stack name and fix rebase error EE-1701

* fix(stack) add front end stack name validation EE-1701

* fix(stack) make stack name regex as a const EE-1701

* fix(stack) reuse stack name regex for compose and swarm EE-1701

* fix(stack) add name validation for stack duplication form EE-1701

Co-authored-by: Simon Meng <simon.meng@portainer.io>
2021-10-01 16:56:34 +13:00
Dmitry Salakhov 2ecc8ab5c9
feat(k8s): support git automated sync for k8s applications [EE-577] (#5548)
* feat(stack): backport changes to CE EE-1189

* feat(stack): front end backport changes to CE EE-1199 (#5455)

* feat(stack): front end backport changes to CE EE-1199

* fix k8s deploy logic

* fixed web editor confirmation message typo. EE-1501

* fix(stack): fixed issue auth detail not remembered EE-1502 (#5459)

* show status in buttons

* removed onChangeRef function.

* moved buttons in git form to its own component

* removed unused variable.

Co-authored-by: ArrisLee <arris_li@hotmail.com>

* moved formvalue to kube app component

* fix(stack): failed to pull and redeploy compose format k8s stack

* fixed form value

* fix(k8s): file content overridden when deployment failed with compose format EE-1548

* updated API response to get IsComposeFormat and show appropriate text.

* feat(k8s): front end backport to CE

* feat(kube): kube app auto update backend (#5547)

* error message updates for different file type

* not display creation source for external application

* added confirmation modal to advanced app created by web editor

* stop showing confirmation modal when updating application

* disable rollback button when application type is not applicatiom form

* only update file after deployment succeded

* Revert "only update file after deployment succeded"

This reverts commit b94bd2e96f.

* fix(k8s): file content overridden when deployment failed with compose format EE-1556

* added analytics-on directive to pull and redeploy button

* fix(kube): don't valide resource control access for kube (#5568)

* added missing question mark to k8s confirmation modal

* fixed webhook format issue

* added question marks to k8s app confirmation modal

* added space in additional file list.

* ignoring error on deletion

* fix(k8s): Git authentication info not persisted

* added RepositoryMechanismTypes constant

* updated analytics functions

* covert RepositoryMechanism to constant

* fixed typo

* removed unused function.

* post tech review updates

* fixed save settings n redeploy button

* refact kub deploy logic

* Revert "refact kub deploy logic"

This reverts commit cbfdd58ece.

* feat(k8s): utilize user token for k8s auto update EE-1594

* feat(k8s): persist kub stack name EE-1630

* feat(k8s): support delete kub stack

* fix(app): updated logic to delete stack for different kind apps. (#5648)

* fix(app): updated logic to delete stack for different kind apps.

* renamed variable

* fix import

* added StackName field.

* fixed stack id not found issue.

* fix(k8s): fixed qusetion mark alignment issue in PAT field. (#5611)

* fix(k8s): fixed qusetion mark alignment issue in PAT field.

* moved inline css to file.

* fix(git-form: made auth input text full width

* add ignore deleted arg

* tech review updates

* typo fix

* fix(k8s): added console error when deleting k8s service.

* fix(console): added no-console config

* fix(deploy): added missing service.

* fix: use stack editor as an owner when exists (#5678)

* fix: tempalte/content based stacks edit/delete

* fix(stack): remove stack when no app. (#5769)

* fix(stack): remove stack when no app.

* support compose format in delete

Co-authored-by: ArrisLee <arris_li@hotmail.com>

Co-authored-by: Hui <arris_li@hotmail.com>
Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
Co-authored-by: Felix Han <felix.han@portainer.io>
2021-09-30 12:58:10 +13:00
zees-dev af98660a55
feat(helm): helm apps deployed by portainer not marked as external EE-1624 (#5637)
* helm lib update

* helm handler requires kubernetes deployer to modify helm deployed resources

* AddAppLabels updated to be more generic - support for adding multiple labels using map

* path installed helm release manifest with portainer labels using kubectl

* updated helm handler unit tests to use mock KubernetesDeployer

* adding labels to manifest retrieved from release

* optional namespace support for k8s raw manifest deployment

* - inline postprocessing support when extracting
- get namespace from yaml support
- added and updated tests

* lowercase error wrapping

* updated libhelm dep
2021-09-29 10:12:45 +10:00
zees-dev 377326085d
feat(db): upgrade auto-backup backup and rollback support EE-867 EE-1158 (#5341)
* backport migration EE code structure

* filesystem copy function

* set db status to updating before migration - reset on completion

* support for auto-backup on version upgrade

* - rollback cli flag support (with confirmation)
- rollback implementation backport from EE

* removed edition as it is not required in CE

* migrated test datastore from bolttest to bolt package to make it usable for testing

* backported failsafe migration

* - backported tests from EE
- refactored tests to use test datastore

* test store implementing datastore interface

* addressed PR issues/improvements

* refactor test

* added backup file removal error logging

* resolved conflicts, updated code

* fixed missing bolttest package - migrated to bolt

* feat(migration): wrap migration errors to provide context for failure EE-1742 (#5711)

* feat(migrator): wrap errors to provide more context to failures EE-1742

* add overall failure back in. diff log file

* updated helm tests pointing to correct teststore

Co-authored-by: Matt Hook <hookenz@gmail.com>
2021-09-27 13:52:50 +13:00
Richard Wei dd808bb7bd
fix(swagger): fix swagger api docs endpoint(s) rename to environment(s) EE-1661 (#5629)
* fix swagger api docs endpoint(s) rename to environment(s)
2021-09-20 12:14:22 +12:00
Yi Chen 1633eceed5
fix(swagger) Fix openapi issues (#5123)
* * fix api version
* fix license info
* fix error response schema
* fix other typos & mistakes

* * remove unused tag

* * fix helm issues
2021-09-13 15:42:53 +12:00
Hui e86a586651
fix(k8s): manifest file not persisted when deploying with manifest URL EE-1586 2021-09-10 13:35:37 +12:00
Richard Wei 9f179fe3ec
feat(ui):rename endpoint(s) to environment(s) EE-1206 (#5588)
* rename endpoints to environments EE-1206
2021-09-08 20:42:17 +12:00
Dmitry Salakhov e49e90f304
feat(kube): advanced apps management [EE-466] (#5446)
* feat(stack): backport changes to CE EE-1189

* feat(stack): front end backport changes to CE EE-1199 (#5455)

* feat(stack): front end backport changes to CE EE-1199

* fix k8s deploy logic

* fixed web editor confirmation message typo. EE-1501

* fix(stack): fixed issue auth detail not remembered EE-1502 (#5459)

* show status in buttons

* removed onChangeRef function.

* moved buttons in git form to its own component

* removed unused variable.

Co-authored-by: ArrisLee <arris_li@hotmail.com>

* moved formvalue to kube app component

* fix(stack): failed to pull and redeploy compose format k8s stack

* fixed form value

* fix(k8s): file content overridden when deployment failed with compose format EE-1548

* updated API response to get IsComposeFormat and show appropriate text.

* error message updates for different file type

* not display creation source for external application

* added confirmation modal to advanced app created by web editor

* stop showing confirmation modal when updating application

* disable rollback button when application type is not applicatiom form

* added analytics-on directive to pull and redeploy button

* fix(kube): don't valide resource control access for kube (#5568)

* added question marks to k8s app confirmation modal

* fix(k8s): Git authentication info not persisted

* removed unused function.

Co-authored-by: Hui <arris_li@hotmail.com>
Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
Co-authored-by: Felix Han <felix.han@portainer.io>
2021-09-07 12:37:26 +12:00
Chaim Lev-Ari 3453735c8b
feat(stacks): support standalone stacks on ARM (#5310) 2021-09-06 10:58:26 +03:00
Anthony Lapenna 70602cf7c8
feat(k8s): add the ability to deploy from a manifest URL (#5550) 2021-09-03 17:37:34 +12:00
Chaim Lev-Ari e4fe4f9a43
feat(kube): introduce custom templates [EE-1125] (#5434)
* feat(kube): introduce custom templates

refactor(customtemplates): use build option

chore(deps): upgrade yaml parser

feat(customtemplates): add and edit RC to kube templates

fix(kube): show docker icon

fix(custom-templates): save rc

* fix(kube/templates): route to correct routes
2021-09-02 17:28:51 +12:00
Dmitry Salakhov bcccdfb669
feat(stacks): support automated sync for stacks [EE-248] (#5340) 2021-08-17 13:12:07 +12:00
cong meng 51ef2c2aa9
fix(advance deploy): EE-1141 A standard user can escalate to cluster administrator privileges on Kubernetes (#5325)
* fix(advance deploy): EE-1141 A standard user can escalate to cluster administrator privileges on Kubernetes

* fix(advance deploy): EE-1141 reuse existing token cache when do deployment

* fix: EE-1141 use user's SA token to exec pod command

* fix: EE-1141 stop advanced-deploy or pod-exec if user's SA token is empty

* fix: EE-1141 resolve merge conflicts

Co-authored-by: Simon Meng <simon.meng@portainer.io>
2021-08-04 11:11:24 +12:00
Richard Wei 09c5bada3e
fix(app): fix create stack with capital letters or space issue EE-908 (#5236)
* fix(app): fix create stack with capital letters or space issue

* replace ComposeWrapper with ComposeStackManager
2021-07-23 09:53:42 +12:00
Hui 68453482af
fix(swagger): add swagger annotation for pull and redeploy stack 2021-07-22 11:40:53 +12:00
Chaim Lev-Ari 72117693fb
feat(stacks): update stopped stack (#5215)
[EE-554]
2021-07-19 10:41:50 +03:00
LP B 179df06267
feat(app): rework private registries and support private registries in kubernetes EE-30 (#5131)
* feat(app): rework private registries and support private registries in kubernetes

[EE-30]

feat(api): backport private registries backend changes (#5072)

* feat(api/bolt): backport bolt changes

* feat(api/exec): backport exec changes

* feat(api/http): backport http/handler/dockerhub changes

* feat(api/http): backport http/handler/endpoints changes

* feat(api/http): backport http/handler/registries changes

* feat(api/http): backport http/handler/stacks changes

* feat(api/http): backport http/handler changes

* feat(api/http): backport http/proxy/factory/azure changes

* feat(api/http): backport http/proxy/factory/docker changes

* feat(api/http): backport http/proxy/factory/utils changes

* feat(api/http): backport http/proxy/factory/kubernetes changes

* feat(api/http): backport http/proxy/factory changes

* feat(api/http): backport http/security changes

* feat(api/http): backport http changes

* feat(api/internal): backport internal changes

* feat(api): backport api changes

* feat(api/kubernetes): backport kubernetes changes

* fix(api/http): changes on backend following backport

feat(app): backport private registries frontend changes (#5056)

* feat(app/docker): backport docker/components changes

* feat(app/docker): backport docker/helpers changes

* feat(app/docker): backport docker/views/container changes

* feat(app/docker): backport docker/views/images changes

* feat(app/docker): backport docker/views/registries changes

* feat(app/docker): backport docker/views/services changes

* feat(app/docker): backport docker changes

* feat(app/kubernetes): backport kubernetes/components changes

* feat(app/kubernetes): backport kubernetes/converters changes

* feat(app/kubernetes): backport kubernetes/models changes

* feat(app/kubernetes): backport kubernetes/registries changes

* feat(app/kubernetes): backport kubernetes/services changes

* feat(app/kubernetes): backport kubernetes/views/applications changes

* feat(app/kubernetes): backport kubernetes/views/configurations changes

* feat(app/kubernetes): backport kubernetes/views/configure changes

* feat(app/kubernetes): backport kubernetes/views/resource-pools changes

* feat(app/kubernetes): backport kubernetes/views changes

* feat(app/portainer): backport portainer/components/accessManagement changes

* feat(app/portainer): backport portainer/components/datatables changes

* feat(app/portainer): backport portainer/components/forms changes

* feat(app/portainer): backport portainer/components/registry-details changes

* feat(app/portainer): backport portainer/models changes

* feat(app/portainer): backport portainer/rest changes

* feat(app/portainer): backport portainer/services changes

* feat(app/portainer): backport portainer/views changes

* feat(app/portainer): backport portainer changes

* feat(app): backport app changes

* config(project): gitignore + jsconfig changes

gitignore all files under api/cmd/portainer but main.go and enable Code Editor autocomplete on import ... from '@/...'

fix(app): fix pull rate limit checker

fix(app/registries): sidebar menus and registry accesses users filtering

fix(api): add missing kube client factory

fix(kube): fetch dockerhub pull limits (#5133)

fix(app): pre review fixes (#5142)

* fix(app/registries): remove checkbox for endpointRegistries view

* fix(endpoints): allow access to default namespace

* fix(docker): fetch pull limits

* fix(kube/ns): show selected registries for non admin

Co-authored-by: Chaim Lev-Ari <chiptus@gmail.com>

chore(webpack): ignore missing sourcemaps

fix(registries): fetch registry config from url

feat(kube/registries): ignore not found when deleting secret

feat(db): move migration to db 31

fix(registries): fix bugs in PR EE-869 (#5169)

* fix(registries): hide role

* fix(endpoints): set empty access policy to edge endpoint

* fix(registry): remove double arguments

* fix(admin): ignore warning

* feat(kube/configurations): tag registry secrets (#5157)

* feat(kube/configurations): tag registry secrets

* feat(kube/secrets): show registry secrets for admins

* fix(registries): move dockerhub to beginning

* refactor(registries): use endpoint scoped registries

feat(registries): filter by namespace if supplied

feat(access-managment): filter users for registry (#5191)

* refactor(access-manage): move users selector to component

* feat(access-managment): filter users for registry

refactor(registries): sync code with CE (#5200)

* refactor(registry): add inspect handler under endpoints

* refactor(endpoint): sync endpoint_registries_list

* refactor(endpoints): sync registry_access

* fix(db): rename migration functions

* fix(registries): show accesses for admin

* fix(kube): set token on transport

* refactor(kube): move secret help to bottom

* fix(kuberentes): remove shouldLog parameter

* style(auth): add description of security.IsAdmin

* feat(security): allow admin access to registry

* feat(edge): connect to edge endpoint when creating client

* style(portainer): change deprecation version

* refactor(sidebar): hide manage

* refactor(containers): revert changes

* style(container): remove whitespace

* fix(endpoint): add handler to registy on endpointService

* refactor(image): use endpointService.registries

* fix(kueb/namespaces): rename resource pool to namespace

* fix(kube/namespace): move selected registries

* fix(api/registries): hide accesses on registry creation

Co-authored-by: LP B <xAt0mZ@users.noreply.github.com>

refactor(api): remove code duplication after rebase

fix(app/registries): replace last registry api usage by endpoint registry api

fix(api/endpoints): update registry access policies on endpoint deletion (#5226)

[EE-1027]

fix(db): update db version

* fix(dockerhub): fetch rate limits

* fix(registry/tests): supply restricred context

* fix(registries): show proget registry only when selected

* fix(registry): create dockerhub registry

* feat(db): move migrations to db 32

Co-authored-by: Chaim Lev-Ari <chiptus@gmail.com>
2021-07-14 21:15:21 +12:00
cong meng a13ad8927f
fix(stack) ignore username and password when authentication is disabled EE-161 (#5222)
* fix(stack) ignore username and password when authentication is disabled EE-161

* fix(stack) ignore username and password when authentication is disabled for stack creation EE-161

Co-authored-by: Simon Meng <simon.meng@portainer.io>
2021-06-22 19:59:05 +12:00
cong meng 8e3751d0b7
fix(stack) Unable to update and redeploy a stack created from a git repository if it has failed once EE-1012 (#5212)
testing passed
2021-06-22 12:58:54 +12:00
Dmitry Salakhov 89f53458c6
fix(stack): allow standard users use advanced deployment (#5205) 2021-06-21 09:53:48 +12:00
Hui caa6c15032
feat(k8s): advanced deployment from Git repo EE-447 (#5166)
* feat(stack): UI updates in git repo deployment method for k8s EE-640. (#5097)

* feat(stack): UI updates in git repo deployment method for k8s EE-640.

* feat(stack): supports the combination of GIT + COMPOSE.

* feat(stack): rename variable

* feat(stack): add git repo deployment method for k8s EE-638

* cleanup

* update payload validation rules

* make repo ref optional in frond end

Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
2021-06-16 23:47:32 +02:00
Alice Groux 7f96220a09
feat(k8s/advanced-deployment): allow standard users to see and use advanced deployment feature EE-446 (#5050) 2021-06-16 17:28:44 +12:00
Dmitry Salakhov 0b93714de4
feat(stacks): redeploy git stack [EE-161] (#5139)
* feat(git): save git config when creating stack (#5048)

* feat(git): save git config when creating stack

* chore(fs): test fileExists

* fix(git): fix tests to use CloneRepository

* refactor(git): move options to new object

* feat(stacks): redeploy git stack api (#5112)

* feat(stacks): redeploy git stacks form

[EE-666]

* feat(stack): show loading after confirmation

* fix(stacks): show same size description

* fix(stacks): reload state when deployed

* feat(stacks): set stopped stacks status to activate when updating

* feat(stacks): backup stack folder before cloning

* feat(stacks): don't accept prune and env on update git

Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com>
Co-authored-by: Chaim Lev-Ari <chiptus@gmail.com>
2021-06-16 09:11:35 +12:00
cong meng 26ead28d7b
Feat(stacks): orphaned stacks #4397 (#4834)
* feat(stack): add the ability for an administrator user to manage orphaned stacks (#4397)

* feat(stack): apply small font size to the information text of associate (#4397)

Co-authored-by: Simon Meng <simon.meng@portainer.io>
2021-06-10 14:52:33 +12:00
cong meng defd929366
Fix(kube) advanced deployment CE-83 (#4866)
* refactor(http/kube): convert compose format

* feat(kube/deploy): deploy to agent

* feat(kube/deploy): show more details about error

* refactor(kube): return string from deploy

* feat(kube/deploy): revert to use local kubectl

* Revert "feat(kube/deploy): revert to use local kubectl"

This reverts commit 7c4a1c70

* feat(kube/deploy): GH#4321 use the v2 version of agent api instead of v3

Co-authored-by: Chaim Lev-Ari <chiptus@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
2021-06-09 01:55:17 +02:00
Lukas Grotz d09ae22ba8
feat(container): add sysctls setting in the container view (#4910)
* feat(container): add sysctls in the container view (#2756)

* feat(container): add setting to restrict sysctl access

* feat(endpoint): move sysctl disable setting to security settings

* feat(container): add sysctls to container edit view

* fix(container) remove unnecessary migration setting

Co-authored-by: Owen Kirby <oskirby@gmail.com>
2021-04-12 19:40:45 +12:00
Dmitry Salakhov 4cbd231a5f
fix: normalize stack name only for libcompose (#4862)
* fix: normilize stack name only for libcompose

* fix
2021-03-14 20:08:31 +01:00
Chaim Lev-Ari 86ad1c6af1
feat(stacks): scope stack names to endpoint (#4520)
* refactor(stack): create unique name function

* refactor(stack): change stack resource control id

* feat(stacks): validate stack unique name in endpoint

* feat(stacks): prevent name collision with external stacks

* refactor(stacks): move resource id util

* refactor(stacks): supply resource id util with name and endpoint

* fix(docker): calculate swarm resource id

* feat(stack): prevent migration if stack name already exist

* feat(authorization): use stackutils
2021-02-23 21:18:05 +01:00
Chaim Lev-Ari 50b57614cf
docs(api): document apis with swagger (#4678)
* feat(api): introduce swagger

* feat(api): anottate api

* chore(api): tag endpoints

* chore(api): remove tags

* chore(api): add docs for oauth auth

* chore(api): document create endpoint api

* chore(api): document endpoint inspect and list

* chore(api): document endpoint update and snapshots

* docs(endpointgroups): document groups api

* docs(auth): document auth api

* chore(build): introduce a yarn script to build api docs

* docs(api): document auth

* docs(customtemplates): document customtemplates api

* docs(tags): document api

* docs(api): document the use of token

* docs(dockerhub): document dockerhub api

* docs(edgegroups): document edgegroups api

* docs(edgejobs): document api

* docs(edgestacks): doc api

* docs(http/upload): add security

* docs(api): document edge templates

* docs(edge): document edge jobs

* docs(endpointgroups): change description

* docs(endpoints): document missing apis

* docs(motd): doc api

* docs(registries): doc api

* docs(resourcecontrol): api doc

* docs(role): add swagger docs

* docs(settings): add swagger docs

* docs(api/status): add swagger docs

* docs(api/teammembership): add swagger docs

* docs(api/teams): add swagger docs

* docs(api/templates): add swagger docs

* docs(api/users): add swagger docs

* docs(api/webhooks): add swagger docs

* docs(api/webscokets): add swagger docs

* docs(api/stacks): swagger

* docs(api): fix missing apis

* docs(swagger): regen

* chore(build): remove docs from build

* docs(api): update tags

* docs(api): document tags

* docs(api): add description

* docs(api): rename jwt token

* docs(api): add info about types

* docs(api): document types

* docs(api): update request types annotation

* docs(api): doc registry and resource control

* chore(docs): add snippet

* docs(api): add description to role

* docs(api): add types for settings

* docs(status): add types

* style(swagger): remove documented code

* docs(http/upload): update docs with types

* docs(http/tags): add types

* docs(api/custom_templates): add types

* docs(api/teammembership): add types

* docs(http/teams): add types

* docs(http/stacks): add types

* docs(edge): add types to edgestack

* docs(http/teammembership): remove double returns

* docs(api/user): add types

* docs(http): fixes to make file built

* chore(snippets): add scope to swagger snippet

* chore(deps): install swag

* chore(swagger): remove handler

* docs(api): add description

* docs(api): ignore docs folder

* docs(api): add contributing guidelines

* docs(api): cleanup handler

* chore(deps): require swaggo

* fix(auth): fix typo

* fix(docs): make http ids pascal case

* feat(edge): add ids to http handlers

* fix(docs): add ids

* fix(docs): show correct api version

* chore(deps): remove swaggo dependency

* chore(docs): add install script for swag
2021-02-23 16:21:39 +13:00
Chaim Lev-Ari 9bef81eef6
fix(stack): show correct error message (#4853) 2021-02-16 22:37:27 +01:00
Chaim Lev-Ari 46dec01fe3
feat(endpoint): relocate docker security settings (#4657)
* feat(endpoint): migrate security settings to endpoint

* feat(endpoint): check for specific endpoint settings

* feat(endpoint): check security settings

* feat(docker): add config page

* feat(endpoint): save settings page

* feat(endpoints): disable features when not agent

* feat(sidebar): hide docker settings for regular user

* fix(docker): small fixes in configs

* fix(volumes): hide browse button for non admins

* refactor(docker): introduce switch component

* refactor(components/switch): seprate label from switch

* feat(app/components): align switch label

* refactor(app/components): move switch css

* fix(docker/settings): add ngijnect

* feat(endpoints): set default security values

* style(portainer): sort types

* fix(endpoint): rename security heading

* fix(endpoints): update endpoints settings
2021-02-09 21:09:06 +13:00
Dmitry Salakhov a71e71f481
feat(compose): add docker-compose wrapper (#4713)
* feat(compose): add docker-compose wrapper

ce-187

* fix(compose): pick compose implementation upon startup

* Add static compose build for linux

* Fix wget

* Fix platofrm specific docker-compose download

* Keep amd64 architecture as download parameter

* Add tmp folder for docker-compose

* fix: line endings

* add proxy server

* logs

* Proxy

* Add lite transport for compose

* Fix local deployment

* refactor: pass proxyManager by ref

* fix: string conversion

* refactor: compose wrapper remove unused code

* fix: tests

* Add edge

* Fix merge issue

* refactor: remove unused code

* Move server to proxy implementation

* Cleanup wrapper and manager

* feat: pass max supported compose syntax version with each endpoint

* fix: pick compose syntax version

* fix: store wrapper version in portainer

* Get and show composeSyntaxMaxVersion at stack creation screen

* Get and show composeSyntaxMaxVersion at stack editor screen

* refactor: proxy server

* Fix used tmp

* Bump docker-compose to 1.28.0

* remove message for docker compose limitation

* fix: markup typo

* Rollback docker compose to 1.27.4

* * attempt to fix the windows build issue

* * attempt to debug grunt issue

* * use console log in grunt file

* fix: try to fix windows build by removing indirect deps from go.mod

* Remove tmp folder

* Remove builder stage

* feat(build/windows): add git for Docker Compose

* feat(build/windows): add git for Docker Compose

* feat(build/windows): add git for Docker Compose

* feat(build/windows): add git for Docker Compose

* feat(build/windows): add git for Docker Compose

* feat(build/windows): add git for Docker Compose - fixed verbose output

* refactor: renames

* fix(stack): get endpoint by EndpointProvider

* fix(stack): use margin to add space between line instead of using br tag

Co-authored-by: Stéphane Busso <stephane.busso@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
Co-authored-by: yi-portainer <yi.chen@portainer.io>
Co-authored-by: Steven Kang <skan070@gmail.com>
2021-01-26 08:16:53 +13:00
cong meng da41dbb79a
fix(stack): stacks created via API are incorrectly marked as private with no owner (#3721) (#4725)
Co-authored-by: Simon Meng <simon.meng@portainer.io>
2021-01-20 15:19:35 +13:00
Chaim Lev-Ari cbd7fdc62e
feat(docker/stacks): introduce date info for stacks (#4660)
* feat(docker/stacks): add creation and update dates

* feat(docker/stacks): put ownership column as the last column

* feat(docker/stacks): fix the no stacks message

* refactor(docker/stacks): make external stacks helpers more readable

* feat(docker/stacks): add updated and created by

* feat(docker/stacks): toggle updated column

* refactor(datatable): create column visibility component

Co-authored-by: alice groux <alice.grx@gmail.com>
2021-01-12 12:38:49 +13:00
Yi Chen 7975ef796d
Revert "feat(docker/stacks): add creation and update dates (#4418)" (#4606)
This reverts commit bd98b8956a.
2020-12-17 13:33:45 +13:00
Alice Groux bd98b8956a
feat(docker/stacks): add creation and update dates (#4418)
* feat(docker/stacks): add creation and update dates

* feat(docker/stacks): put ownership column as the last column

* feat(docker/stacks): fix the no stacks message
2020-12-16 16:11:59 +13:00