You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
portainer/api/apikey/service_test.go

310 lines
8.1 KiB

feat(api-key/backend): introducing support for api-key based auth EE-978 (#6079) * feat(access-token): Multi-auth middleware support EE-1891 (#5936) * AnyAuth middleware initial implementation with tests * using mux.MiddlewareFunc instead of custom definition * removed redundant comments * - ExtractBearerToken bouncer func made private - changed helm token handling functionality to use jwt service to convert token to jwt string - updated tests - fixed helm list broken test due to missing token in request context * rename mwCheckAuthentication -> mwCheckJWTAuthentication * - introduce initial api-key auth support using X-API-KEY header - added tests to validate x-api-key request header presence * updated core mwAuthenticatedUser middleware to support multiple auth paradigms * - simplified anyAuth middleware - enforcing authmiddleware to implement verificationFunc interface - created tests for middleware * simplify bouncer Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * feat(api-key): user-access-token generation endpoint EE-1889 EE-1888 EE-1895 (#6012) * user-access-token generation endpoint * fix comment * - introduction of apikey service - seperation of repository from service logic - called in handler * fixed tests * - fixed api key prefix - added tests * added another test for digest matching * updated swagger spec for access token creation * api key response returns raw key and struct - easing testability * test for api key prefix length * added another TODO to middleware * - api-key prefix rune -> string (rune does not auto-encode when response sent back to client) - digest -> pointer as we want to allow nil values and omit digest in responses (when nil) * - updated apikey struct - updated apikey service to support all common operations - updated apikey repo - integration of apikey service into bouncer - added test for all apikey service functions - boilerplate code for apikey service integration * - user access token generation tests - apiKeyLookup updated to support query params - added api-key tests for query params - added api-key tests for apiKeyLookup * get and remove access token handlers * get and remove access token handler tests * - delete user deletes all associated api keys - tests for this functionality * removed redundant []byte cast * automatic api-key eviction set within cache for 1 hour * fixed bug with loop var using final value * fixed service comment * ignore bolt error responses * case-insensitive query param check * simplified query var assignment * - added GetAPIKey func to get by unique id - updated DeleteAPIKey func to not require user ID - updated tests * GenerateRandomKey helper func from github.com/gorilla/securecookie moved to codebase * json response casing for api-keys fixed * updating api-key will update the cache * updated golang LRU cache * using hashicorps golang-LRU cache for api keys * simplified jwt check in create user access token * fixed api-key update logic on cache miss * Prefix generated api-keys with `ptr_` (#6067) * prefix api-keys with 'ptr_' * updated apikey description * refactor Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * helm list test refactor * fixed user delete test * reduce test nil pointer errors * using correct http 201 created status code for token creation; updated tests * fixed swagger doc user id path param for user access token based endpoints * added api-key security openapi spec to existing jwt secured endpoints (#6091) * fixed flaky test * apikey datecreated and lastused attrs converted to unix timestamp * feat(user): added access token datatable. (#6124) * feat(user): added access token datatable. * feat(tokens): only display lastUsed time when it is not the default date * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/components/datatables/access-tokens-datatable/accessTokensDatatableController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/services/api/userService.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(improvements): proposed datatable improvements to speed up dev time (#6138) * modal code update * updated datatable filenames, updated controller to be default class export * fix(access-token): code improvement. Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(apikeys): create access token view initial implementation EE-1886 (#6129) * CopyButton implementation * Code component implementation * ToolTip component migration to another folder * TextTip component implementation - continued * form Heading component * Button component updated to be more dynamic * copybutton - small size * form control pass tip error * texttip small text * CreateAccessToken react feature initial implementation * create user access token angularjs view implementation * registration of CreateAccessToken component in AngularJS * user token generation API request moved to angular service, method passed down instead * consistent naming of access token operations; clustered similar code together * any user can add access token * create access token page routing * moved code component to the correct location * removed isadmin check as all functionality applicable to all users * create access token angular view moved up a level * fixed PR issues, updated PR * addressed PR issues/improvements * explicit hr for horizontal line * fixed merge conflict storybook build breaking * - apikey test - cache test * addressed testing issues: - description validations - remove token description link on table * fix(api-keys): user role change evicts user keys in cache EE-2113 (#6168) * user role change evicts user api keys in cache * EvictUserKeyCache -> InvalidateUserKeyCache * godoc for InvalidateUserKeyCache func * additional test line * disable add access token button after adding token to prevent spam Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
3 years ago
package apikey
import (
"crypto/sha256"
"log"
"strings"
"testing"
"time"
portainer "github.com/portainer/portainer/api"
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173) * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix app templates symbol (#6221) * feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220) * feat(webhook) EE-2125 add some helpers to registry utils * feat(webhook) EE-2125 persist registryID when creating a webhook * feat(webhook) EE-2125 send registry auth header when executing a webhook * feat(webhook) EE-2125 send registryID to backend when creating a service with webhook * feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen * feat(webhook) EE-2125 update webhook when update registry * feat(webhook) EE-2125 add endpoint of update webhook * feat(webhook) EE-2125 code cleanup * feat(webhook) EE-2125 fix a typo * feat(webhook) EE-2125 fix circle import issue with unit test Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204) Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix data-cy for k8s cluster menu (#6226) LGTM * feat(stack): make stack created from app template editable EE-1941 (#6104) feat(stack): make stack from app template editable * fix(container):disable Duplicate/Edit button when the container is portainer (#6223) * fix/ee-1909/show-pull-image-error (#6195) Co-authored-by: sunportainer <ericsun@SG1.local> * feat(cy): add data-cy to helm install button (#6241) * feat(cy): add data-cy to add registry button (#6242) * refactor(app): convert root folder files to es6 (#4159) * refactor(app): duplicate constants as es6 exports (#4158) * fix(docker): provide workaround to save network name variable (#6080) * fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file * fix/EE-1862/unable-to-stop-or-remove-stack check yaml file * fixed func and var names * wrapper error and used bool for stringset * UT case for createNetworkEnvFile * UT case for %s=%s * powerful StringSet * wrapper error for extract network name * wrapper all the return err * store more env * put to env file * make default value None * feat: gzip static resources (#6258) * fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203) * fix/ee-2106/handle-sslcert-sslkey Co-authored-by: sunportainer <ericsun@SG1.local> * fix(server):support disable https only ee-2068 (#6232) * fix/ee-2068/disable-forcely-https * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * Update missing helm user repo methods * remove redundant comments * add webhook export * update webhooks * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Update missing helm user repo methods * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * remove redundant comments * add webhook export * update webhooks * fix build issues after rebasing * move migratorparams * remove unneeded integer type conversions * disable the db import/export for now Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com> Co-authored-by: cong meng <mcpacino@gmail.com> Co-authored-by: Simon Meng <simon.meng@portainer.io> Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com> Co-authored-by: Hao Zhang <hao.zhang@portainer.io> Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com> Co-authored-by: sunportainer <ericsun@SG1.local> Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com> Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com> Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com> Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: Matt Hook <hookenz@gmail.com>
3 years ago
"github.com/portainer/portainer/api/datastore"
feat(api-key/backend): introducing support for api-key based auth EE-978 (#6079) * feat(access-token): Multi-auth middleware support EE-1891 (#5936) * AnyAuth middleware initial implementation with tests * using mux.MiddlewareFunc instead of custom definition * removed redundant comments * - ExtractBearerToken bouncer func made private - changed helm token handling functionality to use jwt service to convert token to jwt string - updated tests - fixed helm list broken test due to missing token in request context * rename mwCheckAuthentication -> mwCheckJWTAuthentication * - introduce initial api-key auth support using X-API-KEY header - added tests to validate x-api-key request header presence * updated core mwAuthenticatedUser middleware to support multiple auth paradigms * - simplified anyAuth middleware - enforcing authmiddleware to implement verificationFunc interface - created tests for middleware * simplify bouncer Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * feat(api-key): user-access-token generation endpoint EE-1889 EE-1888 EE-1895 (#6012) * user-access-token generation endpoint * fix comment * - introduction of apikey service - seperation of repository from service logic - called in handler * fixed tests * - fixed api key prefix - added tests * added another test for digest matching * updated swagger spec for access token creation * api key response returns raw key and struct - easing testability * test for api key prefix length * added another TODO to middleware * - api-key prefix rune -> string (rune does not auto-encode when response sent back to client) - digest -> pointer as we want to allow nil values and omit digest in responses (when nil) * - updated apikey struct - updated apikey service to support all common operations - updated apikey repo - integration of apikey service into bouncer - added test for all apikey service functions - boilerplate code for apikey service integration * - user access token generation tests - apiKeyLookup updated to support query params - added api-key tests for query params - added api-key tests for apiKeyLookup * get and remove access token handlers * get and remove access token handler tests * - delete user deletes all associated api keys - tests for this functionality * removed redundant []byte cast * automatic api-key eviction set within cache for 1 hour * fixed bug with loop var using final value * fixed service comment * ignore bolt error responses * case-insensitive query param check * simplified query var assignment * - added GetAPIKey func to get by unique id - updated DeleteAPIKey func to not require user ID - updated tests * GenerateRandomKey helper func from github.com/gorilla/securecookie moved to codebase * json response casing for api-keys fixed * updating api-key will update the cache * updated golang LRU cache * using hashicorps golang-LRU cache for api keys * simplified jwt check in create user access token * fixed api-key update logic on cache miss * Prefix generated api-keys with `ptr_` (#6067) * prefix api-keys with 'ptr_' * updated apikey description * refactor Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * helm list test refactor * fixed user delete test * reduce test nil pointer errors * using correct http 201 created status code for token creation; updated tests * fixed swagger doc user id path param for user access token based endpoints * added api-key security openapi spec to existing jwt secured endpoints (#6091) * fixed flaky test * apikey datecreated and lastused attrs converted to unix timestamp * feat(user): added access token datatable. (#6124) * feat(user): added access token datatable. * feat(tokens): only display lastUsed time when it is not the default date * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/components/datatables/access-tokens-datatable/accessTokensDatatableController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/services/api/userService.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(improvements): proposed datatable improvements to speed up dev time (#6138) * modal code update * updated datatable filenames, updated controller to be default class export * fix(access-token): code improvement. Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(apikeys): create access token view initial implementation EE-1886 (#6129) * CopyButton implementation * Code component implementation * ToolTip component migration to another folder * TextTip component implementation - continued * form Heading component * Button component updated to be more dynamic * copybutton - small size * form control pass tip error * texttip small text * CreateAccessToken react feature initial implementation * create user access token angularjs view implementation * registration of CreateAccessToken component in AngularJS * user token generation API request moved to angular service, method passed down instead * consistent naming of access token operations; clustered similar code together * any user can add access token * create access token page routing * moved code component to the correct location * removed isadmin check as all functionality applicable to all users * create access token angular view moved up a level * fixed PR issues, updated PR * addressed PR issues/improvements * explicit hr for horizontal line * fixed merge conflict storybook build breaking * - apikey test - cache test * addressed testing issues: - description validations - remove token description link on table * fix(api-keys): user role change evicts user keys in cache EE-2113 (#6168) * user role change evicts user api keys in cache * EvictUserKeyCache -> InvalidateUserKeyCache * godoc for InvalidateUserKeyCache func * additional test line * disable add access token button after adding token to prevent spam Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
3 years ago
"github.com/stretchr/testify/assert"
)
func Test_SatisfiesAPIKeyServiceInterface(t *testing.T) {
is := assert.New(t)
is.Implements((*APIKeyService)(nil), NewAPIKeyService(nil, nil))
}
func Test_GenerateApiKey(t *testing.T) {
is := assert.New(t)
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173) * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix app templates symbol (#6221) * feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220) * feat(webhook) EE-2125 add some helpers to registry utils * feat(webhook) EE-2125 persist registryID when creating a webhook * feat(webhook) EE-2125 send registry auth header when executing a webhook * feat(webhook) EE-2125 send registryID to backend when creating a service with webhook * feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen * feat(webhook) EE-2125 update webhook when update registry * feat(webhook) EE-2125 add endpoint of update webhook * feat(webhook) EE-2125 code cleanup * feat(webhook) EE-2125 fix a typo * feat(webhook) EE-2125 fix circle import issue with unit test Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204) Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix data-cy for k8s cluster menu (#6226) LGTM * feat(stack): make stack created from app template editable EE-1941 (#6104) feat(stack): make stack from app template editable * fix(container):disable Duplicate/Edit button when the container is portainer (#6223) * fix/ee-1909/show-pull-image-error (#6195) Co-authored-by: sunportainer <ericsun@SG1.local> * feat(cy): add data-cy to helm install button (#6241) * feat(cy): add data-cy to add registry button (#6242) * refactor(app): convert root folder files to es6 (#4159) * refactor(app): duplicate constants as es6 exports (#4158) * fix(docker): provide workaround to save network name variable (#6080) * fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file * fix/EE-1862/unable-to-stop-or-remove-stack check yaml file * fixed func and var names * wrapper error and used bool for stringset * UT case for createNetworkEnvFile * UT case for %s=%s * powerful StringSet * wrapper error for extract network name * wrapper all the return err * store more env * put to env file * make default value None * feat: gzip static resources (#6258) * fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203) * fix/ee-2106/handle-sslcert-sslkey Co-authored-by: sunportainer <ericsun@SG1.local> * fix(server):support disable https only ee-2068 (#6232) * fix/ee-2068/disable-forcely-https * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * Update missing helm user repo methods * remove redundant comments * add webhook export * update webhooks * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Update missing helm user repo methods * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * remove redundant comments * add webhook export * update webhooks * fix build issues after rebasing * move migratorparams * remove unneeded integer type conversions * disable the db import/export for now Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com> Co-authored-by: cong meng <mcpacino@gmail.com> Co-authored-by: Simon Meng <simon.meng@portainer.io> Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com> Co-authored-by: Hao Zhang <hao.zhang@portainer.io> Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com> Co-authored-by: sunportainer <ericsun@SG1.local> Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com> Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com> Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com> Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: Matt Hook <hookenz@gmail.com>
3 years ago
_, store, teardown := datastore.MustNewTestStore(true)
feat(api-key/backend): introducing support for api-key based auth EE-978 (#6079) * feat(access-token): Multi-auth middleware support EE-1891 (#5936) * AnyAuth middleware initial implementation with tests * using mux.MiddlewareFunc instead of custom definition * removed redundant comments * - ExtractBearerToken bouncer func made private - changed helm token handling functionality to use jwt service to convert token to jwt string - updated tests - fixed helm list broken test due to missing token in request context * rename mwCheckAuthentication -> mwCheckJWTAuthentication * - introduce initial api-key auth support using X-API-KEY header - added tests to validate x-api-key request header presence * updated core mwAuthenticatedUser middleware to support multiple auth paradigms * - simplified anyAuth middleware - enforcing authmiddleware to implement verificationFunc interface - created tests for middleware * simplify bouncer Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * feat(api-key): user-access-token generation endpoint EE-1889 EE-1888 EE-1895 (#6012) * user-access-token generation endpoint * fix comment * - introduction of apikey service - seperation of repository from service logic - called in handler * fixed tests * - fixed api key prefix - added tests * added another test for digest matching * updated swagger spec for access token creation * api key response returns raw key and struct - easing testability * test for api key prefix length * added another TODO to middleware * - api-key prefix rune -> string (rune does not auto-encode when response sent back to client) - digest -> pointer as we want to allow nil values and omit digest in responses (when nil) * - updated apikey struct - updated apikey service to support all common operations - updated apikey repo - integration of apikey service into bouncer - added test for all apikey service functions - boilerplate code for apikey service integration * - user access token generation tests - apiKeyLookup updated to support query params - added api-key tests for query params - added api-key tests for apiKeyLookup * get and remove access token handlers * get and remove access token handler tests * - delete user deletes all associated api keys - tests for this functionality * removed redundant []byte cast * automatic api-key eviction set within cache for 1 hour * fixed bug with loop var using final value * fixed service comment * ignore bolt error responses * case-insensitive query param check * simplified query var assignment * - added GetAPIKey func to get by unique id - updated DeleteAPIKey func to not require user ID - updated tests * GenerateRandomKey helper func from github.com/gorilla/securecookie moved to codebase * json response casing for api-keys fixed * updating api-key will update the cache * updated golang LRU cache * using hashicorps golang-LRU cache for api keys * simplified jwt check in create user access token * fixed api-key update logic on cache miss * Prefix generated api-keys with `ptr_` (#6067) * prefix api-keys with 'ptr_' * updated apikey description * refactor Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * helm list test refactor * fixed user delete test * reduce test nil pointer errors * using correct http 201 created status code for token creation; updated tests * fixed swagger doc user id path param for user access token based endpoints * added api-key security openapi spec to existing jwt secured endpoints (#6091) * fixed flaky test * apikey datecreated and lastused attrs converted to unix timestamp * feat(user): added access token datatable. (#6124) * feat(user): added access token datatable. * feat(tokens): only display lastUsed time when it is not the default date * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/components/datatables/access-tokens-datatable/accessTokensDatatableController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/services/api/userService.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(improvements): proposed datatable improvements to speed up dev time (#6138) * modal code update * updated datatable filenames, updated controller to be default class export * fix(access-token): code improvement. Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(apikeys): create access token view initial implementation EE-1886 (#6129) * CopyButton implementation * Code component implementation * ToolTip component migration to another folder * TextTip component implementation - continued * form Heading component * Button component updated to be more dynamic * copybutton - small size * form control pass tip error * texttip small text * CreateAccessToken react feature initial implementation * create user access token angularjs view implementation * registration of CreateAccessToken component in AngularJS * user token generation API request moved to angular service, method passed down instead * consistent naming of access token operations; clustered similar code together * any user can add access token * create access token page routing * moved code component to the correct location * removed isadmin check as all functionality applicable to all users * create access token angular view moved up a level * fixed PR issues, updated PR * addressed PR issues/improvements * explicit hr for horizontal line * fixed merge conflict storybook build breaking * - apikey test - cache test * addressed testing issues: - description validations - remove token description link on table * fix(api-keys): user role change evicts user keys in cache EE-2113 (#6168) * user role change evicts user api keys in cache * EvictUserKeyCache -> InvalidateUserKeyCache * godoc for InvalidateUserKeyCache func * additional test line * disable add access token button after adding token to prevent spam Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
3 years ago
defer teardown()
service := NewAPIKeyService(store.APIKeyRepository(), store.User())
t.Run("Successfully generates API key", func(t *testing.T) {
desc := "test-1"
rawKey, apiKey, err := service.GenerateApiKey(portainer.User{ID: 1}, desc)
is.NoError(err)
is.NotEmpty(rawKey)
is.NotEmpty(apiKey)
is.Equal(desc, apiKey.Description)
})
t.Run("Api key prefix is 7 chars", func(t *testing.T) {
rawKey, apiKey, err := service.GenerateApiKey(portainer.User{ID: 1}, "test-2")
is.NoError(err)
is.Equal(rawKey[:7], apiKey.Prefix)
is.Len(apiKey.Prefix, 7)
})
t.Run("Api key has 'ptr_' as prefix", func(t *testing.T) {
rawKey, _, err := service.GenerateApiKey(portainer.User{ID: 1}, "test-x")
is.NoError(err)
is.Equal(portainerAPIKeyPrefix, "ptr_")
is.True(strings.HasPrefix(rawKey, "ptr_"))
})
t.Run("Successfully caches API key", func(t *testing.T) {
user := portainer.User{ID: 1}
_, apiKey, err := service.GenerateApiKey(user, "test-3")
is.NoError(err)
userFromCache, apiKeyFromCache, ok := service.cache.Get(apiKey.Digest)
is.True(ok)
is.Equal(user, userFromCache)
is.Equal(apiKey, &apiKeyFromCache)
})
t.Run("Decoded raw api-key digest matches generated digest", func(t *testing.T) {
rawKey, apiKey, err := service.GenerateApiKey(portainer.User{ID: 1}, "test-4")
is.NoError(err)
generatedDigest := sha256.Sum256([]byte(rawKey))
is.Equal(apiKey.Digest, generatedDigest[:])
})
}
func Test_GetAPIKey(t *testing.T) {
is := assert.New(t)
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173) * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix app templates symbol (#6221) * feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220) * feat(webhook) EE-2125 add some helpers to registry utils * feat(webhook) EE-2125 persist registryID when creating a webhook * feat(webhook) EE-2125 send registry auth header when executing a webhook * feat(webhook) EE-2125 send registryID to backend when creating a service with webhook * feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen * feat(webhook) EE-2125 update webhook when update registry * feat(webhook) EE-2125 add endpoint of update webhook * feat(webhook) EE-2125 code cleanup * feat(webhook) EE-2125 fix a typo * feat(webhook) EE-2125 fix circle import issue with unit test Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204) Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix data-cy for k8s cluster menu (#6226) LGTM * feat(stack): make stack created from app template editable EE-1941 (#6104) feat(stack): make stack from app template editable * fix(container):disable Duplicate/Edit button when the container is portainer (#6223) * fix/ee-1909/show-pull-image-error (#6195) Co-authored-by: sunportainer <ericsun@SG1.local> * feat(cy): add data-cy to helm install button (#6241) * feat(cy): add data-cy to add registry button (#6242) * refactor(app): convert root folder files to es6 (#4159) * refactor(app): duplicate constants as es6 exports (#4158) * fix(docker): provide workaround to save network name variable (#6080) * fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file * fix/EE-1862/unable-to-stop-or-remove-stack check yaml file * fixed func and var names * wrapper error and used bool for stringset * UT case for createNetworkEnvFile * UT case for %s=%s * powerful StringSet * wrapper error for extract network name * wrapper all the return err * store more env * put to env file * make default value None * feat: gzip static resources (#6258) * fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203) * fix/ee-2106/handle-sslcert-sslkey Co-authored-by: sunportainer <ericsun@SG1.local> * fix(server):support disable https only ee-2068 (#6232) * fix/ee-2068/disable-forcely-https * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * Update missing helm user repo methods * remove redundant comments * add webhook export * update webhooks * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Update missing helm user repo methods * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * remove redundant comments * add webhook export * update webhooks * fix build issues after rebasing * move migratorparams * remove unneeded integer type conversions * disable the db import/export for now Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com> Co-authored-by: cong meng <mcpacino@gmail.com> Co-authored-by: Simon Meng <simon.meng@portainer.io> Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com> Co-authored-by: Hao Zhang <hao.zhang@portainer.io> Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com> Co-authored-by: sunportainer <ericsun@SG1.local> Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com> Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com> Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com> Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: Matt Hook <hookenz@gmail.com>
3 years ago
_, store, teardown := datastore.MustNewTestStore(true)
defer teardown()
service := NewAPIKeyService(store.APIKeyRepository(), store.User())
t.Run("Successfully returns all API keys", func(t *testing.T) {
user := portainer.User{ID: 1}
_, apiKey, err := service.GenerateApiKey(user, "test-1")
is.NoError(err)
apiKeyGot, err := service.GetAPIKey(apiKey.ID)
is.NoError(err)
is.Equal(apiKey, apiKeyGot)
})
}
feat(api-key/backend): introducing support for api-key based auth EE-978 (#6079) * feat(access-token): Multi-auth middleware support EE-1891 (#5936) * AnyAuth middleware initial implementation with tests * using mux.MiddlewareFunc instead of custom definition * removed redundant comments * - ExtractBearerToken bouncer func made private - changed helm token handling functionality to use jwt service to convert token to jwt string - updated tests - fixed helm list broken test due to missing token in request context * rename mwCheckAuthentication -> mwCheckJWTAuthentication * - introduce initial api-key auth support using X-API-KEY header - added tests to validate x-api-key request header presence * updated core mwAuthenticatedUser middleware to support multiple auth paradigms * - simplified anyAuth middleware - enforcing authmiddleware to implement verificationFunc interface - created tests for middleware * simplify bouncer Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * feat(api-key): user-access-token generation endpoint EE-1889 EE-1888 EE-1895 (#6012) * user-access-token generation endpoint * fix comment * - introduction of apikey service - seperation of repository from service logic - called in handler * fixed tests * - fixed api key prefix - added tests * added another test for digest matching * updated swagger spec for access token creation * api key response returns raw key and struct - easing testability * test for api key prefix length * added another TODO to middleware * - api-key prefix rune -> string (rune does not auto-encode when response sent back to client) - digest -> pointer as we want to allow nil values and omit digest in responses (when nil) * - updated apikey struct - updated apikey service to support all common operations - updated apikey repo - integration of apikey service into bouncer - added test for all apikey service functions - boilerplate code for apikey service integration * - user access token generation tests - apiKeyLookup updated to support query params - added api-key tests for query params - added api-key tests for apiKeyLookup * get and remove access token handlers * get and remove access token handler tests * - delete user deletes all associated api keys - tests for this functionality * removed redundant []byte cast * automatic api-key eviction set within cache for 1 hour * fixed bug with loop var using final value * fixed service comment * ignore bolt error responses * case-insensitive query param check * simplified query var assignment * - added GetAPIKey func to get by unique id - updated DeleteAPIKey func to not require user ID - updated tests * GenerateRandomKey helper func from github.com/gorilla/securecookie moved to codebase * json response casing for api-keys fixed * updating api-key will update the cache * updated golang LRU cache * using hashicorps golang-LRU cache for api keys * simplified jwt check in create user access token * fixed api-key update logic on cache miss * Prefix generated api-keys with `ptr_` (#6067) * prefix api-keys with 'ptr_' * updated apikey description * refactor Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * helm list test refactor * fixed user delete test * reduce test nil pointer errors * using correct http 201 created status code for token creation; updated tests * fixed swagger doc user id path param for user access token based endpoints * added api-key security openapi spec to existing jwt secured endpoints (#6091) * fixed flaky test * apikey datecreated and lastused attrs converted to unix timestamp * feat(user): added access token datatable. (#6124) * feat(user): added access token datatable. * feat(tokens): only display lastUsed time when it is not the default date * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/components/datatables/access-tokens-datatable/accessTokensDatatableController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/services/api/userService.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(improvements): proposed datatable improvements to speed up dev time (#6138) * modal code update * updated datatable filenames, updated controller to be default class export * fix(access-token): code improvement. Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(apikeys): create access token view initial implementation EE-1886 (#6129) * CopyButton implementation * Code component implementation * ToolTip component migration to another folder * TextTip component implementation - continued * form Heading component * Button component updated to be more dynamic * copybutton - small size * form control pass tip error * texttip small text * CreateAccessToken react feature initial implementation * create user access token angularjs view implementation * registration of CreateAccessToken component in AngularJS * user token generation API request moved to angular service, method passed down instead * consistent naming of access token operations; clustered similar code together * any user can add access token * create access token page routing * moved code component to the correct location * removed isadmin check as all functionality applicable to all users * create access token angular view moved up a level * fixed PR issues, updated PR * addressed PR issues/improvements * explicit hr for horizontal line * fixed merge conflict storybook build breaking * - apikey test - cache test * addressed testing issues: - description validations - remove token description link on table * fix(api-keys): user role change evicts user keys in cache EE-2113 (#6168) * user role change evicts user api keys in cache * EvictUserKeyCache -> InvalidateUserKeyCache * godoc for InvalidateUserKeyCache func * additional test line * disable add access token button after adding token to prevent spam Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
3 years ago
func Test_GetAPIKeys(t *testing.T) {
is := assert.New(t)
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173) * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix app templates symbol (#6221) * feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220) * feat(webhook) EE-2125 add some helpers to registry utils * feat(webhook) EE-2125 persist registryID when creating a webhook * feat(webhook) EE-2125 send registry auth header when executing a webhook * feat(webhook) EE-2125 send registryID to backend when creating a service with webhook * feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen * feat(webhook) EE-2125 update webhook when update registry * feat(webhook) EE-2125 add endpoint of update webhook * feat(webhook) EE-2125 code cleanup * feat(webhook) EE-2125 fix a typo * feat(webhook) EE-2125 fix circle import issue with unit test Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204) Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix data-cy for k8s cluster menu (#6226) LGTM * feat(stack): make stack created from app template editable EE-1941 (#6104) feat(stack): make stack from app template editable * fix(container):disable Duplicate/Edit button when the container is portainer (#6223) * fix/ee-1909/show-pull-image-error (#6195) Co-authored-by: sunportainer <ericsun@SG1.local> * feat(cy): add data-cy to helm install button (#6241) * feat(cy): add data-cy to add registry button (#6242) * refactor(app): convert root folder files to es6 (#4159) * refactor(app): duplicate constants as es6 exports (#4158) * fix(docker): provide workaround to save network name variable (#6080) * fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file * fix/EE-1862/unable-to-stop-or-remove-stack check yaml file * fixed func and var names * wrapper error and used bool for stringset * UT case for createNetworkEnvFile * UT case for %s=%s * powerful StringSet * wrapper error for extract network name * wrapper all the return err * store more env * put to env file * make default value None * feat: gzip static resources (#6258) * fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203) * fix/ee-2106/handle-sslcert-sslkey Co-authored-by: sunportainer <ericsun@SG1.local> * fix(server):support disable https only ee-2068 (#6232) * fix/ee-2068/disable-forcely-https * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * Update missing helm user repo methods * remove redundant comments * add webhook export * update webhooks * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Update missing helm user repo methods * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * remove redundant comments * add webhook export * update webhooks * fix build issues after rebasing * move migratorparams * remove unneeded integer type conversions * disable the db import/export for now Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com> Co-authored-by: cong meng <mcpacino@gmail.com> Co-authored-by: Simon Meng <simon.meng@portainer.io> Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com> Co-authored-by: Hao Zhang <hao.zhang@portainer.io> Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com> Co-authored-by: sunportainer <ericsun@SG1.local> Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com> Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com> Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com> Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: Matt Hook <hookenz@gmail.com>
3 years ago
_, store, teardown := datastore.MustNewTestStore(true)
feat(api-key/backend): introducing support for api-key based auth EE-978 (#6079) * feat(access-token): Multi-auth middleware support EE-1891 (#5936) * AnyAuth middleware initial implementation with tests * using mux.MiddlewareFunc instead of custom definition * removed redundant comments * - ExtractBearerToken bouncer func made private - changed helm token handling functionality to use jwt service to convert token to jwt string - updated tests - fixed helm list broken test due to missing token in request context * rename mwCheckAuthentication -> mwCheckJWTAuthentication * - introduce initial api-key auth support using X-API-KEY header - added tests to validate x-api-key request header presence * updated core mwAuthenticatedUser middleware to support multiple auth paradigms * - simplified anyAuth middleware - enforcing authmiddleware to implement verificationFunc interface - created tests for middleware * simplify bouncer Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * feat(api-key): user-access-token generation endpoint EE-1889 EE-1888 EE-1895 (#6012) * user-access-token generation endpoint * fix comment * - introduction of apikey service - seperation of repository from service logic - called in handler * fixed tests * - fixed api key prefix - added tests * added another test for digest matching * updated swagger spec for access token creation * api key response returns raw key and struct - easing testability * test for api key prefix length * added another TODO to middleware * - api-key prefix rune -> string (rune does not auto-encode when response sent back to client) - digest -> pointer as we want to allow nil values and omit digest in responses (when nil) * - updated apikey struct - updated apikey service to support all common operations - updated apikey repo - integration of apikey service into bouncer - added test for all apikey service functions - boilerplate code for apikey service integration * - user access token generation tests - apiKeyLookup updated to support query params - added api-key tests for query params - added api-key tests for apiKeyLookup * get and remove access token handlers * get and remove access token handler tests * - delete user deletes all associated api keys - tests for this functionality * removed redundant []byte cast * automatic api-key eviction set within cache for 1 hour * fixed bug with loop var using final value * fixed service comment * ignore bolt error responses * case-insensitive query param check * simplified query var assignment * - added GetAPIKey func to get by unique id - updated DeleteAPIKey func to not require user ID - updated tests * GenerateRandomKey helper func from github.com/gorilla/securecookie moved to codebase * json response casing for api-keys fixed * updating api-key will update the cache * updated golang LRU cache * using hashicorps golang-LRU cache for api keys * simplified jwt check in create user access token * fixed api-key update logic on cache miss * Prefix generated api-keys with `ptr_` (#6067) * prefix api-keys with 'ptr_' * updated apikey description * refactor Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * helm list test refactor * fixed user delete test * reduce test nil pointer errors * using correct http 201 created status code for token creation; updated tests * fixed swagger doc user id path param for user access token based endpoints * added api-key security openapi spec to existing jwt secured endpoints (#6091) * fixed flaky test * apikey datecreated and lastused attrs converted to unix timestamp * feat(user): added access token datatable. (#6124) * feat(user): added access token datatable. * feat(tokens): only display lastUsed time when it is not the default date * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/components/datatables/access-tokens-datatable/accessTokensDatatableController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/services/api/userService.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(improvements): proposed datatable improvements to speed up dev time (#6138) * modal code update * updated datatable filenames, updated controller to be default class export * fix(access-token): code improvement. Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(apikeys): create access token view initial implementation EE-1886 (#6129) * CopyButton implementation * Code component implementation * ToolTip component migration to another folder * TextTip component implementation - continued * form Heading component * Button component updated to be more dynamic * copybutton - small size * form control pass tip error * texttip small text * CreateAccessToken react feature initial implementation * create user access token angularjs view implementation * registration of CreateAccessToken component in AngularJS * user token generation API request moved to angular service, method passed down instead * consistent naming of access token operations; clustered similar code together * any user can add access token * create access token page routing * moved code component to the correct location * removed isadmin check as all functionality applicable to all users * create access token angular view moved up a level * fixed PR issues, updated PR * addressed PR issues/improvements * explicit hr for horizontal line * fixed merge conflict storybook build breaking * - apikey test - cache test * addressed testing issues: - description validations - remove token description link on table * fix(api-keys): user role change evicts user keys in cache EE-2113 (#6168) * user role change evicts user api keys in cache * EvictUserKeyCache -> InvalidateUserKeyCache * godoc for InvalidateUserKeyCache func * additional test line * disable add access token button after adding token to prevent spam Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
3 years ago
defer teardown()
service := NewAPIKeyService(store.APIKeyRepository(), store.User())
t.Run("Successfully returns all API keys", func(t *testing.T) {
user := portainer.User{ID: 1}
_, _, err := service.GenerateApiKey(user, "test-1")
is.NoError(err)
_, _, err = service.GenerateApiKey(user, "test-2")
is.NoError(err)
keys, err := service.GetAPIKeys(user.ID)
is.NoError(err)
is.Len(keys, 2)
})
}
func Test_GetDigestUserAndKey(t *testing.T) {
is := assert.New(t)
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173) * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix app templates symbol (#6221) * feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220) * feat(webhook) EE-2125 add some helpers to registry utils * feat(webhook) EE-2125 persist registryID when creating a webhook * feat(webhook) EE-2125 send registry auth header when executing a webhook * feat(webhook) EE-2125 send registryID to backend when creating a service with webhook * feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen * feat(webhook) EE-2125 update webhook when update registry * feat(webhook) EE-2125 add endpoint of update webhook * feat(webhook) EE-2125 code cleanup * feat(webhook) EE-2125 fix a typo * feat(webhook) EE-2125 fix circle import issue with unit test Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204) Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix data-cy for k8s cluster menu (#6226) LGTM * feat(stack): make stack created from app template editable EE-1941 (#6104) feat(stack): make stack from app template editable * fix(container):disable Duplicate/Edit button when the container is portainer (#6223) * fix/ee-1909/show-pull-image-error (#6195) Co-authored-by: sunportainer <ericsun@SG1.local> * feat(cy): add data-cy to helm install button (#6241) * feat(cy): add data-cy to add registry button (#6242) * refactor(app): convert root folder files to es6 (#4159) * refactor(app): duplicate constants as es6 exports (#4158) * fix(docker): provide workaround to save network name variable (#6080) * fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file * fix/EE-1862/unable-to-stop-or-remove-stack check yaml file * fixed func and var names * wrapper error and used bool for stringset * UT case for createNetworkEnvFile * UT case for %s=%s * powerful StringSet * wrapper error for extract network name * wrapper all the return err * store more env * put to env file * make default value None * feat: gzip static resources (#6258) * fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203) * fix/ee-2106/handle-sslcert-sslkey Co-authored-by: sunportainer <ericsun@SG1.local> * fix(server):support disable https only ee-2068 (#6232) * fix/ee-2068/disable-forcely-https * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * Update missing helm user repo methods * remove redundant comments * add webhook export * update webhooks * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Update missing helm user repo methods * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * remove redundant comments * add webhook export * update webhooks * fix build issues after rebasing * move migratorparams * remove unneeded integer type conversions * disable the db import/export for now Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com> Co-authored-by: cong meng <mcpacino@gmail.com> Co-authored-by: Simon Meng <simon.meng@portainer.io> Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com> Co-authored-by: Hao Zhang <hao.zhang@portainer.io> Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com> Co-authored-by: sunportainer <ericsun@SG1.local> Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com> Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com> Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com> Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: Matt Hook <hookenz@gmail.com>
3 years ago
_, store, teardown := datastore.MustNewTestStore(true)
feat(api-key/backend): introducing support for api-key based auth EE-978 (#6079) * feat(access-token): Multi-auth middleware support EE-1891 (#5936) * AnyAuth middleware initial implementation with tests * using mux.MiddlewareFunc instead of custom definition * removed redundant comments * - ExtractBearerToken bouncer func made private - changed helm token handling functionality to use jwt service to convert token to jwt string - updated tests - fixed helm list broken test due to missing token in request context * rename mwCheckAuthentication -> mwCheckJWTAuthentication * - introduce initial api-key auth support using X-API-KEY header - added tests to validate x-api-key request header presence * updated core mwAuthenticatedUser middleware to support multiple auth paradigms * - simplified anyAuth middleware - enforcing authmiddleware to implement verificationFunc interface - created tests for middleware * simplify bouncer Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * feat(api-key): user-access-token generation endpoint EE-1889 EE-1888 EE-1895 (#6012) * user-access-token generation endpoint * fix comment * - introduction of apikey service - seperation of repository from service logic - called in handler * fixed tests * - fixed api key prefix - added tests * added another test for digest matching * updated swagger spec for access token creation * api key response returns raw key and struct - easing testability * test for api key prefix length * added another TODO to middleware * - api-key prefix rune -> string (rune does not auto-encode when response sent back to client) - digest -> pointer as we want to allow nil values and omit digest in responses (when nil) * - updated apikey struct - updated apikey service to support all common operations - updated apikey repo - integration of apikey service into bouncer - added test for all apikey service functions - boilerplate code for apikey service integration * - user access token generation tests - apiKeyLookup updated to support query params - added api-key tests for query params - added api-key tests for apiKeyLookup * get and remove access token handlers * get and remove access token handler tests * - delete user deletes all associated api keys - tests for this functionality * removed redundant []byte cast * automatic api-key eviction set within cache for 1 hour * fixed bug with loop var using final value * fixed service comment * ignore bolt error responses * case-insensitive query param check * simplified query var assignment * - added GetAPIKey func to get by unique id - updated DeleteAPIKey func to not require user ID - updated tests * GenerateRandomKey helper func from github.com/gorilla/securecookie moved to codebase * json response casing for api-keys fixed * updating api-key will update the cache * updated golang LRU cache * using hashicorps golang-LRU cache for api keys * simplified jwt check in create user access token * fixed api-key update logic on cache miss * Prefix generated api-keys with `ptr_` (#6067) * prefix api-keys with 'ptr_' * updated apikey description * refactor Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * helm list test refactor * fixed user delete test * reduce test nil pointer errors * using correct http 201 created status code for token creation; updated tests * fixed swagger doc user id path param for user access token based endpoints * added api-key security openapi spec to existing jwt secured endpoints (#6091) * fixed flaky test * apikey datecreated and lastused attrs converted to unix timestamp * feat(user): added access token datatable. (#6124) * feat(user): added access token datatable. * feat(tokens): only display lastUsed time when it is not the default date * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/components/datatables/access-tokens-datatable/accessTokensDatatableController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/services/api/userService.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(improvements): proposed datatable improvements to speed up dev time (#6138) * modal code update * updated datatable filenames, updated controller to be default class export * fix(access-token): code improvement. Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(apikeys): create access token view initial implementation EE-1886 (#6129) * CopyButton implementation * Code component implementation * ToolTip component migration to another folder * TextTip component implementation - continued * form Heading component * Button component updated to be more dynamic * copybutton - small size * form control pass tip error * texttip small text * CreateAccessToken react feature initial implementation * create user access token angularjs view implementation * registration of CreateAccessToken component in AngularJS * user token generation API request moved to angular service, method passed down instead * consistent naming of access token operations; clustered similar code together * any user can add access token * create access token page routing * moved code component to the correct location * removed isadmin check as all functionality applicable to all users * create access token angular view moved up a level * fixed PR issues, updated PR * addressed PR issues/improvements * explicit hr for horizontal line * fixed merge conflict storybook build breaking * - apikey test - cache test * addressed testing issues: - description validations - remove token description link on table * fix(api-keys): user role change evicts user keys in cache EE-2113 (#6168) * user role change evicts user api keys in cache * EvictUserKeyCache -> InvalidateUserKeyCache * godoc for InvalidateUserKeyCache func * additional test line * disable add access token button after adding token to prevent spam Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
3 years ago
defer teardown()
service := NewAPIKeyService(store.APIKeyRepository(), store.User())
t.Run("Successfully returns user and api key associated to digest", func(t *testing.T) {
user := portainer.User{ID: 1}
_, apiKey, err := service.GenerateApiKey(user, "test-1")
is.NoError(err)
userGot, apiKeyGot, err := service.GetDigestUserAndKey(apiKey.Digest)
is.NoError(err)
is.Equal(user, userGot)
is.Equal(*apiKey, apiKeyGot)
})
t.Run("Successfully caches user and api key associated to digest", func(t *testing.T) {
user := portainer.User{ID: 1}
_, apiKey, err := service.GenerateApiKey(user, "test-1")
is.NoError(err)
userGot, apiKeyGot, err := service.GetDigestUserAndKey(apiKey.Digest)
is.NoError(err)
is.Equal(user, userGot)
is.Equal(*apiKey, apiKeyGot)
userFromCache, apiKeyFromCache, ok := service.cache.Get(apiKey.Digest)
is.True(ok)
is.Equal(userGot, userFromCache)
is.Equal(apiKeyGot, apiKeyFromCache)
})
}
func Test_UpdateAPIKey(t *testing.T) {
is := assert.New(t)
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173) * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix app templates symbol (#6221) * feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220) * feat(webhook) EE-2125 add some helpers to registry utils * feat(webhook) EE-2125 persist registryID when creating a webhook * feat(webhook) EE-2125 send registry auth header when executing a webhook * feat(webhook) EE-2125 send registryID to backend when creating a service with webhook * feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen * feat(webhook) EE-2125 update webhook when update registry * feat(webhook) EE-2125 add endpoint of update webhook * feat(webhook) EE-2125 code cleanup * feat(webhook) EE-2125 fix a typo * feat(webhook) EE-2125 fix circle import issue with unit test Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204) Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix data-cy for k8s cluster menu (#6226) LGTM * feat(stack): make stack created from app template editable EE-1941 (#6104) feat(stack): make stack from app template editable * fix(container):disable Duplicate/Edit button when the container is portainer (#6223) * fix/ee-1909/show-pull-image-error (#6195) Co-authored-by: sunportainer <ericsun@SG1.local> * feat(cy): add data-cy to helm install button (#6241) * feat(cy): add data-cy to add registry button (#6242) * refactor(app): convert root folder files to es6 (#4159) * refactor(app): duplicate constants as es6 exports (#4158) * fix(docker): provide workaround to save network name variable (#6080) * fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file * fix/EE-1862/unable-to-stop-or-remove-stack check yaml file * fixed func and var names * wrapper error and used bool for stringset * UT case for createNetworkEnvFile * UT case for %s=%s * powerful StringSet * wrapper error for extract network name * wrapper all the return err * store more env * put to env file * make default value None * feat: gzip static resources (#6258) * fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203) * fix/ee-2106/handle-sslcert-sslkey Co-authored-by: sunportainer <ericsun@SG1.local> * fix(server):support disable https only ee-2068 (#6232) * fix/ee-2068/disable-forcely-https * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * Update missing helm user repo methods * remove redundant comments * add webhook export * update webhooks * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Update missing helm user repo methods * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * remove redundant comments * add webhook export * update webhooks * fix build issues after rebasing * move migratorparams * remove unneeded integer type conversions * disable the db import/export for now Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com> Co-authored-by: cong meng <mcpacino@gmail.com> Co-authored-by: Simon Meng <simon.meng@portainer.io> Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com> Co-authored-by: Hao Zhang <hao.zhang@portainer.io> Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com> Co-authored-by: sunportainer <ericsun@SG1.local> Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com> Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com> Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com> Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: Matt Hook <hookenz@gmail.com>
3 years ago
_, store, teardown := datastore.MustNewTestStore(true)
feat(api-key/backend): introducing support for api-key based auth EE-978 (#6079) * feat(access-token): Multi-auth middleware support EE-1891 (#5936) * AnyAuth middleware initial implementation with tests * using mux.MiddlewareFunc instead of custom definition * removed redundant comments * - ExtractBearerToken bouncer func made private - changed helm token handling functionality to use jwt service to convert token to jwt string - updated tests - fixed helm list broken test due to missing token in request context * rename mwCheckAuthentication -> mwCheckJWTAuthentication * - introduce initial api-key auth support using X-API-KEY header - added tests to validate x-api-key request header presence * updated core mwAuthenticatedUser middleware to support multiple auth paradigms * - simplified anyAuth middleware - enforcing authmiddleware to implement verificationFunc interface - created tests for middleware * simplify bouncer Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * feat(api-key): user-access-token generation endpoint EE-1889 EE-1888 EE-1895 (#6012) * user-access-token generation endpoint * fix comment * - introduction of apikey service - seperation of repository from service logic - called in handler * fixed tests * - fixed api key prefix - added tests * added another test for digest matching * updated swagger spec for access token creation * api key response returns raw key and struct - easing testability * test for api key prefix length * added another TODO to middleware * - api-key prefix rune -> string (rune does not auto-encode when response sent back to client) - digest -> pointer as we want to allow nil values and omit digest in responses (when nil) * - updated apikey struct - updated apikey service to support all common operations - updated apikey repo - integration of apikey service into bouncer - added test for all apikey service functions - boilerplate code for apikey service integration * - user access token generation tests - apiKeyLookup updated to support query params - added api-key tests for query params - added api-key tests for apiKeyLookup * get and remove access token handlers * get and remove access token handler tests * - delete user deletes all associated api keys - tests for this functionality * removed redundant []byte cast * automatic api-key eviction set within cache for 1 hour * fixed bug with loop var using final value * fixed service comment * ignore bolt error responses * case-insensitive query param check * simplified query var assignment * - added GetAPIKey func to get by unique id - updated DeleteAPIKey func to not require user ID - updated tests * GenerateRandomKey helper func from github.com/gorilla/securecookie moved to codebase * json response casing for api-keys fixed * updating api-key will update the cache * updated golang LRU cache * using hashicorps golang-LRU cache for api keys * simplified jwt check in create user access token * fixed api-key update logic on cache miss * Prefix generated api-keys with `ptr_` (#6067) * prefix api-keys with 'ptr_' * updated apikey description * refactor Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * helm list test refactor * fixed user delete test * reduce test nil pointer errors * using correct http 201 created status code for token creation; updated tests * fixed swagger doc user id path param for user access token based endpoints * added api-key security openapi spec to existing jwt secured endpoints (#6091) * fixed flaky test * apikey datecreated and lastused attrs converted to unix timestamp * feat(user): added access token datatable. (#6124) * feat(user): added access token datatable. * feat(tokens): only display lastUsed time when it is not the default date * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/components/datatables/access-tokens-datatable/accessTokensDatatableController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/services/api/userService.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(improvements): proposed datatable improvements to speed up dev time (#6138) * modal code update * updated datatable filenames, updated controller to be default class export * fix(access-token): code improvement. Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(apikeys): create access token view initial implementation EE-1886 (#6129) * CopyButton implementation * Code component implementation * ToolTip component migration to another folder * TextTip component implementation - continued * form Heading component * Button component updated to be more dynamic * copybutton - small size * form control pass tip error * texttip small text * CreateAccessToken react feature initial implementation * create user access token angularjs view implementation * registration of CreateAccessToken component in AngularJS * user token generation API request moved to angular service, method passed down instead * consistent naming of access token operations; clustered similar code together * any user can add access token * create access token page routing * moved code component to the correct location * removed isadmin check as all functionality applicable to all users * create access token angular view moved up a level * fixed PR issues, updated PR * addressed PR issues/improvements * explicit hr for horizontal line * fixed merge conflict storybook build breaking * - apikey test - cache test * addressed testing issues: - description validations - remove token description link on table * fix(api-keys): user role change evicts user keys in cache EE-2113 (#6168) * user role change evicts user api keys in cache * EvictUserKeyCache -> InvalidateUserKeyCache * godoc for InvalidateUserKeyCache func * additional test line * disable add access token button after adding token to prevent spam Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
3 years ago
defer teardown()
service := NewAPIKeyService(store.APIKeyRepository(), store.User())
t.Run("Successfully updates the api-key LastUsed time", func(t *testing.T) {
user := portainer.User{ID: 1}
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173) * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix app templates symbol (#6221) * feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220) * feat(webhook) EE-2125 add some helpers to registry utils * feat(webhook) EE-2125 persist registryID when creating a webhook * feat(webhook) EE-2125 send registry auth header when executing a webhook * feat(webhook) EE-2125 send registryID to backend when creating a service with webhook * feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen * feat(webhook) EE-2125 update webhook when update registry * feat(webhook) EE-2125 add endpoint of update webhook * feat(webhook) EE-2125 code cleanup * feat(webhook) EE-2125 fix a typo * feat(webhook) EE-2125 fix circle import issue with unit test Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204) Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix data-cy for k8s cluster menu (#6226) LGTM * feat(stack): make stack created from app template editable EE-1941 (#6104) feat(stack): make stack from app template editable * fix(container):disable Duplicate/Edit button when the container is portainer (#6223) * fix/ee-1909/show-pull-image-error (#6195) Co-authored-by: sunportainer <ericsun@SG1.local> * feat(cy): add data-cy to helm install button (#6241) * feat(cy): add data-cy to add registry button (#6242) * refactor(app): convert root folder files to es6 (#4159) * refactor(app): duplicate constants as es6 exports (#4158) * fix(docker): provide workaround to save network name variable (#6080) * fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file * fix/EE-1862/unable-to-stop-or-remove-stack check yaml file * fixed func and var names * wrapper error and used bool for stringset * UT case for createNetworkEnvFile * UT case for %s=%s * powerful StringSet * wrapper error for extract network name * wrapper all the return err * store more env * put to env file * make default value None * feat: gzip static resources (#6258) * fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203) * fix/ee-2106/handle-sslcert-sslkey Co-authored-by: sunportainer <ericsun@SG1.local> * fix(server):support disable https only ee-2068 (#6232) * fix/ee-2068/disable-forcely-https * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * Update missing helm user repo methods * remove redundant comments * add webhook export * update webhooks * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Update missing helm user repo methods * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * remove redundant comments * add webhook export * update webhooks * fix build issues after rebasing * move migratorparams * remove unneeded integer type conversions * disable the db import/export for now Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com> Co-authored-by: cong meng <mcpacino@gmail.com> Co-authored-by: Simon Meng <simon.meng@portainer.io> Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com> Co-authored-by: Hao Zhang <hao.zhang@portainer.io> Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com> Co-authored-by: sunportainer <ericsun@SG1.local> Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com> Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com> Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com> Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: Matt Hook <hookenz@gmail.com>
3 years ago
store.User().Create(&user)
feat(api-key/backend): introducing support for api-key based auth EE-978 (#6079) * feat(access-token): Multi-auth middleware support EE-1891 (#5936) * AnyAuth middleware initial implementation with tests * using mux.MiddlewareFunc instead of custom definition * removed redundant comments * - ExtractBearerToken bouncer func made private - changed helm token handling functionality to use jwt service to convert token to jwt string - updated tests - fixed helm list broken test due to missing token in request context * rename mwCheckAuthentication -> mwCheckJWTAuthentication * - introduce initial api-key auth support using X-API-KEY header - added tests to validate x-api-key request header presence * updated core mwAuthenticatedUser middleware to support multiple auth paradigms * - simplified anyAuth middleware - enforcing authmiddleware to implement verificationFunc interface - created tests for middleware * simplify bouncer Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * feat(api-key): user-access-token generation endpoint EE-1889 EE-1888 EE-1895 (#6012) * user-access-token generation endpoint * fix comment * - introduction of apikey service - seperation of repository from service logic - called in handler * fixed tests * - fixed api key prefix - added tests * added another test for digest matching * updated swagger spec for access token creation * api key response returns raw key and struct - easing testability * test for api key prefix length * added another TODO to middleware * - api-key prefix rune -> string (rune does not auto-encode when response sent back to client) - digest -> pointer as we want to allow nil values and omit digest in responses (when nil) * - updated apikey struct - updated apikey service to support all common operations - updated apikey repo - integration of apikey service into bouncer - added test for all apikey service functions - boilerplate code for apikey service integration * - user access token generation tests - apiKeyLookup updated to support query params - added api-key tests for query params - added api-key tests for apiKeyLookup * get and remove access token handlers * get and remove access token handler tests * - delete user deletes all associated api keys - tests for this functionality * removed redundant []byte cast * automatic api-key eviction set within cache for 1 hour * fixed bug with loop var using final value * fixed service comment * ignore bolt error responses * case-insensitive query param check * simplified query var assignment * - added GetAPIKey func to get by unique id - updated DeleteAPIKey func to not require user ID - updated tests * GenerateRandomKey helper func from github.com/gorilla/securecookie moved to codebase * json response casing for api-keys fixed * updating api-key will update the cache * updated golang LRU cache * using hashicorps golang-LRU cache for api keys * simplified jwt check in create user access token * fixed api-key update logic on cache miss * Prefix generated api-keys with `ptr_` (#6067) * prefix api-keys with 'ptr_' * updated apikey description * refactor Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * helm list test refactor * fixed user delete test * reduce test nil pointer errors * using correct http 201 created status code for token creation; updated tests * fixed swagger doc user id path param for user access token based endpoints * added api-key security openapi spec to existing jwt secured endpoints (#6091) * fixed flaky test * apikey datecreated and lastused attrs converted to unix timestamp * feat(user): added access token datatable. (#6124) * feat(user): added access token datatable. * feat(tokens): only display lastUsed time when it is not the default date * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/components/datatables/access-tokens-datatable/accessTokensDatatableController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/services/api/userService.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(improvements): proposed datatable improvements to speed up dev time (#6138) * modal code update * updated datatable filenames, updated controller to be default class export * fix(access-token): code improvement. Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(apikeys): create access token view initial implementation EE-1886 (#6129) * CopyButton implementation * Code component implementation * ToolTip component migration to another folder * TextTip component implementation - continued * form Heading component * Button component updated to be more dynamic * copybutton - small size * form control pass tip error * texttip small text * CreateAccessToken react feature initial implementation * create user access token angularjs view implementation * registration of CreateAccessToken component in AngularJS * user token generation API request moved to angular service, method passed down instead * consistent naming of access token operations; clustered similar code together * any user can add access token * create access token page routing * moved code component to the correct location * removed isadmin check as all functionality applicable to all users * create access token angular view moved up a level * fixed PR issues, updated PR * addressed PR issues/improvements * explicit hr for horizontal line * fixed merge conflict storybook build breaking * - apikey test - cache test * addressed testing issues: - description validations - remove token description link on table * fix(api-keys): user role change evicts user keys in cache EE-2113 (#6168) * user role change evicts user api keys in cache * EvictUserKeyCache -> InvalidateUserKeyCache * godoc for InvalidateUserKeyCache func * additional test line * disable add access token button after adding token to prevent spam Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
3 years ago
_, apiKey, err := service.GenerateApiKey(user, "test-x")
is.NoError(err)
apiKey.LastUsed = time.Now().UTC().Unix()
err = service.UpdateAPIKey(apiKey)
is.NoError(err)
_, apiKeyGot, err := service.GetDigestUserAndKey(apiKey.Digest)
is.NoError(err)
log.Println(apiKey)
log.Println(apiKeyGot)
is.Equal(apiKey.LastUsed, apiKeyGot.LastUsed)
})
t.Run("Successfully updates api-key in cache upon api-key update", func(t *testing.T) {
_, apiKey, err := service.GenerateApiKey(portainer.User{ID: 1}, "test-x2")
is.NoError(err)
_, apiKeyFromCache, ok := service.cache.Get(apiKey.Digest)
is.True(ok)
is.Equal(*apiKey, apiKeyFromCache)
apiKey.LastUsed = time.Now().UTC().Unix()
is.NotEqual(*apiKey, apiKeyFromCache)
err = service.UpdateAPIKey(apiKey)
is.NoError(err)
_, updatedAPIKeyFromCache, ok := service.cache.Get(apiKey.Digest)
is.True(ok)
is.Equal(*apiKey, updatedAPIKeyFromCache)
})
}
func Test_DeleteAPIKey(t *testing.T) {
is := assert.New(t)
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173) * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix app templates symbol (#6221) * feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220) * feat(webhook) EE-2125 add some helpers to registry utils * feat(webhook) EE-2125 persist registryID when creating a webhook * feat(webhook) EE-2125 send registry auth header when executing a webhook * feat(webhook) EE-2125 send registryID to backend when creating a service with webhook * feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen * feat(webhook) EE-2125 update webhook when update registry * feat(webhook) EE-2125 add endpoint of update webhook * feat(webhook) EE-2125 code cleanup * feat(webhook) EE-2125 fix a typo * feat(webhook) EE-2125 fix circle import issue with unit test Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204) Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix data-cy for k8s cluster menu (#6226) LGTM * feat(stack): make stack created from app template editable EE-1941 (#6104) feat(stack): make stack from app template editable * fix(container):disable Duplicate/Edit button when the container is portainer (#6223) * fix/ee-1909/show-pull-image-error (#6195) Co-authored-by: sunportainer <ericsun@SG1.local> * feat(cy): add data-cy to helm install button (#6241) * feat(cy): add data-cy to add registry button (#6242) * refactor(app): convert root folder files to es6 (#4159) * refactor(app): duplicate constants as es6 exports (#4158) * fix(docker): provide workaround to save network name variable (#6080) * fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file * fix/EE-1862/unable-to-stop-or-remove-stack check yaml file * fixed func and var names * wrapper error and used bool for stringset * UT case for createNetworkEnvFile * UT case for %s=%s * powerful StringSet * wrapper error for extract network name * wrapper all the return err * store more env * put to env file * make default value None * feat: gzip static resources (#6258) * fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203) * fix/ee-2106/handle-sslcert-sslkey Co-authored-by: sunportainer <ericsun@SG1.local> * fix(server):support disable https only ee-2068 (#6232) * fix/ee-2068/disable-forcely-https * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * Update missing helm user repo methods * remove redundant comments * add webhook export * update webhooks * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Update missing helm user repo methods * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * remove redundant comments * add webhook export * update webhooks * fix build issues after rebasing * move migratorparams * remove unneeded integer type conversions * disable the db import/export for now Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com> Co-authored-by: cong meng <mcpacino@gmail.com> Co-authored-by: Simon Meng <simon.meng@portainer.io> Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com> Co-authored-by: Hao Zhang <hao.zhang@portainer.io> Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com> Co-authored-by: sunportainer <ericsun@SG1.local> Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com> Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com> Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com> Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: Matt Hook <hookenz@gmail.com>
3 years ago
_, store, teardown := datastore.MustNewTestStore(true)
feat(api-key/backend): introducing support for api-key based auth EE-978 (#6079) * feat(access-token): Multi-auth middleware support EE-1891 (#5936) * AnyAuth middleware initial implementation with tests * using mux.MiddlewareFunc instead of custom definition * removed redundant comments * - ExtractBearerToken bouncer func made private - changed helm token handling functionality to use jwt service to convert token to jwt string - updated tests - fixed helm list broken test due to missing token in request context * rename mwCheckAuthentication -> mwCheckJWTAuthentication * - introduce initial api-key auth support using X-API-KEY header - added tests to validate x-api-key request header presence * updated core mwAuthenticatedUser middleware to support multiple auth paradigms * - simplified anyAuth middleware - enforcing authmiddleware to implement verificationFunc interface - created tests for middleware * simplify bouncer Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * feat(api-key): user-access-token generation endpoint EE-1889 EE-1888 EE-1895 (#6012) * user-access-token generation endpoint * fix comment * - introduction of apikey service - seperation of repository from service logic - called in handler * fixed tests * - fixed api key prefix - added tests * added another test for digest matching * updated swagger spec for access token creation * api key response returns raw key and struct - easing testability * test for api key prefix length * added another TODO to middleware * - api-key prefix rune -> string (rune does not auto-encode when response sent back to client) - digest -> pointer as we want to allow nil values and omit digest in responses (when nil) * - updated apikey struct - updated apikey service to support all common operations - updated apikey repo - integration of apikey service into bouncer - added test for all apikey service functions - boilerplate code for apikey service integration * - user access token generation tests - apiKeyLookup updated to support query params - added api-key tests for query params - added api-key tests for apiKeyLookup * get and remove access token handlers * get and remove access token handler tests * - delete user deletes all associated api keys - tests for this functionality * removed redundant []byte cast * automatic api-key eviction set within cache for 1 hour * fixed bug with loop var using final value * fixed service comment * ignore bolt error responses * case-insensitive query param check * simplified query var assignment * - added GetAPIKey func to get by unique id - updated DeleteAPIKey func to not require user ID - updated tests * GenerateRandomKey helper func from github.com/gorilla/securecookie moved to codebase * json response casing for api-keys fixed * updating api-key will update the cache * updated golang LRU cache * using hashicorps golang-LRU cache for api keys * simplified jwt check in create user access token * fixed api-key update logic on cache miss * Prefix generated api-keys with `ptr_` (#6067) * prefix api-keys with 'ptr_' * updated apikey description * refactor Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * helm list test refactor * fixed user delete test * reduce test nil pointer errors * using correct http 201 created status code for token creation; updated tests * fixed swagger doc user id path param for user access token based endpoints * added api-key security openapi spec to existing jwt secured endpoints (#6091) * fixed flaky test * apikey datecreated and lastused attrs converted to unix timestamp * feat(user): added access token datatable. (#6124) * feat(user): added access token datatable. * feat(tokens): only display lastUsed time when it is not the default date * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/components/datatables/access-tokens-datatable/accessTokensDatatableController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/services/api/userService.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(improvements): proposed datatable improvements to speed up dev time (#6138) * modal code update * updated datatable filenames, updated controller to be default class export * fix(access-token): code improvement. Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(apikeys): create access token view initial implementation EE-1886 (#6129) * CopyButton implementation * Code component implementation * ToolTip component migration to another folder * TextTip component implementation - continued * form Heading component * Button component updated to be more dynamic * copybutton - small size * form control pass tip error * texttip small text * CreateAccessToken react feature initial implementation * create user access token angularjs view implementation * registration of CreateAccessToken component in AngularJS * user token generation API request moved to angular service, method passed down instead * consistent naming of access token operations; clustered similar code together * any user can add access token * create access token page routing * moved code component to the correct location * removed isadmin check as all functionality applicable to all users * create access token angular view moved up a level * fixed PR issues, updated PR * addressed PR issues/improvements * explicit hr for horizontal line * fixed merge conflict storybook build breaking * - apikey test - cache test * addressed testing issues: - description validations - remove token description link on table * fix(api-keys): user role change evicts user keys in cache EE-2113 (#6168) * user role change evicts user api keys in cache * EvictUserKeyCache -> InvalidateUserKeyCache * godoc for InvalidateUserKeyCache func * additional test line * disable add access token button after adding token to prevent spam Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
3 years ago
defer teardown()
service := NewAPIKeyService(store.APIKeyRepository(), store.User())
t.Run("Successfully updates the api-key", func(t *testing.T) {
user := portainer.User{ID: 1}
_, apiKey, err := service.GenerateApiKey(user, "test-1")
is.NoError(err)
_, apiKeyGot, err := service.GetDigestUserAndKey(apiKey.Digest)
is.NoError(err)
is.Equal(*apiKey, apiKeyGot)
err = service.DeleteAPIKey(apiKey.ID)
is.NoError(err)
_, _, err = service.GetDigestUserAndKey(apiKey.Digest)
is.Error(err)
})
t.Run("Successfully removes api-key from cache upon deletion", func(t *testing.T) {
user := portainer.User{ID: 1}
_, apiKey, err := service.GenerateApiKey(user, "test-1")
is.NoError(err)
_, apiKeyFromCache, ok := service.cache.Get(apiKey.Digest)
is.True(ok)
is.Equal(*apiKey, apiKeyFromCache)
err = service.DeleteAPIKey(apiKey.ID)
is.NoError(err)
_, _, ok = service.cache.Get(apiKey.Digest)
is.False(ok)
})
}
func Test_InvalidateUserKeyCache(t *testing.T) {
is := assert.New(t)
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173) * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix app templates symbol (#6221) * feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220) * feat(webhook) EE-2125 add some helpers to registry utils * feat(webhook) EE-2125 persist registryID when creating a webhook * feat(webhook) EE-2125 send registry auth header when executing a webhook * feat(webhook) EE-2125 send registryID to backend when creating a service with webhook * feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen * feat(webhook) EE-2125 update webhook when update registry * feat(webhook) EE-2125 add endpoint of update webhook * feat(webhook) EE-2125 code cleanup * feat(webhook) EE-2125 fix a typo * feat(webhook) EE-2125 fix circle import issue with unit test Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204) Co-authored-by: Simon Meng <simon.meng@portainer.io> * fix data-cy for k8s cluster menu (#6226) LGTM * feat(stack): make stack created from app template editable EE-1941 (#6104) feat(stack): make stack from app template editable * fix(container):disable Duplicate/Edit button when the container is portainer (#6223) * fix/ee-1909/show-pull-image-error (#6195) Co-authored-by: sunportainer <ericsun@SG1.local> * feat(cy): add data-cy to helm install button (#6241) * feat(cy): add data-cy to add registry button (#6242) * refactor(app): convert root folder files to es6 (#4159) * refactor(app): duplicate constants as es6 exports (#4158) * fix(docker): provide workaround to save network name variable (#6080) * fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file * fix/EE-1862/unable-to-stop-or-remove-stack check yaml file * fixed func and var names * wrapper error and used bool for stringset * UT case for createNetworkEnvFile * UT case for %s=%s * powerful StringSet * wrapper error for extract network name * wrapper all the return err * store more env * put to env file * make default value None * feat: gzip static resources (#6258) * fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203) * fix/ee-2106/handle-sslcert-sslkey Co-authored-by: sunportainer <ericsun@SG1.local> * fix(server):support disable https only ee-2068 (#6232) * fix/ee-2068/disable-forcely-https * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * Update missing helm user repo methods * remove redundant comments * add webhook export * update webhooks * use the Store interface IsErrObjectNotFound() to avoid revealing internal errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * what happens when you extract the datastore interfaces into their own package Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Start renaming Storage methods Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the boltdb specific code from the Portainer storage code (example, the others need the same) Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extract bolt.Tx from datastore code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * minimise imports by putting moving the struct definition into the file that needs the Service imports Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more extraction of boltdb.Tx Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the use of bucket.SetSequence Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * almost done - just endpoint.Synchonise :/ Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring' Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix test compile errors Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * test compile fixes after rebase Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * set us up to make the connection an interface Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make the db connection a datastore interface, and separate out our datastore services from the bolt ones Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * rename methods to something less oltdb internals specific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * these errors are not boltdb secific Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * start using the db-backend factory method too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * export boltdb raw in case we can't export from the service layer Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add the version info by hand for now Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * actually, the export from services can be fully typed - its the import that needs to do more work Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * redo raw export, and make import capable of using it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add DockerHub Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * fix go test ./... again Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * my goland wasn't setup to gofmt Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the two extremely dubious migration tests down into store, so they can use the test store code Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * the migrator is now free of boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * reverse goland overzealous replcement of internal with boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * more undo over-zealous goland internal->boltdb Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * yay, now bolt is only mentioned inside the api/database/ dir Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * and this might be the last of the boltdb references? Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * add todo Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * extract the store code into a separate module too Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * don't need the fileService in boltdb anymore Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use IsErrObjectNotFound() Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * use a string to select what database backend we use Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * make isNew store an ephemeral bool that doesn't stay true after we've initialised it Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * undo vscode formatting html Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io> * Update missing helm user repo methods * feat(store): implement store tests EE-2112 (#6224) * add store tests * add some more tests * remove redundant comments * add webhook export * update webhooks * fix build issues after rebasing * move migratorparams * remove unneeded integer type conversions * disable the db import/export for now Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com> Co-authored-by: cong meng <mcpacino@gmail.com> Co-authored-by: Simon Meng <simon.meng@portainer.io> Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com> Co-authored-by: Hao Zhang <hao.zhang@portainer.io> Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com> Co-authored-by: sunportainer <ericsun@SG1.local> Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com> Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com> Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com> Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: Matt Hook <hookenz@gmail.com>
3 years ago
_, store, teardown := datastore.MustNewTestStore(true)
feat(api-key/backend): introducing support for api-key based auth EE-978 (#6079) * feat(access-token): Multi-auth middleware support EE-1891 (#5936) * AnyAuth middleware initial implementation with tests * using mux.MiddlewareFunc instead of custom definition * removed redundant comments * - ExtractBearerToken bouncer func made private - changed helm token handling functionality to use jwt service to convert token to jwt string - updated tests - fixed helm list broken test due to missing token in request context * rename mwCheckAuthentication -> mwCheckJWTAuthentication * - introduce initial api-key auth support using X-API-KEY header - added tests to validate x-api-key request header presence * updated core mwAuthenticatedUser middleware to support multiple auth paradigms * - simplified anyAuth middleware - enforcing authmiddleware to implement verificationFunc interface - created tests for middleware * simplify bouncer Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * feat(api-key): user-access-token generation endpoint EE-1889 EE-1888 EE-1895 (#6012) * user-access-token generation endpoint * fix comment * - introduction of apikey service - seperation of repository from service logic - called in handler * fixed tests * - fixed api key prefix - added tests * added another test for digest matching * updated swagger spec for access token creation * api key response returns raw key and struct - easing testability * test for api key prefix length * added another TODO to middleware * - api-key prefix rune -> string (rune does not auto-encode when response sent back to client) - digest -> pointer as we want to allow nil values and omit digest in responses (when nil) * - updated apikey struct - updated apikey service to support all common operations - updated apikey repo - integration of apikey service into bouncer - added test for all apikey service functions - boilerplate code for apikey service integration * - user access token generation tests - apiKeyLookup updated to support query params - added api-key tests for query params - added api-key tests for apiKeyLookup * get and remove access token handlers * get and remove access token handler tests * - delete user deletes all associated api keys - tests for this functionality * removed redundant []byte cast * automatic api-key eviction set within cache for 1 hour * fixed bug with loop var using final value * fixed service comment * ignore bolt error responses * case-insensitive query param check * simplified query var assignment * - added GetAPIKey func to get by unique id - updated DeleteAPIKey func to not require user ID - updated tests * GenerateRandomKey helper func from github.com/gorilla/securecookie moved to codebase * json response casing for api-keys fixed * updating api-key will update the cache * updated golang LRU cache * using hashicorps golang-LRU cache for api keys * simplified jwt check in create user access token * fixed api-key update logic on cache miss * Prefix generated api-keys with `ptr_` (#6067) * prefix api-keys with 'ptr_' * updated apikey description * refactor Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> * helm list test refactor * fixed user delete test * reduce test nil pointer errors * using correct http 201 created status code for token creation; updated tests * fixed swagger doc user id path param for user access token based endpoints * added api-key security openapi spec to existing jwt secured endpoints (#6091) * fixed flaky test * apikey datecreated and lastused attrs converted to unix timestamp * feat(user): added access token datatable. (#6124) * feat(user): added access token datatable. * feat(tokens): only display lastUsed time when it is not the default date * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/views/account/accountController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/components/datatables/access-tokens-datatable/accessTokensDatatableController.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * Update app/portainer/services/api/userService.js Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(improvements): proposed datatable improvements to speed up dev time (#6138) * modal code update * updated datatable filenames, updated controller to be default class export * fix(access-token): code improvement. Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com> * feat(apikeys): create access token view initial implementation EE-1886 (#6129) * CopyButton implementation * Code component implementation * ToolTip component migration to another folder * TextTip component implementation - continued * form Heading component * Button component updated to be more dynamic * copybutton - small size * form control pass tip error * texttip small text * CreateAccessToken react feature initial implementation * create user access token angularjs view implementation * registration of CreateAccessToken component in AngularJS * user token generation API request moved to angular service, method passed down instead * consistent naming of access token operations; clustered similar code together * any user can add access token * create access token page routing * moved code component to the correct location * removed isadmin check as all functionality applicable to all users * create access token angular view moved up a level * fixed PR issues, updated PR * addressed PR issues/improvements * explicit hr for horizontal line * fixed merge conflict storybook build breaking * - apikey test - cache test * addressed testing issues: - description validations - remove token description link on table * fix(api-keys): user role change evicts user keys in cache EE-2113 (#6168) * user role change evicts user api keys in cache * EvictUserKeyCache -> InvalidateUserKeyCache * godoc for InvalidateUserKeyCache func * additional test line * disable add access token button after adding token to prevent spam Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com> Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
3 years ago
defer teardown()
service := NewAPIKeyService(store.APIKeyRepository(), store.User())
t.Run("Successfully updates evicts keys from cache", func(t *testing.T) {
// generate api keys
user := portainer.User{ID: 1}
_, apiKey1, err := service.GenerateApiKey(user, "test-1")
is.NoError(err)
_, apiKey2, err := service.GenerateApiKey(user, "test-2")
is.NoError(err)
// verify api keys are present in cache
_, apiKeyFromCache, ok := service.cache.Get(apiKey1.Digest)
is.True(ok)
is.Equal(*apiKey1, apiKeyFromCache)
_, apiKeyFromCache, ok = service.cache.Get(apiKey2.Digest)
is.True(ok)
is.Equal(*apiKey2, apiKeyFromCache)
// evict cache
ok = service.InvalidateUserKeyCache(user.ID)
is.True(ok)
// verify users keys have been flushed from cache
_, _, ok = service.cache.Get(apiKey1.Digest)
is.False(ok)
_, _, ok = service.cache.Get(apiKey2.Digest)
is.False(ok)
})
t.Run("User key eviction does not affect other users keys", func(t *testing.T) {
// generate keys for 2 users
user1 := portainer.User{ID: 1}
_, apiKey1, err := service.GenerateApiKey(user1, "test-1")
is.NoError(err)
user2 := portainer.User{ID: 2}
_, apiKey2, err := service.GenerateApiKey(user2, "test-2")
is.NoError(err)
// verify keys in cache
_, apiKeyFromCache, ok := service.cache.Get(apiKey1.Digest)
is.True(ok)
is.Equal(*apiKey1, apiKeyFromCache)
_, apiKeyFromCache, ok = service.cache.Get(apiKey2.Digest)
is.True(ok)
is.Equal(*apiKey2, apiKeyFromCache)
// evict key of single user from cache
ok = service.cache.InvalidateUserKeyCache(user1.ID)
is.True(ok)
// verify user1 key has been flushed from cache
_, _, ok = service.cache.Get(apiKey1.Digest)
is.False(ok)
// verify user2 key is still in cache
_, _, ok = service.cache.Get(apiKey2.Digest)
is.True(ok)
})
}