Commit Graph

104 Commits (0141e55936c51a86a31839ee28fa993296574cee)

Author SHA1 Message Date
andres-portainer 20a66fb10f
fix(endpoints): remove global map to avoid panic writes EE-3838 (#7404) 2022-08-03 12:18:33 -03:00
Chaim Lev-Ari 05357ecce5
fix(edge): filtering of edge devices [EE-3210] (#7077)
* fix(edge): filtering of edge devices [EE-3210]

fixes [EE-3210]

changes:
- replaces `edgeDeviceFilter` with two filters:
	- `edgeDevice`
	- `edgeDeviceUntrusted`

these filters will only apply to the edge endpoints in the query (so it's possible to get both regular endpoints and edge devices).

if `edgeDevice` is true, will filter out edge agents which are not an edge device.
			false, will filter out edge devices

`edgeDeviceUntrusted` applies only when `edgeDevice` is true. then false (default) will hide the untrusted edge devices, true will show only untrusted edge devices.

fix(edge/job-create): retrieve only trusted endpoints + fix endpoint selector pagination limits onChange

fix(endpoint-groups): remove listing of untrusted edge envs (aka in waiting room)

refactor(endpoints): move filter to another function

feat(endpoints): separate edge filters

refactor(environments): change getEnv api

refactor(endpoints): use single getEnv

feat(groups): show error when failed loading envs

style(endpoints): remove unused endpointsByGroup

* chore(deps): update go to 1.18

* fix(endpoint): filter out untrusted by default

* fix(edge): show correct endpoints

* style(endpoints): fix typo

* fix(endpoints): fix swagger

* fix(admin): use new getEnv function

Co-authored-by: LP B <xAt0mZ@users.noreply.github.com>
2022-07-19 18:00:45 +02:00
congs 4997e9c7be
feat(gpu) EE-3191 Add GPU support for containers (#7146) 2022-07-18 11:02:14 +12:00
matias-portainer 6b27ba9121
fix(edge): delete endpoint proxy only when updating URL, TLS or is Edge Agent on Kubernetes EE-2759 (#7086) 2022-07-01 11:36:01 -03:00
congs 0522032515
feat(teamleader) EE-294 redesign team leader (#6973)
feat(teamleader) EE-294 redesign team leader (#6973)
2022-06-03 16:44:42 +12:00
Chaim Lev-Ari e686d64011
refactor(docker): strongly type snapshot [EE-3256] (#6990)
* refactor(docker): strongly type snapshot [EE-3256]

fixes [EE-3256]

* fix(endpoints): return empty from association api

* refactor(docker): ignore raw snapshot for swagger
2022-05-31 13:03:10 +03:00
andres-portainer c162e180e0
fix(endpoints): remove global map to avoid panic writes EE-3160 (#6918) 2022-05-30 11:22:37 -03:00
Chaim Lev-Ari 01dc9066b7
refactor(wizard): migrate to react [EE-2305] (#6957) 2022-05-23 17:32:51 +03:00
Chaim Lev-Ari 12cddbd896
feat(demo): disable features on demo env [EE-1874] (#6040) 2022-05-22 08:34:09 +03:00
Matt Hook 840a3ce732
switch natural sort lib for a better one (#6862)
Switched to better natural sorting package
2022-05-02 12:37:26 +12:00
Prabhat Khera 33f433ce45
fix status filter (#6827) 2022-04-27 11:40:23 +12:00
Chaim Lev-Ari bbb096412d
fix(edge): show edge environment in edge views [EE-2997] (#6795) 2022-04-26 14:25:20 +03:00
Prabhat Khera bebee78152
fix(home): fix home page filters EE-2972 (#6789) 2022-04-26 12:17:36 +12:00
andres-portainer 5b77edb76d
fix(aeec): enforce non-empty EdgeIDs for global key environment retrieval EE-3013 (#6808) 2022-04-25 11:35:14 -03:00
Chaim Lev-Ari e217ac7121
feat(edge): show correct heartbeat and sync aeec changes [EE-2876] (#6769) 2022-04-19 21:43:36 +03:00
Chaim Lev-Ari 85a7b7e0fc
refactor(edge): move edge deploy script to react [EE-2689] (#6747) 2022-04-14 13:14:23 +03:00
andres-portainer 3178787bc1
feat(edge): implement automatic edge environment creation EE-2848 (#6754) 2022-04-13 10:08:26 -03:00
LP B aac2aca912
fix(api/endpoint): refresh kubernetes client cache on endpoint update (#6752) 2022-04-12 17:05:15 +02:00
Marcelo Rydel f12c3968f1
feat(edge): edgeStacks and edgeJobs operations small refactors [EE-2744] (#6648) 2022-04-08 11:27:38 -03:00
andres-portainer 9ffaf47741
fix(endpoint-status): add a redirect to keep backwards compatibility EE-2932 (#6735) 2022-04-07 11:33:14 -03:00
Marcelo Rydel f9f937f844
feat(registries): Registry browser for non-admins [EE-2459] (#6549)
* feat(registries): allow non-admin users to see environment registries

* remove unused function

* fix error message

* fix test

* fix imports order

* feat(registry): check access first, add parameters name

* use registryID

* fix(sidebar): allow standard users to see endpoint registries view

Co-authored-by: LP B <xAt0mZ@users.noreply.github.com>
2022-04-07 15:22:31 +02:00
Marcelo Rydel 77e48bfb74
chore(endpoint/edge): small refactor and move endpoint status handler [EE-2710] (#6637) 2022-04-07 09:17:36 -03:00
Chao Geng f59459f936
EE-2463 When add agent to k8s with AGENT_SECRET, AGENT_SECRET should be configured on both side. otherwise, it will get unknown error. this error should be "agent already paired with another Portainer instance" (#6679) 2022-04-01 14:40:44 +08:00
Marcelo Rydel 8860d72f70
fix(edge/jobs): fix get edge job file content [EE-2702] (#6622) 2022-03-28 12:02:09 -03:00
Oscar Zhou 5188ead870
fix(home): fix homepage edge heartbeat judgement [EE-2041] (#6624)
* fix(home): judge LastCheckInDate with QueryDate for heartbeat

* refactor(environments): remove deprecated variable homepageLoadTime

* style(environments): run yarn format

Co-authored-by: sam@gemibook <huapox@126.com>
2022-03-14 14:53:23 +13:00
Chaim Lev-Ari 0f3c7b1424
refactor(home): migrate view to react [EE-1810] (#6314)
* refactor(http): parse axios errors (#6325)

* refactor(home): use endpoint-list as react component [EE-1814] (#6060)

* refactor(home): use endpoint-list as react component

fix(home): add missing features and refactors

- kubebutton
- group name
- poll when endpoint is off
- state management

refactor(endpoints): use stat component

fix(endpoints): add space between items

refactor(endpoints): move stats to components

refactor(endpoints): fetch time

refactor(home): move logic

refactor(home): move fe render logic

refactor(settings): use vanilla js for publicSettings

refactor(kube): remove angular from kube config service

feat(home): add kubeconfig button

feat(home): send analytics when opening kubeconfig modal

fix(home): memoize footer

refactor(home): use react-query for loading

fix(home): show correct control for kubeconfig modal

refactor(home): use debounce

refactor(home): use new components

refactor(home): replace endpoints with environments

refactor(home): move endpoint-list component to home

fix(home): show group name

refactor(home): use switch for environment icon

fix(kubeconfig): fix default case

refactor(axios): use parse axios error

refactor(home): use link components for navigate

fix(home): align azure icon

refactor(home): refactor stats

refactor(home): export envstatusbadge

refactor(home): remove unused bindings

* chore(home): write tests for edge indicator

* chore(home): basic stories for environment item

* style(settings): reformat

* fix(environments): add publicurl

* refactor(home): use table components

* refactor(datatables): merge useSearchBarState

* refactor(home): fetch group in env item

* chore(tests): basic tests

* chore(home): test when no envs

* refactor(tags): use axios for tagService

* refactor(env-groups): use axios for getGroups

* feat(app): ui-state context provider

* refactor(home): create MotdPanel

* refactor(app): create InformationPanel

* feat(endpoints): fetch number of total endpoints

* refactor(app): merge hooks

* refactor(home): migrate view to react [EE-1810]

fixes [EE-1810]

refactor(home): wip use react view

feat(home): show message if no endpoints

refactor(home): show endpoint list

refactor(home): don't use home to manage link

refactor(home): move state

refactor(home): check if edge using util

refactor(home): move inf panels

chore(home): tests

refactor(home): load groups and tags in env-item

refactor(settings): revert publicSettings change

refactor(home): move confirm snapshot method

* fix(home): show tags

* fix(environments): handle missing snapshots

* fix(kube/volumes): fetch pesistent volume claims

* refactor(kube): remove use of endpointProvider

* refactor(endpoints): set current endpoint

* chore(home): add data-cy for tests

* chore(tests): mock axios-progress-bar

* refactor(home): move use env list to env module

* feat(app): sync home view changes with ee

* fix(home): sort page header

* fix(app): fix tests

* chore(github): use yarn cache

* refactor(environments): load list of groups

* chore(babel): remove auto 18n keys extraction

* chore(environments): fix tests

* refactor(k8s/application): use current endpoint

* fix(app/header): add margin to header

* refactor(app): remove unused types

* refactor(app): use rq onError handler

* refactor(home): wrap element with button
2022-03-08 14:14:23 +02:00
Anthony Lapenna 318844226c
refactor(storidge): remove Storidge support from backend [EE-2450] (#6511)
* refactor(storidge): remove Storidge support from backend

* refactor(storidge): remove Storidge support from backend

* refactor(storidge): remove Storidge support from frontend
2022-02-09 05:47:11 +13:00
Marcelo Rydel 2c4c638f46
feat(intel): Enable OpenAMT and FDO capabilities (#6212)
* feat(openamt): add AMT Devices information in Environments view [INT-8] (#6169)

* feat(openamt): add AMT Devices Ouf of Band Managamenet actions  [INT-9] (#6171)

* feat(openamt): add AMT Devices KVM Connection [INT-10] (#6179)

* feat(openamt): Enhance the Environments MX to activate OpenAMT on compatible environments [INT-7] (#6196)

* feat(openamt): Enable KVM by default [INT-25] (#6228)

* feat(fdo): implement the FDO configuration settings INT-19 (#6238)

feat(fdo): implement the FDO configuration settings INT-19

* feat(fdo): implement Owner client INT-17 (#6231)

feat(fdo): implement Owner client INT-17

* feat(openamt): hide wireless config in OpenAMT form (#6250)

* feat(openamt): Increase OpenAMT timeouts [INT-30] (#6253)

* feat(openamt): Disable the ability to use KVM and OOB actions on a MPS disconnected device [INT-36] (#6254)

* feat(fdo): add import device UI [INT-20] (#6240)

feat(fdo): add import device UI INT-20

* refactor(fdo): fix develop merge issues

* feat(openamt): Do not fetch OpenAMT details for an unassociated Edge endpoint (#6273)

* fix(intel): Fix switches params (#6282)

* feat(openamt): preload existing AMT settings (#6283)

* feat(openamt): Better UI/UX for AMT activation loading [INT-39] (#6290)

* feat(openamt): Remove wireless config related code [INT-41] (#6291)

* yarn install

* feat(openamt): change kvm redirection for pop up, always enable features [INT-37] (#6292)

* feat(openamt): change kvm redirection for pop up, always enable features [INT-37] (#6293)

* feat(openmt): use .ts services with axios for OpenAMT (#6312)

* Minor code cleanup.

* fix(fdo): move the FDO client code to the hostmanagement folder INT-44 (#6345)

* refactor(intel): Add Edge Compute Settings view (#6351)

* feat(fdo): add FDO profiles INT-22 (#6363)

feat(fdo): add FDO profiles INT-22

* fix(fdo): fix incorrect profile URL INT-45 (#6377)

* fixed husky version

* fix go.mod with go mod tidy

* feat(edge): migrate OpenAMT devices views to Edge Devices [EE-2322] (#6373)

* feat(intel): OpenAMT UI/UX adjustments (#6394)

* only allow edge agent as edge device

* show all edge agent environments on Edge Devices view

* feat(fdo): add the ability to import multiple ownership vouchers at once EE-2324 (#6395)

* fix(edge): settings edge compute alert (#6402)

* remove pagination, add useMemo for devices result array (#6409)

* feat(edge): minor Edge Devices (AMT) UI fixes (#6410)

* chore(eslint): fix versions

* chore(app): reformat codebase

* change add edge agent modal behaviour, fix yarn.lock

* fix use pagination

* remove extractedTranslations folder

* feat(edge): add FDO Profiles Datatable [EE-2406] (#6415)

* feat(edge): add KVM workaround tooltip (#6441)

* feat(edge): Add default FDO profile (#6450)

* feat(edge): add settings to disable trust on first connect and enforce Edge ID INT-1 EE-2410 (#6429)

Co-authored-by: andres-portainer <91705312+andres-portainer@users.noreply.github.com>
Co-authored-by: Anthony Lapenna <anthony.lapenna@portainer.io>
Co-authored-by: andres-portainer <andres-portainer@users.noreply.github.com>
Co-authored-by: Chaim Lev-Ari <chiptus@gmail.com>
2022-01-24 08:48:04 +13:00
andres-portainer 50b2f789a3
feat(performance): add settings to tune the performance of the database EE-2363 (#6389)
* feat(performance): add settings to tune the performance of the database EE-2363

* Change panics to log.Fatals.

Co-authored-by: andres-portainer <andres-portainer@users.noreply.github.com>
2022-01-18 11:25:29 +13:00
Sven Dowideit f99329eb7e
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173)
* use the Store interface IsErrObjectNotFound() to avoid revealing internal errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* what happens when you extract the datastore interfaces into their own package

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Start renaming Storage methods

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the boltdb specific code from the Portainer storage code (example, the others need the same)

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extract bolt.Tx from datastore code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* minimise imports by putting moving the struct definition into the file that needs the Service imports

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extraction of boltdb.Tx

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the use of bucket.SetSequence

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* almost done - just endpoint.Synchonise :/

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring'

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix test compile errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* test compile fixes after rebase

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* set us up to make the connection an interface

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make the db connection a datastore interface, and separate out our datastore services from the bolt ones

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* rename methods to something less oltdb internals specific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* these errors are not boltdb secific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* start using the db-backend factory method too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* export boltdb raw in case we can't export from the service layer

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add the version info by hand for now

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* actually, the export from services can be fully typed - its the import that needs to do more work

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* redo raw export, and make import capable of using it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add DockerHub

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix go test ./... again

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* my goland wasn't setup to gofmt

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the two extremely dubious migration tests down into store, so they can use the test store code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* the migrator is now free of boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* reverse goland overzealous replcement of internal with boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more undo over-zealous goland internal->boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* yay, now bolt is only mentioned inside the api/database/ dir

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* and this might be the last of the boltdb references?

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add todo

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the store code into a separate module too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* don't need the fileService in boltdb anymore

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use IsErrObjectNotFound()

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use a string to select what database backend we use

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make isNew store an ephemeral bool that doesn't stay true after we've initialised it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* undo vscode formatting html

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix app templates symbol (#6221)

* feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220)

* feat(webhook) EE-2125 add some helpers to registry utils

* feat(webhook) EE-2125 persist registryID when creating a webhook

* feat(webhook) EE-2125 send registry auth header when executing a webhook

* feat(webhook) EE-2125 send registryID to backend when creating a service with webhook

* feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen

* feat(webhook) EE-2125 update webhook when update registry

* feat(webhook) EE-2125 add endpoint of update webhook

* feat(webhook) EE-2125 code cleanup

* feat(webhook) EE-2125 fix a typo

* feat(webhook) EE-2125 fix circle import issue with unit test

Co-authored-by: Simon Meng <simon.meng@portainer.io>

* fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204)

Co-authored-by: Simon Meng <simon.meng@portainer.io>

* fix data-cy for k8s cluster menu (#6226)

LGTM

* feat(stack): make stack created from app template editable EE-1941 (#6104)

feat(stack): make stack from app template editable

* fix(container):disable Duplicate/Edit button when the container is portainer (#6223)

* fix/ee-1909/show-pull-image-error (#6195)

Co-authored-by: sunportainer <ericsun@SG1.local>

* feat(cy): add data-cy to helm install button (#6241)

* feat(cy): add data-cy to add registry button (#6242)

* refactor(app): convert root folder files to es6 (#4159)

* refactor(app): duplicate constants as es6 exports (#4158)

* fix(docker): provide workaround to save network name variable  (#6080)

* fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file

* fix/EE-1862/unable-to-stop-or-remove-stack check yaml file

* fixed func and var names

* wrapper error and used bool for stringset

* UT case for createNetworkEnvFile

* UT case for %s=%s

* powerful StringSet

* wrapper error for extract network name

* wrapper all the return err

* store more env

* put to env file

* make default value None

* feat: gzip static resources (#6258)

* fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203)

* fix/ee-2106/handle-sslcert-sslkey

Co-authored-by: sunportainer <ericsun@SG1.local>

* fix(server):support disable https only ee-2068 (#6232)

* fix/ee-2068/disable-forcely-https

* feat(store): implement store tests EE-2112 (#6224)

* add store tests

* add some more tests

* Update missing helm user repo methods

* remove redundant comments

* add webhook export

* update webhooks

* use the Store interface IsErrObjectNotFound() to avoid revealing internal errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* what happens when you extract the datastore interfaces into their own package

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Start renaming Storage methods

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the boltdb specific code from the Portainer storage code (example, the others need the same)

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extract bolt.Tx from datastore code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* minimise imports by putting moving the struct definition into the file that needs the Service imports

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extraction of boltdb.Tx

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the use of bucket.SetSequence

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* almost done - just endpoint.Synchonise :/

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring'

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix test compile errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* test compile fixes after rebase

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* set us up to make the connection an interface

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make the db connection a datastore interface, and separate out our datastore services from the bolt ones

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* rename methods to something less oltdb internals specific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* these errors are not boltdb secific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* start using the db-backend factory method too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* export boltdb raw in case we can't export from the service layer

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add the version info by hand for now

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* actually, the export from services can be fully typed - its the import that needs to do more work

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* redo raw export, and make import capable of using it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add DockerHub

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix go test ./... again

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* my goland wasn't setup to gofmt

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the two extremely dubious migration tests down into store, so they can use the test store code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* the migrator is now free of boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* reverse goland overzealous replcement of internal with boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more undo over-zealous goland internal->boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* yay, now bolt is only mentioned inside the api/database/ dir

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* and this might be the last of the boltdb references?

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add todo

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the store code into a separate module too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* don't need the fileService in boltdb anymore

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use IsErrObjectNotFound()

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use a string to select what database backend we use

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make isNew store an ephemeral bool that doesn't stay true after we've initialised it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* undo vscode formatting html

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Update missing helm user repo methods

* feat(store): implement store tests EE-2112 (#6224)

* add store tests

* add some more tests

* remove redundant comments

* add webhook export

* update webhooks

* fix build issues after rebasing

* move migratorparams

* remove unneeded integer type conversions

* disable the db import/export for now

Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: cong meng <mcpacino@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com>
Co-authored-by: Hao Zhang <hao.zhang@portainer.io>
Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com>
Co-authored-by: sunportainer <ericsun@SG1.local>
Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com>
Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com>
Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com>
Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>
Co-authored-by: Matt Hook <hookenz@gmail.com>
2021-12-15 15:26:09 +13:00
zees-dev 69c17986d9
feat(api-key/backend): introducing support for api-key based auth EE-978 (#6079)
* feat(access-token): Multi-auth middleware support EE-1891 (#5936)

* AnyAuth middleware initial implementation with tests

* using mux.MiddlewareFunc instead of custom definition

* removed redundant comments

* - ExtractBearerToken bouncer func made private
- changed helm token handling functionality to use jwt service to convert token to jwt string
- updated tests
- fixed helm list broken test due to missing token in request context

* rename mwCheckAuthentication -> mwCheckJWTAuthentication

* - introduce initial api-key auth support using X-API-KEY header
- added tests to validate x-api-key request header presence

* updated core mwAuthenticatedUser middleware to support multiple auth paradigms

* - simplified anyAuth middleware
- enforcing authmiddleware to implement verificationFunc interface
- created tests for middleware

* simplify bouncer

Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>

* feat(api-key): user-access-token generation endpoint EE-1889 EE-1888 EE-1895 (#6012)

* user-access-token generation endpoint

* fix comment

* - introduction of apikey service
- seperation of repository from service logic - called in handler

* fixed tests

* - fixed api key prefix
- added tests

* added another test for digest matching

* updated swagger spec for access token creation

* api key response returns raw key and struct - easing testability

* test for api key prefix length

* added another TODO to middleware

* - api-key prefix rune -> string (rune does not auto-encode when response sent back to client)
- digest -> pointer as we want to allow nil values and omit digest in responses (when nil)

* - updated apikey struct
- updated apikey service to support all common operations
- updated apikey repo
- integration of apikey service into bouncer
- added test for all apikey service functions
- boilerplate code for apikey service integration

* - user access token generation tests
- apiKeyLookup updated to support query params
- added api-key tests for query params
- added api-key tests for apiKeyLookup

* get and remove access token handlers

* get and remove access token handler tests

* - delete user deletes all associated api keys
- tests for this functionality

* removed redundant []byte cast

* automatic api-key eviction set within cache for 1 hour

* fixed bug with loop var using final value

* fixed service comment

* ignore bolt error responses

* case-insensitive query param check

* simplified query var assignment

* - added GetAPIKey func to get by unique id
- updated DeleteAPIKey func to not require user ID
- updated tests

* GenerateRandomKey helper func from github.com/gorilla/securecookie moved to codebase

* json response casing for api-keys fixed

* updating api-key will update the cache

* updated golang LRU cache

* using hashicorps golang-LRU cache for api keys

* simplified jwt check in create user access token

* fixed api-key update logic on cache miss

* Prefix generated api-keys with `ptr_` (#6067)

* prefix api-keys with 'ptr_'

* updated apikey description

* refactor

Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>

* helm list test refactor

* fixed user delete test

* reduce test nil pointer errors

* using correct http 201 created status code for token creation; updated tests

* fixed swagger doc user id path param for user access token based endpoints

* added api-key security openapi spec to existing jwt secured endpoints (#6091)

* fixed flaky test

* apikey datecreated and lastused attrs converted to unix timestamp

* feat(user): added access token datatable. (#6124)

* feat(user): added access token datatable.

* feat(tokens): only display lastUsed time when it is not the default date

* Update app/portainer/views/account/accountController.js

Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>

* Update app/portainer/views/account/accountController.js

Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>

* Update app/portainer/views/account/accountController.js

Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>

* Update app/portainer/components/datatables/access-tokens-datatable/accessTokensDatatableController.js

Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>

* Update app/portainer/services/api/userService.js

Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>

* feat(improvements): proposed datatable improvements to speed up dev time (#6138)

* modal code update

* updated datatable filenames, updated controller to be default class export

* fix(access-token): code improvement.

Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>

* feat(apikeys): create access token view initial implementation EE-1886 (#6129)

* CopyButton implementation

* Code component implementation

* ToolTip component migration to another folder

* TextTip component implementation - continued

* form Heading component

* Button component updated to be more dynamic

* copybutton - small size

* form control pass tip error

* texttip small text

* CreateAccessToken react feature initial implementation

* create user access token angularjs view implementation

* registration of CreateAccessToken component in AngularJS

* user token generation API request moved to angular service, method passed down instead

* consistent naming of access token operations; clustered similar code together

* any user can add access token

* create access token page routing

* moved code component to the correct location

* removed isadmin check as all functionality applicable to all users

* create access token angular view moved up a level

* fixed PR issues, updated PR

* addressed PR issues/improvements

* explicit hr for horizontal line

* fixed merge conflict storybook build breaking

* - apikey test
- cache test

* addressed testing issues:
- description validations
- remove token description link on table

* fix(api-keys): user role change evicts user keys in cache EE-2113 (#6168)

* user role change evicts user api keys in cache

* EvictUserKeyCache -> InvalidateUserKeyCache

* godoc for InvalidateUserKeyCache func

* additional test line

* disable add access token button after adding token to prevent spam

Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>
Co-authored-by: fhanportainer <79428273+fhanportainer@users.noreply.github.com>
2021-11-30 15:31:16 +13:00
Richard Wei cc3b1face2
fix docker pull limit not showed to non admin (#6066) 2021-11-12 15:57:12 +13:00
andres-portainer 28f71e486a
fix(filesystem): harden the filesystem service to avoid path traversal attacks EE-1922 (#5957)
fix(filesystem): harden the filesystem service to avoid path traversal attacks EE-1922
2021-11-01 08:01:03 -03:00
cong meng 41999e149f
fix(edge) EE-1720 activate tunnel and remove proxy cache when needed (#5775)
Co-authored-by: Simon Meng <simon.meng@portainer.io>
2021-10-15 18:13:20 +13:00
Hui e6d690e31e
fix(swagger) swagger annotations fixes and improvements EE-1205 2021-10-12 12:12:08 +13:00
Richard Wei dd808bb7bd
fix(swagger): fix swagger api docs endpoint(s) rename to environment(s) EE-1661 (#5629)
* fix swagger api docs endpoint(s) rename to environment(s)
2021-09-20 12:14:22 +12:00
Yi Chen 1633eceed5
fix(swagger) Fix openapi issues (#5123)
* * fix api version
* fix license info
* fix error response schema
* fix other typos & mistakes

* * remove unused tag

* * fix helm issues
2021-09-13 15:42:53 +12:00
Richard Wei 9f179fe3ec
feat(ui):rename endpoint(s) to environment(s) EE-1206 (#5588)
* rename endpoints to environments EE-1206
2021-09-08 20:42:17 +12:00
Chaim Lev-Ari 1830a80a61
feat(k8s/resource-pool): add the ability to mark/unmark resource pool as system (#5360)
* feat(k8s/resource-pool): add the ability to mark/unmark resource pool as system

fix(kube/ns): check label to see if namespace is system

refactor(k8s/namespaces): rename variables

feat(kubernetes): toggle system state in the server (#5361)

fix(app/resource-pool): UI fixes

feat(app/resource-pool): add confirmation modal when unamrking system namespace

* refactor(app): review changes

* feat(app/namespaces): introduce store to retrieve namespace system status without changing all the kubernetes models

refactor(app/namespaces): remove unused code first introduced for system tagging

fix(app/namespaces): cache namespaces to retrieve system status regardless of namespace reference format

refactor(app): migrate namespace store from helper to a separate singleton

refactor(app): remove KubernetesNamespaceHelper from DI cycle

* refactor(app): normalize usage of KubernetesNamespaceHelper functions

* refactor(app/k8s): change namespace store to functions instead of class

Co-authored-by: LP B <xAt0mZ@users.noreply.github.com>
2021-08-26 16:00:59 +02:00
cong meng 6b37235eb4
feat(edge) EE-947 provide a way to re-associate an Edge endpoint with a new Edge key (#5413)
Co-authored-by: Simon Meng <simon.meng@portainer.io>
2021-08-18 08:25:34 +12:00
cong meng 5652bac004
feat: EE-424 Provide a way to re-associate an Edge endpoint to a new Edge agent (#5266)
Co-authored-by: Simon Meng <simon.meng@portainer.io>
2021-08-02 18:08:40 +12:00
Matt Hook cee7ac26e9
Fix dockerhub pro account rate-limit issue (#5352) 2021-07-27 10:49:28 +12:00
dbuduev 7d6b1edd48
feat(k8s): Introduce the ability to restrict access to default namespace (EE-745) (#5337) 2021-07-23 17:10:46 +12:00
Chaim Lev-Ari 7b2269fbba
feat(endpoints): filter endpoints by a list of types (#5308)
* feat(endpoints): filter endpoints by a list of types

* docs(endpoints): update api docs for endpoint list
2021-07-21 10:16:22 +03:00
LP B 179df06267
feat(app): rework private registries and support private registries in kubernetes EE-30 (#5131)
* feat(app): rework private registries and support private registries in kubernetes

[EE-30]

feat(api): backport private registries backend changes (#5072)

* feat(api/bolt): backport bolt changes

* feat(api/exec): backport exec changes

* feat(api/http): backport http/handler/dockerhub changes

* feat(api/http): backport http/handler/endpoints changes

* feat(api/http): backport http/handler/registries changes

* feat(api/http): backport http/handler/stacks changes

* feat(api/http): backport http/handler changes

* feat(api/http): backport http/proxy/factory/azure changes

* feat(api/http): backport http/proxy/factory/docker changes

* feat(api/http): backport http/proxy/factory/utils changes

* feat(api/http): backport http/proxy/factory/kubernetes changes

* feat(api/http): backport http/proxy/factory changes

* feat(api/http): backport http/security changes

* feat(api/http): backport http changes

* feat(api/internal): backport internal changes

* feat(api): backport api changes

* feat(api/kubernetes): backport kubernetes changes

* fix(api/http): changes on backend following backport

feat(app): backport private registries frontend changes (#5056)

* feat(app/docker): backport docker/components changes

* feat(app/docker): backport docker/helpers changes

* feat(app/docker): backport docker/views/container changes

* feat(app/docker): backport docker/views/images changes

* feat(app/docker): backport docker/views/registries changes

* feat(app/docker): backport docker/views/services changes

* feat(app/docker): backport docker changes

* feat(app/kubernetes): backport kubernetes/components changes

* feat(app/kubernetes): backport kubernetes/converters changes

* feat(app/kubernetes): backport kubernetes/models changes

* feat(app/kubernetes): backport kubernetes/registries changes

* feat(app/kubernetes): backport kubernetes/services changes

* feat(app/kubernetes): backport kubernetes/views/applications changes

* feat(app/kubernetes): backport kubernetes/views/configurations changes

* feat(app/kubernetes): backport kubernetes/views/configure changes

* feat(app/kubernetes): backport kubernetes/views/resource-pools changes

* feat(app/kubernetes): backport kubernetes/views changes

* feat(app/portainer): backport portainer/components/accessManagement changes

* feat(app/portainer): backport portainer/components/datatables changes

* feat(app/portainer): backport portainer/components/forms changes

* feat(app/portainer): backport portainer/components/registry-details changes

* feat(app/portainer): backport portainer/models changes

* feat(app/portainer): backport portainer/rest changes

* feat(app/portainer): backport portainer/services changes

* feat(app/portainer): backport portainer/views changes

* feat(app/portainer): backport portainer changes

* feat(app): backport app changes

* config(project): gitignore + jsconfig changes

gitignore all files under api/cmd/portainer but main.go and enable Code Editor autocomplete on import ... from '@/...'

fix(app): fix pull rate limit checker

fix(app/registries): sidebar menus and registry accesses users filtering

fix(api): add missing kube client factory

fix(kube): fetch dockerhub pull limits (#5133)

fix(app): pre review fixes (#5142)

* fix(app/registries): remove checkbox for endpointRegistries view

* fix(endpoints): allow access to default namespace

* fix(docker): fetch pull limits

* fix(kube/ns): show selected registries for non admin

Co-authored-by: Chaim Lev-Ari <chiptus@gmail.com>

chore(webpack): ignore missing sourcemaps

fix(registries): fetch registry config from url

feat(kube/registries): ignore not found when deleting secret

feat(db): move migration to db 31

fix(registries): fix bugs in PR EE-869 (#5169)

* fix(registries): hide role

* fix(endpoints): set empty access policy to edge endpoint

* fix(registry): remove double arguments

* fix(admin): ignore warning

* feat(kube/configurations): tag registry secrets (#5157)

* feat(kube/configurations): tag registry secrets

* feat(kube/secrets): show registry secrets for admins

* fix(registries): move dockerhub to beginning

* refactor(registries): use endpoint scoped registries

feat(registries): filter by namespace if supplied

feat(access-managment): filter users for registry (#5191)

* refactor(access-manage): move users selector to component

* feat(access-managment): filter users for registry

refactor(registries): sync code with CE (#5200)

* refactor(registry): add inspect handler under endpoints

* refactor(endpoint): sync endpoint_registries_list

* refactor(endpoints): sync registry_access

* fix(db): rename migration functions

* fix(registries): show accesses for admin

* fix(kube): set token on transport

* refactor(kube): move secret help to bottom

* fix(kuberentes): remove shouldLog parameter

* style(auth): add description of security.IsAdmin

* feat(security): allow admin access to registry

* feat(edge): connect to edge endpoint when creating client

* style(portainer): change deprecation version

* refactor(sidebar): hide manage

* refactor(containers): revert changes

* style(container): remove whitespace

* fix(endpoint): add handler to registy on endpointService

* refactor(image): use endpointService.registries

* fix(kueb/namespaces): rename resource pool to namespace

* fix(kube/namespace): move selected registries

* fix(api/registries): hide accesses on registry creation

Co-authored-by: LP B <xAt0mZ@users.noreply.github.com>

refactor(api): remove code duplication after rebase

fix(app/registries): replace last registry api usage by endpoint registry api

fix(api/endpoints): update registry access policies on endpoint deletion (#5226)

[EE-1027]

fix(db): update db version

* fix(dockerhub): fetch rate limits

* fix(registry/tests): supply restricred context

* fix(registries): show proget registry only when selected

* fix(registry): create dockerhub registry

* feat(db): move migrations to db 32

Co-authored-by: Chaim Lev-Ari <chiptus@gmail.com>
2021-07-14 21:15:21 +12:00
cong meng 6b759438b8
fix(k8s) cleaning up namespace access policies when removing users orteams from endpoint or endpoint group EE-718 (#5184)
* fix(k8s) cleaning up namespace access policies when removing users or teams from endpoint or endpoint group EE-718

* fix(k8s) minor code cleanup EE-718

Co-authored-by: Simon Meng <simon.meng@portainer.io>
2021-06-16 20:15:29 +12:00
Hui 49bd139466
fix swagger param (#5183) 2021-06-14 14:45:57 +12:00
Chaim Lev-Ari 9ec1f2ed6d
fix(endpoints): set sysctl setting for new endpoints (#5028) 2021-06-03 11:36:54 +02:00
Lukas Grotz d09ae22ba8
feat(container): add sysctls setting in the container view (#4910)
* feat(container): add sysctls in the container view (#2756)

* feat(container): add setting to restrict sysctl access

* feat(endpoint): move sysctl disable setting to security settings

* feat(container): add sysctls to container edit view

* fix(container) remove unnecessary migration setting

Co-authored-by: Owen Kirby <oskirby@gmail.com>
2021-04-12 19:40:45 +12:00
Chaim Lev-Ari f5aa6c4dc2
feat(docker): show docker pull rate limits (#4666)
* feat(dockerhub): introduce local status endpoint

* feat(proxy): rewrite request with dockerhub credentials

* feat(endpoint): check env type

* feat(endpoint): check for local endpoint

* feat(docker): introduce client side service to get limits

* feat(container): add info about rate limits in container

* feat(dockerhub): load rate limits just for specific endpoints

* feat(images): show specific dockerhub messages for admin

* feat(service-create): show docker rate limits

* feat(service-edit): show rate limit messages

* fix(images): fix loading of page

* refactor(images): move rate limits check to container

* feat(kubernetes): proxy agent requests

* feat(kubernetes/apps): show pull limits in application creation

* refactor(image-registry): move warning to end of field

* fix(image-registry): show right message for admin

* fix(images): silently fail when loading rate limits

* fix(kube/apps): use new rate limits comp

* fix(images): move rate warning to end

* fix(registry): move search to right place

* fix(service): remove service warning

* fix(endpoints): check if kube endpoint is local
2021-03-24 19:27:32 +01:00