Commit Graph

10 Commits (1d42db93f130e7cb4ffd2f63777cc457ea149dd8)

Author SHA1 Message Date
matias-portainer e1b474d04f
feat(edgejobs): support edge groups when using edge jobs EE-3873 (#8099) 2022-12-19 18:54:51 -03:00
andres-portainer 9ef5636718
chore(handlers): replace structs by functions for HTTP errors EE-4227 (#7664) 2022-09-14 20:42:39 -03:00
Marcelo Rydel f12c3968f1
feat(edge): edgeStacks and edgeJobs operations small refactors [EE-2744] (#6648) 2022-04-08 11:27:38 -03:00
Sven Dowideit f99329eb7e
chore(store) EE-1981: Refactor/store/error checking, and other refactoring (#6173)
* use the Store interface IsErrObjectNotFound() to avoid revealing internal errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* what happens when you extract the datastore interfaces into their own package

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Start renaming Storage methods

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the boltdb specific code from the Portainer storage code (example, the others need the same)

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extract bolt.Tx from datastore code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* minimise imports by putting moving the struct definition into the file that needs the Service imports

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extraction of boltdb.Tx

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the use of bucket.SetSequence

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* almost done - just endpoint.Synchonise :/

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring'

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix test compile errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* test compile fixes after rebase

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* set us up to make the connection an interface

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make the db connection a datastore interface, and separate out our datastore services from the bolt ones

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* rename methods to something less oltdb internals specific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* these errors are not boltdb secific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* start using the db-backend factory method too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* export boltdb raw in case we can't export from the service layer

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add the version info by hand for now

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* actually, the export from services can be fully typed - its the import that needs to do more work

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* redo raw export, and make import capable of using it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add DockerHub

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix go test ./... again

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* my goland wasn't setup to gofmt

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the two extremely dubious migration tests down into store, so they can use the test store code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* the migrator is now free of boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* reverse goland overzealous replcement of internal with boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more undo over-zealous goland internal->boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* yay, now bolt is only mentioned inside the api/database/ dir

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* and this might be the last of the boltdb references?

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add todo

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the store code into a separate module too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* don't need the fileService in boltdb anymore

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use IsErrObjectNotFound()

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use a string to select what database backend we use

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make isNew store an ephemeral bool that doesn't stay true after we've initialised it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* undo vscode formatting html

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix app templates symbol (#6221)

* feat(webhook) EE-2125 send registry auth haeder when update swarms service via webhook (#6220)

* feat(webhook) EE-2125 add some helpers to registry utils

* feat(webhook) EE-2125 persist registryID when creating a webhook

* feat(webhook) EE-2125 send registry auth header when executing a webhook

* feat(webhook) EE-2125 send registryID to backend when creating a service with webhook

* feat(webhook) EE-2125 use the initial registry ID to create webhook on editing service screen

* feat(webhook) EE-2125 update webhook when update registry

* feat(webhook) EE-2125 add endpoint of update webhook

* feat(webhook) EE-2125 code cleanup

* feat(webhook) EE-2125 fix a typo

* feat(webhook) EE-2125 fix circle import issue with unit test

Co-authored-by: Simon Meng <simon.meng@portainer.io>

* fix(kubeconfig): show kubeconfig download button for non admin users [EE-2123] (#6204)

Co-authored-by: Simon Meng <simon.meng@portainer.io>

* fix data-cy for k8s cluster menu (#6226)

LGTM

* feat(stack): make stack created from app template editable EE-1941 (#6104)

feat(stack): make stack from app template editable

* fix(container):disable Duplicate/Edit button when the container is portainer (#6223)

* fix/ee-1909/show-pull-image-error (#6195)

Co-authored-by: sunportainer <ericsun@SG1.local>

* feat(cy): add data-cy to helm install button (#6241)

* feat(cy): add data-cy to add registry button (#6242)

* refactor(app): convert root folder files to es6 (#4159)

* refactor(app): duplicate constants as es6 exports (#4158)

* fix(docker): provide workaround to save network name variable  (#6080)

* fix/EE-1862/unable-to-stop-or-remove-stack workaround for var without default value in yaml file

* fix/EE-1862/unable-to-stop-or-remove-stack check yaml file

* fixed func and var names

* wrapper error and used bool for stringset

* UT case for createNetworkEnvFile

* UT case for %s=%s

* powerful StringSet

* wrapper error for extract network name

* wrapper all the return err

* store more env

* put to env file

* make default value None

* feat: gzip static resources (#6258)

* fix(ssl)//handle --sslcert and --sslkey ee-2106 (#6203)

* fix/ee-2106/handle-sslcert-sslkey

Co-authored-by: sunportainer <ericsun@SG1.local>

* fix(server):support disable https only ee-2068 (#6232)

* fix/ee-2068/disable-forcely-https

* feat(store): implement store tests EE-2112 (#6224)

* add store tests

* add some more tests

* Update missing helm user repo methods

* remove redundant comments

* add webhook export

* update webhooks

* use the Store interface IsErrObjectNotFound() to avoid revealing internal errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* what happens when you extract the datastore interfaces into their own package

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Start renaming Storage methods

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the boltdb specific code from the Portainer storage code (example, the others need the same)

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extract bolt.Tx from datastore code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* minimise imports by putting moving the struct definition into the file that needs the Service imports

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more extraction of boltdb.Tx

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the use of bucket.SetSequence

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* almost done - just endpoint.Synchonise :/

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* so, endpoint.Synchonize looks hard, but i can't find where we use it, so 'delete first refactoring'

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix test compile errors

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* test compile fixes after rebase

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix a mis-remembering I had wrt deserialisation - last time i used AnyData - jsoniter's bindTo looks interesting for the same reason

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* set us up to make the connection an interface

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make the db connection a datastore interface, and separate out our datastore services from the bolt ones

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* rename methods to something less oltdb internals specific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* these errors are not boltdb secific

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* start using the db-backend factory method too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* export boltdb raw in case we can't export from the service layer

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add a raw export from boltdb to yaml for broken db's, and an export services to yaml in backup

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add the version info by hand for now

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* actually, the export from services can be fully typed - its the import that needs to do more work

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* redo raw export, and make import capable of using it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add DockerHub

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* migration from anything older than v1.21.0 has been broken for quite a while, deleting the un-tested code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* fix go test ./... again

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* my goland wasn't setup to gofmt

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the two extremely dubious migration tests down into store, so they can use the test store code

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* the migrator is now free of boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* reverse goland overzealous replcement of internal with boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* more undo over-zealous goland internal->boltdb

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* yay, now bolt is only mentioned inside the api/database/ dir

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* and this might be the last of the boltdb references?

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* add todo

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* extract the store code into a separate module too

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* don't need the fileService in boltdb anymore

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use IsErrObjectNotFound()

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* use a string to select what database backend we use

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* make isNew store an ephemeral bool that doesn't stay true after we've initialised it

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* move the import.json wip to a separate file so its more obvious - we'll be using it for testing, emergency fixups, and in the next part of the store work, when we improve migrations and data model lifecycles

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* undo vscode formatting html

Signed-off-by: Sven Dowideit <sven.dowideit@portainer.io>

* Update missing helm user repo methods

* feat(store): implement store tests EE-2112 (#6224)

* add store tests

* add some more tests

* remove redundant comments

* add webhook export

* update webhooks

* fix build issues after rebasing

* move migratorparams

* remove unneeded integer type conversions

* disable the db import/export for now

Co-authored-by: Richard Wei <54336863+WaysonWei@users.noreply.github.com>
Co-authored-by: cong meng <mcpacino@gmail.com>
Co-authored-by: Simon Meng <simon.meng@portainer.io>
Co-authored-by: Marcelo Rydel <marcelorydel26@gmail.com>
Co-authored-by: Hao Zhang <hao.zhang@portainer.io>
Co-authored-by: sunportainer <93502624+sunportainer@users.noreply.github.com>
Co-authored-by: sunportainer <ericsun@SG1.local>
Co-authored-by: wheresolivia <78844659+wheresolivia@users.noreply.github.com>
Co-authored-by: Chaim Lev-Ari <chiptus@users.noreply.github.com>
Co-authored-by: Chao Geng <93526589+chaogeng77977@users.noreply.github.com>
Co-authored-by: Dmitry Salakhov <to@dimasalakhov.com>
Co-authored-by: Matt Hook <hookenz@gmail.com>
2021-12-15 15:26:09 +13:00
Hui e6d690e31e
fix(swagger) swagger annotations fixes and improvements EE-1205 2021-10-12 12:12:08 +13:00
Richard Wei dd808bb7bd
fix(swagger): fix swagger api docs endpoint(s) rename to environment(s) EE-1661 (#5629)
* fix swagger api docs endpoint(s) rename to environment(s)
2021-09-20 12:14:22 +12:00
Richard Wei 9f179fe3ec
feat(ui):rename endpoint(s) to environment(s) EE-1206 (#5588)
* rename endpoints to environments EE-1206
2021-09-08 20:42:17 +12:00
Chaim Lev-Ari 50b57614cf
docs(api): document apis with swagger (#4678)
* feat(api): introduce swagger

* feat(api): anottate api

* chore(api): tag endpoints

* chore(api): remove tags

* chore(api): add docs for oauth auth

* chore(api): document create endpoint api

* chore(api): document endpoint inspect and list

* chore(api): document endpoint update and snapshots

* docs(endpointgroups): document groups api

* docs(auth): document auth api

* chore(build): introduce a yarn script to build api docs

* docs(api): document auth

* docs(customtemplates): document customtemplates api

* docs(tags): document api

* docs(api): document the use of token

* docs(dockerhub): document dockerhub api

* docs(edgegroups): document edgegroups api

* docs(edgejobs): document api

* docs(edgestacks): doc api

* docs(http/upload): add security

* docs(api): document edge templates

* docs(edge): document edge jobs

* docs(endpointgroups): change description

* docs(endpoints): document missing apis

* docs(motd): doc api

* docs(registries): doc api

* docs(resourcecontrol): api doc

* docs(role): add swagger docs

* docs(settings): add swagger docs

* docs(api/status): add swagger docs

* docs(api/teammembership): add swagger docs

* docs(api/teams): add swagger docs

* docs(api/templates): add swagger docs

* docs(api/users): add swagger docs

* docs(api/webhooks): add swagger docs

* docs(api/webscokets): add swagger docs

* docs(api/stacks): swagger

* docs(api): fix missing apis

* docs(swagger): regen

* chore(build): remove docs from build

* docs(api): update tags

* docs(api): document tags

* docs(api): add description

* docs(api): rename jwt token

* docs(api): add info about types

* docs(api): document types

* docs(api): update request types annotation

* docs(api): doc registry and resource control

* chore(docs): add snippet

* docs(api): add description to role

* docs(api): add types for settings

* docs(status): add types

* style(swagger): remove documented code

* docs(http/upload): update docs with types

* docs(http/tags): add types

* docs(api/custom_templates): add types

* docs(api/teammembership): add types

* docs(http/teams): add types

* docs(http/stacks): add types

* docs(edge): add types to edgestack

* docs(http/teammembership): remove double returns

* docs(api/user): add types

* docs(http): fixes to make file built

* chore(snippets): add scope to swagger snippet

* chore(deps): install swag

* chore(swagger): remove handler

* docs(api): add description

* docs(api): ignore docs folder

* docs(api): add contributing guidelines

* docs(api): cleanup handler

* chore(deps): require swaggo

* fix(auth): fix typo

* fix(docs): make http ids pascal case

* feat(edge): add ids to http handlers

* fix(docs): add ids

* fix(docs): show correct api version

* chore(deps): remove swaggo dependency

* chore(docs): add install script for swag
2021-02-23 16:21:39 +13:00
Chaim Lev-Ari db4a5292be
refactor(errors): reorganize errors (#3938)
* refactor(bolt): move ErrObjectNotFound to bolt

* refactor(http): move ErrUnauthorized to http package

* refactor(http): move ErrResourceAccessDenied to http errors

* refactor(http): move security errors to package

* refactor(users): move user errors to users package

* refactor(errors): move single errors to their package

* refactor(schedules): move schedule error to package

* refactor(http): move endpoint error to http package

* refactor(docker): move docker errors to package

* refactor(filesystem): move filesystem errors to package

* refactor(errors): remove portainer.Error

* style(chisel): reorder imports

* fix(stacks): remove portainer.Error
2020-07-08 09:57:52 +12:00
Chaim Lev-Ari 24528ecea8
feat(edge-compute): move host jobs to edge (#3840)
* feat(endpoints): create an associated endpoints selector

* feat(schedules): remove edge specific explanations

* refactor(schedule): replace multi-endpoint-selector

* refactor(schedule): move controller to single file

* refactor(endpoints): remove multi-endpoint-selector

* feat(edge): rename host jobs to edge jobs

* feat(edge-jobs): remove edge warning

* refactor(edge-jobs): move schedule pages to edge

* refactor(edge-jobs): mv views to edgeJobs

* refactor(edge-jobs): rename edge jobs

* refactor(edge-jobs): move services to edge

* refactor(edge-jobs): move tasks datatable

* fix(edge-jobs): fix import

* fix(edge-jobs): use right services

* feat(settings): adjust host management description

* feat(edge-jobs): introduce interfaces and types

* feat(edge-jobs): implement bolt service

* refactor(edge-jobs): replace schedule routes

* refactor(edge-job): replace Schedule service

* refactor(edge-jobs): remove job_script_exec

* refactor(host): remove jobs table

* feat(edge-jobs): replace schedule

* feat(edge-jobs): load file on inspect

* fix(edge-job): parse cron correctly

* feat(edge-jobs): show tasks

* feat(host): rename tooltip

* refactor(host): remove old components

* refactor(main): remove schedule types

* refactor(snapshot): replace job service with snapshot service

* refactor(jobs): remove jobs form and datatable

* feat(edge-jobs): create db migration

* fix(main): start snapshot service with correct interval

* feat(settings): change host tooltip

* feat(edge-jobs): load endpoints

* fix(edge-job): disable form submit when form is invalid

* refactor(edge-compute): use const

* refactor(edge-jobs): use generic controller

* refactor(edge-jobs): replace $scope with controllerAs

* refactor(edge-jobs): replace routes with components

* refactor(edge-jobs): replace functions with classes

* refactor(edge-jobs): use async/await

* refactor(edge-jobs): rename functions

* feat(edge-jobs): introduce beta panel

* feat(edge-jobs): allow single character names

* fix(snapshot): run snapshot in coroutine

* feat(edge-jobs): add logs status

* feat(filesystem): add edge job logs methods

* feat(edge-jobs): intoduce edge jobs tasks api

* feat(edge-jobs): remove schedule task model

* fix(fs): build edge job task file path

* fix(edge-jobs): update task meta

* fix(edge-jobs): return a list of endpoints

* feat(edge-jobs): update logs from agent

* feat(edge-jobs): collect logs

* feat(edge-jobs): rename url

* feat(edge-jobs): refresh to same tab

* feat(edge-jobs): remove old info

* refactor(edge-jobs): rename script path json

* fix(edge-job): save file before adding job

* feat(edge-job): show retrieving logs label

* feat(edge-job): replace cron with 5 places

* refactor(edge-jobs): replace tasks with results

* feat(edge-jobs): add auto refresh until logs are collected

* feat(edge-jobs): fix column size

* feat(edge-job): display editor

* feat(edge-job): add name validation

* feat(edge-job): set default time for 1 hour from now

* feat(edge-job): add validation for cron format

* feat(edge-job): add a note about timezone

* fix(edge-job): replace regex

* fix(edge-job): check for every minute cron

* style(edge-jobs): add reference for cron regex

* refactor(edge-jobs): rename migration name

* refactor(edge-job): rename edge job response

* refactor(snapshot): rename snapshot endpoint method

* refactor(edge-jobs): move tasks handler to edgejobs

* feat(security): introduce a middleware for edge compute operations

* feat(edge-job): use edge compute middleware

* feat(edge-groups): filter http actions based on edge setting

* fix(security): return from edge bouncer if failed

* feat(edge-stacks): filter http actions based on edge setting

* feat(edge-groups): show error when failed to load groups

* refactor(db): remove edge-jobs migration

* refactor(migrator): remove unused dependency

Co-authored-by: Anthony Lapenna <lapenna.anthony@gmail.com>
2020-06-25 15:25:51 +12:00