This commit tries to make the development experience of working on our OIDC support a little more realistic, essentially by creating our own OIDC provider in our application (only during development builds). You can still provide a real OIDC provider to work with via our dev time environment/cookie variables as before, just now we default to the behaviour in this commit. Overall this makes it much easier to verify our OIDC support in the UI, and also opens up avenues for us to be able to test more scenarios that we couldn't before (for example not only successful logins, but also erroneous, potentially with multiple error reasons).
* Add `is` and `test` helpers in a similar vein to `can`
Adds 2 new helpers in a similar vein to ember-cans can:
- `is` allows you to use vocab/phrases such as (is "something model") which calls isSomething() on the models ability.
- `test` allows you to use vocab/phrases such as (test "is something model") or (test "can something model")which calls isSomething() / canSomething() on the models ability. Mostly using the is helper and the can helper. It's basically the is/can helper combined.
* Adds TextInput component + related modifiers/helpers/machines/services (#11189)
Adds a few new components/modifiers/helpers to aid building forms.
- state-chart helper, used in lieu of a more generic approach for requiring our statecharts.
- A few modifications to our existing disabled modifier.
- A new 'validation' modifier, a super small form validation approach built to make use of state charts (optionally). Eventually we should be able to replace our current validation approach (ember-changeset-validations + extra deps) with this.
- A new TextInput component, which is the first of our new components specifically to make it easy to build forms with validations. This is still a WIP, I left some comments in pointing out where this one would be progressed, but as we don't need the planned functionality yet, I left it where it was. All of this will be fleshed out more at a later date.
Documentation is included for all of ^
* ui: Adds initial CRUD for partitions (#11190)
Adds basic CRUD support for partitions. Engineering-wise probably the biggest takeaway here is that we needed to write very little javascript code to add this entire feature, and the little javascript we did need to write was very straightforwards. Everything is pretty much just HTML. Another note to make is that both ember-changeset and ember-data (model layer things) are now completely abstracted away from the view layer of the application.
New components:
- Consul::Partition::Form
- Consul::Partition::List
- Consul::Partition::Notifications
- Consul::Partition::SearchBar
- Consul::Partition::Selector
See additional documentation here for more details
New Route templates:
- index.hbs partition listing/searching/filtering
- edit.hbs partition editing and creation
Additionally:
There is some additional debug work here for better observability and to prevent any errors regarding our href-to usage when a dc is not available in our documentation site.
Our softDelete functionality has been DRYed out a little to be used across two repos.
isLinkable was removed from our ListCollection component for lists like upstream and service listing, and instead use our new is helper from within the ListCollection, meaning we've added a few more lighterweight templateOnly components.
* ui: Exclude all debug-like files from the build (#11211)
This PR adds **/*-debug.* to our test/prod excluded files (realised I needed to add test-support.js also so added that here as its more or less the same thing). Conditionally juggling ES6 static imports (specifically debug ones) for this was also getting a little hairy, so I moved it all to use the same approach as our conditional routes. All in all it brings the vendor build back down to ~430kb gzipped.
From an engineers perspective, whenever specifying colors from now on we should use the form:
```
color: rgb(var(--tone-red-500));
```
Please note:
- Use rgb. This lets us do this like rgb(var(--tone-red-500) / 10%) so we can use a 10% opacity red-500 if we ever need to whilst still making use of our color tokens.
- Use --tone-colorName-000 (so the prefix tone). Previously we could use a mix of --gray-500: $gray-500 (note the left hand CSS prop and right hand SASS var) for the things we need to theme currently. As we no longer use SASS we can't do --gray-500: --gray-500, so we now do --tone-gray-500: --gray-500.
Just for clarity after that, whenever specifying a color anywhere, use rgb and --tone. There is only one reason where you might not use tone, and that is if you never want a color to be affected by a theme (for example a background shadow probably always should use --black)
There are a 2 or 3 left for the code editor, plus our custom-query values
> In the future, this should all be moved to each individual repository now, which will mean we can finally get rid of this service.
This PR moves reconciliation to 'each individual repository'. I stopped short of getting rid of the service, but its so small now we pretty much don't need it. I'd rather wait until I look at the equivalent DataSink service and see if we can get rid of both equivalent services together (this also currently dependant on work soon to be merged)
Reconciliation of models (basically doing the extra work to clean up the ember-data store and bring our frontend 'truth' into line with the actual backend truth) when blocking/long-polling on different views/filters of data is slightly more complicated due to figuring out what should be cleaned up and what should be left in the store. This is especially apparent for KVs.
I built in a such a way to hopefully make sure it will all make sense for the future. I also checked that this all worked nicely with all our models, even KV which has never supported blocking queries. I left all that work in so that if we want to enable blocking queries/live updates for KV it now just involves deleting a couple of lines of code.
There is a tonne of old stuff that we can clean up here now (our 'fake headers' that we pass around) and I've added that to my list of thing for a 'Big Cleanup PR' that will remove lots of code that we no longer require.
This commit addresses some left over admin partition FIXMEs
1. Adds Partition correctly to Service Instances
2. Converts non-important 'we can do this later' FIXMEs to TODOs
3. Removes some FIXMEs that I've double checked and addressed.
Most of the remaining FIXMEs I'm waiting on responses to questions from
the consul core folks for. I'll address those in a separate PR.
Our DataSource came in very iteratively, when we first started using it we specifically tried not to use it for things that would require portions of the @src="" attribute to be URL encoded (so things like service names couldn't be used, but dc etc would be fine). We then gradually added an easy way to url encode the @src="" attributes with a uri helper and began to use the DataSource component more and more. This meant that some DataSource usage continued to be used without our uri helper.
Recently we hit #10901 which was a direct result of us not encoding @src values/URIs (I didn't realise this was one of the places that required URL encoding) and not going back over things to finish things off once we had implemented our uri helper, resulting in ~half of the codebase using it and ~half of it not.
Now that almost all of the UI uses our DataSource component, this PR makes it even harder to not use the uri helper, by wrapping the string that it requires in a private URI class/object, that is then expected/asserted within the DataSource component/service. This means that as a result of this PR you cannot pass a plain string to the DataSource component without seeing an error in your JS console, which in turn means you have to use the uri helper, and it's very very hard to not URL encode any dynamic/user provided values, which otherwise could lead to bugs/errors similar to the one mentioned above.
The error that you see when you don't use the uri helper is currently a 'soft' dev time only error, but like our other functionality that produces a soft error when you mistakenly pass an undefined value to a uri, at some point soon we will make these hard failing "do not do this" errors.
Both of these 'soft error' DX features have been used this to great effect to implement our Admin Partition feature and these kind of things will minimize the amount of these types of bugs moving forwards in a preventative rather than curative manner. Hopefully these are the some of the kinds of things that get added to our codebase that prevent a multitude of problems and therefore are often never noticed/appreciated.
Additionally here we moved the remaining non-uri using DataSources to use uri (that were now super easy to find), and also fixed up a place where I noticed (due to the soft errors) where we were sometimes passing undefined values to a uri call.
The work here also led me to find another couple of non-important 'bugs' that I've PRed already separately, one of which is yet to be merged (#11105), hence the currently failing tests here. I'll rebase that once that PR is in and the tests here should then pass 🤞
Lastly, I didn't go the whole hog here to make DataSink also be this strict with its uri usage, there is a tiny bit more work on DataSink as a result of recently work, so I may (or may not) make DataSink equally as strict as part of that work in a separate PR.
This PR adds a check to policy, role and namespace list pages to make sure the user has can write those things before offering to create them via a button. (The create page/form would then be a read-only form)
* ui: Don't show the CRD menu for read-only intentions
The UI bug here manifests itself only when a user/token is configured to have read-only access to intentions. Instead of only letting folks click to see a read only page of the intention, we would show an additional message saying that the intention was read-only due to it being 'Managed by [a kubernetes] CRD'. Whilst the intention was still read only, this extra message was still confusing for users.
This PR fixes up the conditional logic and further moves the logic to use ember-can - looking at the history of the files in question, this bug snuck itself in partly due to it being 'permission-y type stuff' previous to using ember-can and when something being editable or not was nothing to do with ACLs. Then we moved to start using ember-can without completely realising what IsEditable previously meant. So overall the code here is a tiny bit clearer/cleaner by adding a proper can view CRD intention instead of overloading the idea of 'editability'.
* ui: Gracefully recover from non-existent DC errors
This PR fixes what happens in the UI if you try to navigate to a non-existing DC.
When we received a 500 error from an API response due to a non-existent DC, previously we would show a 404 error, which is what we were trying to convey. But in the spirit of the UI being a 'thin client', its probably best to just show the 500 error from the API response, which may help folks to debug any issues better.
* Automatically set the CONSUL_DATACENTER_LOCAL env var for testing
* ui: Ignore response from API for KV permissions
Currently there is no way for us to use our HTTP authorization API
endpoint to tell us whether a user has access to any KVs (including the
case where a user may not have access to the root KV store, but do have
access to a sub item)
This is a little weird still as in the above case the user would click
on this link and still get a 403 for the root, and then have to manually
type in the URL for the KV they do have access to.
Despite this we think this change makes sense as at least something about KV is
visible in the main navigation.
Once we have the ability to know if any KVs are accessible, we can add
this guard back in.
We'd initially just removed the logic around the button, but then
noticed there may be further related KV issues due to the nested nature
of KVs so we finally decided on simply ignoring the responses from the
HTTP API, essentially reverting the KV area back to being a thin client.
This means when things are revisited in the backend we can undo this
easily change in one place.
* Move acceptance tests to use ACLs perms instead of KV ones
This PR supersedes #10706 and fixes#10686 whilst making sure that saving intentions continues to work.
The original fix in #10706 ignored the change action configured for the change event on the menus, meaning that the selected source/destination namespace could not be set by the user when editing/creating intentions. This, coupled with the fact that using the later intention exact endpoint for API requests endpoint means that you could not use wildcard namespaces for saving intentions.
All in all this meant that intentions could no longer be saved using the UI (whilst using ENT)
This PR reverts #10706 to fix the intention saving issue, and adds a fix for the original visual issue of nspaces doubling up in the menu once clicked. This meant repeating the existing functionality for nspaces aswell as services. It did seem strange to me that the original issue was only apparent for the nspace menus and not the service menus which should all function exactly the same way.
There is potentially more to come here partly related to what the exact functionality should be, but I'm working with other folks to figure out what the best way forwards is longer term. In the meantime this brings us back to the original functionality with the visual issue fixed.
Squashed commits:
* Revert "ui: Fix dropdown option duplications (#10706)"
This reverts commit eb5512fb74.
* ui: Ensure additional nspaces are added to the unique list of nspaces
* Add some acceptance tests
* Add Partition to all our models
* Add partitions into our serializers/fingerprinting
* Make some amends to a few adapters ready for partitions
* Amend blueprints to avoid linting error
* Update all our repositories to include partitions, also
Remove enabled/disable nspace repo and just use a nspace with
conditionals
* Ensure nspace and parition parameters always return '' no matter what
* Ensure data-sink finds the model properly
This will later be replaced by a @dataSink decorator but we are find
kicking that can down the road a little more
* Add all the new partition data layer
* Add a way to set the title of the page from inside the route
and make it accessibile via a route announcer
* Make the Consul Route the default/basic one
* Tweak nspace and partition abilities not to check the length
* Thread partition through all the components that need it
* Some ACL tweaks
* Move the entire app to use partitions
* Delete all the tests we no longer need
* Update some Unit tests to use partition
* Fix up KV title tests
* Fix up a few more acceptance tests
* Fixup and temporarily ignore some acceptance tests
* Stop using ember-cli-page-objects fillable as it doesn't seem to work
* Fix lint error
* Remove old ACL related test
* Add a tick after filling out forms
* Fix token warning modal
* Found some more places where we need a partition var
* Fixup some more acceptance tests
* Tokens still needs a repo service for CRUD
* Remove acceptance tests we no longer need
* Fixup and "FIXME ignore" a few tests
* Remove an s
* Disable blocking queries for KV to revert to previous release for now
* Fixup adapter tests to follow async/function resolving interface
* Fixup all the serializer integration tests
* Fixup service/repo integration tests
* Fixup deleting acceptance test
* Fixup some ent tests
* Make sure nspaces passes the dc through for when thats important
* ...aaaand acceptance nspaces with the extra dc param
This PR mainly adds partition to our HTTP adapter. Additionally and perhaps most importantly, we've also taken the opportunity to move our 'conditional namespaces' deeper into the app.
The reason for doing this was, we like that namespaces should be thought of as required instead of conditional, 'special' things and would like the same thinking to be applied to partitions.
Now, instead of using code throughout the app throughout the adapters to add/remove namespaces or partitions depending on whether they are enabled or not. As a UI engineer you just pretend that namespaces and partitions are always enabled, and we remove them for you deeper in the app, out of the way of you forgetting to treat these properties as a special case.
Notes:
Added a PartitionAbility while we were there (not used as yet)
Started to remove the CONSTANT variables we had just for property names. I prefer that our adapters are as readable and straightforwards as possible, it just looks like HTTP.
We'll probably remove our formatDatacenter method we use also at some point, it was mainly too make it look the same as our previous formatNspace, but now we don't have that, it instead now looks different!
We enable parsing of partition in the UIs URL, but this is feature flagged so still does nothing just yet.
All of the test changes were related to the fact that we were treating client.url as a function rather than a method, and now that we reference this in client.url (etc) it needs binding to client.
This commit fixes a problem where parent Failovers where not showing (subset children were fine).
Seems to have been introduced with a move/glimmer upgrade here #9154 so I'm adding a 1.9.x backport.
During #9617 we added a list view only for AuthMethods, but not a detail view. We did add the Adapter/Serializer that collected/reshaped data for a detail view.
The test for this serializer was skipped here, but I'm not sure why.
We then added #9845 which began to use this AuthMethod Serializer, but we didn't go back to finish up the skipped test here either.
This PR unskips this test and finishes off the test correctly.
This commit fixes 2 problems with our OIDC flow in the UI, the first is straightforwards, the second is relatively more in depth:
1: A typo (1.10.1 only)
During #10503 we injected our settings service into the our oidc-provider service, there are some comments in the PR as to the whys and wherefores for this change (https://github.com/hashicorp/consul/pull/10503/files#diff-aa2ffda6d0a966ba631c079fa3a5f60a2a1bdc7eed5b3a98ee7b5b682f1cb4c3R28)
Fixing the typo so it was no longer looking for an unknown service (repository/settings > settings)
fixed this.
2: URL encoding (1.9.x, 1.10.x)
TL;DR: /oidc/authorize/provider/with/slashes/code/with/slashes/status/with/slashes should be /oidc/authorize/provider%2Fwith%2Fslashes/code%2Fwith%2Fslashes/status%2Fwith%2Fslashes
When we receive our authorization response back from the OIDC 3rd party, we POST the code and status data from that response back to consul via acallback as part of the OIDC flow. From what I remember back when this feature was originally added, the method is a POST request to avoid folks putting secret-like things into API requests/URLs/query params that are more likely to be visible to the human eye, and POSTing is expected behaviour.
Additionally, in the UI we identify all external resources using unique resource identifiers. Our OIDC flow uses these resources and their identifiers to perform the OIDC flow using a declarative state machine. If any information in these identifiers uses non-URL-safe characters then these characters require URL encoding and we added a helper a while back to specifically help us to do this once we started using this for things that required URL encoding.
The final fix here make sure that we URL encode code and status before using them with one of our unique resource identifiers, just like we do with the majority of other places where we use these identifiers.
Chris S. Kim