From 0a56b29dec94d2ce84d2e5ee5c27949cb79e2a75 Mon Sep 17 00:00:00 2001 From: Jossemar Cordero <95245208+jossemargt-3pillar@users.noreply.github.com> Date: Mon, 28 Feb 2022 11:56:23 -0600 Subject: [PATCH] Add contributors documentation (#5154) Add contributor's documentation addressing: - Git workflow and CI - Coding conventions in Go and POSIX shell languages - Development setup and tooling involved --- docs/contrib/code_conventions.md | 37 +++++++++ docs/contrib/continuous_integration.md | 50 ++++++++++++ docs/contrib/development.md | 82 ++++++++++++++++++++ docs/contrib/git_workflow.md | 102 +++++++++++++++++++++++++ 4 files changed, 271 insertions(+) create mode 100644 docs/contrib/code_conventions.md create mode 100644 docs/contrib/continuous_integration.md create mode 100644 docs/contrib/development.md create mode 100644 docs/contrib/git_workflow.md diff --git a/docs/contrib/code_conventions.md b/docs/contrib/code_conventions.md new file mode 100644 index 0000000000..85e24b1d01 --- /dev/null +++ b/docs/contrib/code_conventions.md @@ -0,0 +1,37 @@ +# Code conventions + +- [POSIX shell](#posix-shell) +- [Go](#go) +- [Directory and file conventions](#directory-and-file-conventions) +- [Testing conventions](#testing-conventions) + +## POSIX shell + +- [Style guide](https://google.github.io/styleguide/shell.xml) + +## Go + +- [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments) +- [Effective Go](https://golang.org/doc/effective_go.html) +- Know and avoid [Go landmines](https://gist.github.com/lavalamp/4bd23295a9f32706a48f) +- Comment your code. + - [Go's commenting conventions](http://blog.golang.org/godoc-documenting-go-code) + - If reviewers ask questions about why the code is the way it is, that's a sign that comments might be helpful. +- Command-line flags should use dashes, not underscores +- Naming + - Please consider package name when selecting an interface name, and avoid redundancy. For example, `storage.Interface` is better than `storage.StorageInterface`. + - Do not use uppercase characters, underscores, or dashes in package names. + - Please consider parent directory name when choosing a package name. For example, `pkg/controllers/autoscaler/foo.go` should say `package autoscaler` not `package autoscalercontroller`. + - Unless there's a good reason, the `package foo` line should match the name of the directory in which the `.go` file exists. + - Importers can use a different name if they need to disambiguate. + +## Directory and file conventions + +- Avoid general utility packages. Packages called "util" are suspect. Instead, derive a name that describes your desired function. For example, the utility functions dealing with waiting for operations are in the `wait` package and include functionality like `Poll`. The full name is `wait.Poll`. +- All filenames should be lowercase. +- All source files and directories should use underscores, not dashes. + - Package directories should generally avoid using separators as much as possible. When package names are multiple words, they usually should be in nested subdirectories. + +## Testing conventions + +Please refer to [TESTING.md](../../tests/TESTING.md) document. diff --git a/docs/contrib/continuous_integration.md b/docs/contrib/continuous_integration.md new file mode 100644 index 0000000000..3a6f3f4db5 --- /dev/null +++ b/docs/contrib/continuous_integration.md @@ -0,0 +1,50 @@ +# Continuous Integration and Automation + +Every change on the K3s repository, either made through a pull request or direct push, triggers the continuous integration pipelines defined within the same repository. Needless to say, all the K3s contributions can be merged until all the checks pass (AKA having green builds). + +- [CI Platforms](#ci-platforms) + - [GitHub Actions](#github-actions) + - [Drone Pipelines](#drone-pipelines) +- [Running locally](#running-locally) + +## CI Platforms + +Currently, there are two different platforms involved in running the CI processes: + +- GitHub actions +- Drone pipelines on CNCF infrastructure + +### GitHub Actions + +All the existing GitHub Actions are defined as YAML files under the `.github/workflows` directory. These can be grouped into: + +- **PR Checks**. These actions run all the required validations upon PR creation and update. Covering the DCO compliance check, `x86_64` test batteries (unit, integration, smoke), and code coverage. +- **Repository automation**. Currently, it only covers issues and epic grooming. + +Everything runs on GitHub's provided runners; thus, the tests are limited to run in `x86_64` architectures. + +### Drone Pipelines + +The Drone pipelines are defined in the `.drone.yml` file. These are designed to be run on separated clusters depending on their end goal, being categorized into: + +- **Continuous Integration**. It runs code linting and test batteries for x86_64, arm64, and arm architectures. +- **Publish Artifacts**. It runs the same checks as the CI one but adds the [fossa scan](https://fossa.com/) and the K3s artifacts building, packaging, and publishing steps. This cluster is involved mainly in K3s releases. + +The K3s contributors do not need access to these drone clusters to get their PR's built. Still, for reference, these are their corresponding URLs and . + +## Running locally + +A contributor should verify their changes locally to speed up the pull request process. Fortunately, all the CI steps can be on local environments, except for the publishing ones, through either of the following methods: + +- **Drone local execution**. The drone CLI allows the user to execute a specific pipeline locally through the `drone exec --pipeline ` command. This requires having Docker with volume mount support. +- **Dapper**. The [dapper tool](https://github.com/rancher/dapper) is a "Docker wrapper" that enables the user to run a set of instructions within a Docker container without relying on volume mounts. This tool is always used when any K3s [Makefile](../../Makefile) goal is invoked, which use as an entrypoint for the CI scripts. +- **Step direct invocation**. The CI steps invoked on the drone pipelines are defined in the scripts directory, which can be executed individually in the local environment. Worth noting, these scripts rely on GNU utils, which are not the same as the BSD ones installed on macOS environments, although they have the same name. + +As mentioned above, the scripts within the `scripts` directory are the core of the CI process, being the most relevant ones: + +- **validate**. Executes the `go generate` command and the linting tools, then asserts the K3s components versions. +- **test**. Triggers the unit and integration test batteries. This is further elaborated in the [TESTING.md](../../tests/TESTING.md) document. +- **build**. Builds all the K3s binaries. +- **package**. Triggers the packaging processes for K3s binaries and airgap images. +- **ci**. Use as the CI scripts entrypoint, orchestrating all the required steps. +- **clean**. Remove built binaries and packages. diff --git a/docs/contrib/development.md b/docs/contrib/development.md new file mode 100644 index 0000000000..27e2d6c103 --- /dev/null +++ b/docs/contrib/development.md @@ -0,0 +1,82 @@ +# Development Guide + +Since K3s is written in Go, it is fair to assume that the Go tools are all one needs to contribute to this project. Unfortunately, there is a point where this no longer holds true when required to test or build local changes. This document elaborates on the required tooling for K3s development. + +- [Non-Linux environment prerequisites](#non-linux-environment-prerequisites) + - [Windows Setup](#windows-setup) + - [macOS Setup](#macos-setup) +- [Installing Required Software](#installing-required-software) + - [Go](#go) + - [Docker](#docker) + - [Vagrant](#vagrant) +- [Cloning, Building and Testing K3s](#cloning-building-and-testing-k3s) +- [Dependency management](#dependency-management) + +## Non-Linux environment prerequisites + +All the test and build scripts within this repository were created to be run on GNU Linux development environments. Due to this, it is suggested to use the virtual machine defined on this repository's [Vagrantfile](../../Vagrantfile) to use them. + +Either way, if one still wants to build and test K3s on non-Linux environments, specific setups are to be followed. + +### Windows Setup + +To build K3s on Windows is only possible for versions that support Windows Subsystem for Linux (WSL). If the development environment in question has Windows 10, Version 2004, Build 19041 or higher, [follow these instructions to install WSL2](https://docs.microsoft.com/en-us/windows/wsl/install-win10); otherwise, use a Linux Virtual machine instead. + +### macOS Setup + +The shell scripts in charge of the build and test processes rely on GNU utils (i.e. `sed`), [which slightly differ on macOS](https://unix.stackexchange.com/a/79357), meaning that one must make some adjustments before using them. + +First, install the GNU utils: + +```sh +brew install coreutils findutils gawk gnu-sed gnu-tar grep make +``` + +Then update the shell init script (i.e. `.bashrc`) to prepend the GNU Utils to the `$PATH` variable + +```sh +GNUBINS="$(find /usr/local/opt -type d -follow -name gnubin -print)" + +for bindir in ${GNUBINS[@]}; do + PATH=$bindir:$PATH +done + +export PATH +``` + +## Installing Required Software + +### Go + +It is well known that K3s is written in [Go](http://golang.org). Please follow the [Go Getting Started guide](https://golang.org/doc/install) to install and set up the Go tools used to compile and run the test batteries. + +**Note:** K3s uses the same Go version as the Kubernetes components underneath. The table below lists the required Go versions for supported the Kubernetes releases. + +| Kubernetes | requires Go | +|----------------|-------------| +| 1.19 - 1.20 | 1.15.5 | +| 1.21 - 1.22 | 1.16.7 | +| 1.23+ | 1.17 | + +### Docker + +K3s build and test processes development require Docker to run certain steps. [Follow the Docker website instructions to install Docker](https://docs.docker.com/get-docker/) in the development environment. + +### Vagrant + +As described in the [Testing documentation](../../tests/TESTING.md), all the smoke tests are run in virtual machines managed by Vagrant. To install Vagrant in the development environment, [follow the instructions from the Hashicorp website](https://www.vagrantup.com/downloads), alongside any of the following hypervisors: + +- [VirtualBox](https://www.virtualbox.org/) +- [libvirt](https://libvirt.org/) and the [vagrant-libvirt plugin](https://github.com/vagrant-libvirt/vagrant-libvirt#installation) + +## Cloning, Building and Testing K3s + +These topics already have been addressed on their respective documents: + +- [Git Workflow](./git-workflow.md) +- [Building](../../BUILDING.md) +- [Testing](../../tests/TESTING.md) + +## Dependency management + +K3s uses [go modules](https://github.com/golang/go/wiki/Modules) to manage dependencies. diff --git a/docs/contrib/git_workflow.md b/docs/contrib/git_workflow.md new file mode 100644 index 0000000000..bff898c553 --- /dev/null +++ b/docs/contrib/git_workflow.md @@ -0,0 +1,102 @@ +# Git workflows + +This document is an overview of K3s git workflow. It includes conventions, tips, and how to maintain good repository hygiene. + +- [Branching model](#branching-model) + - [Branch naming conventions](#branch-naming-conventions) + - [Backport policy](#backport-policy) +- [Git operations](#git-operations) + - [Setting up](#setting-up) + - [Branching out](#branching-out) + - [Keeping local branches in sync](#keeping-local-branches-in-sync) + - [Pushing changes](#pushing-changes) + +## Branching model + +K3s project uses the [GitHub flow](https://docs.github.com/en/get-started/quickstart/github-flow) as its branching model, where most of the changes come from repositories forks instead of branches within the same one. + +### Branch naming conventions + +Every forked repository works independently, meaning that any contributor can create branches with the name they see fit. However, it is worth noting that K3s mirrors [Kubernetes version skew policy](https://kubernetes.io/releases/version-skew-policy/) by maintaining release branches for the most recent three minor releases. The only exception is that the main branch mirrors the latest Kubernetes release (1.23) instead of using a `release-` prefixed one. + +```text +master -------------------------------------------. (Kubernetes 1.23) +release-1.21 \---------------|---------------. (Kubernetes 1.21) +release-1.22 \---------------. (Kubernetes 1.22) +``` + +Upon Kubernetes release, each branch will be tagged accordingly to the version they mirror, and the artifacts will be built from there. + +### Backport policy + +All new work happens on the main branch, which means that for most cases, one should branch out from there and create the pull request against it. If the change involves adding a feature or patching K3s, the maintainers will backport it into the supported release branches. + +## Git operations + +There are everyday tasks related to git that every contributor needs to perform, and this section elaborates on them. + +### Setting up + +Creating a K3s fork, cloning it, and setting its upstream remote can be summarized on: + +1. Visit +2. Click the `Fork` button (top right) to establish a cloud-based fork +3. Clone fork to local storage +4. Add to your fork K3s remote as upstream + +Once cloned, in code it would look this way: + +```sh +## Clone fork to local storage +export user="your github profile name" +git clone https://github.com/$user/k3s.git +# or: git clone git@github.com:$user/k3s.git + +## Add k3s as upstream to your fork +cd k3s +git remote add upstream https://github.com/k3s-io/k3s.git +# or: git remote add upstream git@github.com:k3s-io/k3s.git + +## Ensure to never push to upstream directly +git remote set-url --push upstream no_push + +## Confirm that your remotes make sense: +git remote -v +``` + +### Branching out + +Every time one wants to work on a new K3s feature, we do: + +1. Get local main branch up to date +2. Create a new branch from the main one (i.e.: myfeature branch ) + +In code it would look this way: + +```sh +## Get local master up to date +# Assuming the k3s clone is the current working directory +git fetch upstream +git checkout master +git rebase upstream/master + +## Create a new branch from master +git checkout -b myfeature +``` + +### Keeping local branches in sync + +Either when branching out from master or a release one, keep in mind it is worth checking if any change has been pushed upstream by doing: + +```sh +git fetch upstream +git rebase upstream/master +``` + +It is suggested to `fetch` then `rebase` instead of `pull` since the latter does a merge, which leaves merge commits. For this, one can consider changing the local repository configuration by doing `git config branch.autoSetupRebase always` to change the behavior of `git pull`, or another non-merge option such as `git pull --rebase`. + +### Pushing changes + +For commit messages and signatures please refer to the [CONTRIBUTING.md](../../CONTRIBUTING.md) document. + +Nobody should push directly to upstream, even if one has such contributor access; instead, prefer [Github's pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests) mechanism to contribute back into K3s. For expectations and guidelines about pull requests, consult the [CONTRIBUTING.md](../../CONTRIBUTING.md) document.