prometheus

Commit Graph

Author	SHA1	Message	Date
Bryan Boreham	1848623c77	tsdb: re-use iterator when stepping through chunks Saves memory allocations, hence reduces garbage-collection overheads. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	ccea61c7bf	Merge pull request #11717 from bboreham/labels-abstraction Add and use abstractions over labels.Labels	2 years ago
Ganesh Vernekar	6fd89a6fd2	Add chunk encoding for float histogram (#11716 ) Signed-off-by: Marc Tudurí <marctc@protonmail.com> Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Co-authored-by: Marc Tudurí <marctc@protonmail.com>	2 years ago
Bryan Boreham	10b27dfb84	Simplify IndexReader.Series interface Instead of passing in a `ScratchBuilder` and `Labels`, just pass the builder and the caller can extract labels from it. In many cases the caller didn't use the Labels value anyway. Now in `Labels.ScratchBuilder` we need a slightly different API: one to assign what will be the result, instead of overwriting some other `Labels`. This is safer and easier to reason about. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	4b6a4d1425	Update package tsdb tests for new labels.Labels type Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	ce2cfad0cb	Update package tsdb/record for new labels.Labels type Implement decoding via labels.ScratchBuilder, which we retain and re-use to reduce memory allocations. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	543c318ec2	Update package tsdb for new labels.Labels type Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	f0ec81badd	Update package tsdb/test for new labels.Labels type Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	14ad2e780b	Update package tsdb/agent for new labels.Labels type Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	a5bdff414b	Update package tsdb/index tests for new labels.Labels type Note in one cases we needed an extra copy of labels in case they change. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	d3d96ec887	tsdb/index: use ScratchBuilder to create Labels This necessitates a change to the `tsdb.IndexReader` interface: `index.Reader` is used from multiple goroutines concurrently, so we can't have state in it. We do retain a `ScratchBuilder` in `blockBaseSeriesSet` which is iterator-like. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	927a14b0e9	Update package tsdb/index for new labels.Labels type Incomplete - needs further changes to `Decoder.Series()`. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	89bf6e1df9	tsdb: Tidy up some test code Use simpler utility function to create Labels objects, making fewer assumptions about the data structure. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	0853250695	Review feedback Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	463f5cafdd	storage: re-use iterators to save garbage Re-use previous memory if it is already of the correct type. In `NewListSeries` we hoist the conversion to an interface value out so it only allocates once. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	f0866c0774	tsdb: optimise block series iterators Re-use previous memory if it is already of the correct type. Also turn two levels of function closure into a single object that holds the required data. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	3c7de69059	storage: allow re-use of iterators Patterned after `Chunk.Iterator()`: pass the old iterator in so it can be re-used to avoid allocating a new object. (This commit does not do any re-use; it is just changing all the method signatures so re-use is possible in later commits.) Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Julien Pivotto	475cfe8a6b	Merge remote-tracking branch 'origin/release-2.40' Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	2 years ago
Ganesh Vernekar	db99fc43e4	Merge pull request #11632 from bboreham/improve-bbss tsdb: improve blockBaseSeriesSet scan	2 years ago
Ganesh Vernekar	54739a1465	Merge pull request #11674 from bboreham/fix-tsdb-test-mem tsdb tests: allocate more reasonable sample slice	2 years ago
beorn7	5f366e9b62	histograms: Improve tests and fix exposed bugs This adds negative buckets and access of float histograms to TestHistogramChunkSameBuckets and TestHistogramChunkBucketChanges. It also exercises a specific pattern of reusing an iterator (one where no access has happened). This exposes two bugs (where entries for positive buckets where used where the corresponding entries for negative buckets should have been used). One was fixed in #11627 (not merged), which triggered the work in this commit. This commit fixes both issues, so #11627 can be closed. It also simplifies the code in the histogramIterator.Next method that aims to recycle existing slice capacity. Furthermore, this is on top of the release-2.40 branch because we should probably cut a bugfix release for this. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Julien Pivotto	0b302f8a39	Merge pull request #11662 from prometheus/release-2.40 Merge back release-2.40 branch again	2 years ago
Bryan Boreham	9853888f9b	tsdb tests: allocate more reasonable sample slice Typical parameters are one hour by 1 minute step, where the function would allocate a slice of 3.6 million samples instead of 60. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Ganesh Vernekar	72a48321da	Merge pull request #11633 from pstibrany/populate-error Enhance "cannot populate chunk" error message to include source block ID	2 years ago
Ganesh Vernekar	b8b0d45d69	Fix reset of a histogram chunk iterator Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Julien Pivotto	0372e259ba	Merge pull request #11634 from prometheus/release-2.40 Merge release-2.40 branch into main	2 years ago
Bryan Boreham	6bdecf377c	Switch from 'sanity' to more inclusive lanuage (#9376 ) * Switch from 'sanity' to more inclusive lanuage "Removing ableist language in code is important; it helps to create and maintain an environment that welcomes all developers of all backgrounds, while emphasizing that we as developers select the most articulate, precise, descriptive language we can rather than relying on metaphors. The phrase sanity check is ableist, and unnecessarily references mental health in our code bases. It denotes that people with mental illnesses are inferior, wrong, or incorrect, and the phrase sanity continues to be used by employers and other individuals to discriminate against these people." From https://gist.github.com/seanmhanson/fe370c2d8bd2b3228680e38899baf5cc Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Peter Štibraný	af838ccf83	Include source block in error message when loading chunk fails. Signed-off-by: Peter Štibraný <pstibrany@gmail.com>	2 years ago
Bryan Boreham	1226922ff5	tsdb: improve blockBaseSeriesSet scan Inverting the test for chunks deleted by tombstones makes all three rejections consistent, and also avoids the case where a chunk is excluded but still causes `trimFront` or `trimBack` to be set. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	0c05f95e92	tsdb: use smaller allocation in blockBaseSeriesSet This reduces garbage, hence goes faster, when a short time range is required compared to the amount of chunks in the block. For example recording rules and alerts often look only at the last few minutes. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Ganesh Vernekar	ad79fb9f25	Do not error on empty chunk during iteration in populateWithDelChunkSeriesIterator Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	d0e683e26d	Add TestCompactHeadWithDeletion to test compaction failure after deletion Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	42633bd05c	Merge pull request #11485 from t00350320/prometheus-office GetRefByhash() will query a label's ref with hash value rather than lset.Hash().	2 years ago
tanghengjian	982007ecab	GetRefByhash will query a label's ref with hash value rather than lset.Hash(). Signed-off-by: tanghengjian <1040104807@qq.com>	2 years ago
Oleg Zaytsev	8553a98267	Optimize postings offset table reading (#11535 ) * Add BenchmarkOpenBlock * Use specific types when reading offset table Instead of reading a generic-ish []string, we can read a generic type which would be specifically labels.Label. This avoid allocating a slice that escapes to the heap, making it both faster and more efficient in terms of memory management. * Update error message for unexpected number of keys * s/posting offset table/postings offset table/ * Remove useless lastKey assignment * Use two []bytes vars, simplify Applied PR feedback: removed generics, moved the label indices reading to that specific test as we're not using it in production anyway, we're just testing what we've just built. Also using two []bytes variables for name and value that use the backing buffer instead of using strings, this reduces allocations a lot as we only copy them when we store them (this is optimized by the compiler). * Fix the dumb bug Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com> Co-authored-by: Marco Pracucci <marco@pracucci.com>	2 years ago
Julien Pivotto	739494d81b	Fix alignment of atomic int64 (#11547 ) * Fix atomix int64 placement * Test main for 386 Signed-off-by: Julien Pivotto <roidelapluie@o11y.eu>	2 years ago
Ganesh Vernekar	fa6e05903f	Merge pull request #11447 from prometheus/sparsehistogram Add Support for Native Histograms This PR merges all the coding work that has been done in sparsehistogram branch over the last 1 year into main branch. Design doc on native histograms: https://docs.google.com/document/d/1cLNv3aufPZb3fNfaJgdaRBZsInZKKIHo9E6HinJVbpM/edit Some sneak peak: https://www.youtube.com/watch?v=T2GvcYNth9U	2 years ago
Viacheslav Panasovets	3d2e18bad5	Fix time.Since() in defer. Wrap in anonymous function (#11489 ) Function arguments in defer evaluated during definition of defer, not during execution Signed-off-by: Slavik Panasovets <slavik@google.com> Signed-off-by: Slavik Panasovets <slavik@google.com>	2 years ago
Björn Rabenstein	503ffba49a	chunkenc: Slightly optimize xorWrite/xoRead (#11476 ) With these changes, the "happy path" when the leading and trailing number of bits don't need an update, fewer operations are needed. The change is probably very marginal (no change in the benchmark added here, but the benchmark also doesn't cover non-changing values), and an argument could me made that avoiding pointers also has its benefits. However, I think that reducing the number of return values improves readability. Which convinced me that I should at least propose this. Signed-off-by: beorn7 <beorn@grafana.com>	2 years ago
Ganesh Vernekar	8ee4dfd40c	Fix the build after conflict resolution Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	648be89822	Merge remote-tracking branch 'upstream/main' into fix-conflict Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	8e29110949	Add/Improve unit tests for compaction with histogram (#11342 ) Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Ganesh Vernekar	507bfa46fd	Fix HistogramChunk's AtFloatHistogram() Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Signed-off-by: Jesus Vazquez	3362bf6d79	Fix merge conflicts Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com> Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com> Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Jesus Vazquez	775d90d5f8	TSDB: Rename wal package to wlog (#11352 ) The wlog.WL type can now be used to create a Write Ahead Log or a Write Behind Log. Before the prefix for wbl metrics was 'prometheus_tsdb_out_of_order_wal_' and has been replaced with 'prometheus_tsdb_out_of_order_wbl_'. Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com> Signed-off-by: Jesus Vazquez <jesusvazquez@users.noreply.github.com> Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>	2 years ago
Sonali Rajput	9165aedb49	Fixed broken link in tsdb README.md Signed-off-by: Sonali Rajput <sonalirajput1088@gmail.com>	2 years ago
Jesus Vazquez	e934d0f011	Merge 'main' into sparsehistogram Signed-off-by: Jesus Vazquez <jesus.vazquez@grafana.com>	2 years ago
Ganesh Vernekar	d0a6488c74	Update metrics for histograms Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>	2 years ago
Bryan Boreham	9b31adc4e8	tsdb: fix up sort call with faster slices.Sort (#11380 ) This call was added by PR #11075 merged before #11318 which changed all similar calls to `sort.Sort` into a faster one. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago
Bryan Boreham	3330d85ba8	Replace sort.Strings and sort.Ints with faster slices.Sort (#11318 ) Use new experimental package `golang.org/x/exp/slices`. slices.Sort works on values that are directly comparable, like ints, so avoids the overhad of an interface call to `.Less()`. Left tests unchanged, because they don't need the speed and it may be a cross-check that slices.Sort gives the same answer. Signed-off-by: Bryan Boreham <bjboreham@gmail.com>	2 years ago

1 2 3 4 5 ...

647 Commits (d7f0276d880f0a94965aed76dadfa96227e08683)