Skip to content

T
titan
Switch branch/tag
  1. 20 May, 2021 1 commit
  3. 19 Nov, 2020 1 commit
  5. 17 Sep, 2020 1 commit
  7. 04 Jun, 2020 4 commits
  9. 11 May, 2020 6 commits
  11. 07 May, 2020 1 commit
  13. 20 Mar, 2020 3 commits
  15. 19 Feb, 2020 4 commits
    • yiwu-arbug's avatar
      Reconstruct GC stats on reopen (#130) · 92f86d1b
      yiwu-arbug authored 
      Summary:
* On DB reopen, iterate all SST file's properties and sum up live data size of all blob files to use as GC stats.
* Remove previous logic to set GC mark on blob files on reopen.
* Refactor `OnFlushCompleted` and `OnCompactionCompleted`. Check file state is `kPendingLSM` before transit file state. 
* Refactor `BlobFileMeta`.

Test Plan:
Updated unit tests
Signed-off-by: 's avatarYi Wu <yiwu@pingcap.com>
      92f86d1b
    • yiwu-arbug's avatar
      Sync titan directory when adding blob files (#136) · 72d00eff
      yiwu-arbug authored 
      Summary:
When adding new blob file, titan directory need to be fsync, otherwise files can be lost after power outage.

Test Plan:
run `strace -yfe fsync ./titan_titan_db_test --gtest_filter=*.Snapshot` and see titan directory being fsync-ed in the output.
Signed-off-by: 's avatarYi Wu <yiwu@pingcap.com>
      72d00eff
    • kennytm's avatar
      Fix build on Windows (#138) · 4ea03273
      kennytm authored 
      1. use `#include "port/port.h"` instead of `#include "port/port_posix.h"`
2. added an "install" target for CMake to workaround alexcrichton/cmake-rs#18
3. MSVC reported 2 warnings in Titan (and tons of warnings in RocksDB), and these are fixed.
      4ea03273
    • yiwu-arbug's avatar
      Fix GC stat not being updated after DeleteFilesInRange (#135) · bcd8c4d6
      yiwu-arbug authored 
      Summary:
Fix a typo in `TitanDBImpl::DeleteFilesInRange()` which makes GC stats not being updated after `DeleteFilesInRange()` call.

Test Plan:
See the new test
Signed-off-by: 's avatarYi Wu <yiwu@pingcap.com>
      bcd8c4d6
  17. 31 Dec, 2019 1 commit
  19. 06 Dec, 2019 1 commit
  21. 03 Dec, 2019 1 commit
  23. 26 Nov, 2019 1 commit
    • Wu Jiayu's avatar
      Use prefetcher for level merge (#116) · 812e1977
      Wu Jiayu authored 
      since level merge will read blob files sequentially, we should use prefetcher to accelerate merging.

experiment result:

Load 300GB (average value size 1KB) on Huawei NVME device, time to read blob file reduced about 50%.
Signed-off-by: 's avatarwujy-cs <wujy.cs@gmail.com>
      812e1977
  25. 25 Nov, 2019 1 commit
    • yiwu-arbug's avatar
      TitanDBImpl::Open only open base DB once (#114) · 0fc61eec
      yiwu-arbug authored 
      Summary:
Refactor `TitanDBImpl::Open` to open base DB only once. Previously we open the DB twice - the first time only to obtain `cf_id` for each CF, and then initialize internals before open base DB for real. The patch change to open base DB only once, initialize internals after that. Before `TitanDBImpl` is initialized, `BaseDbListener` will be no-op and `TableBuilder` will not output blob files.

Test Plan:
Added new tests
      0fc61eec
  27. 21 Nov, 2019 1 commit
    • Phoebe Bell's avatar
      Update blob file format for ZSTD dictionary compression (#109) · e5731f44
      Phoebe Bell authored 
      Adding a new version to blob file format. v2 blob files come with 4 more bytes in the header to hold future flags. In particular if `kHasUncompressionDictionary` is set, the whole file is compressed using dictionary compression and the per-blob compression type should be ignored.

Also update `BlobEncoder` and `BlobDecoder` to allow passing in a dictionary and use it to  compress/decompress. Add unit test for it.
      e5731f44
  29. 19 Nov, 2019 1 commit
  31. 18 Nov, 2019 1 commit
  33. 15 Nov, 2019 1 commit
  35. 12 Nov, 2019 1 commit
  37. 06 Nov, 2019 1 commit
  39. 04 Nov, 2019 2 commits
  41. 02 Nov, 2019 1 commit
    • Wu Jiayu's avatar
      Implement basic range merge to maintain last level sorted run number (#92) · e8bc9254
      Wu Jiayu authored 
      With level merge enabled, we expect there are no more than 10 sorted runs of blob files in both of last two levels. But 

- Since we force using level_compaction_dynamic_level_bytes, last level blob files won't be merged again, sorted run number in last level will increase infinitely.
- Some hot range may contain more sorted runs.

This patch implement range merge:

- Check sorted run number of compaction range after last level compaction
- Mark blob files to merge in next compaction if sorted run number > max_sorted_runs
      e8bc9254
  43. 31 Oct, 2019 3 commits
  45. 25 Oct, 2019 1 commit
    • yiwu-arbug's avatar
      titandb_stress print git hash (#99) · 69d79667
      yiwu-arbug authored 
      Summary:
Make titandb_stress print rocksdb and titan git hash at the beginning. This is useful in schordinger test to verify which version is being tested.

Test Plan:
run titandb_stress, which prints the following at the beginning:
```
2019/10/25-05:59:22  Initializing db_stress
RocksDB version           : 6.4
RocksDB hash              : rocksdb_build_git_sha:@9ab5a5f9e70d2ea5fc9bd22a455d52a5d082fd57@
Titan hash                : titan_build_git_sha:@b497731e@
```
Signed-off-by: 's avatarYi Wu <yiwu@pingcap.com>
      69d79667
  47. 17 Oct, 2019 1 commit