Skip to content

T
titan
Switch branch/tag
  1. 09 Jun, 2020 1 commit
  3. 04 Jun, 2020 2 commits
  5. 01 Jun, 2020 1 commit
  7. 29 May, 2020 1 commit
  9. 07 May, 2020 1 commit
  11. 06 May, 2020 1 commit
  13. 29 Apr, 2020 1 commit
  15. 02 Apr, 2020 1 commit
  17. 01 Apr, 2020 1 commit
  19. 27 Mar, 2020 2 commits
  21. 20 Mar, 2020 1 commit
    • Wallace's avatar
      fix warn (#147) · f6dc570f
      Wallace authored 
      Signed-off-by: 's avatarLittle-Wallace <bupt2013211450@gmail.com>

Fix warn when titan is compiled with gcc 5.5.0
```
/home/pingcap/.cargo/git/checkouts/rust-rocksdb-e0eeb217423298aa/0e1a085/librocksdb_sys/libtitan_sys/titan/src/db_impl_gc.cc: In member function ‘rocksdb::Status rocksdb::titandb::TitanDBImpl::BackgroundGC(rocksdb::LogBuffer*, uint32_t)’:
/home/pingcap/.cargo/git/checkouts/rust-rocksdb-e0eeb217423298aa/0e1a085/librocksdb_sys/libtitan_sys/titan/src/db_impl_gc.cc:205:55: error: ‘gc_merge_rewrite’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
                           &shuting_down_, stats_.get());
```
      f6dc570f
  23. 04 Mar, 2020 1 commit
  25. 15 Feb, 2020 1 commit
  27. 04 Feb, 2020 2 commits
    • yiwu-arbug's avatar
      Reconstruct GC stats on reopen (#130) · ffaa9d1a
      yiwu-arbug authored 
      Summary:
* On DB reopen, iterate all SST file's properties and sum up live data size of all blob files to use as GC stats.
* Remove previous logic to set GC mark on blob files on reopen.
* Refactor `OnFlushCompleted` and `OnCompactionCompleted`. Check file state is `kPendingLSM` before transit file state. 
* Refactor `BlobFileMeta`.

Test Plan:
Updated unit tests
Signed-off-by: 's avatarYi Wu <yiwu@pingcap.com>
      ffaa9d1a
    • yiwu-arbug's avatar
      Sync titan directory when adding blob files (#136) · 3b267dc5
      yiwu-arbug authored 
      Summary:
When adding new blob file, titan directory need to be fsync, otherwise files can be lost after power outage.

Test Plan:
run `strace -yfe fsync ./titan_titan_db_test --gtest_filter=*.Snapshot` and see titan directory being fsync-ed in the output.
Signed-off-by: 's avatarYi Wu <yiwu@pingcap.com>
      3b267dc5
  29. 31 Jan, 2020 1 commit
    • kennytm's avatar
      Fix build on Windows (#138) · 01f293da
      kennytm authored 
      1. use `#include "port/port.h"` instead of `#include "port/port_posix.h"`
2. added an "install" target for CMake to workaround alexcrichton/cmake-rs#18
3. MSVC reported 2 warnings in Titan (and tons of warnings in RocksDB), and these are fixed.
      01f293da
  31. 30 Jan, 2020 1 commit
  33. 31 Dec, 2019 1 commit
  35. 06 Dec, 2019 1 commit
  37. 03 Dec, 2019 1 commit
  39. 26 Nov, 2019 1 commit
    • Wu Jiayu's avatar
      Use prefetcher for level merge (#116) · 812e1977
      Wu Jiayu authored 
      since level merge will read blob files sequentially, we should use prefetcher to accelerate merging.

experiment result:

Load 300GB (average value size 1KB) on Huawei NVME device, time to read blob file reduced about 50%.
Signed-off-by: 's avatarwujy-cs <wujy.cs@gmail.com>
      812e1977
  41. 25 Nov, 2019 1 commit
    • yiwu-arbug's avatar
      TitanDBImpl::Open only open base DB once (#114) · 0fc61eec
      yiwu-arbug authored 
      Summary:
Refactor `TitanDBImpl::Open` to open base DB only once. Previously we open the DB twice - the first time only to obtain `cf_id` for each CF, and then initialize internals before open base DB for real. The patch change to open base DB only once, initialize internals after that. Before `TitanDBImpl` is initialized, `BaseDbListener` will be no-op and `TableBuilder` will not output blob files.

Test Plan:
Added new tests
      0fc61eec
  43. 21 Nov, 2019 1 commit
    • Phoebe Bell's avatar
      Update blob file format for ZSTD dictionary compression (#109) · e5731f44
      Phoebe Bell authored 
      Adding a new version to blob file format. v2 blob files come with 4 more bytes in the header to hold future flags. In particular if `kHasUncompressionDictionary` is set, the whole file is compressed using dictionary compression and the per-blob compression type should be ignored.

Also update `BlobEncoder` and `BlobDecoder` to allow passing in a dictionary and use it to  compress/decompress. Add unit test for it.
      e5731f44
  45. 19 Nov, 2019 1 commit
  47. 18 Nov, 2019 1 commit
  49. 15 Nov, 2019 1 commit
  51. 12 Nov, 2019 1 commit
  53. 06 Nov, 2019 1 commit
  55. 04 Nov, 2019 2 commits
  57. 02 Nov, 2019 1 commit
    • Wu Jiayu's avatar
      Implement basic range merge to maintain last level sorted run number (#92) · e8bc9254
      Wu Jiayu authored 
      With level merge enabled, we expect there are no more than 10 sorted runs of blob files in both of last two levels. But 

- Since we force using level_compaction_dynamic_level_bytes, last level blob files won't be merged again, sorted run number in last level will increase infinitely.
- Some hot range may contain more sorted runs.

This patch implement range merge:

- Check sorted run number of compaction range after last level compaction
- Mark blob files to merge in next compaction if sorted run number > max_sorted_runs
      e8bc9254
  59. 31 Oct, 2019 3 commits
  61. 25 Oct, 2019 1 commit
    • yiwu-arbug's avatar
      titandb_stress print git hash (#99) · 69d79667
      yiwu-arbug authored 
      Summary:
Make titandb_stress print rocksdb and titan git hash at the beginning. This is useful in schordinger test to verify which version is being tested.

Test Plan:
run titandb_stress, which prints the following at the beginning:
```
2019/10/25-05:59:22  Initializing db_stress
RocksDB version           : 6.4
RocksDB hash              : rocksdb_build_git_sha:@9ab5a5f9e70d2ea5fc9bd22a455d52a5d082fd57@
Titan hash                : titan_build_git_sha:@b497731e@
```
Signed-off-by: 's avatarYi Wu <yiwu@pingcap.com>
      69d79667
  63. 17 Oct, 2019 1 commit
  65. 16 Oct, 2019 2 commits