1. 08 Jul, 2019 1 commit
    • dennis zhuang's avatar
      (feat) Supports partitioned index filters settings for BlockBasedOptions (#294) · 56865683
      dennis zhuang authored
      This PR exports more options in RocksDB `BlockBasedTableOptions`, because we want to use [Partitioned Index Filters](https://github.com/facebook/rocksdb/wiki/Partitioned-Index-Filters#how-to-use-it).
      
      There is an issue [cache_index_and_filter_blocks causes Get slowdown](https://github.com/facebook/rocksdb/issues/3961) in rocksdb, and we test the recommend `Partitioned Index Filters` in java JNI binding, it works well.
      
      So i think these options should be exported to rust-rocksdb users too.
      
      The rocksdb log in test case `test_partitioned_index_filters` as below:
      
      ```
        cache_index_and_filter_blocks: 1
        cache_index_and_filter_blocks_with_high_priority: 1
        pin_l0_filter_and_index_blocks_in_cache: 1
        pin_top_level_index_and_filter: 1
        index_type: 2
        hash_index_allow_collision: 1
        checksum: 1
        no_block_cache: 0
        block_cache: 0x7ff4540060d0
        block_cache_name: LRUCache
        block_cache_options:
          capacity : 8388608
          num_shard_bits : 4
          strict_capacity_limit : 0
          memory_allocator : None
          high_pri_pool_ratio: 0.000
        block_cache_compressed: (nil)
        persistent_cache: (nil)
        block_size: 4096
        block_size_deviation: 10
        block_restart_interval: 16
        index_block_restart_interval: 1
        metadata_block_size: 4096
        partition_filters: 1
        use_delta_encoding: 1
        filter_policy: rocksdb.BuiltinBloomFilter
        whole_key_filtering: 1
        verify_compression: 0
        read_amp_bytes_per_bit: 0
        format_version: 2
        enable_index_compression: 1
        block_align: 0
      ```
      
      It looks working.
      
      Thanks you for this great project.
      56865683
  2. 04 Jul, 2019 1 commit
  3. 03 Jul, 2019 1 commit
    • yiwu-arbug's avatar
      Fix Titan portable build (#312) · c503e736
      yiwu-arbug authored
      Fix build script not passing portable flag to titan.
      
      Tested by build with `cargo build -vv --features portable` and check actual compile command does not contain -march=native flag. e.g.
      ```
      [libtitan_sys 0.0.1] /usr/bin/c++  -DBZIP2 -DHAVE_PCLMUL -DHAVE_SSE42 -DLZ4 -DOS_LINUX -DROCKSDB_FALLOCATE_PRESENT -DROCKSDB_LIB_IO_POSIX -DROCKSDB_MALLOC_USABLE_SIZE -DROCKSDB_PLATFORM_POSIX -DROCKSDB_PTHREAD_ADAPTIVE_MUTEX -DROCKSDB_RANGESYNC_PRESENT -DROCKSDB_SCHED_GETCPU_PRESENT -DROCKSDB_SUPPORT_THREAD_LOCAL -DSNAPPY -DZLIB -DZSTD -I/data/rust-rocksdb    /target/debug/build/snappy-sys-ff485584675450be/out/build -I/data/rust-rocksdb/target/debug/build/bzip2-sys-593753ebc763ecf6/out/include -I/data/rust-rocksdb/target/debug/build/lz4-sys-c5ffb8280e1f82b8/out/include -I/data/rust-rocksdb/target/debug/build/libz-sys-da832d3a86675e51/out/include -I/data/rust-rocksdb/target/debug/build/zstd-sys-ddb8409269514a6a/    out/include -I/data/rust-rocksdb/librocksdb_sys/libtitan_sys/../rocksdb -I/data/rust-rocksdb/librocksdb_sys/libtitan_sys/../rocksdb/include -I/data/rust-rocksdb/librocksdb_sys/libtitan_sys/titan/include -I/data/rust-rocksdb/librocksdb_sys/libtitan_sys/titan/src  -ffunction-sections -fdata-sections -fPIC -m64 -W -Wextra -Wall -Wsign-compare -Wshadow -Wno-un    used-parameter -Wno-unused-variable -Woverloaded-virtual -Wnon-virtual-dtor -Wno-missing-field-initializers -Wno-strict-aliasing -std=c++11 -Werror -fno-builtin-memcmp -g -DROCKSDB_USE_RTTI   -o CMakeFiles/titan.dir/src/blob_file_cache.cc.o -c /data/rust-rocksdb/librocksdb_sys/libtitan_sys/titan/src/blob_file_cache.cc
      ```
      Signed-off-by: 's avatarYi Wu <yiwu@pingcap.com>
      c503e736
  4. 01 Jul, 2019 1 commit
    • yiwu-arbug's avatar
      Fix create_cf not passinig titan specific options to titan (#311) · c17648a5
      yiwu-arbug authored
      Summary:
      `db.create_cf` call `crocksdb_create_column_family`, which doesn't pass titan specific options to titan. Fix it by calling `ctitandb_create_column_family` when titan is enabled.
      
      Test Plan:
      Manual test when enabled titan, tikv pass correct titan options when creating new CFs.
      
      Will add unit test in later patch.
      Signed-off-by: 's avatarYi Wu <yiwu@pingcap.com>
      c17648a5
  5. 24 Jun, 2019 1 commit
  6. 21 Jun, 2019 1 commit
  7. 20 Jun, 2019 1 commit
  8. 19 Jun, 2019 1 commit
  9. 14 Jun, 2019 2 commits
  10. 05 Jun, 2019 2 commits
  11. 10 May, 2019 1 commit
  12. 24 Apr, 2019 1 commit
  13. 23 Apr, 2019 1 commit
  14. 16 Apr, 2019 1 commit
    • yiwu-arbug's avatar
      Update RocksDB (#288) · 44d2a52a
      yiwu-arbug authored
      Update RocksDB to include the following changes:
      018aa1972 Titan: fix error message formatting (#84)
      ee2f17ca8 Fix crash with memtable prefix bloom and key out of prefix extractor domain (#5190)
      16ace7c47 Titan: Add snapshot related test (#80)
      ee8b59301 Titan: Fix iterator fail to pass SuperVersion to underlying iterator (#79)
      
      Signed-off-by: Yi Wu yiwu@pingcap.com
      44d2a52a
  15. 12 Apr, 2019 1 commit
  16. 11 Apr, 2019 1 commit
  17. 03 Apr, 2019 1 commit
  18. 02 Apr, 2019 1 commit
  19. 29 Mar, 2019 1 commit
  20. 25 Mar, 2019 1 commit
  21. 22 Mar, 2019 1 commit
  22. 27 Feb, 2019 1 commit
  23. 19 Feb, 2019 1 commit
  24. 13 Feb, 2019 1 commit
  25. 26 Jan, 2019 1 commit
  26. 21 Jan, 2019 1 commit
  27. 06 Jan, 2019 1 commit
  28. 04 Jan, 2019 1 commit
  29. 02 Jan, 2019 2 commits
  30. 28 Dec, 2018 1 commit
  31. 26 Dec, 2018 1 commit
  32. 17 Dec, 2018 2 commits
  33. 14 Dec, 2018 1 commit
  34. 13 Dec, 2018 1 commit
  35. 11 Dec, 2018 1 commit
    • Huachao Huang's avatar
      tests: fix test_delete_files_in_range (#250) · fc48f1ae
      Huachao Huang authored
      Previous implementation relies on auto compactions to move files to
      the specific level, which is not reliable. This PR controls the
      compaction manually to make sure the files will be placed at the
      expected level.
      fc48f1ae
  36. 10 Dec, 2018 1 commit