* add delete files in ranges API for titan
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

Update rocksdb with fix:
```
849a8c0ae 2019-07-30 yiwu@pingcap.com     fix sign compare warnings
```
Signed-off-by: koushiro <koushiro.cqx@gmail.com>

`register_dep("Z")` correctly set the include path for zlib, but fail to set the path to libz.a. RocksDB cmake script expect libz.a be place under ${DEP_Z_ROOT}/lib, but libz-sys place it under ${DEP_Z_ROOT}/build. Fixed it.

Also added a test to make sure possible compression types are all linked.
Signed-off-by: Yi Wu <yiwu@pingcap.com>

On some platforms jemalloc-sys adds a prefix when building its jemalloc. Thus it is not suitable to use jemalloc-sys to provide a jemalloc in such platforms, otherwise this will lead to a link error.

Update rocksdb with the following changes:

7a03c83ed 2019-07-23 psergey@askmonty.org Fix MyRocks compile warnings-treated-as-errors on Fedora 30, gcc 9.1.1 (#5553)
f21404420 2019-06-19 yiwu@pingcap.com     Fix ingested file and direcotry not being sync (#96) (#99)

When jemalloc feature is on, add jemalloc-sys dependency and pass its lib path to rocksdb.

Tested on ubuntu with libjemalloc-dev uninstalled.
Signed-off-by: Yi Wu <yiwu@pingcap.com>

Signed-off-by: Yi Wu <yiwu@pingcap.com>

Fixing perf context test failure with `cargo test -- --test_threads=1`. The test assume default perf level is `PerfLevel::EnableCount`, which apparently will affected by other tests.

Tested by running `cargo test -- --test_threads=1`.

Closing #318 
Signed-off-by: Yi Wu <yiwu@pingcap.com>

Adding feature "jemalloc". When enabled, rocksdb "WITH_JEMALLOC" will be enabled. This does NOT affect whether jemalloc is use or not. Mostly it only affect how rocksdb calculate block cache size: if "WITH_JEMALLOC" is on, it will use `malloc_usable_size()` (size allocated by jemalloc) to calculate size of each entry of block cache, instead of the requested size of the allocation.

Tested by running `cargo build --features jemalloc -vv` and check rocksdb compile flags from the output.
Signed-off-by: Yi Wu <yiwu@pingcap.com>

This PR exports more options in RocksDB `BlockBasedTableOptions`, because we want to use [Partitioned Index Filters](https://github.com/facebook/rocksdb/wiki/Partitioned-Index-Filters#how-to-use-it).

There is an issue [cache_index_and_filter_blocks causes Get slowdown](https://github.com/facebook/rocksdb/issues/3961) in rocksdb, and we test the recommend `Partitioned Index Filters` in java JNI binding, it works well.

So i think these options should be exported to rust-rocksdb users too.

The rocksdb log in test case `test_partitioned_index_filters` as below:

```
  cache_index_and_filter_blocks: 1
  cache_index_and_filter_blocks_with_high_priority: 1
  pin_l0_filter_and_index_blocks_in_cache: 1
  pin_top_level_index_and_filter: 1
  index_type: 2
  hash_index_allow_collision: 1
  checksum: 1
  no_block_cache: 0
  block_cache: 0x7ff4540060d0
  block_cache_name: LRUCache
  block_cache_options:
    capacity : 8388608
    num_shard_bits : 4
    strict_capacity_limit : 0
    memory_allocator : None
    high_pri_pool_ratio: 0.000
  block_cache_compressed: (nil)
  persistent_cache: (nil)
  block_size: 4096
  block_size_deviation: 10
  block_restart_interval: 16
  index_block_restart_interval: 1
  metadata_block_size: 4096
  partition_filters: 1
  use_delta_encoding: 1
  filter_policy: rocksdb.BuiltinBloomFilter
  whole_key_filtering: 1
  verify_compression: 0
  read_amp_bytes_per_bit: 0
  format_version: 2
  enable_index_compression: 1
  block_align: 0
```

It looks working.

Thanks you for this great project.

Fix typo done in #312

Signed-off-by: Yi Wu yiwu@pingcap.com

Fix build script not passing portable flag to titan.

Tested by build with `cargo build -vv --features portable` and check actual compile command does not contain -march=native flag. e.g.
```
[libtitan_sys 0.0.1] /usr/bin/c++  -DBZIP2 -DHAVE_PCLMUL -DHAVE_SSE42 -DLZ4 -DOS_LINUX -DROCKSDB_FALLOCATE_PRESENT -DROCKSDB_LIB_IO_POSIX -DROCKSDB_MALLOC_USABLE_SIZE -DROCKSDB_PLATFORM_POSIX -DROCKSDB_PTHREAD_ADAPTIVE_MUTEX -DROCKSDB_RANGESYNC_PRESENT -DROCKSDB_SCHED_GETCPU_PRESENT -DROCKSDB_SUPPORT_THREAD_LOCAL -DSNAPPY -DZLIB -DZSTD -I/data/rust-rocksdb    /target/debug/build/snappy-sys-ff485584675450be/out/build -I/data/rust-rocksdb/target/debug/build/bzip2-sys-593753ebc763ecf6/out/include -I/data/rust-rocksdb/target/debug/build/lz4-sys-c5ffb8280e1f82b8/out/include -I/data/rust-rocksdb/target/debug/build/libz-sys-da832d3a86675e51/out/include -I/data/rust-rocksdb/target/debug/build/zstd-sys-ddb8409269514a6a/    out/include -I/data/rust-rocksdb/librocksdb_sys/libtitan_sys/../rocksdb -I/data/rust-rocksdb/librocksdb_sys/libtitan_sys/../rocksdb/include -I/data/rust-rocksdb/librocksdb_sys/libtitan_sys/titan/include -I/data/rust-rocksdb/librocksdb_sys/libtitan_sys/titan/src  -ffunction-sections -fdata-sections -fPIC -m64 -W -Wextra -Wall -Wsign-compare -Wshadow -Wno-un    used-parameter -Wno-unused-variable -Woverloaded-virtual -Wnon-virtual-dtor -Wno-missing-field-initializers -Wno-strict-aliasing -std=c++11 -Werror -fno-builtin-memcmp -g -DROCKSDB_USE_RTTI   -o CMakeFiles/titan.dir/src/blob_file_cache.cc.o -c /data/rust-rocksdb/librocksdb_sys/libtitan_sys/titan/src/blob_file_cache.cc
```
Signed-off-by: Yi Wu <yiwu@pingcap.com>

Summary:
`db.create_cf` call `crocksdb_create_column_family`, which doesn't pass titan specific options to titan. Fix it by calling `ctitandb_create_column_family` when titan is enabled.

Test Plan:
Manual test when enabled titan, tikv pass correct titan options when creating new CFs.

Will add unit test in later patch.
Signed-off-by: Yi Wu <yiwu@pingcap.com>

* add titan read option
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

Update rocksdb with the fix:
```
b4f33387f 2019-06-20 yiwu@pingcap.com     Fix ingested file and direcotry not being sync (#96)
```
Signed-off-by: Yi Wu <yiwu@pingcap.com>

* fix titan config
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

it is possible using a customized comparator.

Signed-off-by: siddontang <siddontang@gmail.com>

* add recycle log configuration
Signed-off-by: siddontang <siddontang@gmail.com>

* add titan blob-run-mode

* checkout latest bug fix and titan stats

* move Titan to separate repository
Signed-off-by: Yi Wu <yiwu@pingcap.com>

This is to get around borrow checker for TiKV side of changes. Other APIs also pass in structs as reference.

Adding BlockBasedTableOptions::set_block_cache so we can set shared block cache instance to different column family and different DB instance.

Also adding LRUCacheOptions and Cache structs.

Update RocksDB to include the following changes:
018aa1972 Titan: fix error message formatting (#84)
ee2f17ca8 Fix crash with memtable prefix bloom and key out of prefix extractor domain (#5190)
16ace7c47 Titan: Add snapshot related test (#80)
ee8b59301 Titan: Fix iterator fail to pass SuperVersion to underlying iterator (#79)

Signed-off-by: Yi Wu yiwu@pingcap.com

Signed-off-by: Connor1996 <zbk602423539@gmail.com>

Update .gitmodules to point to pingcap/rocksdb tikv-3.0 branch. Also update rocksdb to include the following changes:

75133b1b6 2019-03-26 yiwu@pingcap.com Fix SstFileReader not able to open ingested file (#5097)
7ca9eb754 2019-03-26 yiwu@pingcap.com Fix BlockBasedTableIterator construction missing index_key_is_full parameter
Signed-off-by: Yi Wu <yiwu@pingcap.com>

We should not rely on a specific version of rust.

We need to support encryption environment created outside.