This change is to fix ASAN test failure in CI.

The rocksdb logging.h file provides the ROCKS_LOG_XXX logging macro, and it contain a logic to shorten the source file being print. It use the length of its file path (e.g. "/path/to/rocksdb/logging/logging.h")  to truncate other source file path. Since Titan source could be in different directory, Titan source file path could be truncated incorrectly, or even cause out-of-range access to the string buffer. Copying the file into titan repo and make a Titan version of the macros to fix the problem.

Also doing clang format.
Signed-off-by: Yi Wu <yiwu@pingcap.com>
Signed-off-by: Yi Wu <yiwu@pingcap.com>

* Add checkpoint support
Signed-off-by: Myth <caipengbo@outlook.com>

Signed-off-by: Connor1996 <zbk602423539@gmail.com>

Specify IOPriority when creating blob files. Fix #209.
Signed-off-by: BaronStack <2689496754@qq.com>
Co-authored-by: Xinye Tao <xy.tao@outlook.com>

In our CI mac build, homebrew update often timeout. Mitigate this issue by upgrading Xcode image (spec was ineffective before).
Also optimize build time by adding format stage and fail_fast flag so that failed test can trigger immediate termination.
Signed-off-by: tabokie <xy.tao@outlook.com>

Adding a tool to dump Titan manifest file, for easy debugging.

Sample output:
```
next_file_number: 93015
column family: 1

column family: 2

column family: 3

column family: 0
file 92978, size 44644407, level 0
file 92997, size 44513780, level 0
file 93001, size 44639151, level 0
file 93005, size 44420372, level 0
file 93009, size 44595346, level 0
file 93010, size 23167116, level 0
file 93011, size 38966307, level 0
file 93012, size 21745288, level 0
file 93014, size 44181211, level 0
```
Signed-off-by: Yi Wu <yiwu@pingcap.com>

Signed-off-by: Zhenhan Gong <zhenhan.gong@gmail.com>

Do the same cmake change in https://github.com/tikv/rocksdb/pull/219Signed-off-by: Yilin Chen <sticnarf@gmail.com>

This is a subtask of issue: https://github.com/tikv/tikv/issues/5743

> Read blob from file with dictionary compression

Mainly made following changes:

- Refactored `BlobDecoder`, add decompression related components in it.
- Refactored `BlobFileReader` and `BlobFileIterator`, initialize the decoder with corresponding data in blob files.
- add testcases

Issue Number: https://github.com/tikv/tikv/issues/8635

Fixes #118

Briefly I've done following things:
- add compression options in TitanCFOptions
- set flags in header when there is compression dictionary
- add some members(states, sample data, etc.) in BlobFileBuilder for dictionary compression logic
- write the dictionary in meta blocks, when dictionary compression is enabled
- refactor BlobEncoder to support dictionary update

currently for decoding
- update BlobFileReader and BlobFileIterator to return Status::NotSupported if a blob file come with dictionary compression.
- add a test case for BlobFileReader

Issue Number: https://github.com/tikv/tikv/issues/8635Signed-off-by: hexyoungs <chuxdesign@hotmail.com>

Update travis script to enable compression in tests. needed by #189

Also exclude NoSpaceLeft test from travis.
Signed-off-by: Yi Wu <yiwu@pingcap.com>

* separate deleteblobfiles from deletefiles (#185)
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

Looks like it's not a undefined behavior to get a nullptr from compaction filter factory

Fixes #180 
Signed-off-by: DorianZheng <xingzhengde72@gmail.com>

* add more test cases
Signed-off-by: Connor <zbk602423539@gmail.com>

* fix cocurrent compaction while flush
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

Closes #163 
Signed-off-by: Xu Qiaolun <Jamesxql@Gmail.com>

* fix wrong live data size when encounter rewrite failure
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* add deletefilesinrange during gc case
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* check obsolete before apply
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

After creating a new blob file meta, the live data size is 0. The stat is counted when adding the blob file but with live_data_size == 0. So we should update the stat when the live data size is initialized.
Signed-off-by: Connor <zbk602423539@gmail.com>

This is a new interface which is added to RocksDB recently and TiKV will call it if `enable_multi_thread_write` of `DBOptions` is true. So Titan must implement it by self.
Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

exclude metadata size from file size, otherwise discardable ratio would be wrong when the blob file is small.
Signed-off-by: Connor <zbk602423539@gmail.com>

* titan level blob file count
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* store stats in rocksdb statistics
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

Signed-off-by: Little-Wallace <bupt2013211450@gmail.com>

Fix warn when titan is compiled with gcc 5.5.0
```
/home/pingcap/.cargo/git/checkouts/rust-rocksdb-e0eeb217423298aa/0e1a085/librocksdb_sys/libtitan_sys/titan/src/db_impl_gc.cc: In member function ‘rocksdb::Status rocksdb::titandb::TitanDBImpl::BackgroundGC(rocksdb::LogBuffer*, uint32_t)’:
/home/pingcap/.cargo/git/checkouts/rust-rocksdb-e0eeb217423298aa/0e1a085/librocksdb_sys/libtitan_sys/titan/src/db_impl_gc.cc:205:55: error: ‘gc_merge_rewrite’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
                           &shuting_down_, stats_.get());
```

Signed-off-by: liumh8 <571621369@qq.com>

Fixes #144 

* Green Blob GC by merge operator
Signed-off-by: tabokie <xy.tao@outlook.com>

Summary:
* On DB reopen, iterate all SST file's properties and sum up live data size of all blob files to use as GC stats.
* Remove previous logic to set GC mark on blob files on reopen.
* Refactor `OnFlushCompleted` and `OnCompactionCompleted`. Check file state is `kPendingLSM` before transit file state. 
* Refactor `BlobFileMeta`.

Test Plan:
Updated unit tests
Signed-off-by: Yi Wu <yiwu@pingcap.com>

Summary:
When adding new blob file, titan directory need to be fsync, otherwise files can be lost after power outage.

Test Plan:
run `strace -yfe fsync ./titan_titan_db_test --gtest_filter=*.Snapshot` and see titan directory being fsync-ed in the output.
Signed-off-by: Yi Wu <yiwu@pingcap.com>

1. use `#include "port/port.h"` instead of `#include "port/port_posix.h"`
2. added an "install" target for CMake to workaround alexcrichton/cmake-rs#18
3. MSVC reported 2 warnings in Titan (and tons of warnings in RocksDB), and these are fixed.

Summary:
Fix a typo in `TitanDBImpl::DeleteFilesInRange()` which makes GC stats not being updated after `DeleteFilesInRange()` call.

Test Plan:
See the new test
Signed-off-by: Yi Wu <yiwu@pingcap.com>

Signed-off-by: tabokie <xy.tao@outlook.com>

* new thread pool for titan
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

fix compiling error: 
> error: declaration of ‘initialized’ shadows a member of 'this'
Signed-off-by: tabokie <xy.tao@outlook.com>

since level merge will read blob files sequentially, we should use prefetcher to accelerate merging.

experiment result:

Load 300GB (average value size 1KB) on Huawei NVME device, time to read blob file reduced about 50%.
Signed-off-by: wujy-cs <wujy.cs@gmail.com>

Summary:
Refactor `TitanDBImpl::Open` to open base DB only once. Previously we open the DB twice - the first time only to obtain `cf_id` for each CF, and then initialize internals before open base DB for real. The patch change to open base DB only once, initialize internals after that. Before `TitanDBImpl` is initialized, `BaseDbListener` will be no-op and `TableBuilder` will not output blob files.

Test Plan:
Added new tests

Adding a new version to blob file format. v2 blob files come with 4 more bytes in the header to hold future flags. In particular if `kHasUncompressionDictionary` is set, the whole file is compressed using dictionary compression and the per-blob compression type should be ignored.

Also update `BlobEncoder` and `BlobDecoder` to allow passing in a dictionary and use it to  compress/decompress. Add unit test for it.