(cherry picked from commit 2088c8d4)

(cherry picked from commit dc16c9b1)

(cherry picked from commit 5c9bcda8)

* *: update ci and format (#248)

(cherry picked from commit dd413e47)

* *: improve build process and update rocksdb (#243)

Some trivial changes are applied:
- Refactor Makefile
- Refactor README.md
- Add the rust-toolchain
- Update RocksDB submodule
- Reorganize the tests structure

(cherry picked from commit 064ab221)

* make format

* allow nightly version fail

* minor fix

(cherry picked from commit e2e4ac24)

57f5fbe49e2 cherry-pick some upstream fixes (#58)
90d77f07985 Cherry pick CompactFiles related fixes (#23)
8db42be85c1 Fix GitHub issue #3716: gcc-8 warnings (#22)
cf55663fbef Add max_subcompactions as a compaction option (#21)
bbdb3077d6a optimize sub compaction (#20)
9e49fc10b65 Direct I/O writable file should do fsync in Close() (#19)
7474ca91ff4 Fix a leak in FilterBlockBuilder when adding prefix (#18)
7a96f60b99a Fsync after writing global seq number in ExternalSstFileIngestionJob (#17)
ba20fcf0470 Ignore empty filter block when data block is empty (#16)
f38277da0dc Optimize external file ingestion (#15)
169406c1592 Delete files in multiple ranges at once (#13)
bd9f08b0179 add missing config checks to CMakeLists.txt (#12)
80134e957b9 Preserve overlapping file endpoint invariant (#11)
23ba5bb196a fix delete range bug (#10)
bfcf6740305 cherry-pick optimize ingest sst (#5)
7ecb1c06fba cherry-pick table filter (#7)
149f8226e59 Merge pull request #8 from zhangjinpeng1987/zhangjinpeng/5.8
b8655d952df cmake: pass "-msse4.2" to when building crc32c.cc if HAVE_SSE42
fe6e58b2f2f 5.8.fb
aead4041720 fix HISTORY.md typo
a0cdc3ceccf Bump version to 5.8.7
7513f635053 Fix IOError on WAL write doesn't propagate to write group follower
9e47084ce27 Bump version to 5.8.6
36074ba5deb Enable cacheline_aligned_alloc() to allocate from jemalloc if enabled.
aa00523e0e3 Add -DPORTABLE=1 to MSVC CI build
cf2b982375a Bump version to 5.8.5
e8c9350f269 Blob DB: not using PinnableSlice move assignment
4907d2463b6 Bump version to 5.8.4
5d928c795af Blob DB: Fix race condition between flush and write
725bb9d665a Blob DB: Fix release build
b7367fe8442 Bump version to 5.8.3
13b2a9b6ff4 Blob DB: use compression in file header instead of global options
5dc70a15caa Fix PinnableSlice move assignment
9019e912549 dynamically change current memtable size
7f1815c379e Bump version to 5.8.2
2584a18efb2 Blob DB: Fix BlobDBTest::SnapshotAndGarbageCollection asan failure
17f67b5462d PinnableSlice move assignment
6fb56c582cc Blob DB: Add compaction filter to remove expired blob index entries
f90ced92f54 Blob DB: fix snapshot handling
632f36dcd33 Blob DB: option to enable garbage collection
11bacd57876 Blob DB: Fix flaky BlobDBTest::GCExpiredKeyWhileOverwriting test
f98efcb1e34 Blob DB: Evict oldest blob file when close to blob db size limit
c1e99eddc87 Blob DB: cleanup unused options
ffc3c62ca29 Blob DB: Initialize all fields in Blob Header, Footer and Record structs
9e82540901d Blob DB: update blob file format
d66bb21e180 Blob DB: Inline small values in base DB
05d5c575ac1 Return write error on reaching blob dir size limit
2b8893b9e42 Blob DB: Store blob index as kTypeBlobIndex in base db
419b93c56f6 Blob DB: not writing sequence number as blob record footer
8afb0036cac fix lite build
dded348dda7 Blob DB: Move BlobFile definition to a separate file
37473612351 add GetLiveFiles and GetLiveFilesMetaData for BlobDB
8cff6e9456f Enable WAL for blob index
c293472908d Add ValueType::kTypeBlobIndex
eae53de3b5d Make it explicit blob db doesn't support CF
65aec19df18 Fix memory leak on blob db open
30b38c98cf7 TableProperty::oldest_key_time defaults to 0
2879f4bebd9 Bump version to 5.8.1
88595c882a1 Add DB::Properties::kEstimateOldestKeyTime
266ac245aff Bumping version to 5.8
64185c23ad9 update HISTORY.md for DeleteRange bug fix
e83d6a02e31 Not using aligned_alloc with gcc4 + asan
0980dc6c9ab Fix wrong smallest key of delete range tombstones
b7679723136 avoid use-after-move error
c41744270ac CMake: Fix formatting
c21ea8f7a6e CMake: Add support for CMake packages
5444345588a add Erlang to the list of language bindings
2972a702a41 Minor updates to FlushWAL blog
fbfa3e7a435 WriteAtPrepare: Efficient read from snapshot list
b01f426f56c Blog post for FlushWAL
503db684f72 make blob file close synchronous
3c840d1a6dc Allow DB reopen with reduced options.num_levels
92bfd6c5078 Fix DropColumnFamily data race
7fdf735d5dd Pinnableslice examples and blog post
7fbb9eccafb support disabling checksum in block-based table
19cc66dc4f8 fix clang bug in block-based table reader
7eba54eb9b7 test compaction input-level split range tombstone assumption
cd26af34767 Add unit test for WritePrepared skeleton
a12479819d1 Improved transactions support in C API
c10b3913149 LANGUAGE-BINDINGS.md: add another rust binding
90177432e40 Remove leftover references to phutil_module_cache
234f33a3f91 allow nullptr Slice only as sentinel
ccf7f833e32 Use PinnableSlice in Transactions
1dfcdb15f93 Extend pin_l0 to filter partitions
39ef900551a stop calling memcmp with nullptrs
78cb6b61129 Provide byte[] version of SstFileWriter.merge to reduce GC Stall
867fe92e5e6 Scale histogram bucket size by constant factor
f004307e9b5 CMake improvements
09ac6206abf Circumvent ASAN false positive
5b68b114f10 Blob db create a snapshot before every read
4624ae52c91 GC the oldest file when out of space
8ace1f79b56 add counter for deletion dropping optimization
0d8e992b472 Revert the mistake in version update
5358a805681 add VerifyChecksum to HISTORY.md
ed0a4c93eff perf_context measure user bytes read
1efc600ddfa Preload l0 index partitions
bddd5d36306 Added mechanism to track deadlock chain
c1384a70763 fix db_stress uint64_t to int32 cast
29877ec7b44 Fix blob db crash during calculating write amp
8f2598ac9d7 Enable Cassandra merge operator to be called with a single merge operand
9a44b4c32c5 Allow merge operator to be called even with a single operand
ac8fb77afd6 fix some misspellings
23593171c42 minor improvements to db_stress
af012c0f830 fix deleterange with memtable prefix bloom
1c8dbe2aa20 update scores after picking universal compaction
eb6425303eb Update WritePrepared with the pseudo code
132306fbf00 Remove PartialMerge implementation from Cassandra merge operator
71598cdc751 Fix false removal of tombstone issue in FIFO and kCompactionStyleNone
3204a4f64bc Fix missing stdlib include required for abort()
7aa96db7a20 db_stress rolling active window
dfa6c23c4b6 Update RocksDBCommonHelper to use escapeshellarg
e367774d195 Overload new[] to properly align LRUCacheShard
ad42d2fcbbd Remove residual arcanist_util directory
279296f4d83 properly set C[XX]FLAGS during CMake configure-time checks
c5f0c6cc660 compile with correct flags to determine SSE4.2 support
185ade4c0c9 cmake: support more compression type
5449c0990b3 rocksdb: make buildable on aarch64
a144a9782d5 Fix for CMakeLists.txt on Windows for RocksJava
acf935e40f9 fix deletion dropping in intra-L0
8254e9b57c8 make sst_dump compression size command consistent
74f18c13019 db_bench support for non-uniform column family ops
5de98f2d50f approximate histogram stats to save cpu
3f5888430a5 Fix c_test ASAN failure
e5a1b727c0e Fix blob DB transaction usage while GC
6f051e0c71d fix corruption_test valgrind
ac098a4626a expose set_skip_stats_update_on_db_open to C bindings
666a005f9b9 Support prefetch last 512KB with direct I/O in block based file reader
ad77ee0ea02 Revert "Makefile: correct faligned-new test"
b87ee6f7736 Use more keys per lock in daily TSAN crash test
25df24254b7 Add column families related functions (C API)
64f8484356e block_cache_tier: fix gcc-7 warnings
0cecf8155b8 Write batch for `TransactionDB` in C API
6a9de434775 Windows.h macro call fix
23c7d135405 fix comment
1fbad84b694 Makefile: correct faligned-new test
7848f0b24c9 add VerifyChecksum() to db.h
47ed3bfc3b8 fix WinEnv assertions
d97a72d63fb Try to repair db with wal_dir option, avoid leak some WAL files
36375de76fb gcc-7/i386: markup intentional fallthroughs
bdc056f8aa2 Refactor PessimisticTransaction
a9a4e89c386 Fix valgrind complaint about initialization
4ca11b4b07e Update USERS.md
c9804e007a4 Refactor TransactionDBImpl
20dc5e74f27 Optimize range-delete aggregator call in merge helper.
0d4a2b73307 Avoid blob db call Sync() while writing
627c9f1abb2 Don't add -ljemalloc when DISABLE_JEMALLOC is set
dce6d5a8385 db_bench background work thread pool size arguments
4f81ab38bf1 Makefile: fix for GCC 7+ and clang 4+
92afe830f91 Update all blob db TTL and timestamps to uint64_t
5883a1ae24b Fix /bin/bash shebangs
cc01985db09 Introduce bottom-pri thread pool for large universal compactions
0b814ba92d7 Allow concurrent writes to blob db
2c45ada4c4b Blob DB garbage collection should keep keys with newer version
58410aee44e Fix the overflow bug in AwaitState
c3d5c4d38ab Refactor TransactionImpl
060ccd4f84e support multiple CFs with OPTIONS file
3453870677e Fix statistics in RocksJava sample
1900771bd29 Dump Blob DB options to info log
3218edc5734 Fix universal compaction bug
6a36b3a7b9e fix db get/write stats
a84cee8127a Add a missing "once" in .h
21696ba502d Replace dynamic_cast<>
e85f2c64cb2 Prevent empty memtables from using a lot of memory
ac748c57ed7 Fix FIFO Compaction with TTL tests
aaf42fe7750 Move blob_db/ttl_extractor.h into blob_db/blob_db.h
aace46516bd Fix license headers in Cassandra related files
50a969131f6 CacheActivityLogger, component to log cache activity into a file
6083bc79f89 Blob DB TTL extractor
710411aea6a fix asan/valgrind for TableCache cleanup
3a3fb00b7aa TARGETS file not setting sse explicitly
fca4d6da175 Build fewer tests in Travis platform_dependent tests
8f553d3c524 remove unnecessary internal_comparator param in newIterator
7f6d012d70e "ccache -C" in Travis
d12691b86fb move TableCache::EraseHandle outside of db mutex
f33f1136831 fix db_bench argument type
e7697b8ce82 Fix LITE unit tests
3ce20e985b6 Fix use of RocksDBCommonHelper in cont_integration.sh
c281b44829a Revert "CRC32 Power Optimization Changes"
9980de262c9 Fix FIFO compaction picker test
2289d381153 CRC32 Power Optimization Changes
30b58cf71a7 Remove the orphan assert on !need_log_sync
fe1a5559f3e Fix flaky write_callback_test
addbd279c27 5.6.1 release blog post
30edff308ea buckification: remove explicit `-msse*` compiler flags
2b259c9d496 Lower num of iterations in DeadlockCycle test
277f6f23d43 Release note for partitioned index/filters
5e731a13825 Remove unused rocksdb arcanist lib
9b11d4345a0 Fix broken links
06f19174499 add vcpkg as an windows option
ea8ad4f6780 Fix compaction div by zero logging
34112aeffde Added db paths to c
1d8aa2961c4 Gcc 7 ParsedInternalKey replace memset with clear function.
a4c42e80075 Fix UBSAN issue of passing nullptr to memcmp
16e03882054 LRUCacheShard cache line size alignment
216644c61ca enable UBSAN macro in TARGETS
e67b35c0762 Add Iterator::Refresh()
a34b2e388ee Fix caching of compaction picker's next index
72502cf2270 Revert "comment out unused parameters"
1d7048c5985 comment out unused parameters
534c255c7ab Cassandra compaction filter for purge expired columns and rows
63163a8c6eb Remove make_new_version.sh
0302da47a7f Reduce blob db noisy logging
3e5ea29a832 Fix Flaky DeleteSchedulerTest::ImmediateDeleteOn25PercDBSize
a22b9cc6fe2 overlapping endpoint fixes in level compaction picker
ffd2a2eefdc delete ExpandInputsToCleanCut failure log
3e6e863b1f5 Remove arcanist_util directory
36651d14eee Moving static AdaptationContext to outside function
6e3ee015fb1 Update java/rocksjni.pom
ecff9d5e33a Include write_buffer_manager in ImmutableDBOptions::Dump
ae28634e9f7 Remove some left-over BSD headers
33b1de82a7b Remove format compatibility hack
2f375154ead checkout local branch in check_format_compatible.sh
ddb22ac59c0 avoid collision with master branch in check format
0c03a7f17d2 set the remote for git checkout
7ac184c6d79 Revert cmake -DNDEBUG for non-MSVC
0655b585820 enable PinnableSlice for RowCache
00464a3140e Fix column_family_test with LITE build
b2dd192fed1 tools/write_stress.cc: Correct "1204" typos.
cbaab30449f table/block.h: change memset
f1a056e0054 CodeMod: Prefer ADD_FAILURE() over EXPECT_TRUE(false), et cetera
4a2e4891fe4 Add back the LevelDB license file
a7321fc97fc Remove the licensing description in CONTRIBUTING.md
3c327ac2d0f Change RocksDB License

git-subtree-dir: librocksdb_sys/rocksdb
git-subtree-split: 57f5fbe49e2476c43cc7c445906fad694504a8db

* add func: load_latest_options

* modify: retrun value

* modify: expose func

* modify: immutable borrow

* modify: remove warning

* modify: add assert

* modify: s/cf_dec/cf_desc, s/cf_decs/cf_descs

* modify: refactor

* modify: refactor

* modify: expose CColumnFamilyDescriptor

* modify: func name

* modify: cargo fmt the whole project

* fix: ci error

Expose the bottommost level compaction option to make it more flexible.

* *: reformat code

Just reformat code without other changes.

* travis: add cargo fmt

* travis: update rust version

* travis: use stable version

Sync of https://github.com/pingcap/rocksdb/pull/22

Fix the following gcc-8 warnings:
- conflicting C language linkage declaration [-Werror]
- writing to an object with no trivial copy-assignment [-Werror=class-memaccess]
- array subscript -1 is below array bounds [-Werror=array-bounds]

* change block cache capacity

* Add wrapper for CompactFiles