Some trivial changes are applied:
- Refactor Makefile
- Refactor README.md
- Add the rust-toolchain
- Update RocksDB submodule
- Reorganize the tests structure

* export compaction reason
Signed-off-by: zhangjinpeng1987 <zhangjinpeng@pingcap.com>

Signed-off-by: Tong Zhigao <tongzhigao@pingcap.com>

* Add on_stall_conditions_changed to EventListener;
Signed-off-by: Tong Zhigao <tongzhigao@pingcap.com>

* Test stall condition changes from normal to other;
Signed-off-by: Tong Zhigao <tongzhigao@pingcap.com>

* Optimize test;
Signed-off-by: Tong Zhigao <tongzhigao@pingcap.com>

* support DBWithTTL Open Interface

* add DBWithTTL_Open test

* run cargo fmt --all

* small fix to pass travis ci

* fix bug about with_ttl condition and code style

* small fix to pass travis ci

* small fix to pass travis ci

* unify the with_ttl funtion's conditon

* fix condition about with_ttl and add more tests

* little fix to pass travis ci

* little fix to pass travis ci

* fix with_ttl condition and make it more clear

* litte fix to pass travis ci

* Makefile: update rocksdb branch

* Squashed 'librocksdb_sys/rocksdb/' changes from 185a70d50e..22044072fa

22044072fa cmake: fix FORCE_SSE42 (#4490) (#56)
444d3d9e91 Expose column family id to OnCompactionCompleted (#4466) (#54)
7ce226ad72 Fix return status of DBImpl::GetLatestSequenceForKey (#53)
7a99f6261a Fix DBImpl::GetColumnFamilyHandleUnlocked race condition (#4391) (#52)
08a88b17fb c-api: add some missing options
37c392ab7a types: add kEntryBlobIndex for TablePropertiesCollector (#4233)
d156e5ad59 Make sure to call ReleaseFileNumberFromPendingOutputs
9fcb951296 Protect external file when ingesting (#4099)
7e1f37eb4f Bump version and update history
f50476a021 Fix Makefile target 'jtest' on PowerPC (#4357)
c97db5282c Remove warnings caused by unused variables in jni (#4345)
0d4c848325 Download bzip2 packages from Internet Archive (#4306)
02028f9c19 Bump version and update history
0db501141a fix compilation with g++ option `-Wsuggest-override` (#4272)
6c47f8f3d9 bump version and update history for 5.15.8
c00296e9fa Reduce empty SST creation/deletion in compaction (#4336)
39340abf23 BlobDB: Fix expired file not being evicted (#4294)
abbd5a4b93 BlobDB: Can return expiration together with Get() (#4227)
4bd3bc5c4f BlobDB: Fix VisibleToActiveSnapshot() (#4236)
c1d8c1e7dc BlobDB: Cleanup TTLExtractor interface (#4229)
5e01c5a312 bump version and update history
316f30ce02 Reduce empty SST creation/deletion during compaction (#4311)
17d42cc84a Bump version to 5.15.6 and update history
cb2d7fc7bc #3865 followup for fix performance degression introduced by switching order of operands (#4284)
2c7422aa65 Summary: This PR addresses issue #3865 and implements the following approach to fix it:  - adds `MergeContext::GetOperandsDirectionForward` and `MergeContext::GetOperandsDirectionBackward` to query merge operands in a specific order  - `MergeContext::GetOperands` becomes a shortcut for `MergeContext::GetOperandsDirectionForward`  - pass `MergeContext::GetOperandsDirectionBackward` to `MergeOperator::ShouldMerge` and document the order Pull Request resolved: https://github.com/facebook/rocksdb/pull/4266
a0a9829f7d Add SST ingestion to ldb (#4205)
086bae0599 Bump version to 5.15.5 and update history
4842d1ea5f VerifyChecksum() API should preserve options
e28e2fff13 Bump version and update history
0b6123068a Revert changes in PR #4003 (#4263)
ee0cf2f473 Bump patch version number.
3b66d4b617 Fix wrong partitioned index size recorded in properties block (#4259)
421be4af42 Bump version and update HISTORY.md
9810a21baa Return correct usable_size for BlockContents (#4246)
f9fc1f483a bump version and update history
600ca9a439 Skip range deletions at seqno zero when collapsing (#4216)
976212ede4 DBImpl::IngestExternalFile() should grab mutex when releasing file number in failure case (#4189)
516faa0fa0 Fix bug when seeking backward against an out-of-bound iterator (#4187)
9897e42813 Write properties metablock last in block-based tables (#4158)
f132197a93 Windows JNI build fixes (#4015)
87f1325b2b Fix a bug in MANIFEST group commit (#4157)
b79a965f7c Smaller tail readahead when not reading index/filters (#4159)
ad7491e11e Return new operator for Status allocations for Windows (#4128)
c57b91f844 Cap concurrent arena's shard block size to 128KB (#4147)
decf7e52b0 Fix write get stuck when pipelined write is enabled (#4143)
be91970d59 Release 5.15.
ddc07b40fc Remove managed iterator
995fcf7573 Pending output file number should be released after bulkload failure (#4145)
5a59ce4149 Coding.h: Added Fixed16 support (#4142)
fb768a4289 Dump mutable FIFO and Universal compaction options (#4140)
b55da012f6 Refactor IndexBlockIter (#4141)
991120fa10 Allow ttl to be changed dynamically (#4133)
8f06b4fa01 Separate some IndexBlockIter logic from BlockIter (#4136)
ef7815b803 Support range deletion tombstones in IngestExternalFile SSTs (#3778)
91d7c03cdc Exclude time waiting for rate limiter from rocksdb.sst.read.micros (#4102)
90fc40690a Relax VersionStorageInfo::GetOverlappingInputs check (#4050)
21171615c1 Reduce execution time of IngestFileWithGlobalSeqnoRandomized (#4131)
8581a93a6b Per-thread unique test db names (#4135)
23b76252c8 db_bench: enable setting cache_size when loading options file
8527012bb6 Converted db/merge_test.cc to use gtest (#4114)
537a233941 Exclude StackableDB from transaction stress tests (#4132)
e3eba52a5d Re-enable kUniversalSubcompactions option_config (#4125)
7bee48bdbd Add GCC 8 to Travis (#3433)
de98fd88e3 Support compaction filter in db_bench (#4106)
97fe23fc5c Fix unsigned int flag in db_bench (#4129)
520bbb1774 Disable EnvPosixTest.RunImmediately, add EnvPosixTest.RunEventually. (#4126)
90ebf1a257 Reduce execution time of a test. (#4127)
d4ad32d7bd Refactor BlockIter (#4121)
63904434eb db_bench periodically dump stats to info log (#4109)
dbeaa0d397 Reduce #iterations to shorten execution time. (#4123)
5f3088d565 Range deletion performance improvements + cleanup (#4014)
121e321549 Update docs/Gemfile.lock for nokogiri cve (#4116)
a61ff876a1 Remove two CI tests (#4110)
1ea83c5de9 Reduce runtime of compact_on_deletion_collector_test (#4117)
35b38a232c Update comments of WriteBatchWithIndex
5cd8240b86 Test range deletions with more configurations (#4021)
cfee7fb51a Allow storing metadata with backups for Java API (#4111)
1c912196de Remove external tracking of AlignedBuffer's size (#4105)
331cb63641 SetOptions Backup Race Condition (#4108)
440621aab8 Fix Copying of data between buffers in FilePrefetchBuffer (#4100)
926f3a78a6 In delete scheduler, before ftruncate file for slow delete, check whether there is other hard links (#4093)
b9846370e9 WriteUnPrepared: Add support for recovering WriteUnprepared transactions (#4078)
db7ae0a485 Fix a map lookup that may throw exception. (#4098)
d4d9fe8e57 Fix a bug caused by not copying the block trailer. (#4096)
35b83327a7 compaction: fix max_subcompactions option for CompactRange (#4082)
39218a72a4 Increase the size of LRU cache. (#4090)
17027aeffc Change default value of `bytes_max_delete_chunk` to 0 in NewSstFileManager() (#4092)
0d234dfce4 Remove unused arg which causes compilation failure (#4080)
e099c2dd55 check if data size exceeds java array vm limit when it is copied in jni (#3850)
36fa49ceb5 transaction_test: -Wunused-variable with clang-7 (#4074)
2462763b2e Fix mis-spoken assert on prefetch_filter and prefetch_index (#4077)
29ffbb8a50 Charging block cache more accurately (#4073)
b3efb1cbe0 fix clang analyzer warnings (#4072)
8ad63a4b86 WriteUnPrepared: Add new WAL marker kTypeBeginUnprepareXID (#4069)
25403c2265 Prefetch cache lines for filter lookup (#4068)
52d4c9b7f6 Allow DB resume after background errors (#3997)
26d67e357e Support group commits of version edits (#3944)
0a5b5d88b2 Remove ReadOnly part of PinnableSliceAndMmapReads from Lite (#4070)
b557499eee Suppress leak warning for clang(LLVM) asan (#4066)
7f850b889d Remove 'ALIGNAS' from StatisticsImpl. (#4061)
14f409c0f1 PrefixMayMatch: remove unnecessary check for prefix_extractor_ (#4067)
1f6efabe23 Add bottommost_compression_opts to for bottommost_compression (#3985)
235ab9dd32 Pin mmap files in ReadOnlyDB (#4053)
e8f9d7f0d4 Added PingCaps Rust RocksDB and ObjectiveRocks (#4065)
818c84e116 Store timestamp in deadlock detection (#4060)
e5ae1bb465 Remove bogus gcc-8.1 warning (#3870)
a16e00b7b9 WriteUnPrepared Txn: Disable seek to snapshot optimization (#3955)
17339dc2f3 Add table property tracking number of range deletions (#4016)
408205a36b use user_key and iterate_upper_bound to determine compatibility of bloom filters (#3899)
967aa8157a Create lgtm.yml for LGTM.com C/C++ analysis (#4058)
2694b6dc26 Remove unused imports, from python scripts. (#4057)
a8e503e545 Fix universal compaction scheduling conflict with CompactFiles (#4055)
346d1069c3 Align StatisticsImpl / StatisticsData (#4036)
6d454d7376 BlobDB: is_fifo=true also evict non-TTL blob files (#4049)
189f0c27aa Make BlockBasedTableIterator compaction-aware (#4048)
a71e467381 Blob DB: enable readahead for garbage collection (#3648)
2729dd72ad Reclaim memory allocated to backup_engine.
80ade9ad83 Pin top-level index on partitioned index/filter blocks (#4037)
c726f7fda8 Fix dangling checkpoint pointer in db_stress (#4042)
64c85d0d97 Set DEBUG_LEVEL=0 for RocksJava Mac Release (#4040)
795e663df0 option for timing measurement of non-blocking ops during compaction (#4029)
0a5b16c7c5 Cleanup staging directory at start of checkpoint (#4035)
645e57c22d Assert for Direct IO at the beginning in PositionedRead (#3891)
58c221440c Update TARGETS file (#4028)
397495964b Fix a warning (treated as error) caused by type mismatch.
7103559f49 Improve direct IO range scan performance with readahead (#3884)
524c6e6b72 Add file name info to SequentialFileReader. (#4026)
14cee194d6 Support file ingestion in stress test (#4018)
61d69d450d Hide jemalloc aligned allocation functions into .cc (#4025)
28a9d8910b Fix the bug with duplicate prefix in partition filters (#4024)
92ee3350e0 BlockBasedTableIterator to keep BlockIter after out of upper bound (#4004)
7f3a634e06 Support pipelined write in stress/crash tests
8585059ae0 Support backup and checkpoint in db_stress (#4005)
de2c6fb158 Fix stderr processing in crash test (#4006)
c766887458 Fix ExternalSSTFileTest::OverlappingRanges test on Solaris Sparc (#4012)
7b4b43febb zLinux build error with gcc and IBM Java headers (#4013)
e5bee404ce zLinux s390x support in JNI (#4009)
e750dacffb Crash on Windows, because of shared_ptr reinterpret cast (#3999)
80bc35927c Should only decode restart points for uncompressed blocks (#3996)
c48764ba47 Don't generate a notification for a 0 size SST (#4003)
3fbc865cd5 Add kOptionsStatistics to GetProperty() (#3966)
7b5f7ff0b4 travis: osx install zstd lz4 snappy xz (#3893)
906a602c2c Build and tests fixes for Solaris Sparc (#4000)
f23fed19a1 Delay verify compaction output table (#3979)
4faaab70a6 Benchmark sine wave write rate limit (#3914)
f5281a53a4 tools/check_format_compatible.sh to cover forward option reading too (#3994)
fbe3b9e2b6 Udpate db_universal_compaction_test according to PR #3970 (#3995)
1f32dc7d2b Check with PosixEnv before opening LOCK file (#3993)
7497f992e0 Run manual compaction in stress/crash tests (#3936)
dd216dd76a Choose unique keys faster in db_stress (#3990)
a720401877 Avoid acquiring SyncPoint mutex when it is disabled (#3991)
d82f1421b4 Fix regression bug of Prev() with upper bound (#3989)
9d347332fb Fix argument mismatch in BlockBasedTableBuilder (#3974)
9c7da963bc Fix a crash in WinEnvIO::GetSectorSize (#3975)
3593275357 Remove restart point from the properties_block (#3970)
f4502944c3 Change db path for BlockBasedTableTest.BadOptions (#3965)
3470c75852 Fix build errors.
23e1d23675 Fixed the fprintf of uint64_t by using PRIu64 (#3963)
0a0860a5fb Refactoring db_stress.cc (#3902)
45b6bcca98 ZSTD compression: should also expect type = kZSTDNotFinalCompression (#3964)
b73652169e Extend format 3 to partitioned index/filters (#3958)
5504a056f8 Adding advisor Rules and parser scripts with unit tests. (#3934)
4420df4b0e Check conflict at output level in CompactFiles (#3926)
f1592a06c2 run make format for PR 3838 (#3954)
812c7371d3 Fix performance regression in Get() for block-based tables (#3953)
d0c38c0c8c Extend some tests to format_version=3 (#3942)
2210152947 Fix singleton destruction order of PosixEnv and SyncPoint (#3951)
ab2254bedf Fix clang analyze
f4b72d7056 Provide a way to override windows memory allocator with jemalloc for ZSTD
4f297ad05f Fix crash test check for direct I/O
50d7ac0ea3 Fix test for rocksdb_lite: hide incompatible option kDirectIO
fea2b1dfb2 Copy Get() result when file reads use mmap
88c3ee2d31 Configure direct I/O statically in db_stress
01e3c30def Extend existing unit tests to run with WriteUnprepared as well
89b37081a1 add c api rocksdb_sstfilewriter_file_size
2a0dfaa044 fix PrefixExtractorChanged: pass raw pointer instead shared_ptr
44cf84932f Fix the bug of some test scenarios being put after kEnd
2807678b11 c api set bottommost level compaction
82089d59c3 DBImpl::FindObsoleteFiles() not to call GetChildren() on the same path
a35451eaa4 fix deadlock with enable_pipelined_write=true and max_successive_merges > 0
aaac6cd16f Add write unprepared classes by inheriting from write prepared
727eb881a5 Compile error in db bench tool
4dd80debd0 Remove tests from ROCKSDB_VALGRIND_RUN
a736255de8 Delete triggered compaction for universal style
724855c7da Fix LRUCache missing null check on destruct
cf826de3ed Fix compilation error when OPT="-DROCKSDB_LITE".
03cda531e4 Check for rep_->table_properties being nullptr
1c1bafa668 Fix VersionStorageInfo::EstimateLiveDataSize seg fault
402b7aa07f Exclude seq from index keys
8c3bf0801b Check status when reading HashIndexPrefixesMetadataBlock
45434178ee Fix an issue with unnecessary capture in lambda expressions
aa53579d6c Fix segfault caused by object premature destruction
6e08916eb3 Fix Fadvise on closed file when reads use mmap
070319f7bb add flush_before_backup parameter to c api rocksdb_backup_engine_create_new_backup
bc7e8d472e LRUCache midpoint insertion
3db8504cde Catchup with posix features
c465509379 port_posix: use posix_memalign() for aligned_alloc
7a99c04311 refactor constructor of LRUCacheShard
01bcc34896 Introduce library-independent default compression level
4011012d9d Specify the underlying type of enums.
6c73a46693 Fix a backward compatibility problem with table_properties being nullptr
4420cb49da Fix Issue #3771: Slice ctor checks for nullptr and creates empty string
7db721b9a6 Avoid sleep in DBTest.GroupCommitTest to fix flakiness
fcb31016e9 Avoid single-deleting merge operands in db_stress
3db1ada3bf PersistRocksDBOptions() to use WritableFileWriter
c3ebc75843 Move prefix_extractor to MutableCFOptions
263ef52b65 Update ColumnFamilyTest for multi-CF verification
508a09fd62 Print histogram count and sum in statistics string
7b655214d2 Assert keys/values pinned by range deletion meta-block iterators
e410501eeb Add missing test files to src.mk
ed4d3393fb fix a division by zero bug
26da3676d9 class Block to store num_restarts_
a0c7b4d526 Set the default value of max_manifest_file_size.
17af09fcce Implement key shortening functions in ReverseBytewiseComparator
1d7ca20f29 add override to virtual functions
aed7abbcca Reorder field based on esan data
fa43948cbc Update HISTORY and version for upcoming 5.14
7ccb35f653 In instrumented mutex, take timing once for both of perf_context and statistics
8bf555f487 Change and clarify the relationship between Valid(), status() and Seek*() for all iterators. Also fix some bugs
46fde6b653 Fix race condition between log_.erase and log_.back
42cb4775c1 Fix geo_db may seek an error key when they have the same quadkey
12ad711247 Suppress tsan lock-order-inversion on FlushWAL
3d7dc75b36 Bottommost level-based compactions in bottom-pri pool
ebb823f746 Fix db_stress build on mac
718c1c9c1f Pass manual_wal_flush also to the first wal file
66c7aa32fb Clarify the ownership of root db after TransactionDB::Open
3272bc07c6 Fix formatting in log message
072ae671a7 Apply use_direct_io_for_flush_and_compaction to writes only
d19f568abf Refactor argument handling in db_crashtest.py
3690276e74 Disallow to open RandomRW file if the file doesn't exist
ddfd2525d2 Make BlockIter final
f92cd2feb4 Introduce and use the option to disable stall notifications structures
cee138c7d7 Add missing options in BuildColumnfamilyOptions
4bf169f07e Disable readahead when using mmap for reads
1d9f24dc9a Link jemalloc
9470ee45b8 Allows other cmake-specific "true" for USE_RTTI.
6d6e01cdbc Search paths provided by intel's "tbbvars.sh".
d72a51e9e1 Split FaultInjectionTest.FaultTest to avoid timeout
72942ad7a4 Recommit "Avoid adding tombstones of the same file to RangeDelAggregator multiple times"
4c5a3232e4 Fix db_stress memory leak ASAN error
fc522bdb3e Evenly split HarnessTest.Randomized
171f415b30 Rename vars to satisfy unity built
4d40b10e0f Add USE_RTTI and default behavior to CMakeLists
6fc1bccef5 Fix crash test allocation error under TSAN
a703432808 MaxFileSizeForLevel: adjust max_file_size for dynamic level compaction
934f96de27 Better destroydb
a8d77ca381 Speedup ManualCompactionTest.Test
d59549298f Skip deleted WALs during recovery
cfb86659bf WritePrepared Txn: enable rollback in stress test
5bed8a0065 WritePrepared Txn: split SeqAdvanceConcurrentTest
6cab3184f5 avoid double delete on dummy record insertion failure
acb61b7a52 Adjust pread/pwrite to return Status
19fde54841 initialize local variable for UBSAN in PosixEnv function
46152d53bf Second attempt at db_stress crash-recovery verification
282099fc0f fix missing perfcontext destroy declare in C API
c9ace1d81b expose WAL iterator in the C API
6afe22db2e revert db_stress crash-recovery verification
459bb9028f remove prefixscanrandom from db_bench help
ed7a95b28c Add max_subcompactions as a compaction option
7dfbe33532 Rename pending_compaction_ to queued_for_compaction_.
513b5ce618 Rename pending_flush_ to queued_for_flush_.
37cd617b6b Add virtual Truncate method to Env
db36f222d8 Allow options file in db_stress and db_crashtest
7004e45489 Remove block-based table assertion for non-empty filter block
63c965cdb4 Sync parent directory after deleting a file in delete scheduler
7e4e381495 Fix the bloom filter skipping empty prefixes
e5a4dacf6d WritePrepared Txn: disable rollback in stress test
7c9f23e6db Rate limiter should be allowed to share between different rocksdb instances in C API
406b95197c Fix clang build failure with -Wgnu-redeclared-enum
13a0bd90ce cmake: add options for enabling TBB and NUMA support
dfc61e7c24 initialize local variable for UBSAN in PosixEnv function
9b89479e64 Pass -latomic to linker when using clang
a4fb1f8c04 Add crash-recovery correctness check to db_stress
bc0da4b512 Skip duplicate bloom keys when whole_key and prefix are mixed
090c78a0d7 Support lowering CPU priority of background threads
affe01b0d5 Improve write time breakdown stats
d5afa73789 Revert "Skip deleted WALs during recovery"
a8a28da215 Avoid directory renames in BackupEngine
2e72a5899b Disable EnvPosixTest::FilePermission
bb2a2ec731 WritePrepared Txn: rollback via commit
dbdaa4662e Add a stat for MultiGet keys found, update memtable hit/miss stats
c3d1e36cce WritePrepared Txn: enable TryAgain for duplicates at the end of the batch
17e04039dd Propagate fill_cache config to partitioned index iterator
dee95a1afc Fix GitHub issue #3716: gcc-8 warnings
8a9c7f71c9 fix compilation error: implicit conversion loses integer precision
69faddb32e CMake: Read rocksdb version from version.h header file
e1e826b980 check return status for Sync() and Append() calls to avoid corruption
ad511684b2 Add block cache related DB properties
3cea61392f include thread-pool priority in thread names
6d06be22c0 Improve db_stress with transactions
2ee1496c43 Add missing whitespace.
2c2f388897 db_bench fillXXXdeterministic should respect compression type
b4f333922a Improve the comment on TableFactory::NewTableReader()
5e48811844 Initialize a boolean member variable of a struct.
af95aecd01 use delete[] to dealloc an array
954b496b3f fix memory leak in two_level_iterator
9fcd82e987 cmake: append rados to THIRDPARTY_LIBS before appending it to LIBS
81d44f2bc5 fix-typo: add missing periods
28087acd79 Implemented Knuth shuffle to construct permutation for selecting no_o…
a0102aa6d7 Make database files' permissions configurable
31ee4bf240 add kEntryRangeDeletion
1f5457ef21 Merge raw and shared pointer log method impls
c81b0abedd Improve accuracy of I/O stats collection of external SST ingestion.
3be9b36453 comment unused parameters to turn on -Wunused-parameter flag
d15397ba10 WritePrepared Txn: rollback_merge_operands hack
6f5e6445d9 WritePrepared Txn: fix smallest_prep atomicity issue
d42bd041c5 Improve visibility into the reasons for compaction.
019d7894eb fix calling SetOptions on deprecated options
d95014b9df fix some text in comments.
2770a94c42 make MockTimeEnv::current_time_ atomic to fix data race
5ec382b918 Fix up backupable_db stack corruption.
d2bcd7611f Fix the memory leak with pinned partitioned filters
65fe8d6cd6 Change a comment
1c27cbfbd1 fix intra-L0 FIFO for uncompressed use case
f3a1d9e049 fix data race
bde1c1a72a WritePrepared Txn: add stats
eb5a295440 WritePrepared Txn: add write_committed option to dump_wal
ca87aef82d Added support for SstFileManager to RocksJava
74767deec3 Fix typo
faba3fb53d protect valid backup files when max_valid_backups_to_open is set
6571770030 fix shared libary compile on ppc
446b32cfc3 Support for Column family specific paths.
67182678a5 Stats for false positive rate of full filtesr
685912d07f Clock cache should check if deleter is nullptr before calling it
147dfc7bdf Fix pre_release callback argument list.
36a9f22931 Blob DB: blob_dump to show uncompressed values
c827b2dc2a fix build for rocksdb lite
7d9067991e Ttl-triggered and snapshot-release-triggered compactions should not be manual compactions
2a62ca1750 Make Optimistic Tx database stackable
b058a33705 Reduce default --nooverwritepercent in black-box crash tests
12b400e814 Some small improvements to the build_tools
04c11b867d Level Compaction with TTL
df14424410 Fix 3-way SSE4.2 crc32c usage in MSVC with CMake
b225de7e10 WritePrepared Txn: smallest_prepare optimization
1579626d0d Enable cancelling manual compactions if they hit the sfm size limit
44653c7b7a Revert "Avoid adding tombstones of the same file to RangeDelAggregato…
8917eee962 Fixed small typos
d12112d05e Throw NoSpace instead of IOError when out of space.
d9bfb35d31 Update buckifier and TARGETS
c3eb762bb0 Update 64-bit shift in compression.h
73f21a7b21 Skip deleted WALs during recovery
89d989ed75 WritePrepared Txn: fix a bug in publishing recoverable state seq
3cb591954e Allow rocksdbjavastatic to also be built as debug build
0377ff9dea WritePrepared Txn: make recoverable state visible after flush
1f5def1653 Fix race condition causing double deletion of ssts
90c542347a Update comments about MergeOperator::AllowSingleOperand
d687670256 Fix a leak in FilterBlockBuilder when adding prefix
f9f4d40f93 Align SST file data blocks to avoid spanning multiple pages
0999e9b79a WritePrepared Txn: Increase commit cache size to 2^23
35a4469bbf Fix race condition via concurrent FlushWAL
23f9d93f47 Exclude MySQLStyleTransactionTest.TransactionStressTest* from valgrind
3e417a6607 WritePrepared Txn: AddPrepared for all sub-batches
d382ae7de6 Imporve perf of random read and insert compare by suggesting inlining to the compiler
53d66df0c4 Refactor sync_point to make implementation either customizable or replaceable
a993c0139d Add 5.11 and 5.12 to tools/check_format_compatible.sh
e80709a33a Avoid adding tombstones of the same file to RangeDelAggregator multiple times
7ffce2805b Add Java-API-Changes section to History
09b6bf828a InlineSkiplist: don't decode keys unnecessarily during comparisons
1cbc96d236 FlushReason improvement
82137f0ce8 Add unit test for WAL corruption
2e3d407778 Fsync after writing global seq number in ExternalSstFileIngestionJob
4d51feab0b Rename function for handling WAL write error
118058ba69 SstFileManager: add bytes_max_delete_chunk
88c3e26cc0 log value of CompressionOptions::zstd_max_train_bytes
620823f88b parse CompressionOptions::zstd_max_train_bytes in options string
de6cf95a53 Update history for future 5.13 release
7429b20e39 WritePrepared Txn: fix race condition on publishing seq
fa8c050e9f Fixed buffer overrun in BackupEngineImpl::BackupMeta::StoreToFile
7a6353bd1c Ignore empty filter block when data block is empty
70282cf876 fix behavior does not match name for "IsFileDeletionsEnabled"
2ce8f63f81 C API for PerfContext
d5585bb605 Shared block cache in RocksJava
f1b7b790c9 BlobDB: Fix BlobDBImpl::GCFileAndUpdateLSM issues
8823487ff7 doc: fix a typo
6383e42362 benchmark.sh to use --max_background_job
61785c73ed CloseHandle docs says that the return is non-zero, does not say TRUE(1)
93d52696bf Memory Problem Of Destorying ColumnFamilyHandle after deleting the CF
d1b26507bd fix db_compaction_test when compression disabled
ccb761364d Enable compilation on OpenBSD
1139422dfb Fix the command used to generate ctags
bef95be5d8 Improve the output of the RocksJava JUnit runner
cc340268e9 fix wrong length in snprintf
ecfca1ff59 Optimize overlap checking for external file ingestion
da82aab126 allowing CompactFiles to return new file names
cc118b0e6f Update version
0cdaa1a804 Fix WAL corruption from checkpoint/backup race condition
449627f0ea Blob DB: remove unreacheable code
6f7b7f91b5 Optionally create DuplicateDetector
e003d22526 Fix FaultInjectionTestEnv to work with DirectIO
09e5d7af8c add 4th test_group in travis
2256dab135 fix flaky DBSSTTest.DeleteSchedulerMultipleDBPaths
7153153e4b Fix enable_pipelined_write output in OPTIONS file
f6156fb558 Support StringAppendOperator(delimiter_char) constructor in java-api
c5302a8a58 Java wrapper for Native Comparators
e476d0e252 Adding stat to count cancelled compactions
a3a3f5497c Fix some typos in comments and docs.
a277b0f2b7 Clarification regarding record format
b560fc9f62 Fix a block pinning regression introduced in b555ed30a4a93b80a3ac4781c6721ab988e03b5b
e69f6e8629 Fix API name in a comment in db.h
0de710f5b8 Use nullptr instead of NULL / 0 more consistently.
f021f1d9e1 Add rocksdb_open_with_ttl function in C API
0a3db28d98 Disallow compactions if there isn't enough free space
20c508c1ed Enable subcompactions in manual level-based compaction
3462c94be7 Add dual-license info to README.md
6a3eebbab0 support multiple db_paths in SstFileManager
d518fe1da6 uint64_t and size_t changes to compile for iOS
8bc41f4f5d Update TARGETS
c364eb42b5 Windows cumulative patch
b864bc9b5b Blob DB: Improve FIFO eviction
0a2354ca8f Added bytes XOR merge operator
62277e15c3 WritePrepared Txn: Move DuplicateDetector to util
9cb4856dbd Don't need to UpdateFilesByCompactionPri for kCompactionStyleNone
5d68243e61 Comment out unused variables
1ccdc2c337 Fix vagrant build process
92b1a68d87 fix FreeBSD build
680864ae54 WritePrepared Txn: Fix bug with duplicate keys during recovery
15f55e5e06 Fix TSAN timeout in MergeOperatorPinningTest.Randomized/x test
db2445ad24 Brings the Java API for WriteBatch inline with the C++ API
1209b6db5c Blob DB: remove existing garbage collection implementation
2ac988c67e Add TransactionDB and OptimisticTransactionDB to the Java API
d060421c77 Fix a leak in prepared_section_completed_
bf937cf15b Add "rocksdb.live-sst-files-size" DB property
ec5843dca9 Comment typo
3ae0047278 skip CompactRange flush based on memtable contents
c287c098a4 Update comments in DB::Close()
d6336563e5 Adding CentOS 7 Vagrantfile & build script
ad05cbb182 DB:Open should fail on tmpfs when use_direct_reads=true
7eb292da14 Fix a memory leak in WindowsThread
dfbe52e099 Fix the Logger::Close() and DBImpl::Close() design pattern
30649dc6a1 Have a different function when ROCKSDB_JEMALLOC=0
90eca1e616 WritePrepared Txn: optimize SubBatchCnt
243220d08a Update HISTORY.md to 5.12.0
4624edc440 RocksDBOptionsParser::Parse()'s `ignore_unknown_options` argument only ingores options from higher version.
aba3409740 Back out "[codemod] - comment out unused parameters"
f4a030ce81 - comment out unused parameters
b092977643 BackupEngine gluster-friendly file naming convention
828211e901 WritePrepared Txn: fix non-emptied PreparedHeap bug
8ada876dfe Add rocksdb.iterator.internal-key property
e9c31ab159 save redundant key lookup in map of locked keys
1960e73e21 fix handling of empty string as checkpoint directory
5263da6396 fix shift UBSAN error in col_buf_encoder.cc
ab446dc22d Fix build with USE_RTTI=0
c178da053b WritePrepared Txn: optimizations for sysbench update_noindex
97307d888f Fix deadlock in ColumnFamilyData::InstallSuperVersion()
0454f781c2 fix advance reservation of arena block addresses
989d12313c Legocastle job to report lite build binary size to scuba
8eb1d445c3 Unbreak MemTableRep API change
4e7a182d09 Several small "fixes"
c88c57cde1 Tweak external file ingestion seqno logic under universal compaction
6a30b98fdc fix wrong indentation
ba6ee1f749 Fix 2 more unused reference errors VS2017
b3c5351335 Direct I/O writable file should do fsync in Close()
d08d05cb62 fix UBSAN errors in fault_injection_test
dadf01672a Rename one of the two LevelIterator
74748611a8 Suppress UBSAN error in finer guanularity
3c380fdffd Adding blog post for 5.10.2 release
b555ed30a4 Customized BlockBasedTableIterator and LevelIterator
8a04ee4fd1 WritePrepared Txn: use TransactionDBWriteOptimizations (2nd attempt)
ee1c802675 Add delay before flush in CompactRange to avoid write stalling
0a0fad447b db_bench separate options for partition index and filters
3f1bb07351 make flush_reason_ atomic to keep TSAN happy
ef29d2a234 Explictly fail writes if key or value is not smaller than 4GB
fe228da0a9 WritePrepared Txn: Support merge operator
9fc72d6f16 Compilation fixes for powerpc build, -Wparentheses-equality error and missing header guards
d62af7fa4f fix a typo (of a potential vi user)
945f618ba5 log flush reason for better debugging experience
e78715c29a Eliminate a memcpy for uncompressed blocks
a0931b3185 Fix UBSAN Error in WritePreparedTransactionTest
821e0b1683 Disable options_settable_test in UBSAN and fix UBSAN failure in blob_…
1336a7742d Disable alignment check in UBSAN
8feee28020 Add skip_cc option to TransactionDB::Write
8f8eb4f1c0 Fix leak report by asan on DuplicateKeys test
81736d8afe WritePrepared Txn: update compaction_iterator_test and db_iterator_test
2f29991701 split RandomizedHarnessTest more ways
88d8b2a2f5 WritePrepared Txn: Duplicate Keys, Txn Part
4b124fb9d3 Handle error return from WriteBuffer()
c3401846ef Minor typo in comment (s/pro/pri)
cb5b8f2090 Fix use-after-free in tailing iterator with merge operator
cd5092e168 Suppress unused warnings
a247617e6f Java: Add copy constructors for various option classes
6e5b341e8b options: Fix coverity issues
e502839e25 crc32: suppress -Wimplicit-fallthrough warnings
ba8aa8fdc8 Upgrade Appveyor to VS2017
b78ed0460b fix ReadaheadRandomAccessFile/iterator prefetch bug
813719e952 WritePrepared Txn: Duplicate Keys, Memtable part
e62a763752 Blob DB: miscellaneous changes
1edac32b77 Update rocksdb.read.block.get.micros when block cache disabled
5eccf0b9d5 add -fno-sanitize-recover option to force exit on errors
6efa8e270c Update endif/else behavior for unreachable code error on Windows.
5400800a56 Work around VS2017 warning for unused reference
ab5ab36ac2 fix DBTest2.ReadAmpBitmapLiveInCacheAfterDBClose file ID support check
ec225d2e97 Make WithParamInterface virtual in transaction_test
ab43ff58b5 Delete files in multiple ranges at once
77dc069eb9 Change size_t cast in table_test
f3fe6f883b fix for checkpoint directory with trailing slash(es)
4bdf06e78f Fix DBFlushTest::ManualFlushWithMinWriteBufferNumberToMerge dead lock
3073b1c573 Split SnapshotConcurrentAccessTest into 20 sub tests
e6605e5302 Tests for dynamic universal compaction options
3fe0937180 Use block cache to track memory usage when ReadOptions.fill_cache=false
e2d4b0efb1 db_bench: sanity check CuckooTable with mmap_read option
b8eb32f8cf Suppress lint in old files
9f7ccc8445 fix db_bench filluniquerandom key count assertion
3f666f79af Add Nim to the list of language bindings
65cd6cd4b6 Rewrite comments on use_fsync option
46acdc9883 Split HarnessTest_Randomized to avoid timeout
439855a774 StackableDB optionally take shared ownership of the underlying DB
4927b4e662 Rounddown in FilePrefetchBuffer::Prefetch
7fcc1d0ddf Incorrect Universal Compaction reason
0e6e405fec db_bench support for memtable in-place update
d938226af4 Improve performance of long range scans with readahead
65d431639b Update comments about default WALRecoveryMode
1039133f2d BlockBasedTable::NewDataBlockIterator to always return BlockIter
c7226428dd WritePrepared Txn: Fix DBIterator and add test
d6fdd59c63 CMake changes for CRC32 Optimization on PowerPC
35d8e65a04 Make Iterator::SeekForPrev pure virtual
d46e832e94 Assert last reference before destroy ColumnFamilyData
edc258127e DB::DumpSupportInfo should log all supported compression types
ec0167eecb Fix WriteBatch rep_ format for RangeDeletion records
0ea7170d7d Remove old misleading comments
7e3d3326ce Blob DB: dump blob_db_options.min_blob_size
7291a3f813 Improve fallocate size in compaction output
c615689bb5 Support skipping bloom filters for SstFileWriter
6f5ba0bf5b Fix building on FreeBSD
f2f034ef3b Blob DB: fix crash when DB full but no candidate file to evict
5568aec421 Fix DBTest::SoftLimit TSAN failure
47ad6b81ff Add 5.10.fb to tools/check_format_compatible.sh
f1cb83fcf4 Fix Flush() keep waiting after flush finish
b9873162f0 Fixed get version on windows, moved throwing exceptions into cc file.
4decff6fa8 Add possibility to change ttl on open DB
46e599fc6b fix live WALs purged while file deletions disabled
266d85fbec fix DBTest.AutomaticConflictsWithManualCompaction
dc360df81e Fix multiple build failures
bf6f03f3cd Issue #3370 Broken CMakeLists.txt
af92d4ad11 Avoid too frequent MaybeScheduleFlushOrCompaction() call
d0f1b49ab6 Add a Close() method to DB to return status when closing a db
68829ed89c Revert Snappy version upgrade
43549c7d59 Prevent unnecessary calls to PurgeObsoleteFiles
ba295cda29 replace DBTest.HugeNumbersOfLevel with a more targeted test case
e446d14093 Fix PowerPC dynamic java build
6d7e3b9faf fix Gemfile.lock nokogiri dependencies
45828c7215 Consider an increase to buffer size when reading option file, from 4K to 8K.
0a7ba0e548 Fix memleak when DB::DeleteFile()
9c2f64e148 Update Gemfile.lock
204af1eccc add WriteBatch::WriteBatch(std::string&&)
d4da02d147 Add Jenkins for PPC64le build status badge
a53c571d2d FreeBSD build support for RocksDB and RocksJava
b010116d82 Eliminate some redundant block reads.
0c6e8be9e2 Fix directory name for db_basic_test
199405192d Add a BlockBasedTableOption to turn off index block compression.
bafec6bb30 Fix checkpoint_test directory setup/cleanup
a478e85697 Remove GCC parameter "-march=native" for ARM
677f249d6d Fix Travis build failures in CMake RocksJava
6aa95f4d0f Fix a wrong log formatting
0f0d2ab95a fix DBImpl instance variable naming
46ec52499e Fix db_bench write being disabled in lite build
00b33c2474 WritePrepared Txn: address some pending TODOs
24e2c1640d add support for allow_ingest_behind in C API
398d72fa61 Add autotune and #getBytesPerSecond() to RocksJava RateLimiter
30a017feca Blob DB: avoid having a separate read of checksum
3e955fad09 Fix zstd/zdict include path for java static build
84ddbd186a Make Windows dep switches compatible with other builds
90c1d81975 Update javastatic dependencies
1c9ada59cc Remove assert(s.ok()) from ::DeleteFile
ea8ccd2267 fix powerpc java static build
00e5e1ef7f Fixes the build on Windows
ccc095a016 Speed up BlockTest.BlockReadAmpBitmap
b5c99cc908 Disable onboard cache for compaction output
f00e176c5b fix ForwardIterator reference to temporary object
02a2c11732 Blog post for WritePrepared Txn
0ef3fdd732 Disable need_log_sync on bg err
58b841b356 FIXED: string buffers potentially too small to fit formatted write
f54d7f5fea Port 3 way SSE4.2 crc32c implementation from Folly
e763e1b623 BlobDB: dump blob db options on open
48cf8da2bb BlobDB: update blob_db_options.bytes_per_sync behavior
06149429d9 WritePrepared Txn: Return NotSupported on iterator refresh
1563801bce blog post for auto-tuned rate limiter
2190e96727 Remove incorrect comment
0faa026db6 WritePrepared Txn: make buck tests parallel
78c2eedb4f fix release order in validateNumberOfEntries
aa6509d8e4 Fix build for linux
a6d3c762df WritePrepared Txn: non-2pc write in one round
fccc12f386 Add a histogram stat for memtable flush
95583e1532 db_stress: skip snapshot check if cf is dropped
237b292515 BlobDB: Remove the need to get sequence number per write
a79c7c05e8 fix backup meta-file buffer overrun
5a7e08468a fix ThreadStatus for bottom-pri compaction threads
b4d88d7128 Fix the build with MSVC 2017
def6a00740 Print out compression type of new SST files in logging
6b77c07379 NUMBER_BLOCK_COMPRESSED, etc, shouldn't be treated as timer counter
cd2e5cae7f WritePrepared Txn: make db_stress transactional
546a63272f disableWAL with WriteImplWALOnly
35dfbd58dd WritePrepared Txn: GC old_commit_map_
51c2ea0feb Reduce heavy hitter for Get operation
9089373a01 Fix DeleteScheduler::MarkAsTrash() handling existing trash
7393ef779c Fix BlockFetcher ASAN error
4bcb7fb148 Update transaction_test_util.cc
e3a06f12d2 WritePrepared Txn: fix compaction filter snapshot checks
a9c8d4ef15 Fix memory issue introduced by 2f1a3a4d748ea92c282a1302b1523adc6d67ce81
bb5ed4b1d1 exclude DynamicUniversalCompactionOptions from ROCKSDB_LITE
0d5692e02b Switch version to 5.10
2f1a3a4d74 Refactor ReadBlockContents()
9a27ac5d89 Fix drop column family data race
fcc8a6574d Make Universal compaction options dynamic
250a51a3f9 BlobDB: refactor DB open logic
6a183d1ae8 Fix coverity issues compaction_job, compaction_picker
e3814a8608 revert fbcode build behavior
fe608e32ab Fix a race condition in WindowsThread (port::Thread)
34aa245dd8 Fix coverity issues version, write_batch
baff91c1ad table: Fix coverity issues
2e3a00987e fix ASAN for DeleteFilesInRange test case
e1c569c324 Fix clang-analyzer false-positive on ldb_cmd.cc
bbef8c3884 Log GetCurrentTime failures during Flush and Compaction
d51fcb21f4 Blob DB: Add db_bench options
53a516ab58 update history for recent commits
78d1a5ec72 Preserve overlapping file endpoint invariant
a7d32776f0 Fix write_callback_test compile error
7f04af32a5 ldb to allow db with --try_load_options and without an options file
b5798bd324 Add missing recent versions to format compatible test
20995c5729 Make iterator invalid on Merge error
36911f55dd WritePrepared Txn: stress test
1b0c58d7e7 Add Code of Conduct
a37d734596 Add ROCKSDB_DISABLE_* environment variables
4634c735a8 Update DBOptions::IncreaseParallelism to use newer background settings
57056bb606 gflags in cmake on linux
63f1c0a57d fix gflags namespace
18dcf7f98d WritePrepared Txn: PreReleaseCallback
3e40a5e832 add missing config checks to CMakeLists.txt
fad14050ae Remove `import` use from TARGETS
54095d3389 TARGETS file not include tests in opt mode
ffacaaa3ea fix Seek with lower_bound
ed3af9ef99 improve ldb CLI option support
c85f8ccca3 convert null terminator in ascii dump
c1ed005a21 tools: Fix coverity issues
81cf262ff5 utilities/backupable : Fix coverity issues
75d57a5d53 C API: Add some block based table options
b45fbc1175 utilities: Fix coverity issues
7b57510a17 utilities: Fix coverity issues in blob_db and col_buf_decoder
cf0d6aa007 CMake cross platform Java support and add JNI to travis
78279350aa Blob DB: Add statistics
3cf562be31 Fix IOError on WAL write doesn't propagate to write group follower
1bdb44de95 optimize file ingestion checks for range deletion overlap
022c598abb Fix minor typo in comment
4b65cfc723 Support for block_cache num_shards and other config via option string.
2f09524762 Expose all remaining read and write options via the C API
b72b3c6f51 WritePrepared Txn: Add MultiGet to DB
f0dde49cda Blob DB: Fix GC handling for inlined blob
e59cb2a19b Add seq_per_batch to WriteWithCallbackTest
5fac4729cc make compaction_readahead_size_ thread safe
fe187e7e6c Add Elixir to the list of language bindings
8954f830a0 Blob DB: db_bench flag to control BlobDB's garbage collection
d394a6bb48 Add a ticker stat for number of keys skipped during iteration
578f36e431 Blob DB: Remove some redundant log lines
a6c6b8b38c Revert "No need for Restart Interval for meta blocks"
30285ee31c Fix calculating filter partition target size
0996e140bd Added ProfaneDB
0213990b3a Move static variables out of the header file
2d04ed65e4 Make trash-to-DB size ratio limit configurable
32e31d49d1 Make DBOption compaction_readahead_size dynamic
e27f60b1c8 distinguish kZSTDNotFinalCompression in compression string
dd49f89466 Fix TARGETS lint warnings.
bbcd3b0bd2 Suppress valgrind "unimplemented functionality" error
54b43563be WritePrepared Txn: Refactoring WriteCallback
53863b76f9 WritePrepared Txn: fix bug with Rollback seq
175d5d6a9e Properly destruct rebuilding_trx_
9871ea4357 Regression test build binaries with PORTABLE=1
42564ada53 Blob DB: not using PinnableSlice move assignment
2515266725 WritePrepared Txn: Refactoring TrackKeys
2edc92bc28 WritePrepared Txn: cross-compatibility test
857adf388f WritePrepared Txn: Refactor conf params
07c2738ffa prefer enabling cpu features via -march/-mcpu
f8e2db0717 Fix crashes, address test issues and adjust windows test script
eefd75a228 Stream
f1c5eaba56 updated c ingestexternalfileoptions for ingest behind
93f69cb93a use bottommost compression when base level is bottommost
5e9e5a4702 Blob DB: Fix race condition between flush and write
ca75f0a64a Blob DB: Fix release build
4f9f124347 Blob DB: use compression in file header instead of global options
114896c4e0 db_bench compression options
65c95d9c59 support db_bench compact benchmark on bottommost files
e03377c7fd Add lock wait time as a perf context counter
be410dede8 Fix PinnableSlice move assignment
a6d8e30c05 Remove unnecessary status check in TableCache::NewIterator
4c8f336401 util: Fix coverity issues
cfb120f737 fix CopyFile status checks
d956169563 Fix clang build error
2581c0a5a1 Blob DB: Fix BlobDBTest::SnapshotAndGarbageCollection asan failure
24ad430600 pass key/value samples through zstd compression dictionary generator
c4c1f961e7 dynamically change current memtable size
30e4e01e05 add missing else
602fe9454c Fix coverity issues in include/rocksdb
62578d80c1 Blob DB: Add compaction filter to remove expired blob index entries
76c3fbd651 Add Memtable Read Tier to RocksJava
7bfa88037e Blob DB: fix snapshot handling
f662f8f0b6 Blob DB: option to enable garbage collection
167ba599ec Blob DB: Fix flaky BlobDBTest::GCExpiredKeyWhileOverwriting test
25ac1697b4 Blob DB: Evict oldest blob file when close to blob db size limit
3c208e7616 HistogramStat: Handle divide by zero situation
25fbd9a996 Remove the experimental notes about partitioning
60d83df23d WritePrepared Txn: Move DB class to its own file
6778690b51 fix duplicate definition of GetEntryType()
cd124215df release 5.9
02693f64fc WritePrepared Txn: ValidateSnapshot
7fe3b32896 Added support for differential snapshots
17731a43a6 WritePrepared Txn: Optimize for recoverable state
c1cf94c787 WritePrepared Txn: sort indexes before batch collapse
f6082d1944 Blob DB: cleanup unused options
f5078dde2d Blob DB: Initialize all fields in Blob Header, Footer and Record structs
33c7d4ccd9 Make writable_file_max_buffer_size dynamic
c1be8d86c6 Fix removed structurally dead return statement
4d43c6a6a4 db_stress snapshot compatibility with reopens
b7bc9cc038 fix tracking oldest snapshot for bottom-level compaction
792ef10ca8 Return Status::InvalidArgument if user request sync write while disabling WAL
6a9335dbbb always drop tombstones compacted to bottommost level
84a04af9a9 TableProperty::oldest_key_time defaults to 0
05993155ef Mark files as trash by using .trash extension
3ebb7ba7b9 Blob DB: update blob file format
682db81385 Enable cacheline_aligned_alloc() to allocate from jemalloc if enabled.
d9240b548c Fix coverity uninitialized fields warnings in lru_cache
50e95a63dd Fix coverity issues column_family, compaction_db/iterator
47166baeac Fix coverity uninitialized fields warnings
67b29e26be Fix coverity issue for MutableDBOptions default constructor
95667383db implement lower bound for iterators
5a2a6483dc Blob DB: Inline small values in base DB
9b18cc2363 single-file bottom-level compaction when snapshot released
96e3a600ba Return write error on reaching blob dir size limit
addfe1ef4b Fix tombstone scans in SeekForPrev outside prefix
386a57e6ef Fix build on OpenBSD
57fcdc264a added missing subcodes and improved error message for missing enum values
66a2c44ef4 Add DB::Properties::kEstimateOldestKeyTime
d2a65c59e1 Fix unused var warnings in Release mode
63822eb761 Enable two write queues for transactions
a02ed12638 Exclude DBTest.DynamicFIFOCompactionOptions test under RocksDB Lite
3ef55d2c7c Split CompactionFilterWithValueChange
d75793d6b4 db_stress support long-held snapshots
f8b5bb2fd8 remove unused code
57fd4a823b update HISTORY with recent changes
ee2b1ec1e8 Fix unstable floating point exception
f0804db7f7 Make FIFO compaction options dynamically configurable
ebab2e2d42 Enable MSVC W4 with a few exceptions. Fix warnings and bugs
b74999458f Update RocksDB Authors File
7deed2b43c Fix a typo in a comment
7e38238981 WritePrepared Txn: Disable GC during recovery
7891af8b53 expose a hook to skip tables during iteration
eaaef91178 Blob DB: Store blob index as kTypeBlobIndex in base db
0552029b5c Blob DB: not writing sequence number as blob record footer
966b32b57c fix delete range bug
c0208dffbe arena: derive alignment unit from std::max_align_t
b8cea7cc27 VersionBuilder: Erase with iterators for better performance
f7843f30a8 Move ~Comparator define to comparator.h
8e63cad078 fix lite build
731895214b db_bench randomtransaction print throughput
60c09f5fbb print more table_options to info log
10ba50e9eb Blob DB: Move BlobFile definition to a separate file
31d3e41810 PinnableSlice move assignment
cc67b22d6e Add OptionsUtil class to java/CMakeLists.txt
560e984995 Added CompactionFilterFactory support to RocksJava
8dd0a7e11a add comment in SuperVersion referencing logic
fb4ae4d810 fix DBImpl::NewInternalIterator super-version leak on failure
019aa7074c cmake: pass "-msse4.2" to when building crc32c.cc if HAVE_SSE42
38bbc879a5 fix travis failure for ccache command not found
0f3b36964e Fix counter for memtable updates
70aa942153 fix file numbers after repair
1a61ba179e compaction picker to use max_bytes_for_level_multiplier_additional
e2548366e1 add GetLiveFiles and GetLiveFilesMetaData for BlobDB
8c392a31d7 WritePrepared Txn: Iterator
5a38e18627 Make some WriteOptions defaults more explicit
17c6325e8a WritePrepare Txn: Cancel flush/compaction before destruction
ec6c5383d0 WritePrepared Txn: end-to-end tests
da29eba43b Enable WAL for blob index
d1b74b0c82 WritePrepared Txn: Compaction/Flush
01542400a8 Inform caller when rocksdb is stalling writes
cc20ec3689 WritePrepared Txn: Test sequence number 0 is visible
5a6ad9d52a release build treat warnings as errors
4e3c3d8c6a WritePrepared Txn: duplicate keys
1026e794a3 rate limit auto-tuning
75f7f42d41 Added CPU prefetch for skiplist
88ed1f6ea6 Allow upgrades from nullptr to some merge operator
5b2cb64bfb Prevent threads from respawning during joining
821887036e pin L0 filters/indexes for compaction outputs
283d60761e fix valgrind leak report in unit test
377e004048 Fix DBOptionsTest.SetBytesPerSync test when run with no compression
92ccae7123 speedup 'make check'
d1cab2b64e Add ValueType::kTypeBlobIndex
880411f54c disable populating block cache for in-place updates
d27258d3a6 WritePrepared Txn: Rollback
bb38cd03a9 Limit number of merge operands in Cassandra merge operator
cf51d3eb73 Remove an "unused" variable
983028f097 RocksJava build target for Docker on ppc64le
2a3363d52e ldb dump can print histogram of value size
593d3de371 No need for Restart Interval for meta blocks
2b22baf304 Add a template for issues
ab0542f5ec Fix for when block.cache_handle is nullptr
5df172da2f fix deletion-triggered compaction in table builder
385049baf2 WritePrepared Txn: Recovery
8c724f5c7f Default one to rocksdb:x64-windows
93c2b91740 Introduce conditional merge-operator invocation in point lookups
a48a398e7c Use RAII instead of pointers in cf_info_map
c70586621c Blog post for 5.8 release
c2f6e45aa3 prevent nullptr dereference in table reader error case
6a541afcc4 Make bytes_per_sync and wal_bytes_per_sync mutable
ec48e5c77f Add TransactionDB::SingleDelete()
0806801dc8 DestroyDB API
aa67bae6cf Break down PinnedDataIteratorRandomized
4748911357 Add LogDevice to USERS.md
1d6700f9e6 Add test kPointInTimeRecoveryCFConsistency
be97dbb15c Fix WritePreparedTransactionTest::SeqAdvanceTest ASAN failure
4708a6875c Repair DBs with trailing slash in name
fc7476bec1 fix populating range deletions in forward iterator
c8f3606731 Expose LoadLatestOptions, LoadOptionsFromFile and GetLatestOptionsFileName APIs in RocksJava
96a13b4f4b Use jemalloc in rocksdbjni library built via vagrant
65a9cd6168 Use L1 size as estimate for L0 size in LevelCompactionBuilder::GetPathID
8fc3de3c62 make rate limiter a general option
1480e6f7cf Fix TransactionTest::SeqAdvanceTest ASAN failure
3fc08fa88e Expose max_background_jobs option in RocksJava
8ae81684e9 Update cmake_minimum_required to 2.8.12.
b4596c6174 Fix Get does not return super version on error
34ebadf930 Fix MinGW build
e4234fbdcf collecting kValue type tombstone
60beefd6e0 WritePrepared Txn: Advance seq one per batch
c57050b770 Use the default copy constructor in Options
c319792059 Directly refernce perf_context internally.
6b3c71f6ed Fix DBImpl::NotifyOnCompactionCompleted data race
f47b4eeb1e Fix memory leak in OptionsTest::OptionsComposeDecompose
382277d0fe JNI support for ReadOptions::iterate_upper_bound
edcbb36944 Three code-level optimization to Iterator::Next()
885b1c682e Two small refactoring for better inlining
ffac68367f Added save points for transactions C API
9a970c81af Fix WriteBatchWithIndex::GetFromBatchAndDB not allowing StackableDB
a843df668b Fix use-after-free in c_tset
2d6e42122b Remove 'experimental' comment around level_compaction_dynamic_level_bytes option
464fb36de9 fix hanging after CompactFiles with L0 overlap
09713a64b3 WritePrepared Txn: Lock-free CommitMap
72e4190918 Additions for `OptimisticTransactionDB` in C API
9d115d3689 regression test for missing init options
f615f5604b fix missing manual_wal_flush for DBOptions ctor
5785b1fcb8 Fix naming in InternalKey
82860bd55c Use cmake TIMESTAMP function
2d30aaae47 Exclude incompatible options in test
f5148ade10 support opening zero backups during engine init
3c42807794 do not call merge when checking to see if key exists
025b85b4ac speedup DBTest.EncodeDecompressedBlockSizeTest
044a71e27e Add iterator's SeekForPrev functionality to the java-api
64b6452e0c Make InternalKeyComparator final and directly use it in merging iterator
2dd22e5449 Make DBIter class final
2a5915049e Fix missing BYTES_PER_WRITE for pipeline write
f46464d383 write-prepared txn: call IsInSnapshot
9a4df72994 WritePrepared Txn: CommitBatch
fce6c892ab Advance max evicted seq in coarser granularity
dcd36a6aee Make it explicit blob db doesn't support CF
3cd7ea2e8a rename stall-related internal stats
0e99323ac2 Fix CLANG Analyze
ba3c58cab6 specify SSE42 'target' attribute for Fast_CRC32()
7e19a571e9 Remove unused TransactionCallback
10ddd59ba7 fix CompactFiles inclusion of older L0 files
79810e2d49 Skip write_prepared_transaction_test in travis
ab95e293d2 Fix memory leak on blob db open
37ae8cc60f Signal progress of the test to avoid timeout
3b23b1d8c6 fix HistogramWindowingImpl copy-{assignment,constructor}
3b9a000c15 fix inclusive-exclusiveness of histogram ToString
0ec90a7cc2 Add -DPORTABLE=1 to MSVC CI build
4a90cbf429 Include statistics pointer in ImmutableDBOptions::Dump
b97685aef6 fix backup engine when latest backup corrupt
725bc403f8 update dependencies.sh
e612e31740 Updated CRC32 Power Optimization Changes
8fa4d108a2 Try to switch to Stduio 2017
825a22c00c garbage collect tombstones in merge operator
26ac24f199 Add more unit test to write_prepared txns
06b37eef7b Set defaults for high-pri and low-pri thread pools in regression test script
c10cf166fa Dump non-final ZSTD compression type support
8a6708f5f2 Extend property map with compaction stats
dc5f29f32f EnvWrapper: Forward more functions
266ac245af Bumping version to 5.8
64185c23ad update HISTORY.md for DeleteRange bug fix
e83d6a02e3 Not using aligned_alloc with gcc4 + asan
0980dc6c9a Fix wrong smallest key of delete range tombstones
b767972313 avoid use-after-move error
c41744270a CMake: Fix formatting
c21ea8f7a6 CMake: Add support for CMake packages
5444345588 add Erlang to the list of language bindings
2972a702a4 Minor updates to FlushWAL blog
fbfa3e7a43 WriteAtPrepare: Efficient read from snapshot list
b01f426f56 Blog post for FlushWAL
503db684f7 make blob file close synchronous
3c840d1a6d Allow DB reopen with reduced options.num_levels
92bfd6c507 Fix DropColumnFamily data race
7fdf735d5d Pinnableslice examples and blog post
7fbb9eccaf support disabling checksum in block-based table
19cc66dc4f fix clang bug in block-based table reader
7eba54eb9b test compaction input-level split range tombstone assumption
cd26af3476 Add unit test for WritePrepared skeleton
a12479819d Improved transactions support in C API
c10b391314 LANGUAGE-BINDINGS.md: add another rust binding
90177432e4 Remove leftover references to phutil_module_cache
234f33a3f9 allow nullptr Slice only as sentinel
ccf7f833e3 Use PinnableSlice in Transactions
1dfcdb15f9 Extend pin_l0 to filter partitions
39ef900551 stop calling memcmp with nullptrs
78cb6b6112 Provide byte[] version of SstFileWriter.merge to reduce GC Stall
867fe92e5e Scale histogram bucket size by constant factor
f004307e9b CMake improvements
09ac6206ab Circumvent ASAN false positive
5b68b114f1 Blob db create a snapshot before every read
4624ae52c9 GC the oldest file when out of space
8ace1f79b5 add counter for deletion dropping optimization
0d8e992b47 Revert the mistake in version update
5358a80568 add VerifyChecksum to HISTORY.md
ed0a4c93ef perf_context measure user bytes read
1efc600ddf Preload l0 index partitions
bddd5d3630 Added mechanism to track deadlock chain
c1384a7076 fix db_stress uint64_t to int32 cast
29877ec7b4 Fix blob db crash during calculating write amp
8f2598ac9d Enable Cassandra merge operator to be called with a single merge operand
9a44b4c32c Allow merge operator to be called even with a single operand
ac8fb77afd fix some misspellings
23593171c4 minor improvements to db_stress
af012c0f83 fix deleterange with memtable prefix bloom
1c8dbe2aa2 update scores after picking universal compaction
eb6425303e Update WritePrepared with the pseudo code
132306fbf0 Remove PartialMerge implementation from Cassandra merge operator
71598cdc75 Fix false removal of tombstone issue in FIFO and kCompactionStyleNone
3204a4f64b Fix missing stdlib include required for abort()
7aa96db7a2 db_stress rolling active window
dfa6c23c4b Update RocksDBCommonHelper to use escapeshellarg
e367774d19 Overload new[] to properly align LRUCacheShard
ad42d2fcbb Remove residual arcanist_util directory
279296f4d8 properly set C[XX]FLAGS during CMake configure-time checks
c5f0c6cc66 compile with correct flags to determine SSE4.2 support
185ade4c0c cmake: support more compression type
5449c0990b rocksdb: make buildable on aarch64
a144a9782d Fix for CMakeLists.txt on Windows for RocksJava
acf935e40f fix deletion dropping in intra-L0
8254e9b57c make sst_dump compression size command consistent
74f18c1301 db_bench support for non-uniform column family ops
5de98f2d50 approximate histogram stats to save cpu
3f5888430a Fix c_test ASAN failure
e5a1b727c0 Fix blob DB transaction usage while GC
6f051e0c71 fix corruption_test valgrind
ac098a4626 expose set_skip_stats_update_on_db_open to C bindings
666a005f9b Support prefetch last 512KB with direct I/O in block based file reader
ad77ee0ea0 Revert "Makefile: correct faligned-new test"
b87ee6f773 Use more keys per lock in daily TSAN crash test
25df24254b Add column families related functions (C API)
64f8484356 block_cache_tier: fix gcc-7 warnings
0cecf8155b Write batch for `TransactionDB` in C API
6a9de43477 Windows.h macro call fix
23c7d13540 fix comment
1fbad84b69 Makefile: correct faligned-new test
7848f0b24c add VerifyChecksum() to db.h
47ed3bfc3b fix WinEnv assertions
d97a72d63f Try to repair db with wal_dir option, avoid leak some WAL files
36375de76f gcc-7/i386: markup intentional fallthroughs
bdc056f8aa Refactor PessimisticTransaction
a9a4e89c38 Fix valgrind complaint about initialization
4ca11b4b07 Update USERS.md
c9804e007a Refactor TransactionDBImpl
20dc5e74f2 Optimize range-delete aggregator call in merge helper.
0d4a2b7330 Avoid blob db call Sync() while writing
627c9f1abb Don't add -ljemalloc when DISABLE_JEMALLOC is set
dce6d5a838 db_bench background work thread pool size arguments
4f81ab38bf Makefile: fix for GCC 7+ and clang 4+
92afe830f9 Update all blob db TTL and timestamps to uint64_t
5883a1ae24 Fix /bin/bash shebangs
cc01985db0 Introduce bottom-pri thread pool for large universal compactions
0b814ba92d Allow concurrent writes to blob db
2c45ada4c4 Blob DB garbage collection should keep keys with newer version
58410aee44 Fix the overflow bug in AwaitState
c3d5c4d38a Refactor TransactionImpl
060ccd4f84 support multiple CFs with OPTIONS file
3453870677 Fix statistics in RocksJava sample
1900771bd2 Dump Blob DB options to info log
3218edc573 Fix universal compaction bug
6a36b3a7b9 fix db get/write stats
a84cee8127 Add a missing "once" in .h
21696ba502 Replace dynamic_cast<>
e85f2c64cb Prevent empty memtables from using a lot of memory
ac748c57ed Fix FIFO Compaction with TTL tests
aaf42fe775 Move blob_db/ttl_extractor.h into blob_db/blob_db.h
aace46516b Fix license headers in Cassandra related files
50a969131f CacheActivityLogger, component to log cache activity into a file
6083bc79f8 Blob DB TTL extractor
710411aea6 fix asan/valgrind for TableCache cleanup
3a3fb00b7a TARGETS file not setting sse explicitly
fca4d6da17 Build fewer tests in Travis platform_dependent tests
8f553d3c52 remove unnecessary internal_comparator param in newIterator
7f6d012d70 "ccache -C" in Travis
d12691b86f move TableCache::EraseHandle outside of db mutex
f33f113683 fix db_bench argument type
e7697b8ce8 Fix LITE unit tests
3ce20e985b Fix use of RocksDBCommonHelper in cont_integration.sh
c281b44829 Revert "CRC32 Power Optimization Changes"
9980de262c Fix FIFO compaction picker test
2289d38115 CRC32 Power Optimization Changes
30b58cf71a Remove the orphan assert on !need_log_sync
fe1a5559f3 Fix flaky write_callback_test
addbd279c2 5.6.1 release blog post
30edff308e buckification: remove explicit `-msse*` compiler flags
2b259c9d49 Lower num of iterations in DeadlockCycle test
277f6f23d4 Release note for partitioned index/filters
5e731a1382 Remove unused rocksdb arcanist lib
9b11d4345a Fix broken links
06f1917449 add vcpkg as an windows option
ea8ad4f678 Fix compaction div by zero logging
34112aeffd Added db paths to c
1d8aa2961c Gcc 7 ParsedInternalKey replace memset with clear function.
a4c42e8007 Fix UBSAN issue of passing nullptr to memcmp
16e0388205 LRUCacheShard cache line size alignment
216644c61c enable UBSAN macro in TARGETS
e67b35c076 Add Iterator::Refresh()
a34b2e388e Fix caching of compaction picker's next index
72502cf227 Revert "comment out unused parameters"
1d7048c598 comment out unused parameters
534c255c7a Cassandra compaction filter for purge expired columns and rows
63163a8c6e Remove make_new_version.sh
0302da47a7 Reduce blob db noisy logging
3e5ea29a83 Fix Flaky DeleteSchedulerTest::ImmediateDeleteOn25PercDBSize
a22b9cc6fe overlapping endpoint fixes in level compaction picker
ffd2a2eefd delete ExpandInputsToCleanCut failure log
3e6e863b1f Remove arcanist_util directory
36651d14ee Moving static AdaptationContext to outside function
6e3ee015fb Update java/rocksjni.pom
ecff9d5e33 Include write_buffer_manager in ImmutableDBOptions::Dump
ae28634e9f Remove some left-over BSD headers
33b1de82a7 Remove format compatibility hack
2f375154ea checkout local branch in check_format_compatible.sh
ddb22ac59c avoid collision with master branch in check format
0c03a7f17d set the remote for git checkout
7ac184c6d7 Revert cmake -DNDEBUG for non-MSVC
0655b58582 enable PinnableSlice for RowCache
00464a3140 Fix column_family_test with LITE build
b2dd192fed tools/write_stress.cc: Correct "1204" typos.
cbaab30449 table/block.h: change memset
f1a056e005 CodeMod: Prefer ADD_FAILURE() over EXPECT_TRUE(false), et cetera
4a2e4891fe Add back the LevelDB license file
a7321fc97f Remove the licensing description in CONTRIBUTING.md
3c327ac2d0 Change RocksDB License
REVERT: 185a70d50e cherry-pick lower bound for iterator
REVERT: 0335b5231e cherry-pick addfe1ef4b611b8ac47a24f8d7aad92a20d5cb0b Fix tombstone scans in SeekForPrev outside prefix
REVERT: cbb6be8692 cherry-pick lower-bound for ReadOptions
REVERT: 532fc5e664 Merge pull request #4 from BusyJay/adjust-cmakefile
REVERT: 794e69e914 cmake: pass "-msse4.2" to when building crc32c.cc if HAVE_SSE42
REVERT: 8ecc536620 Update cmake_minimum_required to 2.8.12.
REVERT: 8fe21cec85 Use cmake TIMESTAMP function
REVERT: 393b624d66 cmake: support more compression type
REVERT: 33efdde831 Fix missing BYTES_PER_WRITE for pipeline write
REVERT: e196465091 compile with correct flags to determine SSE4.2 support
REVERT: 8946cf270f specify SSE42 'target' attribute for Fast_CRC32()
REVERT: acef93ed65 Bump version to 5.7.3
REVERT: cf38a47a64 update HISTORY.md for DeleteRange bug fix
REVERT: b722a18f67 Fix wrong smallest key of delete range tombstones
REVERT: 2b81d372eb fix deleterange with memtable prefix bloom
REVERT: cfaeb5846b Bump version to 5.7.2
REVERT: 62e9418a70 Fix false removal of tombstone issue in FIFO and kCompactionStyleNone
REVERT: 03df818cff Fix missing stdlib include required for abort()
REVERT: 96a092b1a3 empty commit
REVERT: d59fdc3946 increment patch version for 5.7.1
REVERT: e96d06c816 fix deletion dropping in intra-L0
REVERT: b86f89f25b Move blob_db/ttl_extractor.h into blob_db/blob_db.h
REVERT: 48cf039064 Blob DB TTL extractor
REVERT: c39939d468 Reduce blob db noisy logging
REVERT: 65afd8a1cc Update java/rocksjni.pom
REVERT: 6df239cfee Remove some left-over BSD headers
REVERT: 54c4b3ab4f Add back the LevelDB license file
REVERT: 27768b0ec3 Remove the licensing description in CONTRIBUTING.md
REVERT: 8673a8c567 Change RocksDB License
REVERT: 6124420d5d Remove the orphan assert on !need_log_sync
REVERT: 258fd9ac46 Release note for partitioned index/filters

git-subtree-dir: librocksdb_sys/rocksdb
git-subtree-split: 22044072faae92ffb56855b3abaf0dae0fa38e82

New version rocksdb may change the value of some enums, then we have
to update those enums accordingly. So we need a script to generate
those enums instead of updating them manually.

* add func: load_latest_options

* modify: retrun value

* modify: expose func

* modify: immutable borrow

* modify: remove warning

* modify: add assert

* modify: s/cf_dec/cf_desc, s/cf_decs/cf_descs

* modify: refactor

* modify: refactor

* modify: expose CColumnFamilyDescriptor

* modify: func name

* modify: cargo fmt the whole project

* fix: ci error

Expose the bottommost level compaction option to make it more flexible.

* *: reformat code

Just reformat code without other changes.

* travis: add cargo fmt

* travis: update rust version

* travis: use stable version

Sync of https://github.com/pingcap/rocksdb/pull/22

Fix the following gcc-8 warnings:
- conflicting C language linkage declaration [-Werror]
- writing to an object with no trivial copy-assignment [-Werror=class-memaccess]
- array subscript -1 is below array bounds [-Werror=array-bounds]

* change block cache capacity

* Add wrapper for CompactFiles