Unverified Commit 970a27c2 authored by Brian Anderson's avatar Brian Anderson Committed by GitHub

Fix clippy warnings etc. (#393)

* Fix various unsafe declarations

As reported by clippy. These are all cases where a pointer argument is
dereferenced by a declared safe function. The functions are changed to be unsafe
functions. Though some of these functions are declared pub, they all seem to be
related to internal binding matters and appear to be written correctly, just
declared incorrectly, so there shouldn't be any external impact for callers
that aren't using the crocksdb API directly.
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Fix some clippy style lints
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Fix mutability of crocksdb_load_latest_options errptr

This value can be mutated on the C side. Caught by clippy.
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Cleanup some unit-return functions
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Disable a clippy lint
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* make a transmute more typesafe
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Elide a lifetime
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Remove unnecessary unsafe blocks
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Use repr(C) structs of c_void for opaque types.

These represent opaque RocksDB types. They are used behind pointers, but are
also wrapped in other types in the higher-level bindings.

These use the strategy for opaque C types described in the nomicon [1]:
but with the exception that they contain c_void instead of [u8; 0], thus
making them uninstantiable sized types instead of ZSTs.

The c_void documentation publicly recommends using the ZST pattern from the
nomicon, but in private documentation [2] warns about UB from dereferencing
pointers to uninhabited types, which these bindings do.

Additionally, these bindings wrap some these types directly (not through
pointers) and it's impossible to repr(transparent) a ZST, without which the
unsafe casts within are dubious.

[1]: https://doc.rust-lang.org/nomicon/ffi.html#representing-opaque-structs
[2]: https://doc.rust-lang.org/nightly/src/core/ffi.rs.html#28Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Apply repr(transparent) to various unsafe-cast wrappers

These are wrappers of opaque FFI types, and pointers are being unsafely cast
between the two. repr(transparent) is probably needed to make this well-defined.
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Replace mem::transmute with pointer casts

Slightly more typesafe, and more explicit about what's being cast.
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Replace more mem::transmute with casts
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Remove a no-op mem::transmute
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Remove is_power_of_two

std contains a branchless bitmagic version of this function
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Add clippy makefile target
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Add clippy to CI
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Fix types in memcpy

This code is copying from c_void pointers when it should be copying
from byte pointers.

Fortunately c_void and u8 have the same size, but I don't know if that is
guaranteed.
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Update src/rocksdb.rs
Co-Authored-By: 's avatarkennytm <kennytm@gmail.com>
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Install clippy on travis
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Adjust clippy lints for CI toolchain
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* Make a typesafe cast
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>

* rustfmt
Signed-off-by: 's avatarBrian Anderson <andersrb@gmail.com>
parent 30317230
...@@ -24,8 +24,10 @@ cache: false ...@@ -24,8 +24,10 @@ cache: false
before_script: before_script:
- rustup default nightly-2019-09-05 - rustup default nightly-2019-09-05
- rustup component add rustfmt-preview - rustup component add rustfmt-preview
- rustup component add clippy
script: script:
- make clippy
- cargo fmt --all -- --check - cargo fmt --all -- --check
- cargo build - cargo build
- cargo test --all - cargo test --all
...@@ -14,6 +14,14 @@ clean: ...@@ -14,6 +14,14 @@ clean:
@cargo clean @cargo clean
@cd librocksdb_sys && cargo clean @cd librocksdb_sys && cargo clean
# TODO it could be worth fixing some of these lints
clippy:
@cargo clippy --all -- \
-D warnings \
-A clippy::redundant_field_names -A clippy::single_match \
-A clippy::assign_op_pattern -A clippy::new_without_default -A clippy::useless_let_if_seq \
-A clippy::needless_return -A clippy::len_zero
update_titan: update_titan:
@if [ -n "${TITAN_REPO}" ]; then \ @if [ -n "${TITAN_REPO}" ]; then \
git config --file=.gitmodules submodule.titan.url https://github.com/${TITAN_REPO}/titan.git; \ git config --file=.gitmodules submodule.titan.url https://github.com/${TITAN_REPO}/titan.git; \
......
...@@ -22,60 +22,134 @@ use libc::{c_char, c_double, c_int, c_uchar, c_void, size_t}; ...@@ -22,60 +22,134 @@ use libc::{c_char, c_double, c_int, c_uchar, c_void, size_t};
use std::ffi::CStr; use std::ffi::CStr;
use std::fmt; use std::fmt;
pub enum Options {} // FFI-safe opaque types.
pub enum ColumnFamilyDescriptor {} //
pub enum DBInstance {} // These represent opaque RocksDB types. They are used behind pointers, but are
pub enum DBWriteOptions {} // also wrapped in other types in the higher-level bindings.
pub enum DBReadOptions {} //
pub enum DBMergeOperator {} // These use the strategy for opaque C types described in the nomicon [1]:
pub enum DBBlockBasedTableOptions {} // but with the exception that they contain c_void instead of [u8; 0], thus
pub enum DBMemoryAllocator {} // making them uninstantiable sized types instead of ZSTs.
pub enum DBLRUCacheOptions {} //
pub enum DBCache {} // The c_void documentation publicly recommends using the ZST pattern from the
pub enum DBFilterPolicy {} // nomicon, but in private documentation [2] warns about UB from dereferencing
pub enum DBSnapshot {} // pointers to uninhabited types, which these bindings do.
pub enum DBIterator {} //
pub enum DBCFHandle {} // Additionally, these bindings wrap some these types directly (not through
pub enum DBWriteBatch {} // pointers) and it's impossible to repr(transparent) a ZST, without which the
pub enum DBComparator {} // unsafe casts within are dubious.
pub enum DBFlushOptions {} //
pub enum DBCompactionFilter {} // [1]: https://doc.rust-lang.org/nomicon/ffi.html#representing-opaque-structs
pub enum EnvOptions {} // [2]: https://doc.rust-lang.org/nightly/src/core/ffi.rs.html#28
pub enum SstFileReader {}
pub enum SstFileWriter {} #[repr(C)]
pub enum ExternalSstFileInfo {} pub struct Options(c_void);
pub enum IngestExternalFileOptions {} #[repr(C)]
pub enum DBBackupEngine {} pub struct ColumnFamilyDescriptor(c_void);
pub enum DBRestoreOptions {} #[repr(C)]
pub enum DBSliceTransform {} pub struct DBInstance(c_void);
pub enum DBRateLimiter {} #[repr(C)]
pub enum DBLogger {} pub struct DBWriteOptions(c_void);
pub enum DBCompactOptions {} #[repr(C)]
pub enum DBFifoCompactionOptions {} pub struct DBReadOptions(c_void);
pub enum DBPinnableSlice {} #[repr(C)]
pub enum DBUserCollectedProperties {} pub struct DBMergeOperator(c_void);
pub enum DBUserCollectedPropertiesIterator {} #[repr(C)]
pub enum DBTableProperties {} pub struct DBBlockBasedTableOptions(c_void);
pub enum DBTablePropertiesCollection {} #[repr(C)]
pub enum DBTablePropertiesCollectionIterator {} pub struct DBMemoryAllocator(c_void);
pub enum DBTablePropertiesCollector {} #[repr(C)]
pub enum DBTablePropertiesCollectorFactory {} pub struct DBLRUCacheOptions(c_void);
pub enum DBFlushJobInfo {} #[repr(C)]
pub enum DBCompactionJobInfo {} pub struct DBCache(c_void);
pub enum DBIngestionInfo {} #[repr(C)]
pub enum DBEventListener {} pub struct DBFilterPolicy(c_void);
pub enum DBKeyVersions {} #[repr(C)]
pub enum DBEnv {} pub struct DBSnapshot(c_void);
pub enum DBSequentialFile {} #[repr(C)]
pub enum DBColumnFamilyMetaData {} pub struct DBIterator(c_void);
pub enum DBLevelMetaData {} #[repr(C)]
pub enum DBSstFileMetaData {} pub struct DBCFHandle(c_void);
pub enum DBCompactionOptions {} #[repr(C)]
pub enum DBPerfContext {} pub struct DBWriteBatch(c_void);
pub enum DBIOStatsContext {} #[repr(C)]
pub enum DBWriteStallInfo {} pub struct DBComparator(c_void);
pub enum DBStatusPtr {} #[repr(C)]
pub enum DBMapProperty {} pub struct DBFlushOptions(c_void);
#[repr(C)]
pub struct DBCompactionFilter(c_void);
#[repr(C)]
pub struct EnvOptions(c_void);
#[repr(C)]
pub struct SstFileReader(c_void);
#[repr(C)]
pub struct SstFileWriter(c_void);
#[repr(C)]
pub struct ExternalSstFileInfo(c_void);
#[repr(C)]
pub struct IngestExternalFileOptions(c_void);
#[repr(C)]
pub struct DBBackupEngine(c_void);
#[repr(C)]
pub struct DBRestoreOptions(c_void);
#[repr(C)]
pub struct DBSliceTransform(c_void);
#[repr(C)]
pub struct DBRateLimiter(c_void);
#[repr(C)]
pub struct DBLogger(c_void);
#[repr(C)]
pub struct DBCompactOptions(c_void);
#[repr(C)]
pub struct DBFifoCompactionOptions(c_void);
#[repr(C)]
pub struct DBPinnableSlice(c_void);
#[repr(C)]
pub struct DBUserCollectedProperties(c_void);
#[repr(C)]
pub struct DBUserCollectedPropertiesIterator(c_void);
#[repr(C)]
pub struct DBTableProperties(c_void);
#[repr(C)]
pub struct DBTablePropertiesCollection(c_void);
#[repr(C)]
pub struct DBTablePropertiesCollectionIterator(c_void);
#[repr(C)]
pub struct DBTablePropertiesCollector(c_void);
#[repr(C)]
pub struct DBTablePropertiesCollectorFactory(c_void);
#[repr(C)]
pub struct DBFlushJobInfo(c_void);
#[repr(C)]
pub struct DBCompactionJobInfo(c_void);
#[repr(C)]
pub struct DBIngestionInfo(c_void);
#[repr(C)]
pub struct DBEventListener(c_void);
#[repr(C)]
pub struct DBKeyVersions(c_void);
#[repr(C)]
pub struct DBEnv(c_void);
#[repr(C)]
pub struct DBSequentialFile(c_void);
#[repr(C)]
pub struct DBColumnFamilyMetaData(c_void);
#[repr(C)]
pub struct DBLevelMetaData(c_void);
#[repr(C)]
pub struct DBSstFileMetaData(c_void);
#[repr(C)]
pub struct DBCompactionOptions(c_void);
#[repr(C)]
pub struct DBPerfContext(c_void);
#[repr(C)]
pub struct DBIOStatsContext(c_void);
#[repr(C)]
pub struct DBWriteStallInfo(c_void);
#[repr(C)]
pub struct DBStatusPtr(c_void);
#[repr(C)]
pub struct DBMapProperty(c_void);
#[derive(Copy, Clone, Debug, Eq, PartialEq)] #[derive(Copy, Clone, Debug, Eq, PartialEq)]
#[repr(C)] #[repr(C)]
...@@ -88,8 +162,10 @@ pub enum WriteStallCondition { ...@@ -88,8 +162,10 @@ pub enum WriteStallCondition {
mod generated; mod generated;
pub use generated::*; pub use generated::*;
pub enum DBTitanDBOptions {} #[repr(C)]
pub enum DBTitanReadOptions {} pub struct DBTitanDBOptions(c_void);
#[repr(C)]
pub struct DBTitanReadOptions(c_void);
#[derive(Clone, Debug, Default)] #[derive(Clone, Debug, Default)]
#[repr(C)] #[repr(C)]
...@@ -103,8 +179,8 @@ pub fn new_bloom_filter(bits: c_int) -> *mut DBFilterPolicy { ...@@ -103,8 +179,8 @@ pub fn new_bloom_filter(bits: c_int) -> *mut DBFilterPolicy {
unsafe { crocksdb_filterpolicy_create_bloom(bits) } unsafe { crocksdb_filterpolicy_create_bloom(bits) }
} }
pub fn new_lru_cache(opt: *mut DBLRUCacheOptions) -> *mut DBCache { pub unsafe fn new_lru_cache(opt: *mut DBLRUCacheOptions) -> *mut DBCache {
unsafe { crocksdb_cache_create_lru(opt) } crocksdb_cache_create_lru(opt)
} }
#[derive(Copy, Clone, Debug, Eq, PartialEq)] #[derive(Copy, Clone, Debug, Eq, PartialEq)]
...@@ -302,12 +378,10 @@ pub enum DBBackgroundErrorReason { ...@@ -302,12 +378,10 @@ pub enum DBBackgroundErrorReason {
MemTable = 4, MemTable = 4,
} }
pub fn error_message(ptr: *mut c_char) -> String { pub unsafe fn error_message(ptr: *mut c_char) -> String {
let c_str = unsafe { CStr::from_ptr(ptr) }; let c_str = CStr::from_ptr(ptr);
let s = format!("{}", c_str.to_string_lossy()); let s = format!("{}", c_str.to_string_lossy());
unsafe {
libc::free(ptr as *mut c_void); libc::free(ptr as *mut c_void);
}
s s
} }
...@@ -634,7 +708,7 @@ extern "C" { ...@@ -634,7 +708,7 @@ extern "C" {
cf_descs: *const *mut *mut ColumnFamilyDescriptor, cf_descs: *const *mut *mut ColumnFamilyDescriptor,
cf_descs_len: *mut size_t, cf_descs_len: *mut size_t,
ignore_unknown_options: bool, ignore_unknown_options: bool,
errptr: *const *mut c_char, errptr: *mut *mut c_char,
) -> bool; ) -> bool;
pub fn crocksdb_ratelimiter_create( pub fn crocksdb_ratelimiter_create(
rate_bytes_per_sec: i64, rate_bytes_per_sec: i64,
...@@ -863,8 +937,8 @@ extern "C" { ...@@ -863,8 +937,8 @@ extern "C" {
); );
pub fn crocksdb_mergeoperator_create( pub fn crocksdb_mergeoperator_create(
state: *mut c_void, state: *mut c_void,
destroy: extern "C" fn(*mut c_void) -> (), destroy: unsafe extern "C" fn(*mut c_void) -> (),
full_merge: extern "C" fn( full_merge: unsafe extern "C" fn(
arg: *mut c_void, arg: *mut c_void,
key: *const c_char, key: *const c_char,
key_len: size_t, key_len: size_t,
...@@ -876,7 +950,7 @@ extern "C" { ...@@ -876,7 +950,7 @@ extern "C" {
success: *mut u8, success: *mut u8,
new_value_length: *mut size_t, new_value_length: *mut size_t,
) -> *const c_char, ) -> *const c_char,
partial_merge: extern "C" fn( partial_merge: unsafe extern "C" fn(
arg: *mut c_void, arg: *mut c_void,
key: *const c_char, key: *const c_char,
key_len: size_t, key_len: size_t,
...@@ -887,9 +961,9 @@ extern "C" { ...@@ -887,9 +961,9 @@ extern "C" {
new_value_length: *mut size_t, new_value_length: *mut size_t,
) -> *const c_char, ) -> *const c_char,
delete_value: Option< delete_value: Option<
extern "C" fn(*mut c_void, value: *const c_char, value_len: *mut size_t) -> (), unsafe extern "C" fn(*mut c_void, value: *const c_char, value_len: *mut size_t) -> (),
>, >,
name_fn: extern "C" fn(*mut c_void) -> *const c_char, name_fn: unsafe extern "C" fn(*mut c_void) -> *const c_char,
) -> *mut DBMergeOperator; ) -> *mut DBMergeOperator;
pub fn crocksdb_mergeoperator_destroy(mo: *mut DBMergeOperator); pub fn crocksdb_mergeoperator_destroy(mo: *mut DBMergeOperator);
pub fn crocksdb_options_set_merge_operator(options: *mut Options, mo: *mut DBMergeOperator); pub fn crocksdb_options_set_merge_operator(options: *mut Options, mo: *mut DBMergeOperator);
...@@ -1005,15 +1079,15 @@ extern "C" { ...@@ -1005,15 +1079,15 @@ extern "C" {
pub fn crocksdb_options_set_comparator(options: *mut Options, cb: *mut DBComparator); pub fn crocksdb_options_set_comparator(options: *mut Options, cb: *mut DBComparator);
pub fn crocksdb_comparator_create( pub fn crocksdb_comparator_create(
state: *mut c_void, state: *mut c_void,
destroy: extern "C" fn(*mut c_void) -> (), destroy: unsafe extern "C" fn(*mut c_void) -> (),
compare: extern "C" fn( compare: unsafe extern "C" fn(
arg: *mut c_void, arg: *mut c_void,
a: *const c_char, a: *const c_char,
alen: size_t, alen: size_t,
b: *const c_char, b: *const c_char,
blen: size_t, blen: size_t,
) -> c_int, ) -> c_int,
name_fn: extern "C" fn(*mut c_void) -> *const c_char, name_fn: unsafe extern "C" fn(*mut c_void) -> *const c_char,
) -> *mut DBComparator; ) -> *mut DBComparator;
pub fn crocksdb_comparator_destroy(cmp: *mut DBComparator); pub fn crocksdb_comparator_destroy(cmp: *mut DBComparator);
......
...@@ -15,7 +15,6 @@ ...@@ -15,7 +15,6 @@
use libc::{c_char, c_int, c_void, size_t}; use libc::{c_char, c_int, c_void, size_t};
use std::ffi::CString; use std::ffi::CString;
use std::mem;
use std::slice; use std::slice;
pub struct ComparatorCallback { pub struct ComparatorCallback {
...@@ -23,30 +22,26 @@ pub struct ComparatorCallback { ...@@ -23,30 +22,26 @@ pub struct ComparatorCallback {
pub f: fn(&[u8], &[u8]) -> i32, pub f: fn(&[u8], &[u8]) -> i32,
} }
pub extern "C" fn destructor_callback(raw_cb: *mut c_void) { pub unsafe extern "C" fn destructor_callback(raw_cb: *mut c_void) {
// turn this back into a local variable so rust will reclaim it // turn this back into a local variable so rust will reclaim it
let _: Box<ComparatorCallback> = unsafe { mem::transmute(raw_cb) }; let _ = Box::from_raw(raw_cb as *mut ComparatorCallback);
} }
pub extern "C" fn name_callback(raw_cb: *mut c_void) -> *const c_char { pub unsafe extern "C" fn name_callback(raw_cb: *mut c_void) -> *const c_char {
unsafe {
let cb: &mut ComparatorCallback = &mut *(raw_cb as *mut ComparatorCallback); let cb: &mut ComparatorCallback = &mut *(raw_cb as *mut ComparatorCallback);
let ptr = cb.name.as_ptr(); let ptr = cb.name.as_ptr();
ptr as *const c_char ptr as *const c_char
}
} }
pub extern "C" fn compare_callback( pub unsafe extern "C" fn compare_callback(
raw_cb: *mut c_void, raw_cb: *mut c_void,
a_raw: *const c_char, a_raw: *const c_char,
a_len: size_t, a_len: size_t,
b_raw: *const c_char, b_raw: *const c_char,
b_len: size_t, b_len: size_t,
) -> c_int { ) -> c_int {
unsafe {
let cb: &mut ComparatorCallback = &mut *(raw_cb as *mut ComparatorCallback); let cb: &mut ComparatorCallback = &mut *(raw_cb as *mut ComparatorCallback);
let a: &[u8] = slice::from_raw_parts(a_raw as *const u8, a_len as usize); let a: &[u8] = slice::from_raw_parts(a_raw as *const u8, a_len as usize);
let b: &[u8] = slice::from_raw_parts(b_raw as *const u8, b_len as usize); let b: &[u8] = slice::from_raw_parts(b_raw as *const u8, b_len as usize);
(cb.f)(a, b) (cb.f)(a, b)
}
} }
...@@ -18,7 +18,7 @@ use crocksdb_ffi::{ ...@@ -18,7 +18,7 @@ use crocksdb_ffi::{
}; };
use libc::c_void; use libc::c_void;
use std::path::Path; use std::path::Path;
use std::{mem, slice, str}; use std::{slice, str};
use {TableProperties, TablePropertiesCollectionView}; use {TableProperties, TablePropertiesCollectionView};
macro_rules! fetch_str { macro_rules! fetch_str {
...@@ -30,6 +30,7 @@ macro_rules! fetch_str { ...@@ -30,6 +30,7 @@ macro_rules! fetch_str {
}) })
} }
#[repr(transparent)]
pub struct FlushJobInfo(DBFlushJobInfo); pub struct FlushJobInfo(DBFlushJobInfo);
impl FlushJobInfo { impl FlushJobInfo {
...@@ -58,6 +59,7 @@ impl FlushJobInfo { ...@@ -58,6 +59,7 @@ impl FlushJobInfo {
} }
} }
#[repr(transparent)]
pub struct CompactionJobInfo(DBCompactionJobInfo); pub struct CompactionJobInfo(DBCompactionJobInfo);
impl CompactionJobInfo { impl CompactionJobInfo {
...@@ -128,6 +130,7 @@ impl CompactionJobInfo { ...@@ -128,6 +130,7 @@ impl CompactionJobInfo {
} }
} }
#[repr(transparent)]
pub struct IngestionInfo(DBIngestionInfo); pub struct IngestionInfo(DBIngestionInfo);
impl IngestionInfo { impl IngestionInfo {
...@@ -152,6 +155,7 @@ impl IngestionInfo { ...@@ -152,6 +155,7 @@ impl IngestionInfo {
} }
} }
#[repr(transparent)]
pub struct WriteStallInfo(DBWriteStallInfo); pub struct WriteStallInfo(DBWriteStallInfo);
impl WriteStallInfo { impl WriteStallInfo {
...@@ -199,7 +203,7 @@ extern "C" fn on_flush_completed( ...@@ -199,7 +203,7 @@ extern "C" fn on_flush_completed(
let (ctx, info) = unsafe { let (ctx, info) = unsafe {
( (
&*(ctx as *mut Box<dyn EventListener>), &*(ctx as *mut Box<dyn EventListener>),
mem::transmute(&*info), &*(info as *const FlushJobInfo),
) )
}; };
ctx.on_flush_completed(info); ctx.on_flush_completed(info);
...@@ -213,7 +217,7 @@ extern "C" fn on_compaction_completed( ...@@ -213,7 +217,7 @@ extern "C" fn on_compaction_completed(
let (ctx, info) = unsafe { let (ctx, info) = unsafe {
( (
&*(ctx as *mut Box<dyn EventListener>), &*(ctx as *mut Box<dyn EventListener>),
mem::transmute(&*info), &*(info as *const CompactionJobInfo),
) )
}; };
ctx.on_compaction_completed(info); ctx.on_compaction_completed(info);
...@@ -227,7 +231,7 @@ extern "C" fn on_external_file_ingested( ...@@ -227,7 +231,7 @@ extern "C" fn on_external_file_ingested(
let (ctx, info) = unsafe { let (ctx, info) = unsafe {
( (
&*(ctx as *mut Box<dyn EventListener>), &*(ctx as *mut Box<dyn EventListener>),
mem::transmute(&*info), &*(info as *const IngestionInfo),
) )
}; };
ctx.on_external_file_ingested(info); ctx.on_external_file_ingested(info);
...@@ -254,7 +258,7 @@ extern "C" fn on_stall_conditions_changed(ctx: *mut c_void, info: *const DBWrite ...@@ -254,7 +258,7 @@ extern "C" fn on_stall_conditions_changed(ctx: *mut c_void, info: *const DBWrite
let (ctx, info) = unsafe { let (ctx, info) = unsafe {
( (
&*(ctx as *mut Box<dyn EventListener>), &*(ctx as *mut Box<dyn EventListener>),
mem::transmute(&*info), &*(info as *const WriteStallInfo),
) )
}; };
ctx.on_stall_conditions_changed(info); ctx.on_stall_conditions_changed(info);
......
...@@ -68,4 +68,3 @@ mod table_properties; ...@@ -68,4 +68,3 @@ mod table_properties;
mod table_properties_collector; mod table_properties_collector;
mod table_properties_collector_factory; mod table_properties_collector_factory;
mod titan; mod titan;
mod util;
...@@ -26,20 +26,18 @@ pub struct MergeOperatorCallback { ...@@ -26,20 +26,18 @@ pub struct MergeOperatorCallback {
pub merge_fn: MergeFn, pub merge_fn: MergeFn,
} }
pub extern "C" fn destructor_callback(raw_cb: *mut c_void) { pub unsafe extern "C" fn destructor_callback(raw_cb: *mut c_void) {
// turn this back into a local variable so rust will reclaim it // turn this back into a local variable so rust will reclaim it
let _: Box<MergeOperatorCallback> = unsafe { mem::transmute(raw_cb) }; let _ = Box::from_raw(raw_cb as *mut MergeOperatorCallback);
} }
pub extern "C" fn name_callback(raw_cb: *mut c_void) -> *const c_char { pub unsafe extern "C" fn name_callback(raw_cb: *mut c_void) -> *const c_char {
unsafe {
let cb: &mut MergeOperatorCallback = &mut *(raw_cb as *mut MergeOperatorCallback); let cb: &mut MergeOperatorCallback = &mut *(raw_cb as *mut MergeOperatorCallback);
let ptr = cb.name.as_ptr(); let ptr = cb.name.as_ptr();
ptr as *const c_char ptr as *const c_char
}
} }
pub extern "C" fn full_merge_callback( pub unsafe extern "C" fn full_merge_callback(
raw_cb: *mut c_void, raw_cb: *mut c_void,
raw_key: *const c_char, raw_key: *const c_char,
key_len: size_t, key_len: size_t,
...@@ -51,7 +49,6 @@ pub extern "C" fn full_merge_callback( ...@@ -51,7 +49,6 @@ pub extern "C" fn full_merge_callback(
success: *mut u8, success: *mut u8,
new_value_length: *mut size_t, new_value_length: *mut size_t,
) -> *const c_char { ) -> *const c_char {
unsafe {
let cb: &mut MergeOperatorCallback = &mut *(raw_cb as *mut MergeOperatorCallback); let cb: &mut MergeOperatorCallback = &mut *(raw_cb as *mut MergeOperatorCallback);
let operands = &mut MergeOperands::new(operands_list, operands_list_len, num_operands); let operands = &mut MergeOperands::new(operands_list, operands_list_len, num_operands);
let key: &[u8] = slice::from_raw_parts(raw_key as *const u8, key_len as usize); let key: &[u8] = slice::from_raw_parts(raw_key as *const u8, key_len as usize);
...@@ -61,15 +58,15 @@ pub extern "C" fn full_merge_callback( ...@@ -61,15 +58,15 @@ pub extern "C" fn full_merge_callback(
result.shrink_to_fit(); result.shrink_to_fit();
// TODO(tan) investigate zero-copy techniques to improve performance // TODO(tan) investigate zero-copy techniques to improve performance
let buf = libc::malloc(result.len() as size_t); let buf = libc::malloc(result.len() as size_t);
let buf = buf as *mut u8;
assert!(!buf.is_null()); assert!(!buf.is_null());
*new_value_length = result.len() as size_t; *new_value_length = result.len() as size_t;
*success = 1 as u8; *success = 1 as u8;
ptr::copy(result.as_ptr() as *mut c_void, &mut *buf, result.len()); ptr::copy(result.as_ptr(), &mut *buf, result.len());
buf as *const c_char buf as *const c_char
}
} }
pub extern "C" fn partial_merge_callback( pub unsafe extern "C" fn partial_merge_callback(
raw_cb: *mut c_void, raw_cb: *mut c_void,
raw_key: *const c_char, raw_key: *const c_char,
key_len: size_t, key_len: size_t,
...@@ -79,7 +76,6 @@ pub extern "C" fn partial_merge_callback( ...@@ -79,7 +76,6 @@ pub extern "C" fn partial_merge_callback(
success: *mut u8, success: *mut u8,
new_value_length: *mut size_t, new_value_length: *mut size_t,
) -> *const c_char { ) -> *const c_char {
unsafe {
let cb: &mut MergeOperatorCallback = &mut *(raw_cb as *mut MergeOperatorCallback); let cb: &mut MergeOperatorCallback = &mut *(raw_cb as *mut MergeOperatorCallback);
let operands = &mut MergeOperands::new(operands_list, operands_list_len, num_operands); let operands = &mut MergeOperands::new(operands_list, operands_list_len, num_operands);
let key: &[u8] = slice::from_raw_parts(raw_key as *const u8, key_len as usize); let key: &[u8] = slice::from_raw_parts(raw_key as *const u8, key_len as usize);
...@@ -87,12 +83,12 @@ pub extern "C" fn partial_merge_callback( ...@@ -87,12 +83,12 @@ pub extern "C" fn partial_merge_callback(
result.shrink_to_fit(); result.shrink_to_fit();
// TODO(tan) investigate zero-copy techniques to improve performance // TODO(tan) investigate zero-copy techniques to improve performance
let buf = libc::malloc(result.len() as size_t); let buf = libc::malloc(result.len() as size_t);
let buf = buf as *mut u8;
assert!(!buf.is_null()); assert!(!buf.is_null());
*new_value_length = result.len() as size_t; *new_value_length = result.len() as size_t;
*success = 1 as u8; *success = 1 as u8;
ptr::copy(result.as_ptr() as *mut c_void, &mut *buf, result.len()); ptr::copy(result.as_ptr(), &mut *buf, result.len());
buf as *const c_char buf as *const c_char
}
} }
pub struct MergeOperands { pub struct MergeOperands {
...@@ -133,10 +129,10 @@ impl<'a> Iterator for &'a mut MergeOperands { ...@@ -133,10 +129,10 @@ impl<'a> Iterator for &'a mut MergeOperands {
let len = *len_ptr as usize; let len = *len_ptr as usize;
let ptr = base + (spacing * self.cursor); let ptr = base + (spacing * self.cursor);
self.cursor += 1; self.cursor += 1;
Some(mem::transmute(slice::from_raw_parts( Some(slice::from_raw_parts(
*(ptr as *const *const u8) as *const u8, *(ptr as *const *const u8) as *const u8,
len, len,
))) ))
} }
} }
} }
......
...@@ -39,7 +39,6 @@ use std::str::from_utf8; ...@@ -39,7 +39,6 @@ use std::str::from_utf8;
use std::sync::Arc; use std::sync::Arc;
use std::{fs, ptr, slice}; use std::{fs, ptr, slice};
use table_properties::{TableProperties, TablePropertiesCollection}; use table_properties::{TableProperties, TablePropertiesCollection};
use util::is_power_of_two;
pub struct CFHandle { pub struct CFHandle {
inner: *mut DBCFHandle, inner: *mut DBCFHandle,
...@@ -83,7 +82,7 @@ fn split_descriptors<'a>( ...@@ -83,7 +82,7 @@ fn split_descriptors<'a>(
fn build_cstring_list(str_list: &[&str]) -> Vec<CString> { fn build_cstring_list(str_list: &[&str]) -> Vec<CString> {
str_list str_list
.into_iter() .iter()
.map(|s| CString::new(s.as_bytes()).unwrap()) .map(|s| CString::new(s.as_bytes()).unwrap())
.collect() .collect()
} }
...@@ -262,6 +261,7 @@ impl<D> DBIterator<D> { ...@@ -262,6 +261,7 @@ impl<D> DBIterator<D> {
self.valid() self.valid()
} }
#[allow(clippy::should_implement_trait)]
pub fn next(&mut self) -> bool { pub fn next(&mut self) -> bool {
unsafe { unsafe {
crocksdb_ffi::crocksdb_iter_next(self.inner); crocksdb_ffi::crocksdb_iter_next(self.inner);
...@@ -557,11 +557,7 @@ impl DB { ...@@ -557,11 +557,7 @@ impl DB {
.map(|x| x.titan_inner as *const crocksdb_ffi::DBTitanDBOptions) .map(|x| x.titan_inner as *const crocksdb_ffi::DBTitanDBOptions)
.collect(); .collect();
let readonly = if error_if_log_file_exist.is_some() { let readonly = error_if_log_file_exist.is_some();
true
} else {
false
};
let with_ttl = if ttls_vec.len() > 0 { let with_ttl = if ttls_vec.len() > 0 {
if ttls_vec.len() == cf_names.len() { if ttls_vec.len() == cf_names.len() {
...@@ -2121,11 +2117,9 @@ impl SstFileReader { ...@@ -2121,11 +2117,9 @@ impl SstFileReader {
let path = let path =
CString::new(name.to_owned()).map_err(|e| format!("invalid path {}: {:?}", name, e))?; CString::new(name.to_owned()).map_err(|e| format!("invalid path {}: {:?}", name, e))?;
unsafe { unsafe {
Ok(ffi_try!(crocksdb_sstfilereader_open( ffi_try!(crocksdb_sstfilereader_open(self.inner, path.as_ptr()));
self.inner,
path.as_ptr()
)))
} }
Ok(())
} }
pub fn iter(&self) -> DBIterator<&Self> { pub fn iter(&self) -> DBIterator<&Self> {
...@@ -2188,7 +2182,8 @@ impl SstFileReader { ...@@ -2188,7 +2182,8 @@ impl SstFileReader {
} }
pub fn verify_checksum(&self) -> Result<(), String> { pub fn verify_checksum(&self) -> Result<(), String> {
unsafe { Ok(ffi_try!(crocksdb_sstfilereader_verify_checksum(self.inner))) } unsafe { ffi_try!(crocksdb_sstfilereader_verify_checksum(self.inner)) };
Ok(())
} }
} }
...@@ -2240,11 +2235,9 @@ impl SstFileWriter { ...@@ -2240,11 +2235,9 @@ impl SstFileWriter {
Ok(p) => p, Ok(p) => p,
}; };
unsafe { unsafe {
Ok(ffi_try!(crocksdb_sstfilewriter_open( ffi_try!(crocksdb_sstfilewriter_open(self.inner, path.as_ptr()));
self.inner,
path.as_ptr()
)))
} }
Ok(())
} }
/// Add key, value to currently opened file /// Add key, value to currently opened file
...@@ -2425,7 +2418,7 @@ impl Env { ...@@ -2425,7 +2418,7 @@ impl Env {
// The recommanded block size are 1024, 512 and 256. // The recommanded block size are 1024, 512 and 256.
pub fn new_ctr_encrypted_env(base_env: Arc<Env>, ciphertext: &[u8]) -> Result<Env, String> { pub fn new_ctr_encrypted_env(base_env: Arc<Env>, ciphertext: &[u8]) -> Result<Env, String> {
let len = ciphertext.len(); let len = ciphertext.len();
if len > 2048 || !is_power_of_two(len) { if len > 2048 || !len.is_power_of_two() {
return Err( return Err(
"ciphertext length must be less or equal to 2048, and must be power of 2" "ciphertext length must be less or equal to 2048, and must be power of 2"
.to_owned(), .to_owned(),
...@@ -2434,7 +2427,7 @@ impl Env { ...@@ -2434,7 +2427,7 @@ impl Env {
let env = unsafe { let env = unsafe {
crocksdb_ffi::crocksdb_ctr_encrypted_env_create( crocksdb_ffi::crocksdb_ctr_encrypted_env_create(
base_env.inner, base_env.inner,
mem::transmute(&ciphertext[0]), ciphertext.as_ptr() as *const c_char,
len, len,
) )
}; };
...@@ -2542,10 +2535,13 @@ pub struct Cache { ...@@ -2542,10 +2535,13 @@ pub struct Cache {
impl Cache { impl Cache {
pub fn new_lru_cache(opt: LRUCacheOptions) -> Cache { pub fn new_lru_cache(opt: LRUCacheOptions) -> Cache {
// This is ok because LRUCacheOptions always contains a valid pointer
unsafe {
Cache { Cache {
inner: crocksdb_ffi::new_lru_cache(opt.inner), inner: crocksdb_ffi::new_lru_cache(opt.inner),
} }
} }
}
} }
impl Drop for Cache { impl Drop for Cache {
...@@ -2608,14 +2604,14 @@ pub fn load_latest_options( ...@@ -2608,14 +2604,14 @@ pub fn load_latest_options(
let dbpath = CString::new(dbpath.as_bytes()).map_err(|_| ERR_CONVERT_PATH.to_owned())?; let dbpath = CString::new(dbpath.as_bytes()).map_err(|_| ERR_CONVERT_PATH.to_owned())?;
let db_options = DBOptions::new(); let db_options = DBOptions::new();
unsafe { unsafe {
let mut raw_cf_descs: *mut *mut crocksdb_ffi::ColumnFamilyDescriptor = ptr::null_mut(); let raw_cf_descs: *mut *mut crocksdb_ffi::ColumnFamilyDescriptor = ptr::null_mut();
let mut cf_descs_len: size_t = 0; let mut cf_descs_len: size_t = 0;
let ok = ffi_try!(crocksdb_load_latest_options( let ok = ffi_try!(crocksdb_load_latest_options(
dbpath.as_ptr(), dbpath.as_ptr(),
env.inner, env.inner,
db_options.inner, db_options.inner,
&mut raw_cf_descs, &raw_cf_descs,
&mut cf_descs_len, &mut cf_descs_len,
ignore_unknown_options ignore_unknown_options
)); ));
...@@ -2624,7 +2620,7 @@ pub fn load_latest_options( ...@@ -2624,7 +2620,7 @@ pub fn load_latest_options(
} }
let cf_descs_list = slice::from_raw_parts(raw_cf_descs, cf_descs_len); let cf_descs_list = slice::from_raw_parts(raw_cf_descs, cf_descs_len);
let cf_descs = cf_descs_list let cf_descs = cf_descs_list
.into_iter() .iter()
.map(|raw_cf_desc| CColumnFamilyDescriptor::from_raw(*raw_cf_desc)) .map(|raw_cf_desc| CColumnFamilyDescriptor::from_raw(*raw_cf_desc))
.collect(); .collect();
...@@ -2634,7 +2630,7 @@ pub fn load_latest_options( ...@@ -2634,7 +2630,7 @@ pub fn load_latest_options(
} }
} }
pub fn run_ldb_tool(ldb_args: &Vec<String>, opts: &DBOptions) { pub fn run_ldb_tool(ldb_args: &[String], opts: &DBOptions) {
unsafe { unsafe {
let ldb_args_cstrs: Vec<_> = ldb_args let ldb_args_cstrs: Vec<_> = ldb_args
.iter() .iter()
......
...@@ -31,7 +31,6 @@ use rocksdb::Env; ...@@ -31,7 +31,6 @@ use rocksdb::Env;
use rocksdb::{Cache, MemoryAllocator}; use rocksdb::{Cache, MemoryAllocator};
use slice_transform::{new_slice_transform, SliceTransform}; use slice_transform::{new_slice_transform, SliceTransform};
use std::ffi::{CStr, CString}; use std::ffi::{CStr, CString};
use std::mem;
use std::path::Path; use std::path::Path;
use std::ptr; use std::ptr;
use std::sync::Arc; use std::sync::Arc;
...@@ -454,9 +453,10 @@ impl ReadOptions { ...@@ -454,9 +453,10 @@ impl ReadOptions {
pub fn set_table_filter(&mut self, filter: Box<dyn TableFilter>) { pub fn set_table_filter(&mut self, filter: Box<dyn TableFilter>) {
unsafe { unsafe {
let f = Box::into_raw(Box::new(filter)); let f = Box::into_raw(Box::new(filter));
let f = f as *mut c_void;
crocksdb_ffi::crocksdb_readoptions_set_table_filter( crocksdb_ffi::crocksdb_readoptions_set_table_filter(
self.inner, self.inner,
mem::transmute(f), f,
table_filter, table_filter,
destroy_table_filter, destroy_table_filter,
); );
...@@ -1276,10 +1276,11 @@ impl ColumnFamilyOptions { ...@@ -1276,10 +1276,11 @@ impl ColumnFamilyOptions {
name: CString::new(name.as_bytes()).unwrap(), name: CString::new(name.as_bytes()).unwrap(),
merge_fn: merge_fn, merge_fn: merge_fn,
}); });
let cb = Box::into_raw(cb) as *mut c_void;
unsafe { unsafe {
let mo = crocksdb_ffi::crocksdb_mergeoperator_create( let mo = crocksdb_ffi::crocksdb_mergeoperator_create(
mem::transmute(cb), cb,
merge_operator::destructor_callback, merge_operator::destructor_callback,
full_merge_callback, full_merge_callback,
partial_merge_callback, partial_merge_callback,
...@@ -1295,10 +1296,11 @@ impl ColumnFamilyOptions { ...@@ -1295,10 +1296,11 @@ impl ColumnFamilyOptions {
name: CString::new(name.as_bytes()).unwrap(), name: CString::new(name.as_bytes()).unwrap(),
f: compare_fn, f: compare_fn,
}); });
let cb = Box::into_raw(cb) as *mut c_void;
unsafe { unsafe {
let cmp = crocksdb_ffi::crocksdb_comparator_create( let cmp = crocksdb_ffi::crocksdb_comparator_create(
mem::transmute(cb), cb,
comparator::destructor_callback, comparator::destructor_callback,
compare_callback, compare_callback,
comparator::name_callback, comparator::name_callback,
...@@ -1340,7 +1342,8 @@ impl ColumnFamilyOptions { ...@@ -1340,7 +1342,8 @@ impl ColumnFamilyOptions {
pub fn set_max_bytes_for_level_multiplier(&mut self, mul: i32) { pub fn set_max_bytes_for_level_multiplier(&mut self, mul: i32) {
unsafe { unsafe {
crocksdb_ffi::crocksdb_options_set_max_bytes_for_level_multiplier( crocksdb_ffi::crocksdb_options_set_max_bytes_for_level_multiplier(
self.inner, mul as f64, self.inner,
f64::from(mul),
); );
} }
} }
...@@ -1674,7 +1677,7 @@ impl CColumnFamilyDescriptor { ...@@ -1674,7 +1677,7 @@ impl CColumnFamilyDescriptor {
CColumnFamilyDescriptor { inner } CColumnFamilyDescriptor { inner }
} }
pub fn name<'a>(&'a self) -> &'a str { pub fn name(&self) -> &str {
unsafe { unsafe {
let raw_cf_name = crocksdb_ffi::crocksdb_name_from_column_family_descriptor(self.inner); let raw_cf_name = crocksdb_ffi::crocksdb_name_from_column_family_descriptor(self.inner);
CStr::from_ptr(raw_cf_name).to_str().unwrap() CStr::from_ptr(raw_cf_name).to_str().unwrap()
......
...@@ -13,7 +13,6 @@ ...@@ -13,7 +13,6 @@
use crocksdb_ffi::DBTableProperties; use crocksdb_ffi::DBTableProperties;
use libc::{c_int, c_void}; use libc::{c_int, c_void};
use std::mem;
use table_properties::TableProperties; use table_properties::TableProperties;
pub trait TableFilter { pub trait TableFilter {
...@@ -28,7 +27,8 @@ pub trait TableFilter { ...@@ -28,7 +27,8 @@ pub trait TableFilter {
pub extern "C" fn table_filter(ctx: *mut c_void, props: *const DBTableProperties) -> c_int { pub extern "C" fn table_filter(ctx: *mut c_void, props: *const DBTableProperties) -> c_int {
unsafe { unsafe {
let filter = &*(ctx as *mut Box<dyn TableFilter>); let filter = &*(ctx as *mut Box<dyn TableFilter>);
filter.table_filter(mem::transmute(&*props)) as c_int let props = &*(props as *const TableProperties);
filter.table_filter(props) as c_int
} }
} }
......
...@@ -18,16 +18,16 @@ use crocksdb_ffi::{ ...@@ -18,16 +18,16 @@ use crocksdb_ffi::{
use libc::size_t; use libc::size_t;
use std::marker::PhantomData; use std::marker::PhantomData;
use std::ops::{Deref, Index}; use std::ops::{Deref, Index};
use std::{mem, slice, str}; use std::{slice, str};
#[repr(transparent)]
pub struct TablePropertiesCollectionView(DBTablePropertiesCollection); pub struct TablePropertiesCollectionView(DBTablePropertiesCollection);
impl TablePropertiesCollectionView { impl TablePropertiesCollectionView {
pub unsafe fn from_ptr<'a>( pub unsafe fn from_ptr<'a>(
collection: *const DBTablePropertiesCollection, collection: *const DBTablePropertiesCollection,
) -> &'a TablePropertiesCollectionView { ) -> &'a TablePropertiesCollectionView {
let c = &*collection; &*(collection as *const TablePropertiesCollectionView)
mem::transmute(c)
} }
pub fn iter(&self) -> TablePropertiesCollectionIter { pub fn iter(&self) -> TablePropertiesCollectionIter {
...@@ -130,14 +130,14 @@ impl Deref for TablePropertiesCollection { ...@@ -130,14 +130,14 @@ impl Deref for TablePropertiesCollection {
} }
} }
#[repr(transparent)]
pub struct TableProperties { pub struct TableProperties {
inner: DBTableProperties, inner: DBTableProperties,
} }
impl TableProperties { impl TableProperties {
pub unsafe fn from_ptr<'a>(ptr: *const DBTableProperties) -> &'a TableProperties { pub unsafe fn from_ptr<'a>(ptr: *const DBTableProperties) -> &'a TableProperties {
let res = &*ptr; &*(ptr as *const TableProperties)
mem::transmute(res)
} }
fn get_u64(&self, prop: DBTableProperty) -> u64 { fn get_u64(&self, prop: DBTableProperty) -> u64 {
...@@ -229,14 +229,14 @@ impl TableProperties { ...@@ -229,14 +229,14 @@ impl TableProperties {
} }
} }
#[repr(transparent)]
pub struct UserCollectedProperties { pub struct UserCollectedProperties {
inner: DBUserCollectedProperties, inner: DBUserCollectedProperties,
} }
impl UserCollectedProperties { impl UserCollectedProperties {
unsafe fn from_ptr<'a>(ptr: *const DBUserCollectedProperties) -> &'a UserCollectedProperties { unsafe fn from_ptr<'a>(ptr: *const DBUserCollectedProperties) -> &'a UserCollectedProperties {
let prop = &*ptr; &*(ptr as *const UserCollectedProperties)
mem::transmute(prop)
} }
pub fn get<Q: AsRef<[u8]>>(&self, index: Q) -> Option<&[u8]> { pub fn get<Q: AsRef<[u8]>>(&self, index: Q) -> Option<&[u8]> {
......
// Copyright 2018 PingCAP, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// See the License for the specific language governing permissions and
// limitations under the License.
pub fn is_power_of_two(mut n: usize) -> bool {
if n == 0 {
return false;
}
while n % 2 == 0 {
n = n / 2;
}
n == 1
}
#[cfg(test)]
mod test {
use super::*;
#[test]
fn test_is_power_of_two() {
assert_eq!(is_power_of_two(0), false);
assert_eq!(is_power_of_two(1), true);
assert_eq!(is_power_of_two(2), true);
assert_eq!(is_power_of_two(4), true);
assert_eq!(is_power_of_two(8), true);
assert_eq!(is_power_of_two(5), false);
}
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment