Package com.linkedin.davinci.store.rocksdb

Class RocksDBStoragePartition

  • Direct Known Subclasses:
    ReplicationMetadataRocksDBStoragePartition
    @NotThreadSafe
public class RocksDBStoragePartition
extends AbstractStoragePartition
    In RocksDBStoragePartition, it assumes the update(insert/delete) will happen sequentially. If the batch push is bytewise-sorted by key, this class is leveraging SstFileWriter to generate the SST file directly and ingest all the generated SST files into the RocksDB database at the end of the push. If the ingestion is unsorted, this class is using the regular RocksDB interface to support update operations.

    • Field Detail

      • READ_OPTIONS_DEFAULT

        protected static final org.rocksdb.ReadOptions READ_OPTIONS_DEFAULT

      • writeOptions

        protected final org.rocksdb.WriteOptions writeOptions
        Here RocksDB disables WAL, but relies on the 'flush', which will be invoked through sync() to avoid data loss during recovery.

      • replicaId

        protected final java.lang.String replicaId

      • storeName

        protected final java.lang.String storeName

      • storeNameAndVersion

        protected final java.lang.String storeNameAndVersion

      • blobTransferEnabled

        protected final boolean blobTransferEnabled

      • partitionId

        protected final int partitionId

      • readCloseRWLock

        protected final java.util.concurrent.locks.ReentrantReadWriteLock readCloseRWLock
        Since all the modification functions are synchronized, we don't need any other synchronization for the update path to guard RocksDB closing behavior. The following readCloseRWLock is only used to guard get(byte[]) since we don't want to synchronize get requests.

      • rocksDB

        protected org.rocksdb.RocksDB rocksDB

      • deferredWrite

        protected final boolean deferredWrite
        Whether the input is sorted or not.
        deferredWrite = sortedInput => ingested via batch push which is sorted in VPJ, can use RocksDBSstFileWriter to ingest the input data to RocksDB
        !deferredWrite = !sortedInput => can not use RocksDBSstFileWriter for ingestion

      • readOnly

        protected final boolean readOnly
        Whether the database is read only or not.

      • writeOnly

        protected final boolean writeOnly

      • readWriteLeaderForDefaultCF

        protected final boolean readWriteLeaderForDefaultCF

      • readWriteLeaderForRMDCF

        protected final boolean readWriteLeaderForRMDCF

      • columnFamilyHandleList

        protected final java.util.List<org.rocksdb.ColumnFamilyHandle> columnFamilyHandleList
        Column Family is the concept in RocksDB to create isolation between different value for the same key. All KVs are stored in `DEFAULT` column family, if no column family is specified. If we stores replication metadata in the RocksDB, we stored it in a separated column family. We will insert all the column family descriptors into columnFamilyDescriptors and pass it to RocksDB when opening the store, and it will fill the columnFamilyHandles with handles which will be used when we want to put/get/delete from different RocksDB column families.

      • columnFamilyDescriptors

        protected final java.util.List<org.rocksdb.ColumnFamilyDescriptor> columnFamilyDescriptors