Package com.linkedin.davinci.storage.chunking

Interface ChunkingAdapter<CHUNKS_CONTAINER,VALUE>

All Known Implementing Classes:

AbstractAvroChunkingAdapter, BatchGetChunkingAdapter, GenericChunkingAdapter, GenericRecordChunkingAdapter, RawBytesChunkingAdapter, SingleGetChunkingAdapter, SpecificRecordChunkingAdapter

public interface ChunkingAdapter<CHUNKS_CONTAINER,VALUE>

This interface allows each code path which needs to interact with chunked values to optimize the way the re-assembly is handled so that the final form in which the is consumed is generated as efficiently as possible, via the use of a temporary ChunkingAdapter.

Method Summary

Modifier and Type	Method	Description
void	addChunkIntoContainer(CHUNKS_CONTAINER container, int chunkIndex, byte[] valueChunk)	Used to incrementally add a {@param valueChunk} into the {@param CHUNKS_CONTAINER} container.
CHUNKS_CONTAINER	constructChunksContainer(ChunkedValueManifest chunkedValueManifest)	Used to construct the right kind of {@param CHUNKS_CONTAINER} container (according to the query code) to hold a large value which needs to be incrementally re-assembled from many smaller chunks.
default VALUE	constructValue(byte[] valueOnlyBytes, int offset, int bytesLength, RecordDeserializer<VALUE> recordDeserializer, VeniceCompressor veniceCompressor)
default VALUE	constructValue(byte[] fullBytes, int bytesLength, VALUE reusedValue, org.apache.avro.io.BinaryDecoder reusedDecoder, ReadResponseStats responseStats, int writerSchemaId, int readerSchemaId, StoreDeserializerCache<VALUE> storeDeserializerCache, VeniceCompressor compressor)	Used to wrap a small {@param fullBytes} value fetched from the storage engine into the right type of {@param VALUE} class needed by the query code.
default VALUE	constructValue(int schemaId, byte[] fullBytes)	This function can be implemented by the adapters which need fewer parameters.
default VALUE	constructValue(int schemaId, CHUNKS_CONTAINER chunksContainer)	This function can be implemented by the adapters which need fewer parameters.
default VALUE	constructValue(CHUNKS_CONTAINER chunksContainer, VALUE reusedValue, org.apache.avro.io.BinaryDecoder reusedDecoder, ReadResponseStats responseStats, int writerSchemaId, int readerSchemaId, StoreDeserializerCache<VALUE> storeDeserializerCache, VeniceCompressor compressor)	Used to wrap a large value re-assembled with the use of a {@param CHUNKS_CONTAINER} into the right type of {@param VALUE} class needed by the query code.

Method Detail

constructValue

default VALUE constructValue(byte[] fullBytes,
                             int bytesLength,
                             VALUE reusedValue,
                             org.apache.avro.io.BinaryDecoder reusedDecoder,
                             ReadResponseStats responseStats,
                             int writerSchemaId,
                             int readerSchemaId,
                             StoreDeserializerCache<VALUE> storeDeserializerCache,
                             VeniceCompressor compressor)

Used to wrap a small {@param fullBytes} value fetched from the storage engine into the right type of {@param VALUE} class needed by the query code.

The following parameters are mandatory:

Parameters:

fullBytes - includes both the schema ID header and the payload

The following parameters can be ignored, by implementing constructValue(int, byte[]):

bytesLength -

reusedValue - a previous instance of to be re-used in order to minimize GC

reusedDecoder - a previous instance of BinaryDecoder to be re-used in order to minimize GC

responseStats - the ReadResponseStats which carries certain metrics to be recorded at the end

writerSchemaId - schema used to serialize the value

readerSchemaId -

constructValue

default VALUE constructValue(byte[] valueOnlyBytes,
                             int offset,
                             int bytesLength,
                             RecordDeserializer<VALUE> recordDeserializer,
                             VeniceCompressor veniceCompressor)

constructValue

default VALUE constructValue(int schemaId,
                             byte[] fullBytes)

This function can be implemented by the adapters which need fewer parameters.

addChunkIntoContainer

void addChunkIntoContainer(CHUNKS_CONTAINER container,
                           int chunkIndex,
                           byte[] valueChunk)

Used to incrementally add a {@param valueChunk} into the {@param CHUNKS_CONTAINER} container.

constructChunksContainer

CHUNKS_CONTAINER constructChunksContainer(ChunkedValueManifest chunkedValueManifest)

Used to construct the right kind of {@param CHUNKS_CONTAINER} container (according to the query code) to hold a large value which needs to be incrementally re-assembled from many smaller chunks.

constructValue

default VALUE constructValue(CHUNKS_CONTAINER chunksContainer,
                             VALUE reusedValue,
                             org.apache.avro.io.BinaryDecoder reusedDecoder,
                             ReadResponseStats responseStats,
                             int writerSchemaId,
                             int readerSchemaId,
                             StoreDeserializerCache<VALUE> storeDeserializerCache,
                             VeniceCompressor compressor)

Used to wrap a large value re-assembled with the use of a {@param CHUNKS_CONTAINER} into the right type of {@param VALUE} class needed by the query code.

The following parameters are mandatory:

Parameters:

chunksContainer - temporary , obtained from constructChunksContainer(ChunkedValueManifest)

The following parameters can be ignored, by implementing constructValue(int, Object):

reusedValue - a previous instance of to be re-used in order to minimize GC

reusedDecoder - a previous instance of BinaryDecoder to be re-used in order to minimize GC

responseStats - the ReadResponseStats which carries certain metrics to be recorded at the end

writerSchemaId - of the user's value

readerSchemaId -

constructValue

default VALUE constructValue(int schemaId,
                             CHUNKS_CONTAINER chunksContainer)

This function can be implemented by the adapters which need fewer parameters.