Package com.linkedin.venice.compression

Class ZstdWithDictCompressor

java.lang.Object

com.linkedin.venice.compression.VeniceCompressor

com.linkedin.venice.compression.ZstdWithDictCompressor

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

public class ZstdWithDictCompressor
extends VeniceCompressor

Field Summary

Fields inherited from class com.linkedin.venice.compression.VeniceCompressor

SCHEMA_HEADER_LENGTH

Constructor Summary

Constructors

Constructor	Description
ZstdWithDictCompressor(byte[] dictionary, int level)

Method Summary

Modifier and Type	Method	Description
static byte[]	buildDictionaryOnSyntheticAvroData()	Build a dictionary based on synthetic data.
void	close()
byte[]	compress(byte[] data)
java.nio.ByteBuffer	compress(java.nio.ByteBuffer data, int startPositionOfOutput)
java.nio.ByteBuffer	decompress(byte[] data, int offset, int length)
java.io.InputStream	decompress(java.io.InputStream inputStream)
java.nio.ByteBuffer	decompress(java.nio.ByteBuffer data)
java.nio.ByteBuffer	decompressAndPrependSchemaHeader(byte[] data, int offset, int length, int schemaHeader)	This method tries to decompress data and maybe prepend the schema header.
boolean	equals(java.lang.Object o)
int	hashCode()

Methods inherited from class com.linkedin.venice.compression.VeniceCompressor

getCompressionStrategy

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ZstdWithDictCompressor

public ZstdWithDictCompressor(byte[] dictionary,
                              int level)

Method Detail

compress

public byte[] compress(byte[] data)

Specified by:

compress in class VeniceCompressor

compress

public java.nio.ByteBuffer compress(java.nio.ByteBuffer data,
                                    int startPositionOfOutput)
                             throws java.io.IOException

Specified by:

compress in class VeniceCompressor

Throws:

java.io.IOException

decompress

public java.nio.ByteBuffer decompress(java.nio.ByteBuffer data)
                               throws java.io.IOException

Specified by:

decompress in class VeniceCompressor

Throws:

java.io.IOException

decompress

public java.nio.ByteBuffer decompress(byte[] data,
                                      int offset,
                                      int length)
                               throws java.io.IOException

Specified by:

decompress in class VeniceCompressor

Throws:

java.io.IOException

decompressAndPrependSchemaHeader

public java.nio.ByteBuffer decompressAndPrependSchemaHeader(byte[] data,
                                                            int offset,
                                                            int length,
                                                            int schemaHeader)
                                                     throws java.io.IOException

Description copied from class: VeniceCompressor

This method tries to decompress data and maybe prepend the schema header. The returned ByteBuffer will be backed by byte array that starts with schema header, followed by the decompressed data. The ByteBuffer will be positioned at the beginning of the decompressed data and the remaining of the ByteBuffer will be the length of the decompressed data.

Specified by:

decompressAndPrependSchemaHeader in class VeniceCompressor

Throws:

java.io.IOException

decompress

public java.io.InputStream decompress(java.io.InputStream inputStream)
                               throws java.io.IOException

Specified by:

decompress in class VeniceCompressor

Throws:

java.io.IOException

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable

Overrides:

close in class VeniceCompressor

Throws:

java.io.IOException

buildDictionaryOnSyntheticAvroData

public static byte[] buildDictionaryOnSyntheticAvroData()

Build a dictionary based on synthetic data. Used for empty push where there is no available dictionary to retrieve from a push job.

Returns:

a zstd compression dictionary trained on small amount of avro data

hashCode

public int hashCode()

Overrides:

hashCode in class java.lang.Object

equals

public boolean equals(java.lang.Object o)

Overrides:

equals in class java.lang.Object