Package com.linkedin.davinci.client

Class BlockingDaVinciRecordTransformer<K,V,O>

java.lang.Object

com.linkedin.davinci.client.DaVinciRecordTransformer<K,V,O>

com.linkedin.davinci.client.BlockingDaVinciRecordTransformer<K,V,O>

Type Parameters:

K - type of the input key

V - type of the input value

O - type of the output value

@Experimental
public class BlockingDaVinciRecordTransformer<K,V,O>
extends DaVinciRecordTransformer<K,V,O>

This is an implementation of DaVinciRecordTransformer that implements blocking. It ensures that no puts can proceed until onStartIngestionTask finishes.

Constructor Summary

Constructors

Constructor	Description
BlockingDaVinciRecordTransformer(DaVinciRecordTransformer recordTransformer, boolean storeRecordsInDaVinci)

Method Summary

Modifier and Type	Method	Description
org.apache.avro.Schema	getKeySchema()	Returns the schema for the key used in DaVinciClient's operations.
org.apache.avro.Schema	getOutputValueSchema()	Returns the schema for the output value used in DaVinciClient's operations.
void	onEndVersionIngestion()	Lifecycle event triggered when record consumption is stopped for DaVinciRecordTransformer.storeVersion.
void	onStartVersionIngestion()	Lifecycle event triggered before consuming records for DaVinciRecordTransformer.storeVersion.
void	processDelete(Lazy<K> key)	Override this method to customize the behavior for record deletions.
void	processPut(Lazy<K> key, Lazy<O> value)	Implement this method to manage custom state outside the Da Vinci Client.
DaVinciRecordTransformerResult<O>	transform(Lazy<K> key, Lazy<V> value)	Implement this method to transform records before they are stored.

Methods inherited from class com.linkedin.davinci.client.DaVinciRecordTransformer

getClassHash, getRecordTransformerUtility, getStoreRecordsInDaVinci, getStoreVersion, onRecovery, prependSchemaIdToHeader, prependSchemaIdToHeader, transformAndProcessPut

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BlockingDaVinciRecordTransformer

public BlockingDaVinciRecordTransformer(DaVinciRecordTransformer recordTransformer,
                                        boolean storeRecordsInDaVinci)

Method Detail

getKeySchema

public org.apache.avro.Schema getKeySchema()

Description copied from class: DaVinciRecordTransformer

Returns the schema for the key used in DaVinciClient's operations.

Specified by:

getKeySchema in class DaVinciRecordTransformer<K,V,O>

Returns:

a Schema corresponding to the type of DaVinciRecordTransformer.

getOutputValueSchema

public org.apache.avro.Schema getOutputValueSchema()

Description copied from class: DaVinciRecordTransformer

Returns the schema for the output value used in DaVinciClient's operations.

Specified by:

getOutputValueSchema in class DaVinciRecordTransformer<K,V,O>

Returns:

a Schema corresponding to the type of DaVinciRecordTransformer.

transform

public DaVinciRecordTransformerResult<O> transform(Lazy<K> key,
                                                   Lazy<V> value)

Description copied from class: DaVinciRecordTransformer

Implement this method to transform records before they are stored. This can be useful for tasks such as filtering out unused fields to save storage space.

Specified by:

transform in class DaVinciRecordTransformer<K,V,O>

Parameters:

key - the key of the record to be transformed

value - the value of the record to be transformed

Returns:

DaVinciRecordTransformerResult

processPut

public void processPut(Lazy<K> key,
                       Lazy<O> value)

Description copied from class: DaVinciRecordTransformer

Implement this method to manage custom state outside the Da Vinci Client.

Specified by:

processPut in class DaVinciRecordTransformer<K,V,O>

Parameters:

key - the key of the record to be put

value - the value of the record to be put, derived from the output of DaVinciRecordTransformer.transform(Lazy key, Lazy value)

processDelete

public void processDelete(Lazy<K> key)

Description copied from class: DaVinciRecordTransformer

Override this method to customize the behavior for record deletions. For example, you can use this method to delete records from a custom storage outside the Da Vinci Client. By default, it performs no operation.

Overrides:

processDelete in class DaVinciRecordTransformer<K,V,O>

Parameters:

key - the key of the record to be deleted

onStartVersionIngestion

public void onStartVersionIngestion()

Description copied from class: DaVinciRecordTransformer

Lifecycle event triggered before consuming records for DaVinciRecordTransformer.storeVersion. Use this method to perform setup operations such as opening database connections or creating tables. By default, it performs no operation.

Overrides:

onStartVersionIngestion in class DaVinciRecordTransformer<K,V,O>

onEndVersionIngestion

public void onEndVersionIngestion()

Description copied from class: DaVinciRecordTransformer

Lifecycle event triggered when record consumption is stopped for DaVinciRecordTransformer.storeVersion. Use this method to perform cleanup operations such as closing database connections or dropping tables. By default, it performs no operation.

Overrides:

onEndVersionIngestion in class DaVinciRecordTransformer<K,V,O>