Pinterest Uses Event Time Watermarks to Finalize Petabyte Scale Data Partitions

Image: InfoQ
ad slot · in-content video 16:9
Coverage
More coverage
- Pinterest has introduced event time-based partition finalization for its database ingestion framework. The mechanism uses Apache Flink to track event time and Apache Iceberg metadata to publish a non-regressing watermark, helping downstream jobs determine when continuously ingested CDC data is…