Opinionated Google Cloud Platform projects

I'm glad Google are finally starting to embrace Terraform by creating their own modules. Version 0.1.0 of the project-factory looks really promising.

Managing Flink Jobs

The DA Platform is a huge step forward for running Flink at scale. I was lucky enough to see a demo and was really impressed. Far more advanced that the what can be achieved with Dataflow at the moment.

Taming the stragglers in Google Cloud Dataflow

I'm currently bench-marking Flink against Google Cloud Dataflow using the same Apache Beam pipeline for quantitative analytics. One observation I've seen with Flink is the tail latency associated with some shards. 

Google Cloud Dataflow can optimise away stragglers in large jobs using "Dynamic Workload Rebalancing".  As far as I know, Flink is currently unable to perform similar optimisations.

Latency Sensitive Microservices

Great talk by by Peter Lawrey regarding latency in micro-services.  https://www.infoq.com/presentations/latency-sensitive-microservices/