As a team from Alibaba Cloud, we have developed and open-sourced Apache RocketMQ, a cloud-native “messaging, eventing, streaming” real-time data processing platform that covers cloud-edge-device collaboration scenarios. During the development of RocketMQ, we formulated a log-based storage high availability paradigm that provides a high availability design solution for distributed log storage software used in industrial applications. The paradigm includes six essential components that enable the cluster to recover automatically. Our evaluation shows that this paradigm can achieve high availability, fast recovery, high throughput, and data loss prevention. We hope this paradigm will inspire and guide the development of high-availability solutions for all log-based storage systems.

Juntao Ji

Alibaba Cloud Computing Co. Ltd.

Rongtong Jin

Alibaba Cloud Computing Co. Ltd.

Yubao Fu

Alibaba Cloud Computing Co. Ltd.

Yinyou Gu

Alibaba Cloud Computing Co. Ltd.

Tsung-han Tsai

Alibaba Cloud Computing Co. Ltd.

Qingshan Lin

Alibaba Cloud Computing Co. Ltd.