我的编程空间,编程开发者的网络收藏夹
学习永远不晚

flink cdc 、 canal 、maxwell 的区别

短信预约 -IT技能 免费直播动态提醒
省份

北京

  • 北京
  • 上海
  • 天津
  • 重庆
  • 河北
  • 山东
  • 辽宁
  • 黑龙江
  • 吉林
  • 甘肃
  • 青海
  • 河南
  • 江苏
  • 湖北
  • 湖南
  • 江西
  • 浙江
  • 广东
  • 云南
  • 福建
  • 海南
  • 山西
  • 四川
  • 陕西
  • 贵州
  • 安徽
  • 广西
  • 内蒙
  • 西藏
  • 新疆
  • 宁夏
  • 兵团
手机号立即预约

请填写图片验证码后获取短信验证码

看不清楚,换张图片

免费获取短信验证码

flink cdc 、 canal 、maxwell 的区别

目录

读取数据的格式不同   (CDC是自定义的数据类型  在这里就不进行展示了,主要是展示一下Maxwell和Canal的区别)

1.添加的区别        

1.1 Canal

1.2 Maxwell

2.修改的区别

2.1Canal

2,2Maxwell

3.删除的区别

3.1 Canal

3.2 Maxwell


Flink CDC :

        DataStream:

                优点:多库多表

                缺点:需要自定义反序列化

        FlinkSQL:

                优点:不需要自定义反序列化

                缺点:单表查询

                                FlinkCDC                    Maxwell                Canal

断点续传                 CK                                MySQL                本地磁盘

SQL->数据              无                                 无                         一对一(炸开)

初始化功能              有(多库多表)                有(单表)                无

封装格式                 自定义                          JSON                    JSON(c/s自定义)

高可用                     运行集群高可用           无                         集群(ZK)

读取数据的格式不同   (CDC是自定义的数据类型  在这里就不进行展示了,主要是展示一下Maxwell和Canal的区别)

1.添加的区别        

INSERT INTO z_user_info VALUES(30,'zhang3','13810001010'),(31,'li4','1389999999'); 

1.1 Canal

{"data":[{"id":"30","user_name":"zhang3","tel":"13810001010"},{"id":"31","user_name":"li4","tel":"1389999999"}],"database":"jdmall-2020-04","es":1589385314000,"id":2,"isDdl":false,"mysqlType":{"id":"bigint(20)","user_name":"varchar(20)","tel":"varchar(20)"},"old":null,"pkNames":["id"],"sql":"","sqlType":{"id":-5,"user_name":12,"tel":12},"table":"z_user_info","ts":1589385314116,"type":"INSERT"}

1.2 Maxwell

{"database":"jdmall-2020-04","table":"z_user_info","type":"insert","ts":1589385314,"xid":82982,"xoffset":0,"data":{"id":30,"user_name":"zhang3","tel":"13810001010"}}

{"database":"jdmall-2020-04","table":"z_user_info","type":"insert","ts":1589385314,"xid":82982,"commit":true,"data":{"id":31,"user_name":"li4","tel":"1389999999"}}

2.修改的区别

UPDATE z_user_info SET user_name='wang55' WHERE id IN(30,31)

2.1Canal

{"data":[{"id":"30","user_name":"wang55","tel":"13810001010"},{"id":"31","user_name":"wang55","tel":"1389999999"}],"database":"jdmall-2020-04","es":1589385508000,"id":3,"isDdl":false,"mysqlType":{"id":"bigint(20)","user_name":"varchar(20)","tel":"varchar(20)"},"old":[{"user_name":"zhang3"},{"user_name":"li4"}],"pkNames":["id"],"sql":"","sqlType":{"id":-5,"user_name":12,"tel":12},"table":"z_user_info","ts":1589385508676,"type":"UPDATE"}

2,2Maxwell

{"database":"jdmall-2020-04","table":"z_user_info","type":"update","ts":1589385508,"xid":83206,"xoffset":0,"data":{"id":30,"user_name":"wang55","tel":"13810001010"},"old":{"user_name":"zhang3"}}

{"database":"jdmall-2020-04","table":"z_user_info","type":"update","ts":1589385508,"xid":83206,"commit":true,"data":{"id":31,"user_name":"wang55","tel":"1389999999"},"old":{"user_name":"li4"}}

3.删除的区别

DELETE  FROM z_user_info   WHERE id IN(30,31)

3.1 Canal

{"data":[{"id":"30","user_name":"wang55","tel":"13810001010"},{"id":"31","user_name":"wang55","tel":"1389999999"}],"database":"jdmall-2020-04","es":1589385644000,"id":4,"isDdl":false,"mysqlType":{"id":"bigint(20)","user_name":"varchar(20)","tel":"varchar(20)"},"old":null,"pkNames":["id"],"sql":"","sqlType":{"id":-5,"user_name":12,"tel":12},"table":"z_user_info","ts":1589385644829,"type":"DELETE"}

3.2 Maxwell

{"database":"jdmall-2020-04","table":"z_user_info","type":"delete","ts":1589385644,"xid":83367,"xoffset":0,"data":{"id":30,"user_name":"wang55","tel":"13810001010"}}

{"database":"jdmall-2020-04","table":"z_user_info","type":"delete","ts":1589385644,"xid":83367,"commit":true,"data":{"id":31,"user_name":"wang55","tel":"1389999999"}}

以上是Canal和Maxwell的数据格式区别

来源地址:https://blog.csdn.net/wuhahaq/article/details/130093363

免责声明:

① 本站未注明“稿件来源”的信息均来自网络整理。其文字、图片和音视频稿件的所属权归原作者所有。本站收集整理出于非商业性的教育和科研之目的,并不意味着本站赞同其观点或证实其内容的真实性。仅作为临时的测试数据,供内部测试之用。本站并未授权任何人以任何方式主动获取本站任何信息。

② 本站未注明“稿件来源”的临时测试数据将在测试完成后最终做删除处理。有问题或投稿请发送至: 邮箱/279061341@qq.com QQ/279061341

flink cdc 、 canal 、maxwell 的区别

下载Word文档到电脑,方便收藏和打印~

下载Word文档

编程热搜

目录