wyl/canal
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
1,500 Commits 10 Branches 29 Tags
fb0f274ba81dfc207162e4ffaebbeb1dc67dc9f1
Commit Graph
1500 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
jianghang.loujh fb0f274ba8 fixed logger 2022-05-20 16:09:53 +08:00
jianghang.loujh d71a208123 fixed issue #4178 , support -00:00:00.12 2022-05-20 10:37:53 +08:00
jianghang.loujh 656a1d23c2 instance.properties demo 2022-05-19 19:56:50 +08:00
jianghang.loujh a574b91fdf fixed #3976 , support LogEvent.MYSQL_TYPE_TYPED_ARRAY 2022-05-19 19:53:54 +08:00
jianghang.loujh e41e903984 fixed issue #3538 , compatiable mysql8.0 filed meta 2022-05-17 17:17:32 +08:00
jianghang.loujh 4a22d72ef4 fixed issue #4018, fixed CanalInstanceWithManager thread-safe 2022-05-16 19:46:34 +08:00
jianghang.loujh b7eff723a3 fixed issue #4027, CanalServerWithEmbedded.get support newList 2022-05-16 19:41:46 +08:00
jianghang.loujh 9b9cfe4037 fixed issue #4152, TableMetaTSDB support canal.instance.filter.query.ddl=true 2022-05-16 17:25:43 +08:00
jianghang.loujh ee51f338e0 fixed outAdapterkey autoGen 2022-05-13 15:13:14 +08:00
zhuchao941andagapple 4315f203a2 repair issue #3401 (#3923) 
* 解决OutAdapter单例导致的问题，如 1.多个es7 outAdapter的多线程问题（共享同一个es bulk request），2.多个es7 outAdapter共用一份es配置问题

* 1. 优化adapter和监听器之间的代码结构
2. 监听器只处理匹配的配置文件
3. bugfix-es adapter配置文件回调时漏写esVersion
理论上一个adapter实例对应一个config配置文件

Co-authored-by: agapple <jianghang.loujh@alibaba-inc.com>
 2022-05-13 15:02:06 +08:00
jianghang.loujh b12945d4a9 format yml 2022-05-13 14:39:57 +08:00
Treeyhhai.yuagapple
 		885f05fb6a 支持定义rocketmq client日志路径，及修复一些bug (#3229) 
* 修复接入阿里云RocketMq no route info for this topic 问题

* 支持deployer同步rocketmq时定义rocketmq client日志路径

* 支持deployer同步rocketmq时定义rocketmq client日志路径

* 回退guava版本号，解决adapter的java.lang.NoSuchMethodError: com.google.common.util.concurrent.MoreExecutors.sameThreadExecutor()异常

* 解决adapter启动报com.alibaba.druid.pool.DruidDataSource cannot be cast to com.alibaba.druid.pool.DruidDataSource异常。

* 支持rocketmq client 配置日志路径

Co-authored-by: hai.yu <hai.yu@snowballtech.com>
Co-authored-by: agapple <jianghang.loujh@alibaba-inc.com>
 2022-05-13 14:39:29 +08:00
ChengTie(鐵手) 591126ee6b update (#3326) 2022-05-13 14:37:17 +08:00
He Wang af432bf10a fix data truncated for column issue for type year (#3866) 2022-05-13 14:33:06 +08:00
jumao b532e90ff4 ESAdapter Support the same instance cross -library query (#4161) 
* ESAdapter Support the same instance cross -library query

* fix null point ex
 2022-05-13 14:31:09 +08:00
xiaoyu.zhang e48bc35f35 通过索引别名获取mapping，返回数据为真实索引，而不是别名，导致mappings.get(index).get(type)报空指针异常 (#4122) 2022-05-13 14:28:53 +08:00
He Wang 11a5f2a0fa add druid stat filter (#3985) 2022-05-13 14:24:37 +08:00
He Wangandagapple c775478a56 set backtick by DbType (#3984) 
Co-authored-by: agapple <jianghang.loujh@alibaba-inc.com>
 2022-05-13 14:23:40 +08:00
He Wang bd1f91cd9c fix sync order in RdbMirrorDbSyncService (#3928) 2022-05-13 14:19:40 +08:00
jianghang.loujh 67e85d201c remove client adapter Docker 2022-05-13 12:46:12 +08:00
wanglei 6965726dfe fix bug #2023 (#3871) 
* fix bug #2023

* 配置canal-adapter，docker生成脚本
 2022-05-13 12:39:43 +08:00
willmao c0001afe8f use hostname if register name not set (#3968) 2022-05-13 12:24:33 +08:00
xiayingfeng 977031a666 Improve MAVEN build Performance (#4055) 2022-05-13 12:23:33 +08:00
zhangshaoning bd3224eea1 Upgrade reference log4j package version (#4017) 2022-05-13 12:22:45 +08:00
jianghang.loujh 33733aef25 fixed compatible and code format 2022-05-13 11:59:57 +08:00
zhangjukai 25be0259a4 Some tweaks for pulsarmq-connector (#4060) 
* 1. Add the com.alibaba.otter.canal.connector.core.spi.CanalMsgConsumer file to the META-INF/canal directory under the pulsarmq-connector project
2. Introduce pulsar-client-admin for Canal to automatically create multi-partition topics
3. Add the judgment that roleToken is null
4. The disconnect method in CanalPulsarMQConsumer removes this.pulsarMQConsumer.unsubscribe();, this code will cause data loss during stop
5. When getting Pulsar messages, they are all processed as flat messages, because CanalMessageSerializerUtil.deserializer(data) will deserialize exceptions
6. Use groupId as subscriptName, without the pulsarmq.subscriptName parameter, the entire adapter will be the same subscriber name using pulsarmq.subscriptName

* 逻辑优化

* 代码优化

* 针对pulsar的调整

* 恢复代码

* 恢复代码

* 处理Producer失效的情况
 2022-05-13 11:41:50 +08:00
liteng2430 8114d77a20 Add verification: the user can set a password with a length of less than 6 digits when updating the password, but the password length must be greater than or equal to 6 digits when logging in to canal admin, so the length verification function when updating the login password is added (#4103) 2022-05-13 11:34:36 +08:00
ZhangJian He 571e7f4e25 fix ojdbc6 wrong corrinator (#4132) 2022-05-13 11:33:42 +08:00
温绍锦 ead91433d7 fastjson update 1.x to 2.0.2 (#4160) 2022-05-13 11:32:47 +08:00
xiaoma20082008 bac016f89e <feat>: support visibility columns (#3976) 2022-05-13 11:30:19 +08:00
willmao 8804647c30 avoid print password when logging (#3969) 2022-05-13 11:28:05 +08:00
mclubing 2623d08b8c rdb配置不支持yaml后缀文件 (#3964) 
增加rdb配置支持yaml后缀文件
 2022-05-13 11:26:04 +08:00
wuare 622fed84ce fixed logger name (#3887) 2022-05-13 11:24:00 +08:00
He Wang 9f5e8fd1c4 rollback guava version in adapter module (#3880) 2022-05-13 11:23:30 +08:00
Junwang Zhao 88d472ef1e move protocol test to protocol test directory (#3864) 
1. move the protocol test to right directory
2. fix wrong string format params count

Signed-off-by: 元组 <zhaojunwang.zjw@alibaba-inc.com>
 2022-05-13 11:16:06 +08:00
Aiden.Ao 2987d08161 fix issue #3144 (#3712) 2022-05-13 11:09:09 +08:00
tuple b54bea5e33 polish the code thus make it more stable (#3855) 
Signed-off-by: 元组 <zhaojunwang.zjw@alibaba-inc.com>
 2021-10-11 14:46:22 +08:00
jianghang.loujh c42ba95eed fixed pulsarmq package 2021-10-09 18:06:34 +08:00
jianghang.loujh b48c434ffd fixed phoenix version canal-1.1.6-alpha-1 2021-10-09 17:52:13 +08:00
jianghang.loujh 2c892fce9b fixed issue #3484 , fixed bat script 2021-10-09 17:39:22 +08:00
jianghang.loujh 3bb49e19d2 fixed issue #3576, stop exporter 2021-10-09 17:29:23 +08:00
jianghang.loujh f26b84ffc2 fixed issue #3829 , kafka connector support k8s env 2021-10-09 16:31:59 +08:00
364102729andagapple 0a2254eeaf add tablestore adapter (#3754) 
* add tablestore adapter

* add tablestore adapter

* fix bug add add log when etl fail

* fix bug，tinyint(1)以及rowupdatechange

Co-authored-by: agapple <jianghang115@gmail.com>
 2021-10-09 16:06:13 +08:00
jianghang.loujh 40d840375f fixed MR 2021-10-09 15:59:12 +08:00
Chad2liandagapple 5b6fd8094a 增加Apache Pulsar MQ支持 (#3791) 
* done pulsarmq

* del pulsar consumer

* pulsar partition

* done pulsar consumer&connector

* update pulsar-client version to 2.8.1

Co-authored-by: agapple <jianghang115@gmail.com>
 2021-10-09 15:51:45 +08:00
mikoto 4ce401d219 fix: 多destination的场景下只能生效一个 (#3772) 2021-10-09 15:38:26 +08:00
Jonathan SchneiderandModerne de95a6a128 refactor: Use Java standard library instead of Guava (#3708) 
Co-authored-by: Moderne <team@moderne.io>

Co-authored-by: Moderne <team@moderne.io>
 2021-10-09 15:33:37 +08:00
happyrita 9e11c3c018 定时保存snapshot到DB的逻辑中，schema名称获取错误，在有配置canal.instance.filter.regex时，由于对应的table无法被匹配中，导致compareTableMetaDbAndMemory方法不被执行 (#3693) 
* 避免重复订阅zk变更状态

* Update DatabaseTableMeta.java

获取schema的名称

* Update CanalController.java
 2021-10-09 15:32:20 +08:00
arthasking123 d67bea8f3d repair issue #3678,#3680 (#3684) 
* repair issue #3678,#3680

* repair issue #3678,#3680
 2021-10-09 15:30:32 +08:00
kkjinping 997957d6e0 新增配置 canal.mq.enableDynamicQueuePartition，获取topic对应的队列的数量为分区的数量，以支持动态队列的场景（如阿里云的rocketmq，无法人工设置队列数量，可能会动态伸缩） (#3670) 2021-10-09 15:26:23 +08:00