[maven-release-plugin] prepare release canal-1.1.1

fixed javadoc for release
fixed canal_version release
2018-10-29 14:57:25 +08:00 · 2018-10-29 14:54:17 +08:00 · 2018-10-29 13:49:24 +08:00 · 2018-10-29 12:46:20 +08:00 · 2018-10-29 01:42:30 +08:00 · 2018-10-29 01:40:34 +08:00
257 changed files with 38097 additions and 26115 deletions
@@ -1,14 +1,6 @@
 <div class="blog_content">
    <div class="iteye-blog-content-contain">
-
-<h3>最新更新</h3>
-<ol>
-<li>canal QQ讨论群已经建立，群号：161559791 ，欢迎加入进行技术讨论。</li>
-<li>canal消费端项目开源: Otter(分布式数据库同步系统)，地址：<a href="https://github.com/alibaba/otter">https://github.com/alibaba/otter</a></li>
-<li>Canal已在阿里云推出商业化版本 <a href="https://www.aliyun.com/product/dts?spm=a2c4g.11186623.cloudEssentials.80.srdwr7">数据传输服务DTS</a>， 开通即用，免去部署维护的昂贵使用成本。DTS针对阿里云RDS、DRDS等产品进行了适配，解决了Binlog日志回收，主备切换、VPC网络切换等场景下的订阅高可用问题。同时，针对RDS进行了针对性的性能优化。出于稳定性、性能及成本的考虑，强烈推荐阿里云用户使用DTS产品。<a href="https://help.aliyun.com/document_detail/26592.html?spm=a2c4g.11174283.6.539.t1Y91E">DTS产品使用文档</a></li>
-DTS支持阿里云RDS&DRDS的Binlog日志实时订阅，现推出首月免费体验，限时限量，<a href="https://common-buy.aliyun.com/?commodityCode=dtspre&request=%7b%22dts_function%22%3a%22data_subscribe%22%7d">立即体验>>></a>
-</ol>
-
+    	
 <h1>背景</h1>
 <p style="font-size: 14px;">   早期，阿里巴巴B2B公司因为存在杭州和美国双机房部署，存在跨机房同步的业务需求。不过早期的数据库同步业务，主要是基于trigger的方式获取增量变更，不过从2010年开始，阿里系公司开始逐步的尝试基于数据库的日志解析，获取增量变更进行同步，由此衍生出了增量订阅&amp;消费的业务，从此开启了一段新纪元。</p>
 <p style="font-size: 14px;">   ps. 目前内部版本已经支持mysql和oracle部分版本的日志解析，当前的canal开源版本支持5.7及以下的版本(阿里内部mysql 5.7.13, 5.6.10, mysql 5.5.18和5.1.40/48)</p>
@@ -46,6 +38,16 @@ DTS支持阿里云RDS&DRDS的Binlog日志实时订阅，现推出首月免费体
 <li>canal解析binary log对象(原始为byte流)</li>
 </ol>

+<h1>重要版本更新说明</h1>
+
+canal 1.1.x系列，参考release文档：<a href="https://github.com/alibaba/canal/releases">版本发布信息</a>
+
+1. 整体性能测试&优化,提升了150%. #726 参考: 【[Performance](https://github.com/alibaba/canal/wiki/Performance)】
+2. 原生支持prometheus监控 #765 【[Prometheus QuickStart](https://github.com/alibaba/canal/wiki/Prometheus-QuickStart)】
+3. 原生支持kafka消息投递 #695 【[Canal Kafka/RocketMQ QuickStart](https://github.com/alibaba/canal/wiki/Canal-Kafka-RocketMQ-QuickStart)】
+4. 原生支持aliyun rds的binlog订阅 (解决自动主备切换/oss binlog离线解析) 参考: 【[Aliyun RDS QuickStart](https://github.com/alibaba/canal/wiki/aliyun-RDS-QuickStart)】
+5. 原生支持docker镜像 #801 参考:  【[Docker QuickStart](https://github.com/alibaba/canal/wiki/Docker-QuickStart)】
+
 <h1>相关文档</h1>

 See the wiki page for : <a href="https://github.com/alibaba/canal/wiki" >wiki文档</a>
@@ -54,20 +56,41 @@ See the wiki page for : <a href="https://github.com/alibaba/canal/wiki" >wiki文
 <ul>
 <li><a class="internal present" href="https://github.com/alibaba/canal/wiki/Home">Home</a></li>
 <li><a class="internal present" href="https://github.com/alibaba/canal/wiki/Introduction">Introduction</a></li>
-<li><a class="internal present" href="https://github.com/alibaba/canal/wiki/QuickStart">QuickStart</a></li>
-<li><a class="internal present" href="https://github.com/alibaba/canal/wiki/ClientExample">ClientExample</a></li>
+<li>
+<a class="internal present" href="https://github.com/alibaba/canal/wiki/QuickStart">QuickStart</a>
+<ul>
+<li><a class="internal present" href="https://github.com/alibaba/canal/wiki/Docker-QuickStart">Docker QuickStart</a></li>
+<li><a class="internal present" href="https://github.com/alibaba/canal/wiki/Canal-Kafka-RocketMQ-QuickStart">Canal Kafka/RocketMQ QuickStart</a></li>
+<li><a class="internal present" href="https://github.com/alibaba/canal/wiki/aliyun-RDS-QuickStart">Aliyun RDS QuickStart</a></li>
+<li><a class="internal present" href="https://github.com/alibaba/canal/wiki/Prometheus-QuickStart">Prometheus QuickStart</a></li>
+</ul>
+</li>
 <li><a class="internal present" href="https://github.com/alibaba/canal/wiki/AdminGuide">AdminGuide</a></li>
+<li><a class="internal present" href="https://github.com/alibaba/canal/wiki/ClientExample">ClientExample</a></li>
 <li><a class="internal present" href="https://github.com/alibaba/canal/wiki/ClientAPI">ClientAPI</a></li>
+<li><a class="internal present" href="https://github.com/alibaba/canal/wiki/Performance">Performance</a></li>
 <li><a class="internal present" href="https://github.com/alibaba/canal/wiki/DevGuide">DevGuide</a></li>
 <li><a class="internal present" href="https://github.com/alibaba/canal/wiki/BinlogChange%28mysql5.6%29">BinlogChange(Mysql5.6)</a></li>
+<li><a class="internal present" href="https://github.com/alibaba/canal/wiki/BinlogChange%28MariaDB%29">BinlogChange(MariaDB)</a></li>
+<li><a class="internal present" href="https://github.com/alibaba/canal/wiki/TableMetaTSDB">TableMetaTSDB</a></li>
 <li><a href="http://alibaba.github.com/canal/release.html">ReleaseNotes</a></li>
 <li><a href="https://github.com/alibaba/canal/releases">Download</a></li>
+<li><a class="internal present" href="/alibaba/canal/wiki/FAQ">FAQ</a></li>
 </ul>

+<h1>多语言业务</h1>
+
+1. canal整体交互协议设计上使用了protobuf3.0，理论上可以支持绝大部分的多语言场景，欢迎大家提交多客户端的PR
+    * canal java客户端: <a href="https://github.com/alibaba/canal/wiki/ClientExample"> https://github.com/alibaba/canal/wiki/ClientExample </a>
+    * canal c#客户端开源项目地址：<a href="https://github.com/CanalClient/CanalSharp"> https://github.com/CanalSharp/CanalSharp </a>
+    * canal go客户端开源项目地址：<a href="https://github.com/CanalClient/canal-go"> https://github.com/CanalClient/canal-go </a>
+2. canal作为MySQL binlog的增量获取工具，可以将数据投递到MQ系统中，比如Kafka/RocketMQ，可以借助于MQ的多语言能力 
+    * 参考文档: [Canal Kafka/RocketMQ QuickStart](https://github.com/alibaba/canal/wiki/Canal-Kafka-RocketMQ-QuickStart)
+
 <h1>相关资料</h1>

 * ADC阿里技术嘉年华分享ppt (放在google docs上，可能需要翻墙): <a href="https://docs.google.com/presentation/d/1MkszUPYRDkfVPz9IqOT1LLT5d9tuwde_WC8GZvjaDRg/edit?usp=sharing">ppt下载</href>  
-* [与阿里巴巴的RocketMQ配合使用](https://github.com/alibaba/RocketMQ)
+* [与阿里巴巴的RocketMQ配合使用](https://github.com/apache/RocketMQ)

 <h1>相关开源</h1>
 <ol>
@@ -91,9 +114,13 @@ See the wiki page for : <a href="https://github.com/alibaba/canal/wiki" >wiki文
 <li>报告issue：<a href="https://github.com/alibaba/canal/issues">issues</a></li>
 </ol>

-<pre>
-【招聘】阿里巴巴中间件团队招聘JAVA高级工程师
-岗位主要为技术型内容(非业务部门)，阿里中间件整个体系对于未来想在技术上有所沉淀的同学还是非常有帮助的
-工作地点：杭州、北京均可. ps. 阿里待遇向来都是不错的，有意者可以QQ、微博私聊. 
-具体招聘内容：https://job.alibaba.com/zhaopin/position_detail.htm?positionId=32666
-</pre>
+<h3>最新更新</h3>
+<ol>
+<li>canal发布重大版本更新1.1.0，具体releaseNode参考：<a href="https://github.com/alibaba/canal/releases/tag/canal-1.1.0">https://github.com/alibaba/canal/releases/tag/canal-1.1.0</a></li>
+<li>canal c#客户端开源项目地址：<a href="https://github.com/CanalClient/CanalSharp"> https://github.com/CanalClient/CanalSharp </a>，推荐! </li>
+<li>canal QQ讨论群已经建立，群号：161559791 ，欢迎加入进行技术讨论。</li>
+<li>canal消费端项目开源: Otter(分布式数据库同步系统)，地址：<a href="https://github.com/alibaba/otter">https://github.com/alibaba/otter</a></li>
+
+<li>Canal已在阿里云推出商业化版本 <a href="https://www.aliyun.com/product/dts?spm=a2c4g.11186623.cloudEssentials.80.srdwr7">数据传输服务DTS</a>， 开通即用，免去部署维护的昂贵使用成本。DTS针对阿里云RDS、DRDS等产品进行了适配，解决了Binlog日志回收，主备切换、VPC网络切换等场景下的订阅高可用问题。同时，针对RDS进行了针对性的性能优化。出于稳定性、性能及成本的考虑，强烈推荐阿里云用户使用DTS产品。<a href="https://help.aliyun.com/document_detail/26592.html?spm=a2c4g.11174283.6.539.t1Y91E">DTS产品使用文档</a></li>
+DTS支持阿里云RDS&DRDS的Binlog日志实时订阅，现推出首月免费体验，限时限量，<a href="https://common-buy.aliyun.com/?commodityCode=dtspre&request=%7b%22dts_function%22%3a%22data_subscribe%22%7d">立即体验>>></a>
+</ol>
@@ -0,0 +1,251 @@
+## 基本说明
+canal 1.1.1版本之后, 增加客户端数据落地的适配及启动功能, 目前支持功能:
+* 客户端启动器
+* 同步管理REST接口
+* 日志适配器, 作为DEMO
+* HBase的数据同步(表对表同步), ETL功能
+* (后续支持) ElasticSearch多表数据同步,ETL功能
+
+## 环境版本
+* 操作系统：无要求
+* java版本: jdk1.8 以上 
+* canal 版本: 请下载最新的安装包，本文以当前v1.1.1 的canal.deployer-1.1.1.tar.gz为例
+* MySQL版本 ：5.7.18
+* HBase版本: Apache HBase 1.1.2, 若和服务端版本不一致可自行替换客户端HBase依赖
+
+## 一、适配器启动器
+client-adapter分为适配器和启动器两部分, 适配器为多个fat jar, 每个适配器会将自己所需的依赖打成一个包, 以SPI的方式让启动器动态加载
+
+
+启动器为 SpringBoot 项目, 支持canal-client启动的同时提供相关REST管理接口, 运行目录结构为:
+```
+canal-adapter-launcher.jar
+- lib
+    client-adapter.logger-1.1.1-jar-with-dependencies.jar
+    client-adapter.hbase-1.1.1-jar-with-dependencies.jar
+- config
+    application.yml
+    - hbase
+        mytest_person2.yml
+```
+以上目录结构最终会打包成 canal-adapter-launcher.tar.gz 压缩包
+
+## 二、启动器
+### 2.1 启动器配置 application.yml
+#### canal相关配置部分说明
+```
+canal.conf:
+  canalServerHost: 127.0.0.1:11111  # 对应单机模式下的canal server的ip:port
+  zookeeperHosts: slave1:2181       # 对应集群模式下的zk地址, 如果配置了canalServerHost, 则以canalServerHost为准
+  bootstrapServers: slave1:6667     # kafka或rocketMQ地址, 与canalServerHost不能并存
+  flatMessage: true                 # 扁平message开关, 是否以json字符串形式投递数据, 仅在kafka/rocketMQ模式下有效
+  canalInstances:                   # canal实例组, 如果是tcp模式可配置此项
+  - instance: example               # 对应canal destination
+    groups:                  # 对应适配器分组, 分组间的适配器并行运行
+    - outAdapters:                  # 适配器列表, 分组内的适配串行运行
+      - name: logger                # 适配器SPI名
+      - name: hbase
+        properties:                 # HBase相关连接参数
+          hbase.zookeeper.quorum: slave1
+          hbase.zookeeper.property.clientPort: 2181
+          zookeeper.znode.parent: /hbase
+  mqTopics:                         # MQ topic租, 如果是kafka或者rockeMQ模式可配置此项, 与canalInstances不能并存
+  - mqMode: kafka                   # MQ的模式: kafak/rocketMQ
+    topic: example                  # MQ topic
+    groups:                         # group组
+    - groupId: g2                   # group id
+      outAdapters:                  # 适配器列表, 以下配置和canalInstances中一样
+      - name: logger                
+```
+#### 适配器相关配置部分说明
+```
+adapter.conf:
+  datasourceConfigs:                # 数据源配置列表, 数据源将在适配器中用于ETL、数据同步回查等使用
+    defaultDS:                      # 数据源 dataSourceKey, 适配器中通过该值获取指定数据源
+      url: jdbc:mysql://127.0.0.1:3306/mytest?useUnicode=true
+      username: root
+      password: 121212
+  adapterConfigs:                   # 适配器内部配置列表
+  - hbase/mytest_person2.yml        # 类型/配置文件名, 这里示例就是对应HBase适配器hbase目录下的mytest_person2.yml文件
+```
+## 2.2 同步管理REST接口
+#### 2.2.1 查询所有订阅同步的canal destination或MQ topic
+```
+curl http://127.0.0.1:8081/destinations
+```
+#### 2.2.2 数据同步开关
+```
+curl http://127.0.0.1:8081/syncSwitch/example/off -X PUT
+```
+针对 example 这个canal destination/MQ topic 进行开关操作. off代表关闭, 该destination/topic下的同步将阻塞或者断开连接不再接收数据, on代表开启
+
+注: 如果在配置文件中配置了 zookeeperHosts 项, 则会使用分布式锁来控制HA中的数据同步开关, 如果是单机模式则使用本地锁来控制开关
+#### 2.2.3 数据同步开关状态
+```
+curl http://127.0.0.1:8081/syncSwitch/example
+```
+查看指定 canal destination/MQ topic 的数据同步开关状态
+#### 2.2.4 手动ETL
+```
+curl http://127.0.0.1:8081/etl/hbase/mytest_person2.yml -X POST -d "params=2018-10-21 00:00:00"
+```
+导入数据到指定类型的库
+#### 2.2.5 查看相关库总数据
+```
+curl http://127.0.0.1:8081/count/hbase/mytest_person2.yml
+```
+### 2.3 启动canal-adapter示例
+#### 2.3.1 启动canal server (单机模式), 参考: [Canal QuickStart](https://github.com/alibaba/canal/wiki/QuickStart)
+#### 2.3.2 修改config/application.yml为:
+```
+server:
+  port: 8081
+logging:
+  level:
+    com.alibaba.otter.canal.client.adapter.hbase: DEBUG
+spring:
+  jackson:
+    date-format: yyyy-MM-dd HH:mm:ss
+    time-zone: GMT+8
+    default-property-inclusion: non_null
+
+canal.conf:
+  canalServerHost: 127.0.0.1:11111
+  flatMessage: true
+  canalInstances:
+  - instance: example
+    adapterGroups:
+    - outAdapters:
+      - name: logger
+```
+启动 canal-adapter-launcher.jar
+```
+java -jar canal-adapter-launcher.jar
+```
+
+## 三、HBase适配器
+### 3.1 修改启动器配置: application.yml
+```
+server:
+  port: 8081
+logging:
+  level:
+    com.alibaba.otter.canal.client.adapter.hbase: DEBUG
+spring:
+  jackson:
+    date-format: yyyy-MM-dd HH:mm:ss
+    time-zone: GMT+8
+    default-property-inclusion: non_null
+
+canal.conf:
+  canalServerHost: 127.0.0.1:11111
+  flatMessage: true
+  canalInstances:
+  - instance: example
+    adapterGroups:
+    - outAdapters:
+      - name: hbase
+        properties:
+          hbase.zookeeper.quorum: slave1
+          hbase.zookeeper.property.clientPort: 2181
+          zookeeper.znode.parent: /hbase
+
+adapter.conf:
+  datasourceConfigs:
+    defaultDS:
+      url: jdbc:mysql://127.0.0.1:3306/mytest?useUnicode=true
+      username: root
+      password: 121212
+  adapterConfigs:
+  - hbase/mytest_person.yml 
+```
+其中指定了一个HBase表映射文件: mytest_person.yml
+### 3.2 适配器表映射文件
+修改 config/hbase/mytest_person.yml文件:
+```
+dataSourceKey: defaultDS            # 对应application.yml中的datasourceConfigs下的配置
+hbaseMapping:                       # mysql--HBase的单表映射配置
+  mode: STRING                      # HBase中的存储类型, 默认统一存为String, 可选: #PHOENIX  #NATIVE   #STRING 
+                                    # NATIVE: 以java类型为主, PHOENIX: 将类型转换为Phoenix对应的类型
+  destination: example              # 对应 canal destination/MQ topic 名称
+  database: mytest                  # 数据库名/schema名
+  table: person                     # 表名
+  hbaseTable: MYTEST.PERSON         # HBase表名
+  family: CF                        # 默认统一Column Family名称
+  uppercaseQualifier: true          # 字段名转大写, 默认为true
+  commitBatch: 3000                 # 批量提交的大小, ETL中用到
+  #rowKey: id,type                  # 复合字段rowKey不能和columns中的rowKey并存
+                                    # 复合rowKey会以 '|' 分隔
+  columns:                          # 字段映射, 如果不配置将自动映射所有字段, 
+                                    # 并取第一个字段为rowKey, HBase字段名以mysql字段名为主
+    id: ROWKE                       
+    name: CF:NAME
+    email: EMAIL                    # 如果column family为默认CF, 则可以省略
+    type:                           # 如果HBase字段和mysql字段名一致, 则可以省略
+    c_time: 
+    birthday: 
+```
+如果涉及到类型转换,可以如下形式:
+```
+...
+  columns:                         
+    id: ROWKE$STRING                      
+    ...                   
+    type: TYPE$BYTE                          
+    ...
+```
+类型转换涉及到Java类型和Phoenix类型两种, 分别定义如下:
+```
+#Java 类型转换, 对应配置 mode: NATIVE
+$DEFAULT
+$STRING
+$INTEGER
+$LONG
+$SHORT
+$BOOLEAN
+$FLOAT
+$DOUBLE
+$BIGDECIMAL
+$DATE
+$BYTE
+$BYTES
+```
+```
+#Phoenix 类型转换, 对应配置 mode: PHOENIX
+$DEFAULT                  对应PHOENIX里的VARCHAR
+$UNSIGNED_INT             对应PHOENIX里的UNSIGNED_INT           4字节
+$UNSIGNED_LONG            对应PHOENIX里的UNSIGNED_LONG          8字节
+$UNSIGNED_TINYINT         对应PHOENIX里的UNSIGNED_TINYINT       1字节
+$UNSIGNED_SMALLINT        对应PHOENIX里的UNSIGNED_SMALLINT      2字节
+$UNSIGNED_FLOAT           对应PHOENIX里的UNSIGNED_FLOAT         4字节
+$UNSIGNED_DOUBLE          对应PHOENIX里的UNSIGNED_DOUBLE        8字节
+$INTEGER                  对应PHOENIX里的INTEGER                4字节
+$BIGINT                   对应PHOENIX里的BIGINT                 8字节
+$TINYINT                  对应PHOENIX里的TINYINT                1字节
+$SMALLINT                 对应PHOENIX里的SMALLINT               2字节
+$FLOAT                    对应PHOENIX里的FLOAT                  4字节
+$DOUBLE                   对应PHOENIX里的DOUBLE                 8字节
+$BOOLEAN                  对应PHOENIX里的BOOLEAN                1字节
+$TIME                     对应PHOENIX里的TIME                   8字节
+$DATE                     对应PHOENIX里的DATE                   8字节
+$TIMESTAMP                对应PHOENIX里的TIMESTAMP              12字节
+$UNSIGNED_TIME            对应PHOENIX里的UNSIGNED_TIME          8字节
+$UNSIGNED_DATE            对应PHOENIX里的UNSIGNED_DATE          8字节
+$UNSIGNED_TIMESTAMP       对应PHOENIX里的UNSIGNED_TIMESTAMP     12字节
+$VARCHAR                  对应PHOENIX里的VARCHAR                动态长度
+$VARBINARY                对应PHOENIX里的VARBINARY              动态长度
+$DECIMAL                  对应PHOENIX里的DECIMAL                动态长度
+```
+如果不配置将以java对象原生类型默认映射转换
+### 3.3 启动HBase数据同步
+#### 创建HBase表
+在HBase shell中运行:
+```
+create 'MYTEST.PERSON', {NAME=>'CF'}
+```
+#### 启动canal-adapter启动器
+```
+java -jar canal-adapter-launcher.jar
+```
+#### 验证
+修改mysql mytest.person表的数据, 将会自动同步到HBase的MYTEST.PERSON表下面, 并会打出DML的log
@@ -0,0 +1,36 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <parent>
+        <artifactId>canal.client-adapter</artifactId>
+        <groupId>com.alibaba.otter</groupId>
+        <version>1.1.1</version>
+    </parent>
+    <modelVersion>4.0.0</modelVersion>
+    <groupId>com.alibaba.otter</groupId>
+    <artifactId>client-adapter.common</artifactId>
+    <packaging>jar</packaging>
+    <name>canal client adapter common module for otter ${project.version}</name>
+    <dependencies>
+        <dependency>
+            <groupId>com.alibaba.otter</groupId>
+            <artifactId>canal.protocol</artifactId>
+            <version>1.1.1</version>
+        </dependency>
+        <dependency>
+            <groupId>joda-time</groupId>
+            <artifactId>joda-time</artifactId>
+            <version>2.9.4</version>
+        </dependency>
+        <dependency>
+            <groupId>com.alibaba</groupId>
+            <artifactId>druid</artifactId>
+            <version>1.1.9</version>
+        </dependency>
+        <dependency>
+            <groupId>mysql</groupId>
+            <artifactId>mysql-connector-java</artifactId>
+            <version>5.1.40</version>
+        </dependency>
+    </dependencies>
+
+</project>
@@ -0,0 +1,68 @@
+package com.alibaba.otter.canal.client.adapter;
+
+import java.util.List;
+import java.util.Map;
+
+import com.alibaba.otter.canal.client.adapter.support.Dml;
+import com.alibaba.otter.canal.client.adapter.support.EtlResult;
+import com.alibaba.otter.canal.client.adapter.support.OuterAdapterConfig;
+import com.alibaba.otter.canal.client.adapter.support.SPI;
+
+/**
+ * 外部适配器接口
+ *
+ * @author reweerma 2018-8-18 下午10:14:02
+ * @version 1.0.0
+ */
+@SPI("logger")
+public interface OuterAdapter {
+
+    /**
+     * 外部适配器初始化接口
+     *
+     * @param configuration 外部适配器配置信息
+     */
+    void init(OuterAdapterConfig configuration);
+
+    /**
+     * 往适配器中同步数据
+     *
+     * @param dml 数据包
+     */
+    void sync(Dml dml);
+
+    /**
+     * 外部适配器销毁接口
+     */
+    void destroy();
+
+    /**
+     * Etl操作
+     * 
+     * @param task 任务名, 对应配置名
+     * @param params etl筛选条件
+     */
+    default EtlResult etl(String task, List<String> params) {
+        throw new UnsupportedOperationException("unsupported operation");
+    }
+
+    /**
+     * 计算总数
+     * 
+     * @param task 任务名, 对应配置名
+     * @return 总数
+     */
+    default Map<String, Object> count(String task) {
+        throw new UnsupportedOperationException("unsupported operation");
+    }
+
+    /**
+     * 通过task获取对应的destination
+     * 
+     * @param task 任务名, 对应配置名
+     * @return destination
+     */
+    default String getDestination(String task) {
+        return null;
+    }
+}
@@ -0,0 +1,41 @@
+package com.alibaba.otter.canal.client.adapter.support;
+
+import java.util.HashMap;
+import java.util.LinkedHashSet;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * 适配器配置集合, 用于配置加载, 线程不安全
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+public class AdapterConfigs {
+
+    /**
+     * 类型下对应所有配置名, 如:
+     * hbase
+     *  ┗━ mytest_person.yml
+     *  ┗━ mytest_role.yml
+     *  ┗━ mytest_department.yml
+     */
+    private static Map<String, Set<String>> configs = new HashMap<>();
+
+    public static void put(String key, String value) {
+        Set<String> values = configs.get(key);
+        if (values == null) {
+            values = new LinkedHashSet<>();
+        }
+        values.add(value);
+        configs.put(key, values);
+    }
+
+    public static Set<String> get(String key) {
+        return configs.get(key);
+    }
+
+    public static void clear() {
+        configs.clear();
+    }
+}
@@ -0,0 +1,170 @@
+package com.alibaba.otter.canal.client.adapter.support;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * 配置信息类
+ *
+ * @author rewerma 2018-8-18 下午10:40:12
+ * @version 1.0.0
+ */
+public class CanalClientConfig {
+
+    private String              canalServerHost;    // 单机模式下canal server的 ip:port
+
+    private String              zookeeperHosts;     // 集群模式下的zk地址, 如果配置了单机地址则以单机为准!!
+
+    private String              bootstrapServers;   // kafka or rocket mq 地址
+
+    private Boolean             flatMessage = true; // 是否已flatMessage模式传输, 只适用于mq模式
+
+    private List<MQTopic>       mqTopics;           // mq topic 列表
+
+    private List<CanalInstance> canalInstances;     // tcp 模式下 canal 实例列表, 与mq模式不能共存!!
+
+    public String getCanalServerHost() {
+        return canalServerHost;
+    }
+
+    public void setCanalServerHost(String canalServerHost) {
+        this.canalServerHost = canalServerHost;
+    }
+
+    public String getZookeeperHosts() {
+        return zookeeperHosts;
+    }
+
+    public void setZookeeperHosts(String zookeeperHosts) {
+        this.zookeeperHosts = zookeeperHosts;
+    }
+
+    public String getBootstrapServers() {
+        return bootstrapServers;
+    }
+
+    public void setBootstrapServers(String bootstrapServers) {
+        this.bootstrapServers = bootstrapServers;
+    }
+
+    public List<MQTopic> getMqTopics() {
+        return mqTopics;
+    }
+
+    public void setMqTopics(List<MQTopic> mqTopics) {
+        this.mqTopics = mqTopics;
+    }
+
+    public Boolean getFlatMessage() {
+        return flatMessage;
+    }
+
+    public void setFlatMessage(Boolean flatMessage) {
+        this.flatMessage = flatMessage;
+    }
+
+    public List<CanalInstance> getCanalInstances() {
+        return canalInstances;
+    }
+
+    public void setCanalInstances(List<CanalInstance> canalInstances) {
+        this.canalInstances = canalInstances;
+    }
+
+    public static class CanalInstance {
+
+        private String             instance;      // 实例名
+
+        private List<Group> groups;  // 适配器分组列表
+
+        public String getInstance() {
+            return instance;
+        }
+
+        public void setInstance(String instance) {
+            if (instance != null) {
+                this.instance = instance.trim();
+            }
+        }
+
+        public List<Group> getGroups() {
+            return groups;
+        }
+
+        public void setGroups(List<Group> groups) {
+            this.groups = groups;
+        }
+
+    }
+
+    public static class Group {
+
+        private List<OuterAdapterConfig> outAdapters; // 适配器列表
+
+        public List<OuterAdapterConfig> getOutAdapters() {
+            return outAdapters;
+        }
+
+        public void setOutAdapters(List<OuterAdapterConfig> outAdapters) {
+            this.outAdapters = outAdapters;
+        }
+    }
+
+    public static class MQTopic {
+
+        private String      mqMode;                     // mq模式 kafka or rocketMQ
+
+        private String      topic;                      // topic名
+
+        private List<MQGroup> groups = new ArrayList<>(); // 分组列表
+
+        public String getMqMode() {
+            return mqMode;
+        }
+
+        public void setMqMode(String mqMode) {
+            this.mqMode = mqMode;
+        }
+
+        public String getTopic() {
+            return topic;
+        }
+
+        public void setTopic(String topic) {
+            this.topic = topic;
+        }
+
+        public List<MQGroup> getGroups() {
+            return groups;
+        }
+
+        public void setGroups(List<MQGroup> groups) {
+            this.groups = groups;
+        }
+    }
+
+    public static class MQGroup {
+
+        private String                   groupId;     // group id
+
+        private List<OuterAdapterConfig> outAdapters; // 适配器配置列表
+
+        public String getGroupId() {
+            return groupId;
+        }
+
+        public void setGroupId(String groupId) {
+            this.groupId = groupId;
+        }
+
+        public List<OuterAdapterConfig> getOutAdapters() {
+            return outAdapters;
+        }
+
+        public void setOutAdapters(List<OuterAdapterConfig> outAdapters) {
+            this.outAdapters = outAdapters;
+        }
+
+    }
+
+}
@@ -0,0 +1,81 @@
+package com.alibaba.otter.canal.client.adapter.support;
+
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+
+import com.alibaba.druid.pool.DruidDataSource;
+
+/**
+ * 数据源配置
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+public class DatasourceConfig {
+
+    public final static Map<String, DruidDataSource> DATA_SOURCES = new ConcurrentHashMap<>(); // key对应的数据源
+
+    private String                                   driver       = "com.mysql.jdbc.Driver";   // 默认为mysql jdbc驱动
+    private String                                   url;                                      // jdbc url
+    private String                                   database;                                 // jdbc database
+    private String                                   type         = "mysql";                   // 类型, 默认为mysql
+    private String                                   username;                                 // jdbc username
+    private String                                   password;                                 // jdbc password
+    private Integer                                  maxActive    = 3;                         // 连接池最大连接数,默认为3
+
+    public String getDriver() {
+        return driver;
+    }
+
+    public void setDriver(String driver) {
+        this.driver = driver;
+    }
+
+    public String getUrl() {
+        return url;
+    }
+
+    public void setUrl(String url) {
+        this.url = url;
+    }
+
+    public String getDatabase() {
+        return database;
+    }
+
+    public void setDatabase(String database) {
+        this.database = database;
+    }
+
+    public String getType() {
+        return type;
+    }
+
+    public void setType(String type) {
+        this.type = type;
+    }
+
+    public String getUsername() {
+        return username;
+    }
+
+    public void setUsername(String username) {
+        this.username = username;
+    }
+
+    public String getPassword() {
+        return password;
+    }
+
+    public void setPassword(String password) {
+        this.password = password;
+    }
+
+    public Integer getMaxActive() {
+        return maxActive;
+    }
+
+    public void setMaxActive(Integer maxActive) {
+        this.maxActive = maxActive;
+    }
+}
@@ -0,0 +1,118 @@
+package com.alibaba.otter.canal.client.adapter.support;
+
+import java.io.Serializable;
+import java.util.List;
+import java.util.Map;
+
+/**
+ * DML操作转换对象
+ *
+ * @author rewerma 2018-8-19 下午11:30:49
+ * @version 1.0.0
+ */
+public class Dml implements Serializable {
+
+    private static final long         serialVersionUID = 2611556444074013268L;
+
+    private String                    destination;                            // 对应canal的实例或者MQ的topic
+    private String                    database;                               // 数据库或schema
+    private String                    table;                                  // 表名
+    private String                    type;                                   // 类型: INSERT UPDATE DELETE
+    // binlog executeTime
+    private Long                      es;                                     // 执行耗时
+    // dml build timeStamp
+    private Long                      ts;                                     // 同步时间
+    private String                    sql;                                    // 执行的sql, dml sql为空
+    private List<Map<String, Object>> data;                                   // 数据列表
+    private List<Map<String, Object>> old;                                    // 旧数据列表, 用于update, size和data的size一一对应
+
+    public String getDestination() {
+        return destination;
+    }
+
+    public void setDestination(String destination) {
+        this.destination = destination;
+    }
+
+    public String getDatabase() {
+        return database;
+    }
+
+    public void setDatabase(String database) {
+        this.database = database;
+    }
+
+    public String getTable() {
+        return table;
+    }
+
+    public void setTable(String table) {
+        this.table = table;
+    }
+
+    public String getType() {
+        return type;
+    }
+
+    public void setType(String type) {
+        this.type = type;
+    }
+
+    public Long getTs() {
+        return ts;
+    }
+
+    public void setTs(Long ts) {
+        this.ts = ts;
+    }
+
+    public String getSql() {
+        return sql;
+    }
+
+    public void setSql(String sql) {
+        this.sql = sql;
+    }
+
+    public List<Map<String, Object>> getData() {
+        return data;
+    }
+
+    public void setData(List<Map<String, Object>> data) {
+        this.data = data;
+    }
+
+    public List<Map<String, Object>> getOld() {
+        return old;
+    }
+
+    public void setOld(List<Map<String, Object>> old) {
+        this.old = old;
+    }
+
+    public Long getEs() {
+        return es;
+    }
+
+    public void setEs(Long es) {
+        this.es = es;
+    }
+
+    public void clear() {
+        database = null;
+        table = null;
+        type = null;
+        ts = null;
+        es = null;
+        data = null;
+        old = null;
+        sql = null;
+    }
+
+    @Override
+    public String toString() {
+        return "Dml{" + "destination='" + destination + '\'' + ", database='" + database + '\'' + ", table='" + table
+               + '\'' + ", type='" + type + '\'' + ", es=" + es + ", ts=" + ts + ", sql='" + sql + '\'' + ", data="
+               + data + ", old=" + old + '}';
+    }
+}
@@ -0,0 +1,44 @@
+package com.alibaba.otter.canal.client.adapter.support;
+
+import java.io.Serializable;
+
+/**
+ * ETL的结果对象
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+public class EtlResult implements Serializable {
+
+    private static final long serialVersionUID = 4250522736289866505L;
+
+    private boolean           succeeded        = false;
+
+    private String            resultMessage;
+
+    private String            errorMessage;
+
+    public boolean getSucceeded() {
+        return succeeded;
+    }
+
+    public void setSucceeded(boolean succeeded) {
+        this.succeeded = succeeded;
+    }
+
+    public String getResultMessage() {
+        return resultMessage;
+    }
+
+    public void setResultMessage(String resultMessage) {
+        this.resultMessage = resultMessage;
+    }
+
+    public String getErrorMessage() {
+        return errorMessage;
+    }
+
+    public void setErrorMessage(String errorMessage) {
+        this.errorMessage = errorMessage;
+    }
+}
@@ -0,0 +1,574 @@
+package com.alibaba.otter.canal.client.adapter.support;
+
+import java.io.*;
+import java.net.MalformedURLException;
+import java.net.URL;
+import java.net.URLClassLoader;
+import java.nio.file.Paths;
+import java.util.*;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentMap;
+import java.util.regex.Pattern;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * SPI 类加载器
+ *
+ * @author rewerma 2018-8-19 下午11:30:49
+ * @version 1.0.0
+ */
+public class ExtensionLoader<T> {
+
+    private static final Logger                                      logger                     = LoggerFactory
+        .getLogger(ExtensionLoader.class);
+
+    private static final String                                      SERVICES_DIRECTORY         = "META-INF/services/";
+
+    private static final String                                      CANAL_DIRECTORY            = "META-INF/canal/";
+
+    private static final String                                      DEFAULT_CLASSLOADER_POLICY = "internal";
+
+    private static final Pattern                                     NAME_SEPARATOR             = Pattern
+        .compile("\\s*[,]+\\s*");
+
+    private static final ConcurrentMap<Class<?>, ExtensionLoader<?>> EXTENSION_LOADERS          = new ConcurrentHashMap<>();
+
+    private static final ConcurrentMap<Class<?>, Object>             EXTENSION_INSTANCES        = new ConcurrentHashMap<>();
+
+    private final Class<?>                                           type;
+
+    private final String                                             classLoaderPolicy;
+
+    private final ConcurrentMap<Class<?>, String>                    cachedNames                = new ConcurrentHashMap<>();
+
+    private final Holder<Map<String, Class<?>>>                      cachedClasses              = new Holder<>();
+
+    private final ConcurrentMap<String, Holder<Object>>              cachedInstances            = new ConcurrentHashMap<>();
+
+    private String                                                   cachedDefaultName;
+
+    private ConcurrentHashMap<String, IllegalStateException>         exceptions                 = new ConcurrentHashMap<>();
+
+    private static <T> boolean withExtensionAnnotation(Class<T> type) {
+        return type.isAnnotationPresent(SPI.class);
+    }
+
+    public static <T> ExtensionLoader<T> getExtensionLoader(Class<T> type) {
+        return getExtensionLoader(type, DEFAULT_CLASSLOADER_POLICY);
+    }
+
+    @SuppressWarnings("unchecked")
+    public static <T> ExtensionLoader<T> getExtensionLoader(Class<T> type, String classLoaderPolicy) {
+        if (type == null) throw new IllegalArgumentException("Extension type == null");
+        if (!type.isInterface()) {
+            throw new IllegalArgumentException("Extension type(" + type + ") is not interface!");
+        }
+        if (!withExtensionAnnotation(type)) {
+            throw new IllegalArgumentException("Extension type(" + type + ") is not extension, because WITHOUT @"
+                                               + SPI.class.getSimpleName() + " Annotation!");
+        }
+
+        ExtensionLoader<T> loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
+        if (loader == null) {
+            EXTENSION_LOADERS.putIfAbsent(type, new ExtensionLoader<T>(type, classLoaderPolicy));
+            loader = (ExtensionLoader<T>) EXTENSION_LOADERS.get(type);
+        }
+        return loader;
+    }
+
+    private ExtensionLoader(Class<?> type, String classLoaderPolicy){
+        this.type = type;
+        this.classLoaderPolicy = classLoaderPolicy;
+    }
+
+    public String getExtensionName(T extensionInstance) {
+        return getExtensionName(extensionInstance.getClass());
+    }
+
+    public String getExtensionName(Class<?> extensionClass) {
+        return cachedNames.get(extensionClass);
+    }
+
+    public ConcurrentHashMap<String, IllegalStateException> getExceptions() {
+        return exceptions;
+    }
+
+    /**
+     * 返回扩展点实例，如果没有指定的扩展点或是还没加载（即实例化）则返回<code>null</code>注意：此方法不会触发扩展点的加载
+     * <p/>
+     * 一般应该调用{@link #getExtension(String)}方法获得扩展，这个方法会触发扩展点加载
+     *
+     * @see #getExtension(String)
+     */
+    @SuppressWarnings("unchecked")
+    public T getLoadedExtension(String name) {
+        if (name == null || name.length() == 0) throw new IllegalArgumentException("Extension name == null");
+        Holder<Object> holder = cachedInstances.get(name);
+        if (holder == null) {
+            cachedInstances.putIfAbsent(name, new Holder<>());
+            holder = cachedInstances.get(name);
+        }
+        return (T) holder.get();
+    }
+
+    /**
+     * 返回已经加载的扩展点的名字
+     * <p/>
+     * 一般应该调用{@link #getSupportedExtensions()}方法获得扩展，这个方法会返回所有的扩展点
+     *
+     * @see #getSupportedExtensions()
+     */
+    public Set<String> getLoadedExtensions() {
+        return Collections.unmodifiableSet(new TreeSet<>(cachedInstances.keySet()));
+    }
+
+    /**
+     * 返回指定名字的扩展
+     *
+     * @param name
+     * @return
+     */
+    @SuppressWarnings("unchecked")
+    public T getExtension(String name) {
+        if (name == null || name.length() == 0) throw new IllegalArgumentException("Extension name == null");
+        if ("true".equals(name)) {
+            return getDefaultExtension();
+        }
+        Holder<Object> holder = cachedInstances.get(name);
+        if (holder == null) {
+            cachedInstances.putIfAbsent(name, new Holder<>());
+            holder = cachedInstances.get(name);
+        }
+        Object instance = holder.get();
+        if (instance == null) {
+            synchronized (holder) {
+                instance = holder.get();
+                if (instance == null) {
+                    instance = createExtension(name);
+                    holder.set(instance);
+                }
+            }
+        }
+        return (T) instance;
+    }
+
+    /**
+     * 返回缺省的扩展，如果没有设置则返回<code>null</code>
+     */
+    public T getDefaultExtension() {
+        getExtensionClasses();
+        if (null == cachedDefaultName || cachedDefaultName.length() == 0 || "true".equals(cachedDefaultName)) {
+            return null;
+        }
+        return getExtension(cachedDefaultName);
+    }
+
+    public boolean hasExtension(String name) {
+        if (name == null || name.length() == 0) throw new IllegalArgumentException("Extension name == null");
+        try {
+            return getExtensionClass(name) != null;
+        } catch (Throwable t) {
+            return false;
+        }
+    }
+
+    public Set<String> getSupportedExtensions() {
+        Map<String, Class<?>> clazzes = getExtensionClasses();
+        return Collections.unmodifiableSet(new TreeSet<String>(clazzes.keySet()));
+    }
+
+    /**
+     * 返回缺省的扩展点名，如果没有设置缺省则返回<code>null</code>
+     */
+    public String getDefaultExtensionName() {
+        getExtensionClasses();
+        return cachedDefaultName;
+    }
+
+    /**
+     * 编程方式添加新扩展点
+     *
+     * @param name 扩展点名
+     * @param clazz 扩展点类
+     * @throws IllegalStateException 要添加扩展点名已经存在
+     */
+    public void addExtension(String name, Class<?> clazz) {
+        getExtensionClasses(); // load classes
+
+        if (!type.isAssignableFrom(clazz)) {
+            throw new IllegalStateException("Input type " + clazz + "not implement Extension " + type);
+        }
+        if (clazz.isInterface()) {
+            throw new IllegalStateException("Input type " + clazz + "can not be interface!");
+        }
+
+        if (name == null || "".equals(name)) {
+            throw new IllegalStateException("Extension name is blank (Extension " + type + ")!");
+        }
+        if (cachedClasses.get().containsKey(name)) {
+            throw new IllegalStateException("Extension name " + name + " already existed(Extension " + type + ")!");
+        }
+
+        cachedNames.put(clazz, name);
+        cachedClasses.get().put(name, clazz);
+    }
+
+    /**
+     * 编程方式添加替换已有扩展点
+     *
+     * @param name 扩展点名
+     * @param clazz 扩展点类
+     * @throws IllegalStateException 要添加扩展点名已经存在
+     * @deprecated 不推荐应用使用，一般只在测试时可以使用
+     */
+    @Deprecated
+    public void replaceExtension(String name, Class<?> clazz) {
+        getExtensionClasses(); // load classes
+
+        if (!type.isAssignableFrom(clazz)) {
+            throw new IllegalStateException("Input type " + clazz + "not implement Extension " + type);
+        }
+        if (clazz.isInterface()) {
+            throw new IllegalStateException("Input type " + clazz + "can not be interface!");
+        }
+
+        if (name == null || "".equals(name)) {
+            throw new IllegalStateException("Extension name is blank (Extension " + type + ")!");
+        }
+        if (!cachedClasses.get().containsKey(name)) {
+            throw new IllegalStateException("Extension name " + name + " not existed(Extension " + type + ")!");
+        }
+
+        cachedNames.put(clazz, name);
+        cachedClasses.get().put(name, clazz);
+        cachedInstances.remove(name);
+    }
+
+    @SuppressWarnings("unchecked")
+    private T createExtension(String name) {
+        Class<?> clazz = getExtensionClasses().get(name);
+        if (clazz == null) {
+            throw new IllegalStateException("Extension instance(name: " + name + ", class: " + type
+                                            + ")  could not be instantiated: class could not be found");
+        }
+        try {
+            T instance = (T) EXTENSION_INSTANCES.get(clazz);
+            if (instance == null) {
+                EXTENSION_INSTANCES.putIfAbsent(clazz, (T) clazz.newInstance());
+                instance = (T) EXTENSION_INSTANCES.get(clazz);
+            }
+            return instance;
+        } catch (Throwable t) {
+            throw new IllegalStateException("Extension instance(name: " + name + ", class: " + type
+                                            + ")  could not be instantiated: " + t.getMessage(),
+                t);
+        }
+    }
+
+    private Class<?> getExtensionClass(String name) {
+        if (type == null) throw new IllegalArgumentException("Extension type == null");
+        if (name == null) throw new IllegalArgumentException("Extension name == null");
+        Class<?> clazz = getExtensionClasses().get(name);
+        if (clazz == null)
+            throw new IllegalStateException("No such extension \"" + name + "\" for " + type.getName() + "!");
+        return clazz;
+    }
+
+    private Map<String, Class<?>> getExtensionClasses() {
+        Map<String, Class<?>> classes = cachedClasses.get();
+        if (classes == null) {
+            synchronized (cachedClasses) {
+                classes = cachedClasses.get();
+                if (classes == null) {
+                    classes = loadExtensionClasses();
+                    cachedClasses.set(classes);
+                }
+            }
+        }
+        return classes;
+    }
+
+    private String getJarDirectoryPath() {
+        URL url = Thread.currentThread().getContextClassLoader().getResource("");
+        if (url == null) {
+            throw new IllegalStateException("failed to get class loader resource");
+        }
+        String dirtyPath = url.toString();
+        String jarPath = dirtyPath.replaceAll("^.*file:/", ""); // removes
+                                                                // file:/ and
+                                                                // everything
+                                                                // before it
+        jarPath = jarPath.replaceAll("jar!.*", "jar"); // removes everything
+                                                       // after .jar, if .jar
+                                                       // exists in dirtyPath
+        jarPath = jarPath.replaceAll("%20", " "); // necessary if path has
+                                                  // spaces within
+        if (!jarPath.endsWith(".jar")) { // this is needed if you plan to run
+                                         // the app using Spring Tools Suit play
+                                         // button.
+            jarPath = jarPath.replaceAll("/classes/.*", "/classes/");
+        }
+        return Paths.get(jarPath).getParent().toString(); // Paths - from java 8
+    }
+
+    private Map<String, Class<?>> loadExtensionClasses() {
+        final SPI defaultAnnotation = type.getAnnotation(SPI.class);
+        if (defaultAnnotation != null) {
+            String value = defaultAnnotation.value();
+            if ((value = value.trim()).length() > 0) {
+                String[] names = NAME_SEPARATOR.split(value);
+                if (names.length > 1) {
+                    throw new IllegalStateException("more than 1 default extension name on extension " + type.getName()
+                                                    + ": " + Arrays.toString(names));
+                }
+                if (names.length == 1) cachedDefaultName = names[0];
+            }
+        }
+
+        Map<String, Class<?>> extensionClasses = new HashMap<String, Class<?>>();
+
+        // 1. lib folder，customized extension classLoader （jar_dir/lib）
+        String dir = File.separator + this.getJarDirectoryPath() + File.separator + "lib";
+        logger.info("extension classpath dir: " + dir);
+        File externalLibDir = new File(dir);
+        if (!externalLibDir.exists()) {
+            externalLibDir = new File(
+                File.separator + this.getJarDirectoryPath() + File.separator + "canal_client" + File.separator + "lib");
+        }
+        if (externalLibDir.exists()) {
+            File[] files = externalLibDir.listFiles(new FilenameFilter() {
+
+                @Override
+                public boolean accept(File dir, String name) {
+                    return name.endsWith(".jar");
+                }
+            });
+            if (files != null) {
+                for (File f : files) {
+                    URL url = null;
+                    try {
+                        url = f.toURI().toURL();
+                    } catch (MalformedURLException e) {
+                        throw new RuntimeException("load extension jar failed!", e);
+                    }
+
+                    ClassLoader parent = Thread.currentThread().getContextClassLoader();
+                    URLClassLoader localClassLoader;
+                    if (classLoaderPolicy == null || "".equals(classLoaderPolicy)
+                        || DEFAULT_CLASSLOADER_POLICY.equalsIgnoreCase(classLoaderPolicy)) {
+                        localClassLoader = new URLClassLoader(new URL[] { url }, parent) {
+
+                            @Override
+                            public Class<?> loadClass(String name) throws ClassNotFoundException {
+                                Class<?> c = findLoadedClass(name);
+                                if (c != null) {
+                                    return c;
+                                }
+
+                                if (name.startsWith("java.") || name.startsWith("org.slf4j.")
+                                    || name.startsWith("org.apache.logging")
+                                    || name.startsWith("org.apache.commons.logging.")) {
+                                    // || name.startsWith("org.apache.hadoop."))
+                                    // {
+                                    c = super.loadClass(name);
+                                }
+                                if (c != null) return c;
+
+                                try {
+                                    // 先加载jar内的class，可避免jar冲突
+                                    c = findClass(name);
+                                } catch (ClassNotFoundException e) {
+                                    c = null;
+                                }
+                                if (c != null) {
+                                    return c;
+                                }
+
+                                return super.loadClass(name);
+                            }
+
+                            @Override
+                            public Enumeration<URL> getResources(String name) throws IOException {
+                                @SuppressWarnings("unchecked")
+                                Enumeration<URL>[] tmp = (Enumeration<URL>[]) new Enumeration<?>[2];
+
+                                tmp[0] = findResources(name); // local class
+                                                              // path first
+                                // tmp[1] = super.getResources(name);
+
+                                return new CompoundEnumeration<>(tmp);
+                            }
+                        };
+                    } else {
+                        localClassLoader = new URLClassLoader(new URL[] { url }, parent);
+                    }
+
+                    loadFile(extensionClasses, CANAL_DIRECTORY, localClassLoader);
+                    loadFile(extensionClasses, SERVICES_DIRECTORY, localClassLoader);
+                }
+            }
+        }
+        // 2. load inner extension class with default classLoader
+        ClassLoader classLoader = findClassLoader();
+        loadFile(extensionClasses, CANAL_DIRECTORY, classLoader);
+        loadFile(extensionClasses, SERVICES_DIRECTORY, classLoader);
+
+        return extensionClasses;
+    }
+
+    public static class CompoundEnumeration<E> implements Enumeration<E> {
+
+        private Enumeration<E>[] enums;
+        private int              index = 0;
+
+        public CompoundEnumeration(Enumeration<E>[] enums){
+            this.enums = enums;
+        }
+
+        private boolean next() {
+            while (this.index < this.enums.length) {
+                if (this.enums[this.index] != null && this.enums[this.index].hasMoreElements()) {
+                    return true;
+                }
+
+                ++this.index;
+            }
+
+            return false;
+        }
+
+        public boolean hasMoreElements() {
+            return this.next();
+        }
+
+        public E nextElement() {
+            if (!this.next()) {
+                throw new NoSuchElementException();
+            } else {
+                return this.enums[this.index].nextElement();
+            }
+        }
+    }
+
+    private void loadFile(Map<String, Class<?>> extensionClasses, String dir, ClassLoader classLoader) {
+        String fileName = dir + type.getName();
+        try {
+            Enumeration<URL> urls;
+            if (classLoader != null) {
+                urls = classLoader.getResources(fileName);
+            } else {
+                urls = ClassLoader.getSystemResources(fileName);
+            }
+            if (urls != null) {
+                while (urls.hasMoreElements()) {
+                    URL url = urls.nextElement();
+                    try {
+                        BufferedReader reader = null;
+                        try {
+                            reader = new BufferedReader(new InputStreamReader(url.openStream(), "utf-8"));
+                            String line = null;
+                            while ((line = reader.readLine()) != null) {
+                                final int ci = line.indexOf('#');
+                                if (ci >= 0) line = line.substring(0, ci);
+                                line = line.trim();
+                                if (line.length() > 0) {
+                                    try {
+                                        String name = null;
+                                        int i = line.indexOf('=');
+                                        if (i > 0) {
+                                            name = line.substring(0, i).trim();
+                                            line = line.substring(i + 1).trim();
+                                        }
+                                        if (line.length() > 0) {
+                                            Class<?> clazz = classLoader.loadClass(line);
+                                            // Class<?> clazz =
+                                            // Class.forName(line, true,
+                                            // classLoader);
+                                            if (!type.isAssignableFrom(clazz)) {
+                                                throw new IllegalStateException(
+                                                    "Error when load extension class(interface: " + type
+                                                                                + ", class line: " + clazz.getName()
+                                                                                + "), class " + clazz.getName()
+                                                                                + "is not subtype of interface.");
+                                            } else {
+                                                try {
+                                                    clazz.getConstructor(type);
+                                                } catch (NoSuchMethodException e) {
+                                                    clazz.getConstructor();
+                                                    String[] names = NAME_SEPARATOR.split(name);
+                                                    if (names != null && names.length > 0) {
+                                                        for (String n : names) {
+                                                            if (!cachedNames.containsKey(clazz)) {
+                                                                cachedNames.put(clazz, n);
+                                                            }
+                                                            Class<?> c = extensionClasses.get(n);
+                                                            if (c == null) {
+                                                                extensionClasses.put(n, clazz);
+                                                            } else if (c != clazz) {
+                                                                cachedNames.remove(clazz);
+                                                                throw new IllegalStateException(
+                                                                    "Duplicate extension " + type.getName() + " name "
+                                                                                                + n + " on "
+                                                                                                + c.getName() + " and "
+                                                                                                + clazz.getName());
+                                                            }
+                                                        }
+                                                    }
+                                                }
+                                            }
+                                        }
+                                    } catch (Throwable t) {
+                                        IllegalStateException e = new IllegalStateException(
+                                            "Failed to load extension class(interface: " + type + ", class line: "
+                                                                                            + line + ") in " + url
+                                                                                            + ", cause: "
+                                                                                            + t.getMessage(),
+                                            t);
+                                        exceptions.put(line, e);
+                                    }
+                                }
+                            } // end of while read lines
+                        } finally {
+                            if (reader != null) {
+                                reader.close();
+                            }
+                        }
+                    } catch (Throwable t) {
+                        logger.error("Exception when load extension class(interface: " + type + ", class file: " + url
+                                     + ") in " + url,
+                            t);
+                    }
+                } // end of while urls
+            }
+        } catch (Throwable t) {
+            logger.error(
+                "Exception when load extension class(interface: " + type + ", description file: " + fileName + ").",
+                t);
+        }
+    }
+
+    private static ClassLoader findClassLoader() {
+        return ExtensionLoader.class.getClassLoader();
+    }
+
+    @Override
+    public String toString() {
+        return this.getClass().getName() + "[" + type.getName() + "]";
+    }
+
+    private static class Holder<T> {
+
+        private volatile T value;
+
+        private void set(T value) {
+            this.value = value;
+        }
+
+        private T get() {
+            return value;
+        }
+
+    }
+}
@@ -0,0 +1,149 @@
+package com.alibaba.otter.canal.client.adapter.support;
+
+import java.math.BigDecimal;
+import java.math.BigInteger;
+import java.sql.*;
+
+import org.joda.time.DateTime;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * 类型转换工具类
+ *
+ * @author rewerma 2018-8-19 下午06:14:23
+ * @version 1.0.0
+ */
+public class JdbcTypeUtil {
+
+    private static Logger logger = LoggerFactory.getLogger(JdbcTypeUtil.class);
+
+    public static Object getRSData(ResultSet rs, String columnName, int jdbcType) throws SQLException {
+        if (jdbcType == Types.BIT || jdbcType == Types.BOOLEAN) {
+            return rs.getByte(columnName);
+        } else {
+            return rs.getObject(columnName);
+        }
+    }
+
+    public static Class<?> jdbcType2javaType(int jdbcType) {
+        switch (jdbcType) {
+            case Types.BIT:
+            case Types.BOOLEAN:
+                // return Boolean.class;
+            case Types.TINYINT:
+                return Byte.TYPE;
+            case Types.SMALLINT:
+                return Short.class;
+            case Types.INTEGER:
+                return Integer.class;
+            case Types.BIGINT:
+                return Long.class;
+            case Types.DECIMAL:
+            case Types.NUMERIC:
+                return BigDecimal.class;
+            case Types.REAL:
+                return Float.class;
+            case Types.FLOAT:
+            case Types.DOUBLE:
+                return Double.class;
+            case Types.CHAR:
+            case Types.VARCHAR:
+            case Types.LONGVARCHAR:
+                return String.class;
+            case Types.BINARY:
+            case Types.VARBINARY:
+            case Types.LONGVARBINARY:
+            case Types.BLOB:
+                return byte[].class;
+            case Types.DATE:
+                return java.sql.Date.class;
+            case Types.TIME:
+                return Time.class;
+            case Types.TIMESTAMP:
+                return Timestamp.class;
+            default:
+                return String.class;
+        }
+    }
+
+    public static Object typeConvert(String columnName, String value, int sqlType, String mysqlType) {
+        if (value == null || value.equals("")) {
+            return null;
+        }
+
+        try {
+            Object res;
+            switch (sqlType) {
+                case Types.INTEGER:
+                    res = Integer.parseInt(value);
+                    break;
+                case Types.SMALLINT:
+                    res = Short.parseShort(value);
+                    break;
+                case Types.BIT:
+                case Types.TINYINT:
+                    res = Byte.parseByte(value);
+                    break;
+                case Types.BIGINT:
+                    if (mysqlType.startsWith("bigint") && mysqlType.endsWith("unsigned")) {
+                        res = new BigInteger(value);
+                    } else {
+                        res = Long.parseLong(value);
+                    }
+                    break;
+                // case Types.BIT:
+                case Types.BOOLEAN:
+                    res = !"0".equals(value);
+                    break;
+                case Types.DOUBLE:
+                case Types.FLOAT:
+                    res = Double.parseDouble(value);
+                    break;
+                case Types.REAL:
+                    res = Float.parseFloat(value);
+                    break;
+                case Types.DECIMAL:
+                case Types.NUMERIC:
+                    res = new BigDecimal(value);
+                    break;
+                case Types.BINARY:
+                case Types.VARBINARY:
+                case Types.LONGVARBINARY:
+                case Types.BLOB:
+                    res = value.getBytes("ISO-8859-1");
+                    break;
+                case Types.DATE:
+                    if (!value.startsWith("0000-00-00")) {
+                        value = value.trim().replace(" ", "T");
+                        DateTime dt = new DateTime(value);
+                        res = new Date(dt.toDate().getTime());
+                    } else {
+                        res = null;
+                    }
+                    break;
+                case Types.TIME:
+                    value = "T" + value;
+                    DateTime dt = new DateTime(value);
+                    res = new Time(dt.toDate().getTime());
+                    break;
+                case Types.TIMESTAMP:
+                    if (!value.startsWith("0000-00-00")) {
+                        value = value.trim().replace(" ", "T");
+                        dt = new DateTime(value);
+                        res = new Timestamp(dt.toDate().getTime());
+                    } else {
+                        res = null;
+                    }
+                    break;
+                case Types.CLOB:
+                default:
+                    res = value;
+            }
+            return res;
+        } catch (Exception e) {
+            logger.error("table: {} column: {}, failed convert type {} to {}", columnName, value, sqlType);
+            return value;
+        }
+    }
+}
@@ -0,0 +1,173 @@
+package com.alibaba.otter.canal.client.adapter.support;
+
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import com.alibaba.otter.canal.protocol.CanalEntry;
+import com.alibaba.otter.canal.protocol.FlatMessage;
+import com.alibaba.otter.canal.protocol.Message;
+
+/**
+ * Message对象解析工具类
+ *
+ * @author rewerma 2018-8-19 下午06:14:23
+ * @version 1.0.0
+ */
+public class MessageUtil {
+
+    public static void parse4Dml(String destination, Message message, Consumer<Dml> consumer) {
+        if (message == null) {
+            return;
+        }
+        List<CanalEntry.Entry> entries = message.getEntries();
+
+        for (CanalEntry.Entry entry : entries) {
+            if (entry.getEntryType() == CanalEntry.EntryType.TRANSACTIONBEGIN
+                || entry.getEntryType() == CanalEntry.EntryType.TRANSACTIONEND) {
+                continue;
+            }
+
+            CanalEntry.RowChange rowChange;
+            try {
+                rowChange = CanalEntry.RowChange.parseFrom(entry.getStoreValue());
+            } catch (Exception e) {
+                throw new RuntimeException("ERROR ## parser of eromanga-event has an error , data:" + entry.toString(),
+                    e);
+            }
+
+            CanalEntry.EventType eventType = rowChange.getEventType();
+
+            final Dml dml = new Dml();
+            dml.setDestination(destination);
+            dml.setDatabase(entry.getHeader().getSchemaName());
+            dml.setTable(entry.getHeader().getTableName());
+            dml.setType(eventType.toString());
+            dml.setEs(entry.getHeader().getExecuteTime());
+            dml.setTs(System.currentTimeMillis());
+            dml.setSql(rowChange.getSql());
+            List<Map<String, Object>> data = new ArrayList<>();
+            List<Map<String, Object>> old = new ArrayList<>();
+
+            if (!rowChange.getIsDdl()) {
+                Set<String> updateSet = new HashSet<>();
+                for (CanalEntry.RowData rowData : rowChange.getRowDatasList()) {
+                    if (eventType != CanalEntry.EventType.INSERT && eventType != CanalEntry.EventType.UPDATE
+                        && eventType != CanalEntry.EventType.DELETE) {
+                        continue;
+                    }
+
+                    Map<String, Object> row = new LinkedHashMap<>();
+                    List<CanalEntry.Column> columns;
+
+                    if (eventType == CanalEntry.EventType.DELETE) {
+                        columns = rowData.getBeforeColumnsList();
+                    } else {
+                        columns = rowData.getAfterColumnsList();
+                    }
+
+                    for (CanalEntry.Column column : columns) {
+                        row.put(column.getName(),
+                            JdbcTypeUtil.typeConvert(column.getName(),
+                                column.getValue(),
+                                column.getSqlType(),
+                                column.getMysqlType()));
+                        // 获取update为true的字段
+                        if (column.getUpdated()) {
+                            updateSet.add(column.getName());
+                        }
+                    }
+                    if (!row.isEmpty()) {
+                        data.add(row);
+                    }
+
+                    if (eventType == CanalEntry.EventType.UPDATE) {
+                        Map<String, Object> rowOld = new LinkedHashMap<>();
+                        for (CanalEntry.Column column : rowData.getBeforeColumnsList()) {
+                            if (updateSet.contains(column.getName())) {
+                                rowOld.put(column.getName(),
+                                    JdbcTypeUtil.typeConvert(column.getName(),
+                                        column.getValue(),
+                                        column.getSqlType(),
+                                        column.getMysqlType()));
+                            }
+                        }
+                        // update操作将记录修改前的值
+                        if (!rowOld.isEmpty()) {
+                            old.add(rowOld);
+                        }
+                    }
+                }
+                if (!data.isEmpty()) {
+                    dml.setData(data);
+                }
+                if (!old.isEmpty()) {
+                    dml.setOld(old);
+                }
+            }
+
+            consumer.accept(dml);
+        }
+    }
+
+    public static Dml flatMessage2Dml(String destination, FlatMessage flatMessage) {
+        if (flatMessage == null) {
+            return null;
+        }
+        Dml dml = new Dml();
+        dml.setDestination(destination);
+        dml.setDatabase(flatMessage.getDatabase());
+        dml.setTable(flatMessage.getTable());
+        dml.setType(flatMessage.getType());
+        dml.setTs(flatMessage.getTs());
+        dml.setEs(flatMessage.getEs());
+        dml.setSql(flatMessage.getSql());
+        if (flatMessage.getSqlType() == null || flatMessage.getMysqlType() == null) {
+            throw new RuntimeException("SqlType or mysqlType is null");
+        }
+        List<Map<String, String>> data = flatMessage.getData();
+        if (data != null) {
+            dml.setData(changeRows(data, flatMessage.getSqlType(), flatMessage.getMysqlType()));
+        }
+        List<Map<String, String>> old = flatMessage.getOld();
+        if (old != null) {
+            dml.setOld(changeRows(old, flatMessage.getSqlType(), flatMessage.getMysqlType()));
+        }
+        return dml;
+    }
+
+    private static List<Map<String, Object>> changeRows(List<Map<String, String>> rows, Map<String, Integer> sqlTypes,
+                                                        Map<String, String> mysqlTypes) {
+        List<Map<String, Object>> result = new ArrayList<>();
+        for (Map<String, String> row : rows) {
+            Map<String, Object> resultRow = new LinkedHashMap<>();
+            for (Map.Entry<String, String> entry : row.entrySet()) {
+                String columnName = entry.getKey();
+                String columnValue = entry.getValue();
+
+                Integer sqlType = sqlTypes.get(columnName);
+                if (sqlType == null) {
+                    continue;
+                }
+
+                String mysqlType = mysqlTypes.get(columnName);
+                if (mysqlType == null) {
+                    continue;
+                }
+
+                Object finalValue = JdbcTypeUtil.typeConvert(columnName, columnValue, sqlType, mysqlType);
+                resultRow.put(columnName, finalValue);
+            }
+            result.add(resultRow);
+        }
+        return result;
+    }
+
+    public interface Consumer<T> {
+
+        void accept(T t);
+    }
+}
@@ -0,0 +1,52 @@
+package com.alibaba.otter.canal.client.adapter.support;
+
+import java.util.Map;
+
+/**
+ * 外部适配器配置信息类
+ *
+ * @author rewerma 2018-8-18 下午10:15:12
+ * @version 1.0.0
+ */
+public class OuterAdapterConfig {
+
+    private String              name;       // 适配器名称, 如: logger, hbase, es
+
+    private String              hosts;      // 适配器内部的地址, 比如对应es该参数可以填写es的server地址
+
+    private String              zkHosts;    // 适配器内部的ZK地址, 比如对应HBase该参数可以填写HBase对应的ZK地址
+
+    private Map<String, String> properties; // 其余参数, 可填写适配器中的所需的配置信息
+
+    public String getName() {
+        return name;
+    }
+
+    public void setName(String name) {
+        this.name = name;
+    }
+
+    public String getHosts() {
+        return hosts;
+    }
+
+    public void setHosts(String hosts) {
+        this.hosts = hosts;
+    }
+
+    public Map<String, String> getProperties() {
+        return properties;
+    }
+
+    public void setProperties(Map<String, String> properties) {
+        this.properties = properties;
+    }
+
+    public String getZkHosts() {
+        return zkHosts;
+    }
+
+    public void setZkHosts(String zkHosts) {
+        this.zkHosts = zkHosts;
+    }
+}
@@ -0,0 +1,57 @@
+package com.alibaba.otter.canal.client.adapter.support;
+
+import java.io.Serializable;
+import java.util.Date;
+
+/**
+ * 用于rest的结果返回对象
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+public class Result implements Serializable {
+
+    private static final long serialVersionUID = -3276409502352405716L;
+    public Integer code = 20000;
+    public Object  data;
+    public String  message;
+    public Date    sysTime;
+
+    public static Result createSuccess(String message) {
+        Result result = new Result();
+        result.setMessage(message);
+        return result;
+    }
+
+    public Integer getCode() {
+        return code;
+    }
+
+    public void setCode(Integer code) {
+        this.code = code;
+    }
+
+    public Object getData() {
+        return data;
+    }
+
+    public void setData(Object data) {
+        this.data = data;
+    }
+
+    public String getMessage() {
+        return message;
+    }
+
+    public void setMessage(String message) {
+        this.message = message;
+    }
+
+    public Date getSysTime() {
+        return sysTime;
+    }
+
+    public void setSysTime(Date sysTime) {
+        this.sysTime = sysTime;
+    }
+}
@@ -0,0 +1,22 @@
+package com.alibaba.otter.canal.client.adapter.support;
+
+import java.lang.annotation.Documented;
+import java.lang.annotation.ElementType;
+import java.lang.annotation.Retention;
+import java.lang.annotation.RetentionPolicy;
+import java.lang.annotation.Target;
+
+/**
+ * SPI装载器注解
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+@Documented
+@Retention(RetentionPolicy.RUNTIME)
+@Target({ ElementType.TYPE })
+public @interface SPI {
+
+    // Default SPI name
+    String value() default "";
+}
@@ -0,0 +1,68 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <parent>
+        <artifactId>canal.client-adapter</artifactId>
+        <groupId>com.alibaba.otter</groupId>
+        <version>1.1.1</version>
+    </parent>
+    <modelVersion>4.0.0</modelVersion>
+    <groupId>com.alibaba.otter</groupId>
+    <artifactId>client-adapter.hbase</artifactId>
+    <packaging>jar</packaging>
+    <name>canal client adapter hbase module for otter ${project.version}</name>
+    <dependencies>
+        <dependency>
+            <groupId>com.alibaba.otter</groupId>
+            <artifactId>client-adapter.common</artifactId>
+            <version>${project.version}</version>
+            <scope>provided</scope>
+        </dependency>
+        <dependency>
+            <groupId>org.yaml</groupId>
+            <artifactId>snakeyaml</artifactId>
+            <version>1.19</version>
+            <scope>provided</scope>
+        </dependency>
+        <dependency>
+            <groupId>org.apache.hbase</groupId>
+            <artifactId>hbase-client</artifactId>
+            <version>1.1.2</version>
+            <exclusions>
+                <exclusion>
+                    <groupId>org.slf4j</groupId>
+                    <artifactId>slf4j-log4j12</artifactId>
+                </exclusion>
+            </exclusions>
+        </dependency>
+        <dependency>
+            <groupId>com.google.guava</groupId>
+            <artifactId>guava</artifactId>
+            <version>12.0.1</version>
+        </dependency>
+    </dependencies>
+
+    <build>
+        <plugins>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-assembly-plugin</artifactId>
+                <version>2.4</version>
+                <configuration>
+                    <descriptorRefs>
+                        <descriptorRef>jar-with-dependencies</descriptorRef>
+                    </descriptorRefs>
+                </configuration>
+                <executions>
+                    <execution>
+                        <id>make-assembly</id>
+                        <phase>package</phase>
+                        <goals>
+                            <goal>single</goal>
+                        </goals>
+                    </execution>
+                </executions>
+            </plugin>
+        </plugins>
+    </build>
+
+</project>
@@ -0,0 +1,177 @@
+package com.alibaba.otter.canal.client.adapter.hbase;
+
+import java.io.IOException;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+
+import javax.sql.DataSource;
+
+import org.apache.commons.lang.StringUtils;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.hbase.HBaseConfiguration;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.*;
+import org.apache.hadoop.hbase.client.Result;
+import org.apache.hadoop.hbase.filter.FirstKeyOnlyFilter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.alibaba.otter.canal.client.adapter.OuterAdapter;
+import com.alibaba.otter.canal.client.adapter.hbase.config.MappingConfig;
+import com.alibaba.otter.canal.client.adapter.hbase.config.MappingConfigLoader;
+import com.alibaba.otter.canal.client.adapter.hbase.service.HbaseEtlService;
+import com.alibaba.otter.canal.client.adapter.hbase.service.HbaseSyncService;
+import com.alibaba.otter.canal.client.adapter.hbase.support.HbaseTemplate;
+import com.alibaba.otter.canal.client.adapter.support.*;
+
+/**
+ * HBase外部适配器
+ *
+ * @author machengyuan 2018-8-21 下午8:45:38
+ * @version 1.0.0
+ */
+@SPI("hbase")
+public class HbaseAdapter implements OuterAdapter {
+
+    private static Logger                              logger             = LoggerFactory.getLogger(HbaseAdapter.class);
+
+    private static volatile Map<String, MappingConfig> hbaseMapping       = null;                                       // 文件名对应配置
+    private static volatile Map<String, MappingConfig> mappingConfigCache = null;                                       // 库名-表名对应配置
+
+    private Connection                                 conn;
+    private HbaseSyncService                           hbaseSyncService;
+    private HbaseTemplate                              hbaseTemplate;
+
+    @Override
+    public void init(OuterAdapterConfig configuration) {
+        try {
+            if (mappingConfigCache == null) {
+                synchronized (MappingConfig.class) {
+                    if (mappingConfigCache == null) {
+                        hbaseMapping = MappingConfigLoader.load();
+                        mappingConfigCache = new HashMap<>();
+                        for (MappingConfig mappingConfig : hbaseMapping.values()) {
+                            mappingConfigCache.put(StringUtils.trimToEmpty(mappingConfig.getHbaseMapping().getDestination())
+                                                   + "." + mappingConfig.getHbaseMapping().getDatabase() + "."
+                                                   + mappingConfig.getHbaseMapping().getTable(),
+                                mappingConfig);
+                        }
+                    }
+                }
+            }
+
+            Map<String, String> propertites = configuration.getProperties();
+
+            Configuration hbaseConfig = HBaseConfiguration.create();
+            propertites.forEach(hbaseConfig::set);
+            conn = ConnectionFactory.createConnection(hbaseConfig);
+            hbaseTemplate = new HbaseTemplate(conn);
+            hbaseSyncService = new HbaseSyncService(hbaseTemplate);
+        } catch (Exception e) {
+            throw new RuntimeException(e);
+        }
+    }
+
+    @Override
+    public void sync(Dml dml) {
+        if (dml == null) {
+            return;
+        }
+        String destination = StringUtils.trimToEmpty(dml.getDestination());
+        String database = dml.getDatabase();
+        String table = dml.getTable();
+        MappingConfig config = mappingConfigCache.get(destination + "." + database + "." + table);
+        hbaseSyncService.sync(config, dml);
+    }
+
+    @Override
+    public EtlResult etl(String task, List<String> params) {
+        EtlResult etlResult = new EtlResult();
+        MappingConfig config = hbaseMapping.get(task);
+        if (config != null) {
+            DataSource dataSource = DatasourceConfig.DATA_SOURCES.get(config.getDataSourceKey());
+            if (dataSource != null) {
+                return HbaseEtlService.importData(dataSource, hbaseTemplate, config, params);
+            } else {
+                etlResult.setSucceeded(false);
+                etlResult.setErrorMessage("DataSource not found");
+                return etlResult;
+            }
+        } else {
+            DataSource dataSource = DatasourceConfig.DATA_SOURCES.get(task);
+            if (dataSource != null) {
+                StringBuilder resultMsg = new StringBuilder();
+                boolean resSucc = true;
+                // ds不为空说明传入的是datasourceKey
+                for (MappingConfig configTmp : hbaseMapping.values()) {
+                    // 取所有的datasourceKey为task的配置
+                    if (configTmp.getDataSourceKey().equals(task)) {
+                        EtlResult etlRes = HbaseEtlService.importData(dataSource, hbaseTemplate, configTmp, params);
+                        if (!etlRes.getSucceeded()) {
+                            resSucc = false;
+                            resultMsg.append(etlRes.getErrorMessage()).append("\n");
+                        } else {
+                            resultMsg.append(etlRes.getResultMessage()).append("\n");
+                        }
+                    }
+                }
+                if (resultMsg.length() > 0) {
+                    etlResult.setSucceeded(resSucc);
+                    if (resSucc) {
+                        etlResult.setResultMessage(resultMsg.toString());
+                    } else {
+                        etlResult.setErrorMessage(resultMsg.toString());
+                    }
+                    return etlResult;
+                }
+            }
+        }
+        etlResult.setSucceeded(false);
+        etlResult.setErrorMessage("Task not found");
+        return etlResult;
+    }
+
+    @Override
+    public Map<String, Object> count(String task) {
+        MappingConfig config = hbaseMapping.get(task);
+        String hbaseTable = config.getHbaseMapping().getHbaseTable();
+        long rowCount = 0L;
+        try {
+            HTable table = (HTable) conn.getTable(TableName.valueOf(hbaseTable));
+            Scan scan = new Scan();
+            scan.setFilter(new FirstKeyOnlyFilter());
+            ResultScanner resultScanner = table.getScanner(scan);
+            for (Result result : resultScanner) {
+                rowCount += result.size();
+            }
+        } catch (IOException e) {
+            logger.error(e.getMessage(), e);
+        }
+        Map<String, Object> res = new LinkedHashMap<>();
+        res.put("hbaseTable", hbaseTable);
+        res.put("count", rowCount);
+        return res;
+    }
+
+    @Override
+    public void destroy() {
+        if (conn != null) {
+            try {
+                conn.close();
+            } catch (IOException e) {
+                throw new RuntimeException(e);
+            }
+        }
+    }
+
+    @Override
+    public String getDestination(String task) {
+        MappingConfig config = hbaseMapping.get(task);
+        if (config != null && config.getHbaseMapping() != null) {
+            return config.getHbaseMapping().getDestination();
+        }
+        return null;
+    }
+}
@@ -0,0 +1,365 @@
+package com.alibaba.otter.canal.client.adapter.hbase.config;
+
+import java.util.*;
+
+/**
+ * HBase表映射配置
+ *
+ * @author rewerma 2018-8-21 下午06:45:49
+ * @version 1.0.0
+ */
+public class MappingConfig {
+
+    private String       dataSourceKey; // 数据源key
+
+    private HbaseMapping hbaseMapping;  // hbase映射配置
+
+    public String getDataSourceKey() {
+        return dataSourceKey;
+    }
+
+    public void setDataSourceKey(String dataSourceKey) {
+        this.dataSourceKey = dataSourceKey;
+    }
+
+    public HbaseMapping getHbaseMapping() {
+        return hbaseMapping;
+    }
+
+    public void setHbaseMapping(HbaseMapping hbaseMapping) {
+        this.hbaseMapping = hbaseMapping;
+    }
+
+    public void validate() {
+        if (hbaseMapping.database == null || hbaseMapping.database.isEmpty()) {
+            throw new NullPointerException("hbaseMapping.database");
+        }
+        if (hbaseMapping.table == null || hbaseMapping.table.isEmpty()) {
+            throw new NullPointerException("hbaseMapping.table");
+        }
+        if (hbaseMapping.hbaseTable == null || hbaseMapping.hbaseTable.isEmpty()) {
+            throw new NullPointerException("hbaseMapping.hbaseTable");
+        }
+        if (hbaseMapping.mode == null) {
+            throw new NullPointerException("hbaseMapping.mode");
+        }
+        if (hbaseMapping.rowKey != null && hbaseMapping.rowKeyColumn != null) {
+            throw new RuntimeException("已配置了复合主键作为RowKey，无需再指定RowKey列");
+        }
+    }
+
+    @Override
+    public boolean equals(Object o) {
+        if (this == o) return true;
+        if (o == null || getClass() != o.getClass()) return false;
+
+        MappingConfig config = (MappingConfig) o;
+
+        return hbaseMapping != null ? hbaseMapping.equals(config.hbaseMapping) : config.hbaseMapping == null;
+    }
+
+    @Override
+    public int hashCode() {
+        return hbaseMapping != null ? hbaseMapping.hashCode() : 0;
+    }
+
+    public static class ColumnItem {
+
+        private boolean isRowKey = false;
+        private String  column;
+        private String  family;
+        private String  qualifier;
+        private String  type;
+
+        public boolean isRowKey() {
+            return isRowKey;
+        }
+
+        public void setRowKey(boolean rowKey) {
+            isRowKey = rowKey;
+        }
+
+        public String getColumn() {
+            return column;
+        }
+
+        public void setColumn(String column) {
+            this.column = column;
+        }
+
+        public String getFamily() {
+            return family;
+        }
+
+        public void setFamily(String family) {
+            this.family = family;
+        }
+
+        public String getQualifier() {
+            return qualifier;
+        }
+
+        public void setQualifier(String qualifier) {
+            this.qualifier = qualifier;
+        }
+
+        public String getType() {
+            return type;
+        }
+
+        public void setType(String type) {
+            this.type = type;
+        }
+
+        @Override
+        public boolean equals(Object o) {
+            if (this == o) return true;
+            if (o == null || getClass() != o.getClass()) return false;
+            ColumnItem that = (ColumnItem) o;
+            return Objects.equals(column, that.column);
+        }
+
+        @Override
+        public int hashCode() {
+
+            return Objects.hash(column);
+        }
+    }
+
+    public enum Mode {
+                      STRING("STRING"), NATIVE("NATIVE"), PHOENIX("PHOENIX");
+
+        private String type;
+
+        public String getType() {
+            return type;
+        }
+
+        Mode(String type){
+            this.type = type;
+        }
+    }
+
+    public static class HbaseMapping {
+
+        private Mode                    mode               = Mode.STRING;           // hbase默认转换格式
+        private String                  destination;                                // canal实例或MQ的topic
+        private String                  database;                                   // 数据库名或schema名
+        private String                  table;                                      // 表面名
+        private String                  hbaseTable;                                 // hbase表名
+        private String                  family             = "CF";                  // 默认统一column family
+        private boolean                 uppercaseQualifier = true;                  // 是否转大写
+        private boolean                 autoCreateTable    = false;                 // 同步时HBase中表不存在的情况下自动建表
+        private String                  rowKey;                                     // 指定复合主键为rowKey
+        private Map<String, String>     columns;                                    // 字段映射
+        private List<String>            excludeColumns;                             // 不映射的字段
+        private ColumnItem              rowKeyColumn;                               // rowKey字段
+        private String                  etlCondition;                               // etl条件sql
+
+        private Map<String, ColumnItem> columnItems        = new LinkedHashMap<>(); // 转换后的字段映射列表
+        private Set<String>             families           = new LinkedHashSet<>(); // column family列表
+        private int                     readBatch          = 5000;
+        private int                     commitBatch        = 5000;                  // etl等批量提交大小
+
+        public Mode getMode() {
+            return mode;
+        }
+
+        public void setMode(Mode mode) {
+            this.mode = mode;
+        }
+
+        public String getDestination() {
+            return destination;
+        }
+
+        public void setDestination(String destination) {
+            this.destination = destination;
+        }
+
+        public String getDatabase() {
+            return database;
+        }
+
+        public void setDatabase(String database) {
+            this.database = database;
+        }
+
+        public String getTable() {
+            return table;
+        }
+
+        public void setTable(String table) {
+            this.table = table;
+        }
+
+        public String getHbaseTable() {
+            return hbaseTable;
+        }
+
+        public void setHbaseTable(String hbaseTable) {
+            this.hbaseTable = hbaseTable;
+        }
+
+        public Map<String, String> getColumns() {
+            return columns;
+        }
+
+        public boolean isAutoCreateTable() {
+            return autoCreateTable;
+        }
+
+        public void setAutoCreateTable(boolean autoCreateTable) {
+            this.autoCreateTable = autoCreateTable;
+        }
+
+        public int getReadBatch() {
+            return readBatch;
+        }
+
+        public void setReadBatch(int readBatch) {
+            this.readBatch = readBatch;
+        }
+
+        public int getCommitBatch() {
+            return commitBatch;
+        }
+
+        public void setCommitBatch(int commitBatch) {
+            this.commitBatch = commitBatch;
+        }
+
+        public String getRowKey() {
+            return rowKey;
+        }
+
+        public void setRowKey(String rowKey) {
+            this.rowKey = rowKey;
+        }
+
+        public String getEtlCondition() {
+            return etlCondition;
+        }
+
+        public void setEtlCondition(String etlCondition) {
+            this.etlCondition = etlCondition;
+        }
+
+        public void setColumns(Map<String, String> columns) {
+            this.columns = columns;
+
+            if (columns != null) {
+                for (Map.Entry<String, String> columnField : columns.entrySet()) {
+                    String field = columnField.getValue();
+                    String type = null;
+                    if (field != null) {
+                        // 解析类型
+                        int i = field.indexOf("$");
+                        if (i > -1) {
+                            type = field.substring(i + 1);
+                            field = field.substring(0, i);
+                        }
+                    }
+                    ColumnItem columnItem = new ColumnItem();
+                    columnItem.setColumn(columnField.getKey());
+                    columnItem.setType(type);
+                    if ("rowKey".equalsIgnoreCase(field)) {
+                        columnItem.setRowKey(true);
+                        rowKeyColumn = columnItem;
+                    } else {
+                        if (field == null || field.equals("")) {
+                            columnItem.setFamily(family);
+                            columnItem.setQualifier(columnField.getKey());
+                        } else {
+                            int len = field.indexOf(":");
+                            if (len > -1) {
+                                columnItem.setFamily(field.substring(0, len));
+                                columnItem.setQualifier(field.substring(len + 1));
+                            } else {
+                                columnItem.setFamily(family);
+                                columnItem.setQualifier(field);
+                            }
+                        }
+                        if (uppercaseQualifier) {
+                            columnItem.setQualifier(columnItem.getQualifier().toUpperCase());
+                        }
+                        families.add(columnItem.getFamily());
+                    }
+
+                    columnItems.put(columnField.getKey(), columnItem);
+                }
+            } else {
+                this.columns = new LinkedHashMap<>();
+            }
+        }
+
+        public List<String> getExcludeColumns() {
+            return excludeColumns;
+        }
+
+        public void setExcludeColumns(List<String> excludeColumns) {
+            this.excludeColumns = excludeColumns;
+        }
+
+        public String getFamily() {
+            return family;
+        }
+
+        public void setFamily(String family) {
+            this.family = family;
+            if (family == null) {
+                this.family = "CF";
+            }
+        }
+
+        public boolean isUppercaseQualifier() {
+            return uppercaseQualifier;
+        }
+
+        public void setUppercaseQualifier(boolean uppercaseQualifier) {
+            this.uppercaseQualifier = uppercaseQualifier;
+        }
+
+        public ColumnItem getRowKeyColumn() {
+            return rowKeyColumn;
+        }
+
+        public void setRowKeyColumn(ColumnItem rowKeyColumn) {
+            this.rowKeyColumn = rowKeyColumn;
+        }
+
+        public Map<String, ColumnItem> getColumnItems() {
+            return columnItems;
+        }
+
+        public void setColumnItems(Map<String, ColumnItem> columnItems) {
+            this.columnItems = columnItems;
+        }
+
+        public Set<String> getFamilies() {
+            return families;
+        }
+
+        public void setFamilies(Set<String> families) {
+            this.families = families;
+        }
+
+        @Override
+        public boolean equals(Object o) {
+            if (this == o) return true;
+            if (o == null || getClass() != o.getClass()) return false;
+
+            HbaseMapping hbaseMapping = (HbaseMapping) o;
+
+            if (table != null ? !table.equals(hbaseMapping.table) : hbaseMapping.table != null) return false;
+            return hbaseTable != null ? hbaseTable.equals(hbaseMapping.hbaseTable) : hbaseMapping.hbaseTable == null;
+        }
+
+        @Override
+        public int hashCode() {
+            int result = table != null ? table.hashCode() : 0;
+            result = 31 * result + (hbaseTable != null ? hbaseTable.hashCode() : 0);
+            return result;
+        }
+    }
+}
@@ -0,0 +1,149 @@
+package com.alibaba.otter.canal.client.adapter.hbase.config;
+
+import java.io.File;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Collection;
+import java.util.LinkedHashMap;
+import java.util.Map;
+
+import org.apache.commons.lang.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.yaml.snakeyaml.Yaml;
+
+import com.alibaba.otter.canal.client.adapter.support.AdapterConfigs;
+
+/**
+ * HBase表映射配置加载器
+ *
+ * @author rewerma 2018-8-21 下午06:45:49
+ * @version 1.0.0
+ */
+public class MappingConfigLoader {
+
+    private static Logger       logger    = LoggerFactory.getLogger(MappingConfigLoader.class);
+
+    private static final String BASE_PATH = "hbase";
+
+    /**
+     * 加载HBase表映射配置
+     * 
+     * @return 配置名/配置文件名--对象
+     */
+    public static Map<String, MappingConfig> load() {
+        logger.info("## Start loading mapping config ... ");
+
+        Map<String, MappingConfig> result = new LinkedHashMap<>();
+
+        Collection<String> configs = AdapterConfigs.get("hbase");
+        for (String c : configs) {
+            if (c == null) {
+                continue;
+            }
+            c = c.trim();
+            if (c.equals("") || c.startsWith("#")) {
+                continue;
+            }
+
+            MappingConfig config;
+            String configContent = null;
+
+            if (c.endsWith(".yml")) {
+                configContent = readConfigContent(BASE_PATH + "/" + c);
+            }
+
+            // 简单配置database.table@datasourcekey?rowKey=key1,key2
+            if (StringUtils.isEmpty(configContent)) {
+                String[] mapping = c.split("\\?");
+                String params = mapping.length == 2 ? mapping[1] : null;
+                String rowKey = null;
+                String srcMeta = mapping[0];
+                //
+                if (params != null) {
+                    for (String entry : params.split("&")) {
+                        if ("rowKey".equals(entry.split("=")[0])) {
+                            rowKey = entry.split("=")[1];
+                        }
+                    }
+                }
+                String dsKey = srcMeta.split("@").length == 2 ? srcMeta.split("@")[1] : null;
+                String[] dbTable;
+                if (dsKey == null) {
+                    dbTable = srcMeta.split("\\.");
+
+                } else {
+                    dbTable = srcMeta.split("@")[0].split("\\.");
+                }
+
+                if (dbTable.length == 2) {
+                    config = new MappingConfig();
+
+                    MappingConfig.HbaseMapping hbaseMapping = new MappingConfig.HbaseMapping();
+                    hbaseMapping.setHbaseTable(dbTable[0].toUpperCase() + "." + dbTable[1].toUpperCase());
+                    hbaseMapping.setAutoCreateTable(true);
+                    hbaseMapping.setDatabase(dbTable[0]);
+                    hbaseMapping.setTable(dbTable[1]);
+                    hbaseMapping.setMode(MappingConfig.Mode.PHOENIX);
+                    hbaseMapping.setRowKey(rowKey);
+                    // 有定义rowKey
+                    if (rowKey != null) {
+                        MappingConfig.ColumnItem columnItem = new MappingConfig.ColumnItem();
+                        columnItem.setRowKey(true);
+                        columnItem.setColumn(rowKey);
+                        hbaseMapping.setRowKeyColumn(columnItem);
+                    }
+                    config.setHbaseMapping(hbaseMapping);
+                    config.setDataSourceKey(dsKey);
+
+                } else {
+                    throw new RuntimeException(String.format("配置项[%s]内容为空, 或格式不符合database.table", c));
+                }
+
+            } else { // 配置文件配置
+                config = new Yaml().loadAs(configContent, MappingConfig.class);
+            }
+
+            try {
+                config.validate();
+            } catch (Exception e) {
+                throw new RuntimeException("ERROR Config: " + c + " " + e.getMessage(), e);
+            }
+            result.put(c, config);
+        }
+
+        logger.info("## Mapping config loaded");
+        return result;
+    }
+
+    public static String readConfigContent(String config) {
+        InputStream in = null;
+        try {
+            // 先取本地文件，再取类路径
+            File configFile = new File("config/" + config);
+            if (configFile.exists()) {
+                in = new FileInputStream(configFile);
+            } else {
+                in = MappingConfigLoader.class.getClassLoader().getResourceAsStream(config);
+            }
+            if (in == null) {
+                throw new RuntimeException("Config file not found.");
+            }
+
+            byte[] bytes = new byte[in.available()];
+            in.read(bytes);
+            return new String(bytes, "UTF-8");
+        } catch (IOException e) {
+            throw new RuntimeException("Read ds-config.yml or hbase-mappings.conf error. ", e);
+        } finally {
+            try {
+                if (in != null) {
+                    in.close();
+                }
+            } catch (IOException e) {
+                // ignore
+            }
+        }
+    }
+}
@@ -0,0 +1,385 @@
+package com.alibaba.otter.canal.client.adapter.hbase.service;
+
+import java.sql.*;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Date;
+import java.util.List;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.concurrent.atomic.AtomicLong;
+import java.util.function.Function;
+
+import javax.sql.DataSource;
+
+import org.apache.hadoop.hbase.util.Bytes;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.alibaba.otter.canal.client.adapter.hbase.config.MappingConfig;
+import com.alibaba.otter.canal.client.adapter.hbase.support.*;
+import com.alibaba.otter.canal.client.adapter.support.EtlResult;
+import com.alibaba.otter.canal.client.adapter.support.JdbcTypeUtil;
+import com.google.common.base.Joiner;
+
+/**
+ * HBase ETL 操作业务类
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+public class HbaseEtlService {
+
+    private static Logger logger = LoggerFactory.getLogger(HbaseEtlService.class);
+
+    public static Object sqlRS(DataSource ds, String sql, Function<ResultSet, Object> fun) throws SQLException {
+        Connection conn = null;
+        Statement stmt = null;
+        ResultSet rs = null;
+        try {
+            conn = ds.getConnection();
+            stmt = conn.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
+            stmt.setFetchSize(Integer.MIN_VALUE);
+            rs = stmt.executeQuery(sql);
+            return fun.apply(rs);
+        } finally {
+            if (rs != null) {
+                try {
+                    rs.close();
+                } catch (SQLException e) {
+                    logger.error(e.getMessage(), e);
+                }
+            }
+            if (stmt != null) {
+                try {
+                    stmt.close();
+                } catch (SQLException e) {
+                    logger.error(e.getMessage(), e);
+                }
+            }
+            if (conn != null) {
+                try {
+                    conn.close();
+                } catch (SQLException e) {
+                    logger.error(e.getMessage(), e);
+                }
+            }
+            rs = null;
+            stmt = null;
+            conn = null;
+        }
+    }
+
+    /**
+     * 建表
+     * 
+     * @param hbaseTemplate
+     * @param config
+     */
+    public static void createTable(HbaseTemplate hbaseTemplate, MappingConfig config) {
+        try {
+            // 判断hbase表是否存在，不存在则建表
+            MappingConfig.HbaseMapping hbaseMapping = config.getHbaseMapping();
+            if (!hbaseTemplate.tableExists(hbaseMapping.getHbaseTable())) {
+                hbaseTemplate.createTable(hbaseMapping.getHbaseTable(), hbaseMapping.getFamily());
+            }
+        } catch (Exception e) {
+            logger.error(e.getMessage(), e);
+            throw new RuntimeException(e);
+        }
+    }
+
+    /**
+     * 导入数据
+     * 
+     * @param ds 数据源
+     * @param hbaseTemplate hbaseTemplate
+     * @param config 配置
+     * @param params 筛选条件
+     * @return 导入结果
+     */
+    public static EtlResult importData(DataSource ds, HbaseTemplate hbaseTemplate, MappingConfig config,
+                                       List<String> params) {
+        EtlResult etlResult = new EtlResult();
+        AtomicLong successCount = new AtomicLong();
+        List<String> errMsg = new ArrayList<>();
+        String hbaseTable = "";
+        try {
+            if (config == null) {
+                logger.error("Config is null!");
+                etlResult.setSucceeded(false);
+                etlResult.setErrorMessage("Config is null!");
+                return etlResult;
+            }
+            MappingConfig.HbaseMapping hbaseMapping = config.getHbaseMapping();
+            hbaseTable = hbaseMapping.getHbaseTable();
+
+            long start = System.currentTimeMillis();
+
+            if (params != null && params.size() == 1 && "rebuild".equalsIgnoreCase(params.get(0))) {
+                logger.info(hbaseMapping.getHbaseTable() + " rebuild is starting!");
+                // 如果表存在则删除
+                if (hbaseTemplate.tableExists(hbaseMapping.getHbaseTable())) {
+                    hbaseTemplate.disableTable(hbaseMapping.getHbaseTable());
+                    hbaseTemplate.deleteTable(hbaseMapping.getHbaseTable());
+                }
+                params = null;
+            } else {
+                logger.info(hbaseMapping.getHbaseTable() + " etl is starting!");
+            }
+            createTable(hbaseTemplate, config);
+
+            // 拼接sql
+            String sql = "SELECT * FROM " + config.getHbaseMapping().getDatabase() + "." + hbaseMapping.getTable();
+
+            // 拼接条件
+            if (params != null && params.size() == 1 && hbaseMapping.getEtlCondition() == null) {
+                AtomicBoolean stExists = new AtomicBoolean(false);
+                // 验证是否有SYS_TIME字段
+                sqlRS(ds, sql, rs -> {
+                    try {
+                        ResultSetMetaData rsmd = rs.getMetaData();
+                        int cnt = rsmd.getColumnCount();
+                        for (int i = 1; i <= cnt; i++) {
+                            String columnName = rsmd.getColumnName(i);
+                            if ("SYS_TIME".equalsIgnoreCase(columnName)) {
+                                stExists.set(true);
+                                break;
+                            }
+                        }
+                    } catch (Exception e) {
+                        // ignore
+                    }
+                    return null;
+                });
+                if (stExists.get()) {
+                    sql += " WHERE SYS_TIME >= '" + params.get(0) + "' ";
+                }
+            } else if (hbaseMapping.getEtlCondition() != null && params != null) {
+                String etlCondition = hbaseMapping.getEtlCondition();
+                int size = params.size();
+                for (int i = 0; i < size; i++) {
+                    etlCondition = etlCondition.replace("{" + i + "}", params.get(i));
+                }
+
+                sql += " " + etlCondition;
+            }
+
+            // 获取总数
+            String countSql = "SELECT COUNT(1) FROM ( " + sql + ") _CNT ";
+            long cnt = (Long) sqlRS(ds, countSql, rs -> {
+                Long count = null;
+                try {
+                    if (rs.next()) {
+                        count = ((Number) rs.getObject(1)).longValue();
+                    }
+                } catch (Exception e) {
+                    logger.error(e.getMessage(), e);
+                }
+                return count == null ? 0 : count;
+            });
+
+            // 当大于1万条记录时开启多线程
+            if (cnt >= 10000) {
+                int threadCount = 3;
+                long perThreadCnt = cnt / threadCount;
+                ExecutorService executor = Executors.newFixedThreadPool(threadCount);
+                List<Future<Boolean>> futures = new ArrayList<>(threadCount);
+                for (int i = 0; i < threadCount; i++) {
+                    long offset = i * perThreadCnt;
+                    Long size = null;
+                    if (i != threadCount - 1) {
+                        size = perThreadCnt;
+                    }
+                    String sqlFinal;
+                    if (size != null) {
+                        sqlFinal = sql + " LIMIT " + offset + "," + size;
+                    } else {
+                        sqlFinal = sql + " LIMIT " + offset + "," + cnt;
+                    }
+                    Future<Boolean> future = executor
+                        .submit(() -> executeSqlImport(ds, sqlFinal, hbaseMapping, hbaseTemplate, successCount, errMsg));
+                    futures.add(future);
+                }
+
+                for (Future<Boolean> future : futures) {
+                    future.get();
+                }
+
+                executor.shutdown();
+            } else {
+                executeSqlImport(ds, sql, hbaseMapping, hbaseTemplate, successCount, errMsg);
+            }
+
+            logger.info(
+                hbaseMapping.getHbaseTable() + " etl completed in: " + (System.currentTimeMillis() - start) / 1000 + "s!");
+
+            etlResult.setResultMessage("导入HBase表 " + hbaseMapping.getHbaseTable() + " 数据：" + successCount.get() + " 条");
+        } catch (Exception e) {
+            logger.error(e.getMessage(), e);
+            errMsg.add(hbaseTable + " etl failed! ==>" + e.getMessage());
+        }
+
+        if (errMsg.isEmpty()) {
+            etlResult.setSucceeded(true);
+        } else {
+            etlResult.setErrorMessage(Joiner.on("\n").join(errMsg));
+        }
+        return etlResult;
+    }
+
+    /**
+     * 执行导入
+     * 
+     * @param ds
+     * @param sql
+     * @param hbaseMapping
+     * @param hbaseTemplate
+     * @param successCount
+     * @param errMsg
+     * @return
+     */
+    private static boolean executeSqlImport(DataSource ds, String sql, MappingConfig.HbaseMapping hbaseMapping,
+                                            HbaseTemplate hbaseTemplate, AtomicLong successCount, List<String> errMsg) {
+        try {
+            sqlRS(ds, sql, rs -> {
+                int i = 1;
+
+                try {
+                    boolean complete = false;
+                    List<HRow> rows = new ArrayList<>();
+                    String[] rowKeyColumns = null;
+                    if (hbaseMapping.getRowKey() != null) {
+                        rowKeyColumns = hbaseMapping.getRowKey().trim().split(",");
+                    }
+                    while (rs.next()) {
+                        int cc = rs.getMetaData().getColumnCount();
+                        int[] jdbcTypes = new int[cc];
+                        Class<?>[] classes = new Class[cc];
+                        for (int j = 1; j <= cc; j++) {
+                            int jdbcType = rs.getMetaData().getColumnType(j);
+                            jdbcTypes[j - 1] = jdbcType;
+                            classes[j - 1] = JdbcTypeUtil.jdbcType2javaType(jdbcType);
+                        }
+                        HRow row = new HRow();
+
+                        if (rowKeyColumns != null) {
+                            // 取rowKey字段拼接
+                            StringBuilder rowKeyVale = new StringBuilder();
+                            for (String rowKeyColumnName : rowKeyColumns) {
+                                Object obj = rs.getObject(rowKeyColumnName);
+                                if (obj != null) {
+                                    rowKeyVale.append(obj.toString());
+                                }
+                                rowKeyVale.append("|");
+                            }
+                            int len = rowKeyVale.length();
+                            if (len > 0) {
+                                rowKeyVale.delete(len - 1, len);
+                            }
+                            row.setRowKey(Bytes.toBytes(rowKeyVale.toString()));
+                        }
+
+                        for (int j = 1; j <= cc; j++) {
+                            String columnName = rs.getMetaData().getColumnName(j);
+
+                            Object val = JdbcTypeUtil.getRSData(rs, columnName, jdbcTypes[j - 1]);
+                            if (val == null) {
+                                continue;
+                            }
+
+                            MappingConfig.ColumnItem columnItem = hbaseMapping.getColumnItems().get(columnName);
+                            // 没有配置映射
+                            if (columnItem == null) {
+                                String family = hbaseMapping.getFamily();
+                                String qualifile = columnName;
+                                if (hbaseMapping.isUppercaseQualifier()) {
+                                    qualifile = qualifile.toUpperCase();
+                                }
+                                if (MappingConfig.Mode.STRING == hbaseMapping.getMode()) {
+                                    if (hbaseMapping.getRowKey() == null && j == 1) {
+                                        row.setRowKey(Bytes.toBytes(val.toString()));
+                                    } else {
+                                        row.addCell(family, qualifile, Bytes.toBytes(val.toString()));
+                                    }
+                                } else if (MappingConfig.Mode.NATIVE == hbaseMapping.getMode()) {
+                                    Type type = Type.getType(classes[j - 1]);
+                                    if (hbaseMapping.getRowKey() == null && j == 1) {
+                                        row.setRowKey(TypeUtil.toBytes(val, type));
+                                    } else {
+                                        row.addCell(family, qualifile, TypeUtil.toBytes(val, type));
+                                    }
+                                } else if (MappingConfig.Mode.PHOENIX == hbaseMapping.getMode()) {
+                                    PhType phType = PhType.getType(classes[j - 1]);
+                                    if (hbaseMapping.getRowKey() == null && j == 1) {
+                                        row.setRowKey(PhTypeUtil.toBytes(val, phType));
+                                    } else {
+                                        row.addCell(family, qualifile, PhTypeUtil.toBytes(val, phType));
+                                    }
+                                }
+                            } else {
+                                // 如果不需要类型转换
+                                if (columnItem.getType() == null || "".equals(columnItem.getType())) {
+                                    if (val instanceof java.sql.Date) {
+                                        SimpleDateFormat dateFmt = new SimpleDateFormat("yyyy-MM-dd");
+                                        val = dateFmt.format((Date) val);
+                                    } else if (val instanceof Timestamp) {
+                                        SimpleDateFormat datetimeFmt = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+                                        val = datetimeFmt.format((Date) val);
+                                    }
+
+                                    byte[] valBytes = Bytes.toBytes(val.toString());
+                                    if (columnItem.isRowKey()) {
+                                        row.setRowKey(valBytes);
+                                    } else {
+                                        row.addCell(columnItem.getFamily(), columnItem.getQualifier(), valBytes);
+                                    }
+                                } else {
+                                    PhType phType = PhType.getType(columnItem.getType());
+                                    if (columnItem.isRowKey()) {
+                                        row.setRowKey(PhTypeUtil.toBytes(val, phType));
+                                    } else {
+                                        row.addCell(columnItem.getFamily(),
+                                            columnItem.getQualifier(),
+                                            PhTypeUtil.toBytes(val, phType));
+                                    }
+                                }
+                            }
+                        }
+
+                        if (row.getRowKey() == null) throw new RuntimeException("RowKey 值为空");
+
+                        rows.add(row);
+                        complete = false;
+                        if (i % hbaseMapping.getCommitBatch() == 0 && !rows.isEmpty()) {
+                            hbaseTemplate.puts(hbaseMapping.getHbaseTable(), rows);
+                            rows.clear();
+                            complete = true;
+                        }
+                        i++;
+                        successCount.incrementAndGet();
+                        if (logger.isDebugEnabled()) {
+                            logger.debug("successful import count:" + successCount.get());
+                        }
+                    }
+
+                    if (!complete && !rows.isEmpty()) {
+                        hbaseTemplate.puts(hbaseMapping.getHbaseTable(), rows);
+                    }
+
+                } catch (Exception e) {
+                    logger.error(hbaseMapping.getHbaseTable() + " etl failed! ==>" + e.getMessage(), e);
+                    errMsg.add(hbaseMapping.getHbaseTable() + " etl failed! ==>" + e.getMessage());
+                    // throw new RuntimeException(e);
+                }
+                return i;
+            });
+            return true;
+        } catch (Exception e) {
+            logger.error(e.getMessage(), e);
+            return false;
+        }
+    }
+}
@@ -0,0 +1,426 @@
+package com.alibaba.otter.canal.client.adapter.hbase.service;
+
+import java.util.*;
+
+import org.apache.hadoop.hbase.util.Bytes;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.alibaba.otter.canal.client.adapter.hbase.config.MappingConfig;
+import com.alibaba.otter.canal.client.adapter.hbase.support.*;
+import com.alibaba.otter.canal.client.adapter.support.Dml;
+
+/**
+ * HBase同步操作业务
+ *
+ * @author rewerma 2018-8-21 下午06:45:49
+ * @version 1.0.0
+ */
+public class HbaseSyncService {
+
+    private Logger        logger = LoggerFactory.getLogger(this.getClass());
+
+    private HbaseTemplate hbaseTemplate;                                    // HBase操作模板
+
+    public HbaseSyncService(HbaseTemplate hbaseTemplate){
+        this.hbaseTemplate = hbaseTemplate;
+    }
+
+    public void sync(MappingConfig config, Dml dml) {
+        try {
+            if (config != null) {
+                String type = dml.getType();
+                if (type != null && type.equalsIgnoreCase("INSERT")) {
+                    insert(config, dml);
+                } else if (type != null && type.equalsIgnoreCase("UPDATE")) {
+                    update(config, dml);
+                } else if (type != null && type.equalsIgnoreCase("DELETE")) {
+                    delete(config, dml);
+                }
+                if (logger.isDebugEnabled()) {
+                    String res = dml.toString();
+                    logger.debug(res);
+                }
+            }
+        } catch (Exception e) {
+            logger.error(e.getMessage(), e);
+        }
+    }
+
+    /**
+     * 插入操作
+     * 
+     * @param config 配置项
+     * @param dml DML数据
+     */
+    private void insert(MappingConfig config, Dml dml) {
+        List<Map<String, Object>> data = dml.getData();
+        if (data == null || data.isEmpty()) {
+            return;
+        }
+
+        MappingConfig.HbaseMapping hbaseMapping = config.getHbaseMapping();
+
+        // if (!validHTable(config)) {
+        // logger.error("HBase table '{}' not exists",
+        // hbaseMapping.getHbaseTable());
+        // return;
+        // }
+        int i = 1;
+        boolean complete = false;
+        List<HRow> rows = new ArrayList<>();
+        for (Map<String, Object> r : data) {
+            HRow hRow = new HRow();
+
+            // 拼接复合rowKey
+            if (hbaseMapping.getRowKey() != null) {
+                String[] rowKeyColumns = hbaseMapping.getRowKey().trim().split(",");
+                String rowKeyVale = getRowKeys(rowKeyColumns, r);
+                // params.put("rowKey", Bytes.toBytes(rowKeyVale));
+                hRow.setRowKey(Bytes.toBytes(rowKeyVale));
+            }
+
+            convertData2Row(hbaseMapping, hRow, r);
+            if (hRow.getRowKey() == null) {
+                throw new RuntimeException("empty rowKey");
+            }
+            rows.add(hRow);
+            complete = false;
+            if (i % config.getHbaseMapping().getCommitBatch() == 0 && !rows.isEmpty()) {
+                hbaseTemplate.puts(hbaseMapping.getHbaseTable(), rows);
+                rows.clear();
+                complete = true;
+            }
+            i++;
+        }
+        if (!complete && !rows.isEmpty()) {
+            hbaseTemplate.puts(hbaseMapping.getHbaseTable(), rows);
+        }
+
+    }
+
+    /**
+     * 将Map数据转换为HRow行数据
+     * 
+     * @param hbaseMapping hbase映射配置
+     * @param hRow 行对象
+     * @param data Map数据
+     */
+    private static void convertData2Row(MappingConfig.HbaseMapping hbaseMapping, HRow hRow, Map<String, Object> data) {
+        Map<String, MappingConfig.ColumnItem> columnItems = hbaseMapping.getColumnItems();
+        int i = 0;
+        for (Map.Entry<String, Object> entry : data.entrySet()) {
+            if (hbaseMapping.getExcludeColumns() != null && hbaseMapping.getExcludeColumns().contains(entry.getKey())) {
+                continue;
+            }
+            if (entry.getValue() != null) {
+                MappingConfig.ColumnItem columnItem = columnItems.get(entry.getKey());
+
+                byte[] bytes = typeConvert(columnItem, hbaseMapping, entry.getValue());
+
+                if (columnItem == null) {
+                    String familyName = hbaseMapping.getFamily();
+                    String qualifier = entry.getKey();
+                    if (hbaseMapping.isUppercaseQualifier()) {
+                        qualifier = qualifier.toUpperCase();
+                    }
+
+                    if (hbaseMapping.getRowKey() == null && i == 0) {
+                        hRow.setRowKey(bytes);
+                    } else {
+                        hRow.addCell(familyName, qualifier, bytes);
+                    }
+                } else {
+                    if (columnItem.isRowKey()) {
+                        // row.put("rowKey", bytes);
+                        hRow.setRowKey(bytes);
+                    } else {
+                        hRow.addCell(columnItem.getFamily(), columnItem.getQualifier(), bytes);
+                    }
+                }
+            }
+            i++;
+        }
+    }
+
+    /**
+     * 更新操作
+     * 
+     * @param config
+     * @param dml
+     */
+    private void update(MappingConfig config, Dml dml) {
+        List<Map<String, Object>> data = dml.getData();
+        List<Map<String, Object>> old = dml.getOld();
+        if (old == null || old.isEmpty() || data == null || data.isEmpty()) {
+            return;
+        }
+
+        MappingConfig.HbaseMapping hbaseMapping = config.getHbaseMapping();
+
+        // if (!validHTable(config)) {
+        // logger.error("HBase table '{}' not exists",
+        // hbaseMapping.getHbaseTable());
+        // return;
+        // }
+
+        MappingConfig.ColumnItem rowKeyColumn = hbaseMapping.getRowKeyColumn();
+        int index = 0;
+        int i = 1;
+        boolean complete = false;
+        List<HRow> rows = new ArrayList<>();
+        out: for (Map<String, Object> r : data) {
+            byte[] rowKeyBytes;
+
+            if (hbaseMapping.getRowKey() != null) {
+                String[] rowKeyColumns = hbaseMapping.getRowKey().trim().split(",");
+
+                // 判断是否有复合主键修改
+                for (String updateColumn : old.get(index).keySet()) {
+                    for (String rowKeyColumnName : rowKeyColumns) {
+                        if (rowKeyColumnName.equalsIgnoreCase(updateColumn)) {
+                            // 调用删除插入操作
+                            deleteAndInsert(config, dml);
+                            continue out;
+                        }
+                    }
+                }
+
+                String rowKeyVale = getRowKeys(rowKeyColumns, r);
+                rowKeyBytes = Bytes.toBytes(rowKeyVale);
+            } else if (rowKeyColumn == null) {
+                Map<String, Object> rowKey = data.get(0);
+                rowKeyBytes = typeConvert(null, hbaseMapping, rowKey.values().iterator().next());
+            } else {
+                rowKeyBytes = typeConvert(rowKeyColumn, hbaseMapping, r.get(rowKeyColumn.getColumn()));
+            }
+            if (rowKeyBytes == null) throw new RuntimeException("rowKey值为空");
+
+            Map<String, MappingConfig.ColumnItem> columnItems = hbaseMapping.getColumnItems();
+            HRow hRow = new HRow(rowKeyBytes);
+            for (String updateColumn : old.get(index).keySet()) {
+                if (hbaseMapping.getExcludeColumns() != null && hbaseMapping.getExcludeColumns().contains(updateColumn)) {
+                    continue;
+                }
+                MappingConfig.ColumnItem columnItem = columnItems.get(updateColumn);
+                if (columnItem == null) {
+                    String family = hbaseMapping.getFamily();
+                    String qualifier = updateColumn;
+                    if (hbaseMapping.isUppercaseQualifier()) {
+                        qualifier = qualifier.toUpperCase();
+                    }
+
+                    Object newVal = r.get(updateColumn);
+
+                    if (newVal == null) {
+                        hRow.addCell(family, qualifier, null);
+                    } else {
+                        hRow.addCell(family, qualifier, typeConvert(null, hbaseMapping, newVal));
+                    }
+                } else {
+                    // 排除修改id的情况
+                    if (columnItem.isRowKey()) continue;
+
+                    Object newVal = r.get(updateColumn);
+                    if (newVal == null) {
+                        hRow.addCell(columnItem.getFamily(), columnItem.getQualifier(), null);
+                    } else {
+                        hRow.addCell(columnItem.getFamily(),
+                            columnItem.getQualifier(),
+                            typeConvert(columnItem, hbaseMapping, newVal));
+                    }
+                }
+            }
+            rows.add(hRow);
+            complete = false;
+            if (i % config.getHbaseMapping().getCommitBatch() == 0 && !rows.isEmpty()) {
+                hbaseTemplate.puts(hbaseMapping.getHbaseTable(), rows);
+                rows.clear();
+                complete = true;
+            }
+            i++;
+            index++;
+        }
+        if (!complete && !rows.isEmpty()) {
+            hbaseTemplate.puts(hbaseMapping.getHbaseTable(), rows);
+        }
+    }
+
+    private void delete(MappingConfig config, Dml dml) {
+        List<Map<String, Object>> data = dml.getData();
+        if (data == null || data.isEmpty()) {
+            return;
+        }
+
+        MappingConfig.HbaseMapping hbaseMapping = config.getHbaseMapping();
+
+        // if (!validHTable(config)) {
+        // logger.error("HBase table '{}' not exists",
+        // hbaseMapping.getHbaseTable());
+        // return;
+        // }
+
+        MappingConfig.ColumnItem rowKeyColumn = hbaseMapping.getRowKeyColumn();
+        boolean complete = false;
+        int i = 1;
+        Set<byte[]> rowKeys = new HashSet<>();
+        for (Map<String, Object> r : data) {
+            byte[] rowKeyBytes;
+
+            if (hbaseMapping.getRowKey() != null) {
+                String[] rowKeyColumns = hbaseMapping.getRowKey().trim().split(",");
+                String rowKeyVale = getRowKeys(rowKeyColumns, r);
+                rowKeyBytes = Bytes.toBytes(rowKeyVale);
+            } else if (rowKeyColumn == null) {
+                // 如果不需要类型转换
+                Map<String, Object> rowKey = data.get(0);
+                rowKeyBytes = typeConvert(null, hbaseMapping, rowKey.values().iterator().next());
+            } else {
+                Object val = r.get(rowKeyColumn.getColumn());
+                rowKeyBytes = typeConvert(rowKeyColumn, hbaseMapping, val);
+            }
+            if (rowKeyBytes == null) throw new RuntimeException("rowKey值为空");
+            rowKeys.add(rowKeyBytes);
+            complete = false;
+            if (i % config.getHbaseMapping().getCommitBatch() == 0 && !rowKeys.isEmpty()) {
+                hbaseTemplate.deletes(hbaseMapping.getHbaseTable(), rowKeys);
+                rowKeys.clear();
+                complete = true;
+            }
+            i++;
+        }
+        if (!complete && !rowKeys.isEmpty()) {
+            hbaseTemplate.deletes(hbaseMapping.getHbaseTable(), rowKeys);
+        }
+    }
+
+    private void deleteAndInsert(MappingConfig config, Dml dml) {
+        List<Map<String, Object>> data = dml.getData();
+        List<Map<String, Object>> old = dml.getOld();
+        if (old == null || old.isEmpty() || data == null || data.isEmpty()) {
+            return;
+        }
+        MappingConfig.HbaseMapping hbaseMapping = config.getHbaseMapping();
+
+        String[] rowKeyColumns = hbaseMapping.getRowKey().trim().split(",");
+
+        int index = 0;
+        int i = 1;
+        boolean complete = false;
+        Set<byte[]> rowKeys = new HashSet<>();
+        List<HRow> rows = new ArrayList<>();
+        for (Map<String, Object> r : data) {
+            // 拼接老的rowKey
+            List<String> updateSubRowKey = new ArrayList<>();
+            for (String rowKeyColumnName : rowKeyColumns) {
+                for (String updateColumn : old.get(index).keySet()) {
+                    if (rowKeyColumnName.equalsIgnoreCase(updateColumn)) {
+                        updateSubRowKey.add(rowKeyColumnName);
+                    }
+                }
+            }
+            if (updateSubRowKey.isEmpty()) {
+                throw new RuntimeException("没有更新复合主键的RowKey");
+            }
+            StringBuilder oldRowKey = new StringBuilder();
+            StringBuilder newRowKey = new StringBuilder();
+            for (String rowKeyColumnName : rowKeyColumns) {
+                newRowKey.append(r.get(rowKeyColumnName).toString()).append("|");
+                if (!updateSubRowKey.contains(rowKeyColumnName)) {
+                    // 从data取
+                    oldRowKey.append(r.get(rowKeyColumnName).toString()).append("|");
+                } else {
+                    // 从old取
+                    oldRowKey.append(old.get(index).get(rowKeyColumnName).toString()).append("|");
+                }
+            }
+            int len = newRowKey.length();
+            newRowKey.delete(len - 1, len);
+            len = oldRowKey.length();
+            oldRowKey.delete(len - 1, len);
+            byte[] newRowKeyBytes = Bytes.toBytes(newRowKey.toString());
+            byte[] oldRowKeyBytes = Bytes.toBytes(oldRowKey.toString());
+
+            rowKeys.add(oldRowKeyBytes);
+            HRow row = new HRow(newRowKeyBytes);
+            convertData2Row(hbaseMapping, row, r);
+            rows.add(row);
+            complete = false;
+            if (i % config.getHbaseMapping().getCommitBatch() == 0 && !rows.isEmpty()) {
+                hbaseTemplate.deletes(hbaseMapping.getHbaseTable(), rowKeys);
+
+                hbaseTemplate.puts(hbaseMapping.getHbaseTable(), rows);
+                rowKeys.clear();
+                rows.clear();
+                complete = true;
+            }
+            i++;
+            index++;
+        }
+        if (!complete && !rows.isEmpty()) {
+            hbaseTemplate.deletes(hbaseMapping.getHbaseTable(), rowKeys);
+            hbaseTemplate.puts(hbaseMapping.getHbaseTable(), rows);
+        }
+    }
+
+    /**
+     * 根据对应的类型进行转换
+     * 
+     * @param columnItem 列项配置
+     * @param hbaseMapping hbase映射配置
+     * @param value 值
+     * @return 复合字段rowKey
+     */
+    private static byte[] typeConvert(MappingConfig.ColumnItem columnItem, MappingConfig.HbaseMapping hbaseMapping,
+                                      Object value) {
+        if (value == null) {
+            return null;
+        }
+        byte[] bytes = null;
+        if (columnItem == null || columnItem.getType() == null || "".equals(columnItem.getType())) {
+            if (MappingConfig.Mode.STRING == hbaseMapping.getMode()) {
+                bytes = Bytes.toBytes(value.toString());
+            } else if (MappingConfig.Mode.NATIVE == hbaseMapping.getMode()) {
+                bytes = TypeUtil.toBytes(value);
+            } else if (MappingConfig.Mode.PHOENIX == hbaseMapping.getMode()) {
+                PhType phType = PhType.getType(value.getClass());
+                bytes = PhTypeUtil.toBytes(value, phType);
+            }
+        } else {
+            if (hbaseMapping.getMode() == MappingConfig.Mode.STRING) {
+                bytes = Bytes.toBytes(value.toString());
+            } else if (hbaseMapping.getMode() == MappingConfig.Mode.NATIVE) {
+                Type type = Type.getType(columnItem.getType());
+                bytes = TypeUtil.toBytes(value, type);
+            } else if (hbaseMapping.getMode() == MappingConfig.Mode.PHOENIX) {
+                PhType phType = PhType.getType(columnItem.getType());
+                bytes = PhTypeUtil.toBytes(value, phType);
+            }
+        }
+        return bytes;
+    }
+
+    /**
+     * 获取复合字段作为rowKey的拼接
+     *
+     * @param rowKeyColumns 复合rowK对应的字段
+     * @param data 数据
+     * @return
+     */
+    private static String getRowKeys(String[] rowKeyColumns, Map<String, Object> data) {
+        StringBuilder rowKeyValue = new StringBuilder();
+        for (String rowKeyColumnName : rowKeyColumns) {
+            Object obj = data.get(rowKeyColumnName);
+            if (obj != null) {
+                rowKeyValue.append(obj.toString());
+            }
+            rowKeyValue.append("|");
+        }
+        int len = rowKeyValue.length();
+        if (len > 0) {
+            rowKeyValue.delete(len - 1, len);
+        }
+        return rowKeyValue.toString();
+    }
+
+}
@@ -0,0 +1,84 @@
+package com.alibaba.otter.canal.client.adapter.hbase.support;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ * HBase操作对象类
+ *
+ * @author machengyuan 2018-8-21 下午10:12:34
+ * @version 1.0.0
+ */
+public class HRow {
+
+    private byte[]      rowKey;
+    private List<HCell> cells = new ArrayList<>();
+
+    public HRow(){
+    }
+
+    public HRow(byte[] rowKey){
+        this.rowKey = rowKey;
+    }
+
+    public byte[] getRowKey() {
+        return rowKey;
+    }
+
+    public void setRowKey(byte[] rowKey) {
+        this.rowKey = rowKey;
+    }
+
+    public List<HCell> getCells() {
+        return cells;
+    }
+
+    public void setCells(List<HCell> cells) {
+        this.cells = cells;
+    }
+
+    public void addCell(String family, String qualifier, byte[] value) {
+        HCell hCell = new HCell(family, qualifier, value);
+        cells.add(hCell);
+    }
+
+    public static class HCell {
+
+        private String family;
+        private String qualifier;
+        private byte[] value;
+
+        public HCell(){
+        }
+
+        public HCell(String family, String qualifier, byte[] value){
+            this.family = family;
+            this.qualifier = qualifier;
+            this.value = value;
+        }
+
+        public String getFamily() {
+            return family;
+        }
+
+        public void setFamily(String family) {
+            this.family = family;
+        }
+
+        public String getQualifier() {
+            return qualifier;
+        }
+
+        public void setQualifier(String qualifier) {
+            this.qualifier = qualifier;
+        }
+
+        public byte[] getValue() {
+            return value;
+        }
+
+        public void setValue(byte[] value) {
+            this.value = value;
+        }
+    }
+}
@@ -0,0 +1,163 @@
+package com.alibaba.otter.canal.client.adapter.hbase.support;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Set;
+
+import org.apache.hadoop.hbase.HColumnDescriptor;
+import org.apache.hadoop.hbase.HTableDescriptor;
+import org.apache.hadoop.hbase.TableName;
+import org.apache.hadoop.hbase.client.Connection;
+import org.apache.hadoop.hbase.client.Delete;
+import org.apache.hadoop.hbase.client.HBaseAdmin;
+import org.apache.hadoop.hbase.client.HTable;
+import org.apache.hadoop.hbase.client.Put;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * HBase操作模板
+ *
+ * @author machengyuan 2018-8-21 下午10:12:34
+ * @version 1.0.0
+ */
+public class HbaseTemplate {
+
+    private Logger     logger = LoggerFactory.getLogger(this.getClass());
+
+    private Connection conn;
+
+    public HbaseTemplate(Connection conn){
+        this.conn = conn;
+    }
+
+    public boolean tableExists(String tableName) {
+        try (HBaseAdmin admin = (HBaseAdmin) conn.getAdmin()) {
+
+            return admin.tableExists(TableName.valueOf(tableName));
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
+    public void createTable(String tableName, String... familyNames) {
+        try (HBaseAdmin admin = (HBaseAdmin) conn.getAdmin()) {
+
+            HTableDescriptor desc = new HTableDescriptor(TableName.valueOf(tableName));
+            // 添加列簇
+            if (familyNames != null) {
+                for (String familyName : familyNames) {
+                    HColumnDescriptor hcd = new HColumnDescriptor(familyName);
+                    desc.addFamily(hcd);
+                }
+            }
+            admin.createTable(desc);
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
+    public void disableTable(String tableName) {
+        try (HBaseAdmin admin = (HBaseAdmin) conn.getAdmin()) {
+            admin.disableTable(tableName);
+        } catch (IOException e) {
+            logger.error(e.getMessage(), e);
+            throw new RuntimeException(e);
+        }
+    }
+
+    public void deleteTable(String tableName) {
+        try (HBaseAdmin admin = (HBaseAdmin) conn.getAdmin()) {
+            if (admin.isTableEnabled(tableName)) {
+                disableTable(tableName);
+            }
+            admin.deleteTable(tableName);
+        } catch (IOException e) {
+            logger.error(e.getMessage(), e);
+            throw new RuntimeException(e);
+        }
+    }
+
+    /**
+     * 插入一行数据
+     * 
+     * @param tableName 表名
+     * @param hRow 行数据对象
+     * @return 是否成功
+     */
+    public Boolean put(String tableName, HRow hRow) {
+        boolean flag = false;
+        try {
+            HTable table = (HTable) conn.getTable(TableName.valueOf(tableName));
+            Put put = new Put(hRow.getRowKey());
+            for (HRow.HCell hCell : hRow.getCells()) {
+                put.addColumn(Bytes.toBytes(hCell.getFamily()), Bytes.toBytes(hCell.getQualifier()), hCell.getValue());
+            }
+            table.put(put);
+            flag = true;
+        } catch (Exception e) {
+            logger.error(e.getMessage(), e);
+        }
+        return flag;
+
+    }
+
+    /**
+     * 批量插入
+     * 
+     * @param tableName 表名
+     * @param rows 行数据对象集合
+     * @return 是否成功
+     */
+    public Boolean puts(String tableName, List<HRow> rows) {
+        boolean flag = false;
+        try {
+            HTable table = (HTable) conn.getTable(TableName.valueOf(tableName));
+            List<Put> puts = new ArrayList<>();
+            for (HRow hRow : rows) {
+                Put put = new Put(hRow.getRowKey());
+                for (HRow.HCell hCell : hRow.getCells()) {
+                    put.addColumn(Bytes.toBytes(hCell.getFamily()),
+                        Bytes.toBytes(hCell.getQualifier()),
+                        hCell.getValue());
+                }
+                puts.add(put);
+            }
+            if (!puts.isEmpty()) {
+                table.put(puts);
+            }
+            flag = true;
+        } catch (Exception e) {
+            logger.error(e.getMessage(), e);
+        }
+        return flag;
+    }
+
+    /**
+     * 批量删除数据
+     * 
+     * @param tableName 表名
+     * @param rowKeys rowKey集合
+     * @return 是否成功
+     */
+    public Boolean deletes(String tableName, Set<byte[]> rowKeys) {
+        boolean flag = false;
+        try {
+            HTable table = (HTable) conn.getTable(TableName.valueOf(tableName));
+            List<Delete> deletes = new ArrayList<>();
+            for (byte[] rowKey : rowKeys) {
+                Delete delete = new Delete(rowKey);
+                deletes.add(delete);
+            }
+            if (!deletes.isEmpty()) {
+                table.delete(deletes);
+            }
+            flag = true;
+        } catch (Exception e) {
+            logger.error(e.getMessage(), e);
+        }
+        return flag;
+    }
+}
@@ -0,0 +1,73 @@
+package com.alibaba.otter.canal.client.adapter.hbase.support;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.math.BigDecimal;
+import java.math.BigInteger;
+import java.sql.Time;
+import java.sql.Timestamp;
+import java.util.Date;
+
+/**
+ * Phoenix类型
+ *
+ * @author machengyuan 2018-8-21 下午06:12:34
+ * @version 1.0.0
+ */
+public enum PhType {
+                    DEFAULT, UNSIGNED_INT, UNSIGNED_LONG, UNSIGNED_TINYINT, UNSIGNED_SMALLINT, UNSIGNED_FLOAT,
+                    UNSIGNED_DOUBLE, INTEGER, BIGINT, TINYINT, SMALLINT, FLOAT, DOUBLE, DECIMAL, BOOLEAN, UNSIGNED_TIME,
+                    UNSIGNED_DATE, UNSIGNED_TIMESTAMP, TIME, DATE, TIMESTAMP, VARCHAR, VARBINARY;
+
+    private static Logger logger = LoggerFactory.getLogger(PhType.class);
+
+    public static PhType getType(Class<?> javaType) {
+        if (javaType == null) return DEFAULT;
+        PhType phType;
+        if (Integer.class == javaType || int.class == javaType) {
+            phType = INTEGER;
+        } else if (Long.class == javaType || long.class == javaType) {
+            phType = BIGINT;
+        } else if (Byte.class == javaType || byte.class == javaType) {
+            phType = TINYINT;
+        } else if (Short.class == javaType || short.class == javaType) {
+            phType = SMALLINT;
+        } else if (Float.class == javaType || float.class == javaType) {
+            phType = FLOAT;
+        } else if (Double.class == javaType || double.class == javaType) {
+            phType = DOUBLE;
+        } else if (Boolean.class == javaType || boolean.class == javaType) {
+            phType = BOOLEAN;
+        } else if (java.sql.Date.class == javaType) {
+            phType = DATE;
+        } else if (Time.class == javaType) {
+            phType = DATE;
+        } else if (Timestamp.class == javaType) {
+            phType = TIMESTAMP;
+        } else if (Date.class == javaType) {
+            phType = DATE;
+        } else if (byte[].class == javaType) {
+            phType = VARBINARY;
+        } else if (String.class == javaType) {
+            phType = VARCHAR;
+        } else if (BigDecimal.class == javaType) {
+            phType = DECIMAL;
+        } else if (BigInteger.class == javaType) {
+            phType = UNSIGNED_LONG;
+        } else {
+            phType = DEFAULT;
+        }
+        return phType;
+    }
+
+    public static PhType getType(String type) {
+        if (type == null) return DEFAULT;
+        try {
+            return PhType.valueOf(type.toUpperCase());
+        } catch (Exception e) {
+            logger.error(e.getMessage(), e);
+            return DEFAULT;
+        }
+    }
+}
@@ -0,0 +1,620 @@
+package com.alibaba.otter.canal.client.adapter.hbase.support;
+
+import java.math.BigDecimal;
+import java.math.BigInteger;
+import java.math.MathContext;
+import java.math.RoundingMode;
+import java.sql.Timestamp;
+import java.util.Date;
+
+import org.apache.hadoop.hbase.util.Bytes;
+import org.joda.time.DateTime;
+
+import com.google.common.math.LongMath;
+
+/**
+ * Phoenix类型转换工具类
+ *
+ * @author machengyuan 2018-8-21 下午06:14:26
+ * @version 1.0.0
+ */
+public class PhTypeUtil {
+
+    public static byte[] toBytes(Object v, PhType phType) {
+        if (v == null) return null;
+        byte[] b = null;
+        if (phType == PhType.DEFAULT) {
+            PhType phType1 = PhType.getType(v.getClass());
+            if (phType1 != null && phType1 != PhType.DEFAULT) {
+                toBytes(v, phType1);
+            }
+        } else if (phType == PhType.INTEGER) {
+            b = new byte[Bytes.SIZEOF_INT];
+            encodeInt(((Number) v).intValue(), b, 0);
+        } else if (phType == PhType.UNSIGNED_INT) {
+            b = new byte[Bytes.SIZEOF_INT];
+            encodeUnsignedInt(((Number) v).intValue(), b, 0);
+        } else if (phType == PhType.BIGINT) {
+            b = new byte[Bytes.SIZEOF_LONG];
+            encodeLong(((Number) v).longValue(), b, 0);
+        } else if (phType == PhType.UNSIGNED_LONG) {
+            b = new byte[Bytes.SIZEOF_LONG];
+            encodeUnsignedLong(((Number) v).longValue(), b, 0);
+        } else if (phType == PhType.SMALLINT) {
+            b = new byte[Bytes.SIZEOF_SHORT];
+            encodeShort(((Number) v).shortValue(), b, 0);
+        } else if (phType == PhType.UNSIGNED_SMALLINT) {
+            b = new byte[Bytes.SIZEOF_SHORT];
+            encodeUnsignedShort(((Number) v).shortValue(), b, 0);
+        } else if (phType == PhType.TINYINT) {
+            b = new byte[Bytes.SIZEOF_BYTE];
+            encodeByte(((Number) v).byteValue(), b, 0);
+        } else if (phType == PhType.UNSIGNED_TINYINT) {
+            b = new byte[Bytes.SIZEOF_BYTE];
+            encodeUnsignedByte(((Number) v).byteValue(), b, 0);
+        } else if (phType == PhType.FLOAT) {
+            b = new byte[Bytes.SIZEOF_FLOAT];
+            encodeFloat(((Number) v).floatValue(), b, 0);
+        } else if (phType == PhType.UNSIGNED_FLOAT) {
+            b = new byte[Bytes.SIZEOF_FLOAT];
+            encodeUnsignedFloat(((Number) v).floatValue(), b, 0);
+        } else if (phType == PhType.DOUBLE) {
+            b = new byte[Bytes.SIZEOF_DOUBLE];
+            encodeDouble(((Number) v).doubleValue(), b, 0);
+        } else if (phType == PhType.UNSIGNED_DOUBLE) {
+            b = new byte[Bytes.SIZEOF_DOUBLE];
+            encodeUnsignedDouble(((Number) v).doubleValue(), b, 0);
+        } else if (phType == PhType.BOOLEAN) {
+            if ((Boolean) v) {
+                b = new byte[] { 1 };
+            } else {
+                b = new byte[] { 0 };
+            }
+        } else if (phType == PhType.TIME || phType == PhType.DATE) {
+            b = new byte[Bytes.SIZEOF_LONG];
+            encodeDate(v, b, 0);
+        } else if (phType == PhType.TIMESTAMP) {
+            b = new byte[Bytes.SIZEOF_LONG + Bytes.SIZEOF_INT];
+            encodeTimestamp(v, b, 0);
+        } else if (phType == PhType.UNSIGNED_TIME || phType == PhType.UNSIGNED_DATE) {
+            b = new byte[Bytes.SIZEOF_LONG];
+            encodeUnsignedDate(v, b, 0);
+        } else if (phType == PhType.UNSIGNED_TIMESTAMP) {
+            b = new byte[Bytes.SIZEOF_LONG + Bytes.SIZEOF_INT];
+            encodeUnsignedTimestamp(v, b, 0);
+        } else if (phType == PhType.VARBINARY) {
+            b = (byte[]) v;
+        } else if (phType == PhType.VARCHAR) {
+            b = Bytes.toBytes(v.toString());
+        } else if (phType == PhType.DECIMAL) {
+            if (v instanceof BigDecimal) {
+                b = encodeDecimal(v);
+            } else if (v instanceof Number) {
+                b = encodeDecimal(new BigDecimal(v.toString()));
+            }
+        }
+        return b;
+    }
+
+    public static Object toObject(byte[] b, PhType phType) {
+        if (b == null) return null;
+        Object v = null;
+        if (phType == PhType.INTEGER) {
+            v = decodeInt(b, 0);
+        } else if (phType == PhType.UNSIGNED_INT) {
+            v = decodeUnsignedInt(b, 0);
+        } else if (phType == PhType.BIGINT) {
+            v = decodeLong(b, 0);
+        } else if (phType == PhType.UNSIGNED_LONG) {
+            v = decodeUnsignedLong(b, 0);
+        } else if (phType == PhType.SMALLINT) {
+            v = decodeShort(b, 0);
+        } else if (phType == PhType.UNSIGNED_SMALLINT) {
+            v = decodeUnsignedShort(b, 0);
+        } else if (phType == PhType.TINYINT) {
+            v = decodeByte(b, 0);
+        } else if (phType == PhType.UNSIGNED_TINYINT) {
+            v = decodeUnsignedByte(b, 0);
+        } else if (phType == PhType.FLOAT) {
+            v = decodeFloat(b, 0);
+        } else if (phType == PhType.UNSIGNED_FLOAT) {
+            v = decodeUnsignedFloat(b, 0);
+        } else if (phType == PhType.DOUBLE) {
+            v = decodeDouble(b, 0);
+        } else if (phType == PhType.UNSIGNED_DOUBLE) {
+            v = decodeUnsignedDouble(b, 0);
+        } else if (phType == PhType.BOOLEAN) {
+            checkForSufficientLength(b, 0, Bytes.SIZEOF_BOOLEAN);
+            if (b[0] == 1) {
+                v = true;
+            } else if (b[0] == 0) {
+                v = false;
+            }
+        } else if (phType == PhType.TIME || phType == PhType.DATE) {
+            v = new Date(decodeLong(b, 0));
+        } else if (phType == PhType.TIMESTAMP) {
+            long millisDeserialized = decodeLong(b, 0);
+            Timestamp ts = new Timestamp(millisDeserialized);
+            int nanosDeserialized = decodeUnsignedInt(b, Bytes.SIZEOF_LONG);
+            ts.setNanos(nanosDeserialized < 1000000 ? ts.getNanos() + nanosDeserialized : nanosDeserialized);
+            v = ts;
+        } else if (phType == PhType.UNSIGNED_TIME || phType == PhType.UNSIGNED_DATE) {
+            v = new Date(decodeUnsignedLong(b, 0));
+        } else if (phType == PhType.UNSIGNED_TIMESTAMP) {
+            long millisDeserialized = decodeUnsignedLong(b, 0);
+            Timestamp ts = new Timestamp(millisDeserialized);
+            int nanosDeserialized = decodeUnsignedInt(b, Bytes.SIZEOF_LONG);
+            ts.setNanos(nanosDeserialized < 1000000 ? ts.getNanos() + nanosDeserialized : nanosDeserialized);
+            v = ts;
+        } else if (phType == PhType.VARBINARY) {
+            v = b;
+        } else if (phType == PhType.VARCHAR || phType == PhType.DEFAULT) {
+            v = Bytes.toString(b);
+        } else if (phType == PhType.DECIMAL) {
+            v = decodeDecimal(b, 0, b.length);
+        }
+
+        return v;
+    }
+
+    private static int decodeInt(byte[] bytes, int o) {
+        checkForSufficientLength(bytes, o, Bytes.SIZEOF_INT);
+        int v;
+        v = bytes[o] ^ 0x80; // Flip sign bit back
+        for (int i = 1; i < Bytes.SIZEOF_INT; i++) {
+            v = (v << 8) + (bytes[o + i] & 0xff);
+        }
+        return v;
+    }
+
+    private static int encodeInt(int v, byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_INT);
+        b[o + 0] = (byte) ((v >> 24) ^ 0x80); // Flip sign bit so that INTEGER
+                                              // is binary comparable
+        b[o + 1] = (byte) (v >> 16);
+        b[o + 2] = (byte) (v >> 8);
+        b[o + 3] = (byte) v;
+        return Bytes.SIZEOF_INT;
+    }
+
+    private static int decodeUnsignedInt(byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_INT);
+
+        int v = Bytes.toInt(b, o);
+        if (v < 0) {
+            throw new RuntimeException();
+        }
+        return v;
+    }
+
+    private static int encodeUnsignedInt(int v, byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_INT);
+        if (v < 0) {
+            throw new RuntimeException();
+        }
+        Bytes.putInt(b, o, v);
+        return Bytes.SIZEOF_INT;
+    }
+
+    private static long decodeLong(byte[] bytes, int o) {
+        checkForSufficientLength(bytes, o, Bytes.SIZEOF_LONG);
+        long v;
+        byte b = bytes[o];
+        v = b ^ 0x80; // Flip sign bit back
+        for (int i = 1; i < Bytes.SIZEOF_LONG; i++) {
+            b = bytes[o + i];
+            v = (v << 8) + (b & 0xff);
+        }
+        return v;
+    }
+
+    private static int encodeLong(long v, byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_LONG);
+        b[o + 0] = (byte) ((v >> 56) ^ 0x80); // Flip sign bit so that INTEGER
+                                              // is binary comparable
+        b[o + 1] = (byte) (v >> 48);
+        b[o + 2] = (byte) (v >> 40);
+        b[o + 3] = (byte) (v >> 32);
+        b[o + 4] = (byte) (v >> 24);
+        b[o + 5] = (byte) (v >> 16);
+        b[o + 6] = (byte) (v >> 8);
+        b[o + 7] = (byte) v;
+        return Bytes.SIZEOF_LONG;
+    }
+
+    private static long decodeUnsignedLong(byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_LONG);
+        long v = 0;
+        for (int i = o; i < o + Bytes.SIZEOF_LONG; i++) {
+            v <<= 8;
+            v ^= b[i] & 0xFF;
+        }
+
+        if (v < 0) {
+            throw new RuntimeException();
+        }
+        return v;
+    }
+
+    private static int encodeUnsignedLong(long v, byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_LONG);
+        if (v < 0) {
+            throw new RuntimeException();
+        }
+        Bytes.putLong(b, o, v);
+        return Bytes.SIZEOF_LONG;
+    }
+
+    private static short decodeShort(byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_SHORT);
+        int v;
+        v = b[o] ^ 0x80; // Flip sign bit back
+        for (int i = 1; i < Bytes.SIZEOF_SHORT; i++) {
+            v = (v << 8) + (b[o + i] & 0xff);
+        }
+        return (short) v;
+    }
+
+    private static int encodeShort(short v, byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_SHORT);
+        b[o + 0] = (byte) ((v >> 8) ^ 0x80); // Flip sign bit so that Short is
+                                             // binary comparable
+        b[o + 1] = (byte) v;
+        return Bytes.SIZEOF_SHORT;
+    }
+
+    private static short decodeUnsignedShort(byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_SHORT);
+        short v = Bytes.toShort(b, o);
+        if (v < 0) {
+            throw new RuntimeException();
+        }
+        return v;
+    }
+
+    private static int encodeUnsignedShort(short v, byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_SHORT);
+        if (v < 0) {
+            throw new RuntimeException();
+        }
+        Bytes.putShort(b, o, v);
+        return Bytes.SIZEOF_SHORT;
+    }
+
+    private static byte decodeByte(byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_BYTE);
+        int v;
+        v = b[o] ^ 0x80; // Flip sign bit back
+        return (byte) v;
+    }
+
+    private static int encodeByte(byte v, byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_BYTE);
+        b[o] = (byte) (v ^ 0x80); // Flip sign bit so that Short is binary
+                                  // comparable
+        return Bytes.SIZEOF_BYTE;
+    }
+
+    private static byte decodeUnsignedByte(byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_BYTE);
+        byte v = b[o];
+        if (v < 0) {
+            throw new RuntimeException();
+        }
+        return v;
+    }
+
+    private static int encodeUnsignedByte(byte v, byte[] b, int o) {
+        if (v < 0) {
+            throw new RuntimeException();
+        }
+        Bytes.putByte(b, o, v);
+        return Bytes.SIZEOF_BYTE;
+    }
+
+    private static float decodeFloat(byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_INT);
+        int value;
+        value = Bytes.toInt(b, o);
+        value--;
+        value ^= (~value >> Integer.SIZE - 1) | Integer.MIN_VALUE;
+        return Float.intBitsToFloat(value);
+    }
+
+    private static int encodeFloat(float v, byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_FLOAT);
+        int i = Float.floatToIntBits(v);
+        i = (i ^ ((i >> Integer.SIZE - 1) | Integer.MIN_VALUE)) + 1;
+        Bytes.putInt(b, o, i);
+        return Bytes.SIZEOF_FLOAT;
+    }
+
+    private static float decodeUnsignedFloat(byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_FLOAT);
+        float v = Bytes.toFloat(b, o);
+        if (v < 0) {
+            throw new RuntimeException();
+        }
+        return v;
+    }
+
+    private static int encodeUnsignedFloat(float v, byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_FLOAT);
+        if (v < 0) {
+            throw new RuntimeException();
+        }
+        Bytes.putFloat(b, o, v);
+        return Bytes.SIZEOF_FLOAT;
+    }
+
+    private static double decodeDouble(byte[] bytes, int o) {
+        checkForSufficientLength(bytes, o, Bytes.SIZEOF_LONG);
+        long l;
+        l = Bytes.toLong(bytes, o);
+        l--;
+        l ^= (~l >> Long.SIZE - 1) | Long.MIN_VALUE;
+        return Double.longBitsToDouble(l);
+    }
+
+    private static int encodeDouble(double v, byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_LONG);
+        long l = Double.doubleToLongBits(v);
+        l = (l ^ ((l >> Long.SIZE - 1) | Long.MIN_VALUE)) + 1;
+        Bytes.putLong(b, o, l);
+        return Bytes.SIZEOF_LONG;
+    }
+
+    private static double decodeUnsignedDouble(byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_DOUBLE);
+        double v = Bytes.toDouble(b, o);
+        if (v < 0) {
+            throw new RuntimeException();
+        }
+        return v;
+    }
+
+    private static int encodeUnsignedDouble(double v, byte[] b, int o) {
+        checkForSufficientLength(b, o, Bytes.SIZEOF_DOUBLE);
+        if (v < 0) {
+            throw new RuntimeException();
+        }
+        Bytes.putDouble(b, o, v);
+        return Bytes.SIZEOF_DOUBLE;
+    }
+
+    private static int encodeDate(Object v, byte[] b, int o) {
+        if (v instanceof Date) {
+            encodeLong(((Date) v).getTime(), b, 0);
+        } else if (v instanceof String) {
+            String dateStr = (String) v;
+            Date date;
+            try {
+                date = parseDatetime(dateStr);
+                if (date != null) {
+                    encodeLong(date.getTime(), b, 0);
+                }
+            } catch (Exception e) {
+                throw new RuntimeException(e);
+            }
+        }
+        return Bytes.SIZEOF_LONG;
+    }
+
+    private static int encodeTimestamp(Object v, byte[] b, int o) {
+        if (v instanceof Timestamp) {
+            Timestamp ts = (Timestamp) v;
+            encodeLong(ts.getTime(), b, o);
+            Bytes.putInt(b, Bytes.SIZEOF_LONG, ts.getNanos() % 1000000);
+        } else {
+            encodeDate(v, b, o);
+        }
+        return Bytes.SIZEOF_LONG + Bytes.SIZEOF_INT;
+    }
+
+    private static int encodeUnsignedDate(Object v, byte[] b, int o) {
+        if (v instanceof Date) {
+            encodeUnsignedLong(((Date) v).getTime(), b, 0);
+        } else if (v instanceof String) {
+            String dateStr = (String) v;
+            Date date;
+            try {
+                date = parseDatetime(dateStr);
+                if (date != null) {
+                    encodeUnsignedLong(date.getTime(), b, 0);
+                }
+            } catch (Exception e) {
+                throw new RuntimeException(e);
+            }
+        }
+        return Bytes.SIZEOF_LONG;
+    }
+
+    private static int encodeUnsignedTimestamp(Object v, byte[] b, int o) {
+        if (v instanceof Timestamp) {
+            Timestamp ts = (Timestamp) v;
+            encodeUnsignedLong(ts.getTime(), b, o);
+            Bytes.putInt(b, Bytes.SIZEOF_LONG, ts.getNanos() % 1000000);
+        } else {
+            encodeUnsignedDate(v, b, o);
+        }
+        return Bytes.SIZEOF_LONG + Bytes.SIZEOF_INT;
+    }
+
+    private static byte[] encodeDecimal(Object object) {
+        if (object == null) {
+            return new byte[0];
+        }
+        BigDecimal v = (BigDecimal) object;
+        v = v.round(DEFAULT_MATH_CONTEXT).stripTrailingZeros();
+        int len = getLength(v);
+        byte[] result = new byte[Math.min(len, 21)];
+        decimalToBytes(v, result, 0, len);
+        return result;
+    }
+
+    private static BigDecimal decodeDecimal(byte[] bytes, int offset, int length) {
+        if (length == 1 && bytes[offset] == ZERO_BYTE) {
+            return BigDecimal.ZERO;
+        }
+        int signum = ((bytes[offset] & 0x80) == 0) ? -1 : 1;
+        int scale;
+        int index;
+        int digitOffset;
+        long multiplier = 100L;
+        int begIndex = offset + 1;
+        if (signum == 1) {
+            scale = (byte) (((bytes[offset] & 0x7F) - 65) * -2);
+            index = offset + length;
+            digitOffset = POS_DIGIT_OFFSET;
+        } else {
+            scale = (byte) ((~bytes[offset] - 65 - 128) * -2);
+            index = offset + length - (bytes[offset + length - 1] == NEG_TERMINAL_BYTE ? 1 : 0);
+            digitOffset = -NEG_DIGIT_OFFSET;
+        }
+        length = index - offset;
+        long l = signum * bytes[--index] - digitOffset;
+        if (l % 10 == 0) { // trailing zero
+            scale--; // drop trailing zero and compensate in the scale
+            l /= 10;
+            multiplier = 10;
+        }
+        // Use long arithmetic for as long as we can
+        while (index > begIndex) {
+            if (l >= MAX_LONG_FOR_DESERIALIZE || multiplier >= Long.MAX_VALUE / 100) {
+                multiplier = LongMath.divide(multiplier, 100L, RoundingMode.UNNECESSARY);
+                break; // Exit loop early so we don't overflow our multiplier
+            }
+            int digit100 = signum * bytes[--index] - digitOffset;
+            l += digit100 * multiplier;
+            multiplier = LongMath.checkedMultiply(multiplier, 100);
+        }
+
+        BigInteger bi;
+        // If still more digits, switch to BigInteger arithmetic
+        if (index > begIndex) {
+            bi = BigInteger.valueOf(l);
+            BigInteger biMultiplier = BigInteger.valueOf(multiplier).multiply(ONE_HUNDRED);
+            do {
+                int digit100 = signum * bytes[--index] - digitOffset;
+                bi = bi.add(biMultiplier.multiply(BigInteger.valueOf(digit100)));
+                biMultiplier = biMultiplier.multiply(ONE_HUNDRED);
+            } while (index > begIndex);
+            if (signum == -1) {
+                bi = bi.negate();
+            }
+        } else {
+            bi = BigInteger.valueOf(l * signum);
+        }
+        // Update the scale based on the precision
+        scale += (length - 2) * 2;
+        BigDecimal v = new BigDecimal(bi, scale);
+        return v;
+    }
+
+    private static int getLength(BigDecimal v) {
+        int signum = v.signum();
+        if (signum == 0) { // Special case for zero
+            return 1;
+        }
+        return (signum < 0 ? 2 : 1) + (v.precision() + 1 + (v.scale() % 2 == 0 ? 0 : 1)) / 2;
+    }
+
+    private static final int         MAX_PRECISION            = 38;
+    private static final MathContext DEFAULT_MATH_CONTEXT     = new MathContext(MAX_PRECISION, RoundingMode.HALF_UP);
+    private static final Integer     MAX_BIG_DECIMAL_BYTES    = 21;
+    private static final byte        ZERO_BYTE                = (byte) 0x80;
+    private static final byte        NEG_TERMINAL_BYTE        = (byte) 102;
+    private static final int         EXP_BYTE_OFFSET          = 65;
+    private static final int         POS_DIGIT_OFFSET         = 1;
+    private static final int         NEG_DIGIT_OFFSET         = 101;
+    private static final BigInteger  MAX_LONG                 = BigInteger.valueOf(Long.MAX_VALUE);
+    private static final BigInteger  MIN_LONG                 = BigInteger.valueOf(Long.MIN_VALUE);
+    private static final BigInteger  ONE_HUNDRED              = BigInteger.valueOf(100);
+    private static final long        MAX_LONG_FOR_DESERIALIZE = Long.MAX_VALUE / 1000;
+
+    private static int decimalToBytes(BigDecimal v, byte[] result, final int offset, int length) {
+        int signum = v.signum();
+        if (signum == 0) {
+            result[offset] = ZERO_BYTE;
+            return 1;
+        }
+        int index = offset + length;
+        int scale = v.scale();
+        int expOffset = scale % 2 * (scale < 0 ? -1 : 1);
+        int multiplyBy;
+        BigInteger divideBy;
+        if (expOffset == 0) {
+            multiplyBy = 1;
+            divideBy = ONE_HUNDRED;
+        } else {
+            multiplyBy = 10;
+            divideBy = BigInteger.TEN;
+        }
+        // Normalize the scale based on what is necessary to end up with a base
+        // 100
+        // decimal (i.e. 10.123e3)
+        int digitOffset;
+        BigInteger compareAgainst;
+        if (signum == 1) {
+            digitOffset = POS_DIGIT_OFFSET;
+            compareAgainst = MAX_LONG;
+            scale -= (length - 2) * 2;
+            result[offset] = (byte) ((-(scale + expOffset) / 2 + EXP_BYTE_OFFSET) | 0x80);
+        } else {
+            digitOffset = NEG_DIGIT_OFFSET;
+            compareAgainst = MIN_LONG;
+            // Scale adjustment shouldn't include terminal byte in length
+            scale -= (length - 2 - 1) * 2;
+            result[offset] = (byte) (~(-(scale + expOffset) / 2 + EXP_BYTE_OFFSET + 128) & 0x7F);
+            if (length <= MAX_BIG_DECIMAL_BYTES) {
+                result[--index] = NEG_TERMINAL_BYTE;
+            } else {
+                // Adjust length and offset down because we don't have enough
+                // room
+                length = MAX_BIG_DECIMAL_BYTES;
+                index = offset + length;
+            }
+        }
+        BigInteger bi = v.unscaledValue();
+        // Use BigDecimal arithmetic until we can fit into a long
+        while (bi.compareTo(compareAgainst) * signum > 0) {
+            BigInteger[] dandr = bi.divideAndRemainder(divideBy);
+            bi = dandr[0];
+            int digit = dandr[1].intValue();
+            result[--index] = (byte) (digit * multiplyBy + digitOffset);
+            multiplyBy = 1;
+            divideBy = ONE_HUNDRED;
+        }
+        long l = bi.longValue();
+        do {
+            long divBy = 100 / multiplyBy;
+            long digit = l % divBy;
+            l /= divBy;
+            result[--index] = (byte) (digit * multiplyBy + digitOffset);
+            multiplyBy = 1;
+        } while (l != 0);
+
+        return length;
+    }
+
+    private static void checkForSufficientLength(byte[] b, int offset, int requiredLength) {
+        if (b.length < offset + requiredLength) {
+            throw new RuntimeException("Expected length of at least " + requiredLength + " bytes, but had "
+                                       + (b.length - offset));
+        }
+    }
+
+    private static Date parseDatetime(String dateStr) {
+        Date date = null;
+        int len = dateStr.length();
+        if (len == 10 && dateStr.charAt(4) == '-' && dateStr.charAt(7) == '-') {
+            date = new DateTime(dateStr).toDate();
+        } else if (len == 8 && dateStr.charAt(2) == ':' && dateStr.charAt(5) == ':') {
+            date = new DateTime("T" + dateStr).toDate();
+        } else if (len >= 19 && dateStr.charAt(4) == '-' && dateStr.charAt(7) == '-' && dateStr.charAt(13) == ':'
+                   && dateStr.charAt(16) == ':') {
+            date = new DateTime(dateStr.replace(" ", "T")).toDate();
+        }
+        return date;
+    }
+}
@@ -0,0 +1,67 @@
+package com.alibaba.otter.canal.client.adapter.hbase.support;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.math.BigDecimal;
+import java.math.BigInteger;
+import java.util.Date;
+
+/**
+ * Java类型
+ *
+ * @author machengyuan 2018-8-21 下午06:11:36
+ * @version 1.0.0
+ */
+public enum Type {
+                  DEFAULT, STRING, INTEGER, LONG, SHORT, BOOLEAN, FLOAT, DOUBLE, BIGDECIMAL, DATE, BYTE, BYTES;
+
+    private static Logger logger = LoggerFactory.getLogger(Type.class);
+
+    public static Type getType(String type) {
+        if (type == null) {
+            return DEFAULT;
+        }
+        try {
+            return Type.valueOf(type);
+        } catch (Exception e) {
+            logger.error(e.getMessage(), e);
+            return DEFAULT;
+        }
+    }
+
+    public static Type getType(Class<?> javaType) {
+        if (javaType == null) {
+            return DEFAULT;
+        }
+        Type type;
+        if (Integer.class == javaType || int.class == javaType) {
+            type = INTEGER;
+        } else if (Long.class == javaType || long.class == javaType) {
+            type = LONG;
+        } else if (Byte.class == javaType || byte.class == javaType) {
+            type = BYTE;
+        } else if (Short.class == javaType || short.class == javaType) {
+            type = SHORT;
+        } else if (Float.class == javaType || float.class == javaType) {
+            type = FLOAT;
+        } else if (Double.class == javaType || double.class == javaType) {
+            type = DOUBLE;
+        } else if (Boolean.class == javaType || boolean.class == javaType) {
+            type = BOOLEAN;
+        } else if (Date.class == javaType) {
+            type = DATE;
+        } else if (byte[].class == javaType) {
+            type = BYTES;
+        } else if (String.class == javaType) {
+            type = STRING;
+        } else if (BigDecimal.class == javaType) {
+            type = BIGDECIMAL;
+        } else if (BigInteger.class == javaType) {
+            type = LONG;
+        } else {
+            type = DEFAULT;
+        }
+        return type;
+    }
+}
@@ -0,0 +1,189 @@
+package com.alibaba.otter.canal.client.adapter.hbase.support;
+
+import java.math.BigDecimal;
+import java.math.BigInteger;
+import java.sql.Time;
+import java.sql.Timestamp;
+import java.util.Date;
+
+import org.apache.hadoop.hbase.util.Bytes;
+
+/**
+ * Java类型转换工具类
+ *
+ * @author machengyuan 2018-8-21 下午06:12:34
+ * @version 1.0.0
+ */
+public class TypeUtil {
+
+    public static byte[] toBytes(Object obj) {
+        if (obj == null) {
+            return null;
+        }
+        byte[] bytes;
+        Class<?> clazz = obj.getClass();
+        if (clazz == String.class) {
+            bytes = Bytes.toBytes((String) obj);
+        } else if (clazz == Integer.class || clazz == int.class) {
+            bytes = Bytes.toBytes((Integer) obj);
+        } else if (clazz == Long.class || clazz == long.class) {
+            bytes = Bytes.toBytes((Long) obj);
+        } else if (clazz == Short.class || clazz == short.class) {
+            bytes = Bytes.toBytes((Short) obj);
+        } else if (clazz == Boolean.class || clazz == boolean.class) {
+            bytes = Bytes.toBytes((Boolean) obj);
+        } else if (clazz == Float.class || clazz == float.class) {
+            bytes = Bytes.toBytes((Float) obj);
+        } else if (clazz == Double.class || clazz == double.class) {
+            bytes = Bytes.toBytes((Double) obj);
+        } else if (clazz == Byte.class || clazz == byte.class) {
+            bytes = new byte[] { (byte) obj };
+        } else if (clazz == BigDecimal.class) {
+            bytes = Bytes.toBytes((BigDecimal) obj);
+        } else if (clazz == BigInteger.class) {
+            bytes = Bytes.toBytes(((BigInteger) obj).longValue());
+        } else if (clazz == Date.class) {
+            bytes = Bytes.toBytes(((Date) obj).getTime());
+        } else if (clazz == byte[].class) {
+            bytes = (byte[]) obj;
+        } else {
+            // 其余类型统一转换为string
+            bytes = Bytes.toBytes(obj.toString());
+        }
+        return bytes;
+    }
+
+    public static byte[] toBytes(Object v, Type type) {
+        if (v == null) {
+            return null;
+        }
+        byte[] b = null;
+        if (type == Type.DEFAULT) {
+            Type type1 = Type.getType(v.getClass());
+            if (type1 != null && type1 != Type.DEFAULT) {
+                b = toBytes(v, type1);
+            }
+        } else if (type == Type.STRING) {
+            b = Bytes.toBytes(v.toString());
+        } else if (type == Type.INTEGER) {
+            b = Bytes.toBytes(((Number) v).intValue());
+        } else if (type == Type.LONG) {
+            b = Bytes.toBytes(((Number) v).longValue());
+        } else if (type == Type.SHORT) {
+            b = Bytes.toBytes(((Number) v).shortValue());
+        } else if (type == Type.BYTE) {
+            b = Bytes.toBytes(((Number) v).byteValue());
+        } else if (type == Type.FLOAT) {
+            b = Bytes.toBytes(((Number) v).floatValue());
+        } else if (type == Type.DOUBLE) {
+            b = Bytes.toBytes(((Number) v).doubleValue());
+        } else if (type == Type.BOOLEAN) {
+            b = Bytes.toBytes(((Boolean) v));
+        } else if (type == Type.DATE) {
+            b = Bytes.toBytes(((Date) v).getTime());
+        } else if (type == Type.BYTES) {
+            b = (byte[]) v;
+        } else if (type == Type.BIGDECIMAL) {
+            if (v instanceof BigDecimal) {
+                b = Bytes.toBytes((BigDecimal) v);
+            } else {
+                b = Bytes.toBytes(new BigDecimal(v.toString()));
+            }
+        }
+        return b;
+    }
+
+    @SuppressWarnings("unchecked")
+    public static <T> T toObject(byte[] bytes, Class<T> clazz) {
+        if (bytes == null) {
+            return null;
+        }
+        Object res;
+        if (String.class == clazz) {
+            res = Bytes.toString(bytes);
+        } else if (Integer.class == clazz || int.class == clazz) {
+            res = Bytes.toInt(bytes);
+        } else if (Long.class == clazz || long.class == clazz) {
+            res = Bytes.toLong(bytes);
+        } else if (Short.class == clazz || short.class == clazz) {
+            res = Bytes.toShort(bytes);
+        } else if (Boolean.class == clazz || boolean.class == clazz) {
+            res = Bytes.toBoolean(bytes);
+        } else if (Float.class == clazz || float.class == clazz) {
+            res = Bytes.toFloat(bytes);
+        } else if (Double.class == clazz || double.class == clazz) {
+            res = Bytes.toDouble(bytes);
+        } else if (Byte.class == clazz || byte.class == clazz) {
+            res = bytes[0];
+        } else if (BigDecimal.class == clazz) {
+            res = Bytes.toBigDecimal(bytes);
+        } else if (BigInteger.class == clazz) {
+            res = Bytes.toLong(bytes);
+        } else if (java.sql.Date.class == clazz) {
+            long ts = Bytes.toLong(bytes);
+            res = new java.sql.Date(ts);
+        } else if (Time.class == clazz) {
+            long ts = Bytes.toLong(bytes);
+            res = new Time(ts);
+        } else if (Timestamp.class == clazz) {
+            long ts = Bytes.toLong(bytes);
+            res = new Timestamp(ts);
+        } else if (Date.class == clazz) {
+            long ts = Bytes.toLong(bytes);
+            res = new Date(ts);
+        } else {
+            throw new IllegalArgumentException("mismatch class type");
+        }
+        return (T) res;
+    }
+
+    @SuppressWarnings("unchecked")
+    public static <T> T toObject(byte[] bytes, Type type) {
+        if (bytes == null) {
+            return null;
+        }
+        Object res = null;
+        if (type == Type.STRING || type == Type.DEFAULT) {
+            res = Bytes.toString(bytes);
+        } else if (type == Type.INTEGER) {
+            if (bytes.length == Bytes.SIZEOF_INT) {
+                res = Bytes.toInt(bytes);
+            }
+        } else if (type == Type.LONG) {
+            if (bytes.length == Bytes.SIZEOF_LONG) {
+                res = Bytes.toLong(bytes);
+            }
+        } else if (type == Type.SHORT) {
+            if (bytes.length == Bytes.SIZEOF_SHORT) {
+                res = Bytes.toShort(bytes);
+            }
+        } else if (type == Type.BYTE) {
+            if (bytes.length == Bytes.SIZEOF_BYTE) {
+                res = bytes[0];
+            }
+        } else if (type == Type.FLOAT) {
+            if (bytes.length == Bytes.SIZEOF_FLOAT) {
+                res = Bytes.toFloat(bytes);
+            }
+        } else if (type == Type.DOUBLE) {
+            if (bytes.length == Bytes.SIZEOF_DOUBLE) {
+                res = Bytes.toDouble(bytes);
+            }
+        } else if (type == Type.BOOLEAN) {
+            if (bytes.length == Bytes.SIZEOF_BOOLEAN) {
+                res = Bytes.toBoolean(bytes);
+            }
+        } else if (type == Type.DATE) {
+            if (bytes.length == Bytes.SIZEOF_LONG) {
+                res = new Date(Bytes.toLong(bytes));
+            }
+        } else if (type == Type.BYTES) {
+            res = bytes;
+        } else if (type == Type.BIGDECIMAL) {
+            res = Bytes.toBigDecimal(bytes);
+        } else {
+            throw new IllegalArgumentException("mismatch class type");
+        }
+        return (T) res;
+    }
+}
@@ -0,0 +1 @@
+hbase=com.alibaba.otter.canal.client.adapter.hbase.HbaseAdapter
@@ -0,0 +1,60 @@
+dataSourceKey: defaultDS
+hbaseMapping:
+  mode: PHOENIX  #NATIVE   #STRING
+  destination: example
+  database: mytest  # 数据库名
+  table: person2     # 数据库表名
+  hbaseTable: MYTEST.PERSON2   # HBase表名
+  family: CF  # 默认统一Family名称
+  uppercaseQualifier: true  # 字段名转大写, 默认为true
+  commitBatch: 3000 # 批量提交的大小
+  #rowKey: id,type  # 复合字段rowKey不能和columns中的rowKey重复
+  columns:
+    # 数据库字段:HBase对应字段
+    id: ROWKEY$UNSIGNED_LONG
+    name: NAME
+    email: EMAIL
+    type: $DECIMAL
+    c_time: C_TIME$UNSIGNED_TIMESTAMP
+    birthday: BIRTHDAY$DATE
+  excludeColumns:
+    - lat   # 忽略字段
+
+# -- NATIVE类型
+# $DEFAULT
+# $STRING
+# $INTEGER
+# $LONG
+# $SHORT
+# $BOOLEAN
+# $FLOAT
+# $DOUBLE
+# $BIGDECIMAL
+# $DATE
+# $BYTE
+# $BYTES
+
+# -- PHOENIX类型
+# $DEFAULT                  对应PHOENIX里的VARCHAR
+# $UNSIGNED_INT             对应PHOENIX里的UNSIGNED_INT           4字节
+# $UNSIGNED_LONG            对应PHOENIX里的UNSIGNED_LONG          8字节
+# $UNSIGNED_TINYINT         对应PHOENIX里的UNSIGNED_TINYINT       1字节
+# $UNSIGNED_SMALLINT        对应PHOENIX里的UNSIGNED_SMALLINT      2字节
+# $UNSIGNED_FLOAT           对应PHOENIX里的UNSIGNED_FLOAT         4字节
+# $UNSIGNED_DOUBLE          对应PHOENIX里的UNSIGNED_DOUBLE        8字节
+# $INTEGER                  对应PHOENIX里的INTEGER                4字节
+# $BIGINT                   对应PHOENIX里的BIGINT                 8字节
+# $TINYINT                  对应PHOENIX里的TINYINT                1字节
+# $SMALLINT                 对应PHOENIX里的SMALLINT               2字节
+# $FLOAT                    对应PHOENIX里的FLOAT                  4字节
+# $DOUBLE                    对应PHOENIX里的DOUBLE                 8字节
+# $BOOLEAN                  对应PHOENIX里的BOOLEAN                1字节
+# $TIME                     对应PHOENIX里的TIME                   8字节
+# $DATE                     对应PHOENIX里的DATE                   8字节
+# $TIMESTAMP                对应PHOENIX里的TIMESTAMP              12字节
+# $UNSIGNED_TIME            对应PHOENIX里的UNSIGNED_TIME          8字节
+# $UNSIGNED_DATE            对应PHOENIX里的UNSIGNED_DATE          8字节
+# $UNSIGNED_TIMESTAMP       对应PHOENIX里的UNSIGNED_TIMESTAMP     12字节
+# $VARCHAR                  对应PHOENIX里的VARCHAR                动态长度
+# $VARBINARY                对应PHOENIX里的VARBINARY              动态长度
+# $DECIMAL                  对应PHOENIX里的DECIMAL                动态长度
@@ -0,0 +1,207 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <parent>
+        <artifactId>canal.client-adapter</artifactId>
+        <groupId>com.alibaba.otter</groupId>
+        <version>1.1.1</version>
+    </parent>
+    <modelVersion>4.0.0</modelVersion>
+    <groupId>com.alibaba.otter</groupId>
+    <artifactId>client-adapter.launcher</artifactId>
+    <packaging>jar</packaging>
+    <name>canal client adapter launcher module for otter ${project.version}</name>
+
+    <dependencyManagement>
+        <dependencies>
+            <dependency>
+                <groupId>org.springframework.boot</groupId>
+                <artifactId>spring-boot-dependencies</artifactId>
+                <version>2.0.1.RELEASE</version>
+                <type>pom</type>
+                <scope>import</scope>
+            </dependency>
+        </dependencies>
+    </dependencyManagement>
+
+    <dependencies>
+        <dependency>
+            <groupId>com.alibaba.otter</groupId>
+            <artifactId>client-adapter.common</artifactId>
+            <version>${project.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>com.alibaba.otter</groupId>
+            <artifactId>canal.client</artifactId>
+            <version>1.1.1</version>
+        </dependency>
+        <dependency>
+            <groupId>org.yaml</groupId>
+            <artifactId>snakeyaml</artifactId>
+            <version>1.19</version>
+        </dependency>
+        <dependency>
+            <groupId>org.springframework.boot</groupId>
+            <artifactId>spring-boot-starter-web</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>org.springframework.boot</groupId>
+            <artifactId>spring-boot-configuration-processor</artifactId>
+            <optional>true</optional>
+        </dependency>
+        <dependency>
+            <groupId>org.apache.curator</groupId>
+            <artifactId>curator-recipes</artifactId>
+            <version>2.10.0</version>
+        </dependency>
+        <!-- 单独引入rocketmq依赖 -->
+        <dependency>
+            <groupId>org.apache.rocketmq</groupId>
+            <artifactId>rocketmq-client</artifactId>
+            <version>4.3.0</version>
+        </dependency>
+        <!-- 单独引入kafka依赖 -->
+        <dependency>
+            <groupId>org.apache.kafka</groupId>
+            <artifactId>kafka-clients</artifactId>
+            <version>1.1.1</version>
+        </dependency>
+
+        <!-- outer adapter jar with dependencies-->
+        <dependency>
+            <groupId>com.alibaba.otter</groupId>
+            <artifactId>client-adapter.logger</artifactId>
+            <version>${project.version}</version>
+            <exclusions>
+                <exclusion>
+                    <artifactId>*</artifactId>
+                    <groupId>*</groupId>
+                </exclusion>
+            </exclusions>
+            <classifier>jar-with-dependencies</classifier>
+            <optional>true</optional>
+        </dependency>
+        <dependency>
+            <groupId>com.alibaba.otter</groupId>
+            <artifactId>client-adapter.hbase</artifactId>
+            <version>${project.version}</version>
+            <exclusions>
+                <exclusion>
+                    <artifactId>*</artifactId>
+                    <groupId>*</groupId>
+                </exclusion>
+            </exclusions>
+            <classifier>jar-with-dependencies</classifier>
+            <optional>true</optional>
+        </dependency>
+    </dependencies>
+
+    <build>
+        <finalName>canal-adapter-launcher</finalName>
+        <plugins>
+            <plugin>
+                <groupId>org.springframework.boot</groupId>
+                <artifactId>spring-boot-maven-plugin</artifactId>
+                <version>2.0.1.RELEASE</version>
+                <configuration>
+                    <excludes>
+                        <exclude>
+                            <groupId>com.alibaba.otter</groupId>
+                            <artifactId>client-adapter.hbase</artifactId>
+                        </exclude>
+                        <exclude>
+                            <groupId>com.alibaba.otter</groupId>
+                            <artifactId>client-adapter.logger</artifactId>
+                        </exclude>
+                    </excludes>
+                </configuration>
+                <executions>
+                    <execution>
+                        <goals>
+                            <goal>repackage</goal>
+                        </goals>
+                    </execution>
+                </executions>
+            </plugin>
+            <plugin>
+                <artifactId>maven-antrun-plugin</artifactId>
+                <executions>
+                    <execution>
+                        <phase>package</phase>
+                        <goals>
+                            <goal>run</goal>
+                        </goals>
+                        <configuration>
+                            <tasks>
+                                <copy todir="${project.basedir}/target/config" overwrite="true">
+                                    <fileset dir="${project.basedir}/src/main/resources" erroronmissingdir="true">
+                                        <include name="*.yml" />
+                                    </fileset>
+                                </copy>
+                                <copy todir="${project.basedir}/target/config/hbase" overwrite="true">
+                                    <fileset dir="${project.basedir}/../hbase/src/main/resources/hbase" erroronmissingdir="true">
+                                        <include name="*.yml" />
+                                    </fileset>
+                                </copy>
+                            </tasks>
+                        </configuration>
+                    </execution>
+                </executions>
+            </plugin>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-dependency-plugin</artifactId>
+                <version>2.10</version>
+                <executions>
+                    <execution>
+                        <id>copy-dependencies-to-canal-client-service</id>
+                        <phase>package</phase>
+                        <goals>
+                            <goal>copy-dependencies</goal>
+                        </goals>
+                        <configuration>
+                            <includeClassifiers>jar-with-dependencies</includeClassifiers>
+                            <outputDirectory>${project.basedir}/target/lib</outputDirectory>
+                        </configuration>
+                    </execution>
+                </executions>
+            </plugin>
+        </plugins>
+    </build>
+
+    <profiles>
+        <profile>
+            <id>release</id>
+            <activation>
+                <property>
+                    <name>env</name>
+                    <value>release</value>
+                </property>
+            </activation>
+
+            <build>
+                <plugins>
+                    <plugin>
+                        <artifactId>maven-assembly-plugin</artifactId>
+                        <configuration>
+                            <appendAssemblyId>false</appendAssemblyId>
+                            <descriptors>
+                                <descriptor>${basedir}/src/main/assembly/release.xml</descriptor>
+                            </descriptors>
+                            <finalName>${project.artifactId}-${project.version}</finalName>
+                            <outputDirectory>${project.basedir}/../../target</outputDirectory>
+                        </configuration>
+                        <executions>
+                            <execution>
+                                <id>make-assembly</id>
+                                <phase>package</phase>
+                                <goals>
+                                    <goal>single</goal>
+                                </goals>
+                            </execution>
+                        </executions>
+                    </plugin>
+                </plugins>
+            </build>
+        </profile>
+    </profiles>
+</project>
@@ -0,0 +1,34 @@
+<assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
+          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+          xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd">
+    <id>canal-adapter</id>
+    <formats>
+        <format>tar.gz</format>
+    </formats>
+    <fileSets>
+        <fileSet>
+            <directory>${project.build.directory}</directory>
+            <outputDirectory>/</outputDirectory>
+            <includes>
+                <include>*.jar</include>
+            </includes>
+        </fileSet>
+
+        <fileSet>
+            <directory>${project.build.directory}/lib</directory>
+            <outputDirectory>/lib/</outputDirectory>
+            <includes>
+                <include>*.jar</include>
+            </includes>
+        </fileSet>
+
+        <fileSet>
+            <directory>${project.build.directory}/config</directory>
+            <outputDirectory>/config/</outputDirectory>
+            <includes>
+                <include>**</include>
+            </includes>
+        </fileSet>
+    </fileSets>
+    <baseDirectory>/</baseDirectory>
+</assembly>
@@ -0,0 +1,18 @@
+package com.alibaba.otter.canal.adapter.launcher;
+
+import org.springframework.boot.autoconfigure.SpringBootApplication;
+import org.springframework.boot.builder.SpringApplicationBuilder;
+
+/**
+ * 启动入口
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+@SpringBootApplication
+public class CanalAdapterApplication {
+
+    public static void main(String[] args) {
+        new SpringApplicationBuilder(CanalAdapterApplication.class).run(args);
+    }
+}
@@ -0,0 +1,120 @@
+package com.alibaba.otter.canal.adapter.launcher.common;
+
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.locks.ReentrantLock;
+
+import javax.annotation.PostConstruct;
+import javax.annotation.Resource;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.recipes.locks.InterProcessMutex;
+import org.springframework.stereotype.Component;
+
+import com.alibaba.otter.canal.adapter.launcher.config.CuratorClient;
+
+/**
+ * Etl 同步锁
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+@Component
+public class EtlLock {
+
+    private static final Map<String, ReentrantLock>     LOCAL_LOCK       = new ConcurrentHashMap<>();
+
+    private static final Map<String, InterProcessMutex> DISTRIBUTED_LOCK = new ConcurrentHashMap<>();
+
+    private static Mode                                 mode             = Mode.LOCAL;
+
+    @Resource
+    private CuratorClient                               curatorClient;
+
+    @PostConstruct
+    public void init() {
+        CuratorFramework curator = curatorClient.getCurator();
+        if (curator != null) {
+            mode = Mode.DISTRIBUTED;
+        } else {
+            mode = Mode.LOCAL;
+        }
+    }
+
+    private ReentrantLock getLock(String key) {
+        ReentrantLock lock = LOCAL_LOCK.get(key);
+        if (lock == null) {
+            synchronized (EtlLock.class) {
+                lock = LOCAL_LOCK.get(key);
+                if (lock == null) {
+                    lock = new ReentrantLock();
+                    LOCAL_LOCK.put(key, lock);
+                }
+            }
+        }
+        return lock;
+    }
+
+    private InterProcessMutex getRemoteLock(String key) {
+        InterProcessMutex lock = DISTRIBUTED_LOCK.get(key);
+        if (lock == null) {
+            synchronized (EtlLock.class) {
+                lock = DISTRIBUTED_LOCK.get(key);
+                if (lock == null) {
+                    lock = new InterProcessMutex(curatorClient.getCurator(), key);
+                    DISTRIBUTED_LOCK.put(key, lock);
+                }
+            }
+        }
+        return lock;
+    }
+
+    public void lock(String key) throws Exception {
+        if (mode == Mode.LOCAL) {
+            getLock(key).lock();
+        } else {
+            InterProcessMutex lock = getRemoteLock(key);
+            lock.acquire();
+        }
+    }
+
+    public boolean tryLock(String key, long timeout, TimeUnit unit) {
+        try {
+            if (mode == Mode.LOCAL) {
+                return getLock(key).tryLock(timeout, unit);
+            } else {
+                InterProcessMutex lock = getRemoteLock(key);
+                return lock.acquire(timeout, unit);
+            }
+        } catch (Exception e) {
+            return false;
+        }
+    }
+
+    public boolean tryLock(String key) {
+        try {
+            if (mode == Mode.LOCAL) {
+                return getLock(key).tryLock();
+            } else {
+                InterProcessMutex lock = getRemoteLock(key);
+                return lock.acquire(500, TimeUnit.MILLISECONDS);
+            }
+        } catch (Exception e) {
+            return false;
+        }
+    }
+
+    public void unlock(String key) {
+        if (mode == Mode.LOCAL) {
+            getLock(key).unlock();
+        } else {
+            InterProcessMutex lock = getRemoteLock(key);
+            try {
+                lock.release();
+            } catch (Exception e) {
+                // ignore
+            }
+        }
+    }
+}
@@ -0,0 +1,6 @@
+package com.alibaba.otter.canal.adapter.launcher.common;
+
+public enum Mode {
+                  LOCAL, // 本地模式
+                  DISTRIBUTED // 分布式
+}
@@ -0,0 +1,213 @@
+package com.alibaba.otter.canal.adapter.launcher.common;
+
+import java.nio.charset.StandardCharsets;
+import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+import javax.annotation.PostConstruct;
+import javax.annotation.Resource;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.recipes.cache.NodeCache;
+import org.apache.zookeeper.CreateMode;
+import org.apache.zookeeper.data.Stat;
+import org.springframework.stereotype.Component;
+
+import com.alibaba.otter.canal.adapter.launcher.config.AdapterCanalConfig;
+import com.alibaba.otter.canal.adapter.launcher.config.CuratorClient;
+import com.alibaba.otter.canal.common.utils.BooleanMutex;
+
+/**
+ * 同步开关
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+@Component
+public class SyncSwitch {
+
+    private static final String                    SYN_SWITCH_ZK_NODE = "/sync-switch/";
+
+    private static final Map<String, BooleanMutex> LOCAL_LOCK         = new ConcurrentHashMap<>();
+
+    private static final Map<String, BooleanMutex> DISTRIBUTED_LOCK   = new ConcurrentHashMap<>();
+
+    private static Mode                            mode               = Mode.LOCAL;
+
+    @Resource
+    private AdapterCanalConfig                     adapterCanalConfig;
+    @Resource
+    private CuratorClient                          curatorClient;
+
+    @PostConstruct
+    public void init() {
+        CuratorFramework curator = curatorClient.getCurator();
+        if (curator != null) {
+            mode = Mode.DISTRIBUTED;
+            DISTRIBUTED_LOCK.clear();
+            for (String destination : adapterCanalConfig.DESTINATIONS) {
+                // 对应每个destination注册锁
+                BooleanMutex mutex = new BooleanMutex(true);
+                initMutex(curator, destination, mutex);
+                DISTRIBUTED_LOCK.put(destination, mutex);
+                startListen(destination, mutex);
+            }
+        } else {
+            mode = Mode.LOCAL;
+            LOCAL_LOCK.clear();
+            for (String destination : adapterCanalConfig.DESTINATIONS) {
+                // 对应每个destination注册锁
+                LOCAL_LOCK.put(destination, new BooleanMutex(true));
+            }
+        }
+    }
+
+    private synchronized void startListen(String destination, BooleanMutex mutex) {
+        try {
+            String path = SYN_SWITCH_ZK_NODE + destination;
+            CuratorFramework curator = curatorClient.getCurator();
+            final NodeCache nodeCache = new NodeCache(curator, path);
+            nodeCache.start();
+            nodeCache.getListenable().addListener(() -> initMutex(curator, destination, mutex));
+        } catch (Exception e) {
+            throw new RuntimeException(e.getMessage());
+        }
+    }
+
+    private synchronized void initMutex(CuratorFramework curator, String destination, BooleanMutex mutex) {
+        try {
+            String path = SYN_SWITCH_ZK_NODE + destination;
+            Stat stat = curator.checkExists().forPath(path);
+            if (stat == null) {
+                if (!mutex.state()) {
+                    mutex.set(true);
+                }
+            } else {
+                String data = new String(curator.getData().forPath(path), StandardCharsets.UTF_8);
+                if ("on".equals(data)) {
+                    if (!mutex.state()) {
+                        mutex.set(true);
+                    }
+                } else {
+                    if (mutex.state()) {
+                        mutex.set(false);
+                    }
+                }
+            }
+        } catch (Exception e) {
+            throw new RuntimeException(e.getMessage());
+        }
+    }
+
+    public synchronized void off(String destination) {
+        if (mode == Mode.LOCAL) {
+            BooleanMutex mutex = LOCAL_LOCK.get(destination);
+            if (mutex != null && mutex.state()) {
+                mutex.set(false);
+            }
+        } else {
+            try {
+                String path = SYN_SWITCH_ZK_NODE + destination;
+                try {
+                    curatorClient.getCurator()
+                        .create()
+                        .creatingParentContainersIfNeeded()
+                        .withMode(CreateMode.PERSISTENT)
+                        .forPath(path, "off".getBytes(StandardCharsets.UTF_8));
+                } catch (Exception e) {
+                    curatorClient.getCurator().setData().forPath(path, "off".getBytes(StandardCharsets.UTF_8));
+                }
+            } catch (Exception e) {
+                throw new RuntimeException(e);
+            }
+        }
+    }
+
+    public synchronized void on(String destination) {
+        if (mode == Mode.LOCAL) {
+            BooleanMutex mutex = LOCAL_LOCK.get(destination);
+            if (mutex != null && !mutex.state()) {
+                mutex.set(true);
+            }
+        } else {
+            try {
+                String path = SYN_SWITCH_ZK_NODE + destination;
+                try {
+                    curatorClient.getCurator()
+                        .create()
+                        .creatingParentContainersIfNeeded()
+                        .withMode(CreateMode.PERSISTENT)
+                        .forPath(path, "on".getBytes(StandardCharsets.UTF_8));
+                } catch (Exception e) {
+                    curatorClient.getCurator().setData().forPath(path, "on".getBytes(StandardCharsets.UTF_8));
+                }
+            } catch (Exception e) {
+                throw new RuntimeException(e);
+            }
+        }
+    }
+
+    public synchronized void release(String destination) {
+        if (mode == Mode.LOCAL) {
+            BooleanMutex mutex = LOCAL_LOCK.get(destination);
+            if (mutex != null && !mutex.state()) {
+                mutex.set(true);
+            }
+        }
+        if (mode == Mode.DISTRIBUTED) {
+            BooleanMutex mutex = DISTRIBUTED_LOCK.get(destination);
+            if (mutex != null && !mutex.state()) {
+                mutex.set(true);
+            }
+        }
+    }
+
+    public Boolean status(String destination) {
+        if (mode == Mode.LOCAL) {
+            BooleanMutex mutex = LOCAL_LOCK.get(destination);
+            if (mutex != null) {
+                return mutex.state();
+            } else {
+                return null;
+            }
+        } else {
+            BooleanMutex mutex = DISTRIBUTED_LOCK.get(destination);
+            if (mutex != null) {
+                return mutex.state();
+            } else {
+                return null;
+            }
+        }
+    }
+
+    public void get(String destination) throws InterruptedException {
+        if (mode == Mode.LOCAL) {
+            BooleanMutex mutex = LOCAL_LOCK.get(destination);
+            if (mutex != null) {
+                mutex.get();
+            }
+        } else {
+            BooleanMutex mutex = DISTRIBUTED_LOCK.get(destination);
+            if (mutex != null) {
+                mutex.get();
+            }
+        }
+    }
+
+    public void get(String destination, long timeout, TimeUnit unit) throws InterruptedException, TimeoutException {
+        if (mode == Mode.LOCAL) {
+            BooleanMutex mutex = LOCAL_LOCK.get(destination);
+            if (mutex != null) {
+                mutex.get(timeout, unit);
+            }
+        } else {
+            BooleanMutex mutex = DISTRIBUTED_LOCK.get(destination);
+            if (mutex != null) {
+                mutex.get(timeout, unit);
+            }
+        }
+    }
+
+}
@@ -0,0 +1,51 @@
+package com.alibaba.otter.canal.adapter.launcher.config;
+
+import java.util.LinkedHashSet;
+import java.util.List;
+import java.util.Set;
+
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.stereotype.Component;
+
+import com.alibaba.otter.canal.client.adapter.support.CanalClientConfig;
+
+/**
+ * canal 的相关配置类
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+@Component
+@ConfigurationProperties(prefix = "canal.conf")
+public class AdapterCanalConfig extends CanalClientConfig {
+
+    public final Set<String> DESTINATIONS = new LinkedHashSet<>();
+
+    @Override
+    public void setCanalInstances(List<CanalInstance> canalInstances) {
+        super.setCanalInstances(canalInstances);
+
+        if (canalInstances != null) {
+            synchronized (DESTINATIONS) {
+                DESTINATIONS.clear();
+                for (CanalInstance canalInstance : canalInstances) {
+                    DESTINATIONS.add(canalInstance.getInstance());
+                }
+            }
+        }
+    }
+
+    @Override
+    public void setMqTopics(List<MQTopic> mqTopics) {
+        super.setMqTopics(mqTopics);
+
+        if (mqTopics != null) {
+            synchronized (DESTINATIONS) {
+                DESTINATIONS.clear();
+                for (MQTopic mqTopic : mqTopics) {
+                    DESTINATIONS.add(mqTopic.getTopic());
+                }
+            }
+        }
+    }
+}
@@ -0,0 +1,86 @@
+package com.alibaba.otter.canal.adapter.launcher.config;
+
+import java.sql.SQLException;
+import java.util.List;
+import java.util.Map;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.stereotype.Component;
+
+import com.alibaba.druid.pool.DruidDataSource;
+import com.alibaba.otter.canal.client.adapter.support.AdapterConfigs;
+import com.alibaba.otter.canal.client.adapter.support.DatasourceConfig;
+
+/**
+ * 适配器数据源及配置文件列表配置类
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+@Component
+@ConfigurationProperties(prefix = "adapter.conf")
+public class AdapterConfig {
+
+    private static Logger                 logger = LoggerFactory.getLogger(AdapterConfig.class);
+
+    private Map<String, DatasourceConfig> datasourceConfigs;
+
+    private List<String>                  adapterConfigs;
+
+    public List<String> getAdapterConfigs() {
+        return adapterConfigs;
+    }
+
+    public Map<String, DatasourceConfig> getDatasourceConfigs() {
+        return datasourceConfigs;
+    }
+
+    public void setDatasourceConfigs(Map<String, DatasourceConfig> datasourceConfigs) {
+        this.datasourceConfigs = datasourceConfigs;
+
+        if (datasourceConfigs != null) {
+            for (Map.Entry<String, DatasourceConfig> entry : datasourceConfigs.entrySet()) {
+                DatasourceConfig datasourceConfig = entry.getValue();
+                // 加载数据源连接池
+                DruidDataSource ds = new DruidDataSource();
+                ds.setDriverClassName(datasourceConfig.getDriver());
+                ds.setUrl(datasourceConfig.getUrl());
+                ds.setUsername(datasourceConfig.getUsername());
+                ds.setPassword(datasourceConfig.getPassword());
+                ds.setInitialSize(1);
+                ds.setMinIdle(1);
+                ds.setMaxActive(datasourceConfig.getMaxActive());
+                ds.setMaxWait(60000);
+                ds.setTimeBetweenEvictionRunsMillis(60000);
+                ds.setMinEvictableIdleTimeMillis(300000);
+                ds.setPoolPreparedStatements(false);
+                ds.setMaxPoolPreparedStatementPerConnectionSize(20);
+                ds.setValidationQuery("select 1");
+                try {
+                    ds.init();
+                } catch (SQLException e) {
+                    logger.error("ERROR ## failed to initial datasource: " + datasourceConfig.getUrl(), e);
+                }
+                DatasourceConfig.DATA_SOURCES.put(entry.getKey(), ds);
+            }
+        }
+    }
+
+    public void setAdapterConfigs(List<String> adapterConfigs) {
+        this.adapterConfigs = adapterConfigs;
+
+        if (adapterConfigs != null) {
+            AdapterConfigs.clear();
+            for (String adapterConfig : adapterConfigs) {
+                int idx = adapterConfig.indexOf("/");
+                if (idx > -1) {
+                    String type = adapterConfig.substring(0, idx);
+                    String ymlFile = adapterConfig.substring(idx + 1);
+                    AdapterConfigs.put(type, ymlFile);
+                }
+            }
+        }
+    }
+}
@@ -0,0 +1,42 @@
+package com.alibaba.otter.canal.adapter.launcher.config;
+
+import javax.annotation.PostConstruct;
+import javax.annotation.Resource;
+
+import org.apache.curator.framework.CuratorFramework;
+import org.apache.curator.framework.CuratorFrameworkFactory;
+import org.apache.curator.retry.ExponentialBackoffRetry;
+import org.springframework.stereotype.Component;
+
+/**
+ * curator 配置类
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+@Component
+public class CuratorClient {
+
+    @Resource
+    private AdapterCanalConfig adapterCanalConfig;
+
+    private CuratorFramework   curator = null;
+
+    @PostConstruct
+    public void init() {
+        if (adapterCanalConfig.getZookeeperHosts() != null) {
+            curator = CuratorFrameworkFactory.builder()
+                .connectString(adapterCanalConfig.getZookeeperHosts())
+                .retryPolicy(new ExponentialBackoffRetry(1000, 3))
+                .sessionTimeoutMs(6000)
+                .connectionTimeoutMs(3000)
+                .namespace("canal-adapter")
+                .build();
+            curator.start();
+        }
+    }
+
+    public CuratorFramework getCurator() {
+        return curator;
+    }
+}
@@ -0,0 +1,34 @@
+package com.alibaba.otter.canal.adapter.launcher.config;
+
+import org.springframework.beans.BeansException;
+import org.springframework.context.ApplicationContext;
+import org.springframework.context.ApplicationContextAware;
+import org.springframework.stereotype.Component;
+
+/**
+ * spring util配置类
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+@Component
+public class SpringContext implements ApplicationContextAware {
+
+    private static ApplicationContext context;
+
+    /*
+     * 注入ApplicationContext
+     */
+    public void setApplicationContext(final ApplicationContext context) throws BeansException {
+        // 在加载Spring时自动获得context
+        SpringContext.context = context;
+    }
+
+    public static Object getBean(final String beanName) {
+        return SpringContext.context.getBean(beanName);
+    }
+
+    public static Object getBean(final Class<?> clz) {
+        return context.getBean(clz);
+    }
+}
@@ -0,0 +1,158 @@
+package com.alibaba.otter.canal.adapter.launcher.loader;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.ExecutionException;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.Future;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.alibaba.otter.canal.adapter.launcher.common.SyncSwitch;
+import com.alibaba.otter.canal.adapter.launcher.config.SpringContext;
+import com.alibaba.otter.canal.client.adapter.OuterAdapter;
+import com.alibaba.otter.canal.client.adapter.support.Dml;
+import com.alibaba.otter.canal.client.adapter.support.MessageUtil;
+import com.alibaba.otter.canal.protocol.FlatMessage;
+import com.alibaba.otter.canal.protocol.Message;
+
+/**
+ * 适配器工作线程抽象类
+ *
+ * @author rewerma 2018-8-19 下午11:30:49
+ * @version 1.0.0
+ */
+public abstract class AbstractCanalAdapterWorker {
+
+    protected final Logger                    logger  = LoggerFactory.getLogger(this.getClass());
+
+    protected String                          canalDestination;                                                // canal实例
+    protected List<List<OuterAdapter>>        canalOuterAdapters;                                              // 外部适配器
+    protected ExecutorService                 groupInnerExecutorService;                                       // 组内工作线程池
+    protected volatile boolean                running = false;                                                 // 是否运行中
+    protected Thread                          thread  = null;
+    protected Thread.UncaughtExceptionHandler handler = (t, e) -> logger.error("parse events has an error", e);
+
+    protected SyncSwitch                      syncSwitch;
+
+    public AbstractCanalAdapterWorker(List<List<OuterAdapter>> canalOuterAdapters){
+        this.canalOuterAdapters = canalOuterAdapters;
+        this.groupInnerExecutorService = Executors.newFixedThreadPool(canalOuterAdapters.size());
+        syncSwitch = (SyncSwitch) SpringContext.getBean(SyncSwitch.class);
+    }
+
+    protected void writeOut(final Message message) {
+        List<Future<Boolean>> futures = new ArrayList<>();
+        // 组间适配器并行运行
+        canalOuterAdapters.forEach(outerAdapters -> {
+            final List<OuterAdapter> adapters = outerAdapters;
+            futures.add(groupInnerExecutorService.submit(() -> {
+                try {
+                    // 组内适配器穿行运行，尽量不要配置组内适配器
+                    adapters.forEach(adapter -> {
+                        long begin = System.currentTimeMillis();
+                        MessageUtil.parse4Dml(canalDestination, message, adapter::sync);
+                        if (logger.isDebugEnabled()) {
+                            logger.debug("{} elapsed time: {}",
+                                adapter.getClass().getName(),
+                                (System.currentTimeMillis() - begin));
+                        }
+                    });
+                    return true;
+                } catch (Exception e) {
+                    return false;
+                }
+            }));
+
+            // 等待所有适配器写入完成
+            // 由于是组间并发操作，所以将阻塞直到耗时最久的工作组操作完成
+            futures.forEach(future -> {
+                try {
+                    if (!future.get()) {
+                        logger.error("Outer adapter write failed");
+                    }
+                } catch (InterruptedException | ExecutionException e) {
+                    // ignore
+                }
+            });
+        });
+    }
+
+    protected void writeOut(final FlatMessage flatMessage) {
+        List<Future<Boolean>> futures = new ArrayList<>();
+        // 组间适配器并行运行
+        canalOuterAdapters.forEach(outerAdapters -> {
+            futures.add(groupInnerExecutorService.submit(() -> {
+                try {
+                    // 组内适配器穿行运行，尽量不要配置组内适配器
+                    outerAdapters.forEach(adapter -> {
+                        long begin = System.currentTimeMillis();
+                        Dml dml = MessageUtil.flatMessage2Dml(canalDestination, flatMessage);
+                        adapter.sync(dml);
+                        if (logger.isDebugEnabled()) {
+                            logger.debug("{} elapsed time: {}",
+                                adapter.getClass().getName(),
+                                (System.currentTimeMillis() - begin));
+                        }
+                    });
+                    return true;
+                } catch (Exception e) {
+                    return false;
+                }
+            }));
+
+            // 等待所有适配器写入完成
+            // 由于是组间并发操作，所以将阻塞直到耗时最久的工作组操作完成
+            futures.forEach(future -> {
+                try {
+                    if (!future.get()) {
+                        logger.error("Outer adapter write failed");
+                    }
+                } catch (InterruptedException | ExecutionException e) {
+                    // ignore
+                }
+            });
+        });
+    }
+
+    public void start() {
+        if (!running) {
+            thread = new Thread(this::process);
+            thread.setUncaughtExceptionHandler(handler);
+            thread.start();
+            running = true;
+        }
+    }
+
+    protected abstract void process();
+
+    public void stop() {
+        try {
+            if (!running) {
+                return;
+            }
+
+            running = false;
+
+            syncSwitch.release(canalDestination);
+
+            logger.info("destination {} is waiting for adapters' worker thread die!", canalDestination);
+            if (thread != null) {
+                try {
+                    thread.join();
+                } catch (InterruptedException e) {
+                    // ignore
+                }
+            }
+            groupInnerExecutorService.shutdown();
+            logger.info("destination {} adapters worker thread dead!", canalDestination);
+            canalOuterAdapters.forEach(outerAdapters -> outerAdapters.forEach(OuterAdapter::destroy));
+            logger.info("destination {} all adapters destroyed!", canalDestination);
+        } catch (Exception e) {
+            logger.error(e.getMessage(), e);
+        }
+    }
+
+}
@@ -0,0 +1,93 @@
+package com.alibaba.otter.canal.adapter.launcher.loader;
+
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.kafka.clients.consumer.CommitFailedException;
+import org.apache.kafka.common.errors.WakeupException;
+
+import com.alibaba.otter.canal.client.adapter.OuterAdapter;
+import com.alibaba.otter.canal.client.kafka.KafkaCanalConnector;
+import com.alibaba.otter.canal.client.kafka.KafkaCanalConnectors;
+import com.alibaba.otter.canal.protocol.FlatMessage;
+import com.alibaba.otter.canal.protocol.Message;
+
+/**
+ * kafka对应的client适配器工作线程
+ *
+ * @author rewerma 2018-8-19 下午11:30:49
+ * @version 1.0.0
+ */
+public class CanalAdapterKafkaWorker extends AbstractCanalAdapterWorker {
+
+    private KafkaCanalConnector connector;
+    private String              topic;
+    private boolean             flatMessage;
+
+    public CanalAdapterKafkaWorker(String bootstrapServers, String topic, String groupId,
+                                   List<List<OuterAdapter>> canalOuterAdapters, boolean flatMessage){
+        super(canalOuterAdapters);
+        this.topic = topic;
+        this.canalDestination = topic;
+        this.flatMessage = flatMessage;
+        this.connector = KafkaCanalConnectors.newKafkaConnector(bootstrapServers, topic, null, groupId, flatMessage);
+        // connector.setSessionTimeout(1L, TimeUnit.MINUTES);
+    }
+
+    @Override
+    protected void process() {
+        while (!running)
+            ;
+        while (running) {
+            try {
+                syncSwitch.get(canalDestination);
+                logger.info("=============> Start to connect topic: {} <=============", this.topic);
+                connector.connect();
+                logger.info("=============> Start to subscribe topic: {} <=============", this.topic);
+                connector.subscribe();
+                logger.info("=============> Subscribe topic: {} succeed <=============", this.topic);
+                while (running) {
+                    try {
+                        Boolean status = syncSwitch.status(canalDestination);
+                        if (status != null && !status) {
+                            connector.disconnect();
+                            break;
+                        }
+
+                        List<?> messages;
+                        if (!flatMessage) {
+                            messages = connector.getListWithoutAck(100L, TimeUnit.MILLISECONDS);
+                        } else {
+                            messages = connector.getFlatListWithoutAck(100L, TimeUnit.MILLISECONDS);
+                        }
+                        if (messages != null) {
+                            for (final Object message : messages) {
+                                if (message instanceof FlatMessage) {
+                                    writeOut((FlatMessage) message);
+                                } else {
+                                    writeOut((Message) message);
+                                }
+                            }
+                        }
+                        connector.ack();
+                    } catch (CommitFailedException e) {
+                        logger.warn(e.getMessage());
+                    } catch (Throwable e) {
+                        logger.error(e.getMessage(), e);
+                        TimeUnit.SECONDS.sleep(1L);
+                    }
+                }
+            } catch (Exception e) {
+                logger.error(e.getMessage(), e);
+            }
+        }
+
+        try {
+            connector.unsubscribe();
+        } catch (WakeupException e) {
+            // No-op. Continue process
+        }
+        connector.disconnect();
+        logger.info("=============> Disconnect topic: {} <=============", this.topic);
+    }
+}
@@ -0,0 +1,152 @@
+package com.alibaba.otter.canal.adapter.launcher.loader;
+
+import java.net.InetSocketAddress;
+import java.net.SocketAddress;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+
+import org.apache.commons.lang.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.alibaba.otter.canal.client.adapter.OuterAdapter;
+import com.alibaba.otter.canal.client.adapter.support.CanalClientConfig;
+import com.alibaba.otter.canal.client.adapter.support.ExtensionLoader;
+import com.alibaba.otter.canal.client.adapter.support.OuterAdapterConfig;
+
+/**
+ * 外部适配器的加载器
+ *
+ * @version 1.0.0
+ */
+public class CanalAdapterLoader {
+
+    private static final Logger                     logger        = LoggerFactory.getLogger(CanalAdapterLoader.class);
+
+    private CanalClientConfig                       canalClientConfig;
+
+    private Map<String, CanalAdapterWorker>         canalWorkers  = new HashMap<>();
+
+    private Map<String, AbstractCanalAdapterWorker> canalMQWorker = new HashMap<>();
+
+    private ExtensionLoader<OuterAdapter>           loader;
+
+    public CanalAdapterLoader(CanalClientConfig canalClientConfig){
+        this.canalClientConfig = canalClientConfig;
+    }
+
+    /**
+     * 初始化canal-client
+     */
+    public void init() {
+        loader = ExtensionLoader.getExtensionLoader(OuterAdapter.class);
+
+        String canalServerHost = this.canalClientConfig.getCanalServerHost();
+        SocketAddress sa = null;
+        if (canalServerHost != null) {
+            String[] ipPort = canalServerHost.split(":");
+            sa = new InetSocketAddress(ipPort[0], Integer.parseInt(ipPort[1]));
+        }
+        String zkHosts = this.canalClientConfig.getZookeeperHosts();
+
+        // 初始化canal-client的适配器
+        if (canalClientConfig.getCanalInstances() != null) {
+            for (CanalClientConfig.CanalInstance instance : canalClientConfig.getCanalInstances()) {
+                List<List<OuterAdapter>> canalOuterAdapterGroups = new ArrayList<>();
+
+                for (CanalClientConfig.Group connectorGroup : instance.getGroups()) {
+                    List<OuterAdapter> canalOutConnectors = new ArrayList<>();
+                    for (OuterAdapterConfig c : connectorGroup.getOutAdapters()) {
+                        loadConnector(c, canalOutConnectors);
+                    }
+                    canalOuterAdapterGroups.add(canalOutConnectors);
+                }
+                CanalAdapterWorker worker;
+                if (sa != null) {
+                    worker = new CanalAdapterWorker(instance.getInstance(), sa, canalOuterAdapterGroups);
+                } else if (zkHosts != null) {
+                    worker = new CanalAdapterWorker(instance.getInstance(), zkHosts, canalOuterAdapterGroups);
+                } else {
+                    throw new RuntimeException("No canal server connector found");
+                }
+                canalWorkers.put(instance.getInstance(), worker);
+                worker.start();
+                logger.info("Start adapter for canal instance: {} succeed", instance.getInstance());
+            }
+        }
+
+        // 初始化canal-client-mq的适配器
+        if (canalClientConfig.getMqTopics() != null) {
+            for (CanalClientConfig.MQTopic topic : canalClientConfig.getMqTopics()) {
+                for (CanalClientConfig.MQGroup group : topic.getGroups()) {
+                    List<List<OuterAdapter>> canalOuterAdapterGroups = new ArrayList<>();
+                    List<OuterAdapter> canalOuterAdapters = new ArrayList<>();
+                    for (OuterAdapterConfig config : group.getOutAdapters()) {
+                        loadConnector(config, canalOuterAdapters);
+                    }
+                    canalOuterAdapterGroups.add(canalOuterAdapters);
+                    if (StringUtils.isBlank(topic.getMqMode()) || "rocketmq".equalsIgnoreCase(topic.getMqMode())) {
+                        CanalAdapterRocketMQWorker rocketMQWorker = new CanalAdapterRocketMQWorker(canalClientConfig.getBootstrapServers(),
+                            topic.getTopic(),
+                            group.getGroupId(),
+                            canalOuterAdapterGroups,
+                            canalClientConfig.getFlatMessage());
+                        canalMQWorker.put(topic.getTopic() + "-rocketmq-" + group.getGroupId(), rocketMQWorker);
+                        rocketMQWorker.start();
+                    } else if ("kafka".equalsIgnoreCase(topic.getMqMode())) {
+                        CanalAdapterKafkaWorker canalKafkaWorker = new CanalAdapterKafkaWorker(canalClientConfig.getBootstrapServers(),
+                            topic.getTopic(),
+                            group.getGroupId(),
+                            canalOuterAdapterGroups,
+                            canalClientConfig.getFlatMessage());
+                        canalMQWorker.put(topic.getTopic() + "-kafka-" + group.getGroupId(), canalKafkaWorker);
+                        canalKafkaWorker.start();
+                    }
+                    logger.info("Start adapter for canal-client mq topic: {} succeed",
+                        topic.getTopic() + "-" + group.getGroupId());
+                }
+            }
+        }
+    }
+
+    private void loadConnector(OuterAdapterConfig config, List<OuterAdapter> canalOutConnectors) {
+        try {
+            OuterAdapter adapter = loader.getExtension(config.getName());
+            ClassLoader cl = Thread.currentThread().getContextClassLoader();
+            // 替换ClassLoader
+            Thread.currentThread().setContextClassLoader(adapter.getClass().getClassLoader());
+            adapter.init(config);
+            Thread.currentThread().setContextClassLoader(cl);
+            canalOutConnectors.add(adapter);
+            logger.info("Load canal adapter: {} succeed", config.getName());
+        } catch (Exception e) {
+            logger.error("Load canal adapter: {} failed", config.getName(), e);
+        }
+    }
+
+    /**
+     * 销毁所有适配器 为防止canal实例太多造成销毁阻塞, 并行销毁
+     */
+    public void destroy() {
+        if (canalWorkers.size() > 0) {
+            ExecutorService stopExecutorService = Executors.newFixedThreadPool(canalWorkers.size());
+            for (CanalAdapterWorker canalAdapterWorker : canalWorkers.values()) {
+                stopExecutorService.submit(canalAdapterWorker::stop);
+            }
+            stopExecutorService.shutdown();
+        }
+
+        if (canalMQWorker.size() > 0) {
+            ExecutorService stopMQWorkerService = Executors.newFixedThreadPool(canalMQWorker.size());
+            for (AbstractCanalAdapterWorker canalAdapterMQWorker : canalMQWorker.values()) {
+                stopMQWorkerService.submit(canalAdapterMQWorker::stop);
+            }
+            stopMQWorkerService.shutdown();
+        }
+        logger.info("All canal adapters destroyed");
+    }
+}
@@ -0,0 +1,89 @@
+package com.alibaba.otter.canal.adapter.launcher.loader;
+
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.kafka.common.errors.WakeupException;
+
+import com.alibaba.otter.canal.client.adapter.OuterAdapter;
+import com.alibaba.otter.canal.client.rocketmq.RocketMQCanalConnector;
+import com.alibaba.otter.canal.client.rocketmq.RocketMQCanalConnectors;
+import com.alibaba.otter.canal.protocol.FlatMessage;
+import com.alibaba.otter.canal.protocol.Message;
+
+/**
+ * rocketmq对应的client适配器工作线程
+ *
+ * @version 1.0.0
+ */
+public class CanalAdapterRocketMQWorker extends AbstractCanalAdapterWorker {
+
+    private RocketMQCanalConnector connector;
+    private String                 topic;
+    private boolean                flatMessage;
+
+    public CanalAdapterRocketMQWorker(String nameServers, String topic, String groupId,
+                                      List<List<OuterAdapter>> canalOuterAdapters, boolean flatMessage){
+        super(canalOuterAdapters);
+        this.topic = topic;
+        this.flatMessage = flatMessage;
+        this.canalDestination = topic;
+        this.connector = RocketMQCanalConnectors.newRocketMQConnector(nameServers, topic, groupId, flatMessage);
+        logger.info("RocketMQ consumer config topic:{}, nameServer:{}, groupId:{}", topic, nameServers, groupId);
+    }
+
+    @Override
+    protected void process() {
+        while (!running)
+            ;
+        while (running) {
+            try {
+                syncSwitch.get(canalDestination);
+                logger.info("=============> Start to connect topic: {} <=============", this.topic);
+                connector.connect();
+                logger.info("=============> Start to subscribe topic: {}<=============", this.topic);
+                connector.subscribe();
+                logger.info("=============> Subscribe topic: {} succeed<=============", this.topic);
+                while (running) {
+                    try {
+                        Boolean status = syncSwitch.status(canalDestination);
+                        if (status != null && !status) {
+                            connector.disconnect();
+                            break;
+                        }
+
+                        List<?> messages;
+                        if (!flatMessage) {
+                            messages = connector.getListWithoutAck(100L, TimeUnit.MILLISECONDS);
+                        } else {
+                            messages = connector.getFlatListWithoutAck(100L, TimeUnit.MILLISECONDS);
+                        }
+                        if (messages != null) {
+                            for (final Object message : messages) {
+                                if (message instanceof FlatMessage) {
+                                    writeOut((FlatMessage) message);
+                                } else {
+                                    writeOut((Message) message);
+                                }
+                            }
+                        }
+                        connector.ack();
+                    } catch (Throwable e) {
+                        logger.error(e.getMessage(), e);
+                        TimeUnit.SECONDS.sleep(1L);
+                    }
+                }
+            } catch (Exception e) {
+                logger.error(e.getMessage(), e);
+            }
+        }
+
+        try {
+            connector.unsubscribe();
+        } catch (WakeupException e) {
+            // No-op. Continue process
+        }
+        connector.disconnect();
+        logger.info("=============> Disconnect topic: {} <=============", this.topic);
+    }
+}
@@ -0,0 +1,77 @@
+package com.alibaba.otter.canal.adapter.launcher.loader;
+
+import javax.annotation.PostConstruct;
+import javax.annotation.PreDestroy;
+import javax.annotation.Resource;
+
+import com.alibaba.otter.canal.adapter.launcher.common.SyncSwitch;
+import com.alibaba.otter.canal.adapter.launcher.config.SpringContext;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.stereotype.Component;
+
+import com.alibaba.druid.pool.DruidDataSource;
+import com.alibaba.otter.canal.adapter.launcher.config.AdapterCanalConfig;
+import com.alibaba.otter.canal.adapter.launcher.config.AdapterConfig;
+import com.alibaba.otter.canal.client.adapter.support.DatasourceConfig;
+
+/**
+ * 适配器启动业务类
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+@Component
+public class CanalAdapterService {
+
+    private static final Logger       logger = LoggerFactory.getLogger(CanalAdapterService.class);
+
+    private static CanalAdapterLoader adapterLoader;
+
+    @Resource
+    private AdapterCanalConfig        adapterCanalConfig;
+
+    // 注入bean保证优先注册
+    @Resource
+    private AdapterConfig             adapterConfig;
+    @Resource
+    private SpringContext             springContext;
+    @Resource
+    private SyncSwitch                syncSwitch;
+
+    @PostConstruct
+    public void init() {
+        if (adapterLoader == null) {
+            try {
+                logger.info("## start the canal client adapters.");
+                adapterLoader = new CanalAdapterLoader(adapterCanalConfig);
+                adapterLoader.init();
+                logger.info("## the canal client adapters are running now ......");
+            } catch (Throwable e) {
+                logger.error("## something goes wrong when starting up the canal client adapters:", e);
+                System.exit(0);
+            }
+        }
+    }
+
+    @PreDestroy
+    public void destroy() {
+        try {
+            logger.info("## stop the canal client adapters");
+            if (adapterLoader != null) {
+                adapterLoader.destroy();
+            }
+            for (DruidDataSource druidDataSource : DatasourceConfig.DATA_SOURCES.values()) {
+                try {
+                    druidDataSource.close();
+                } catch (Exception e) {
+                    logger.error(e.getMessage(), e);
+                }
+            }
+        } catch (Throwable e) {
+            logger.warn("## something goes wrong when stopping canal client adapters:", e);
+        } finally {
+            logger.info("## canal client adapters are down.");
+        }
+    }
+}
@@ -0,0 +1,162 @@
+package com.alibaba.otter.canal.adapter.launcher.loader;
+
+import java.net.SocketAddress;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
+
+import com.alibaba.otter.canal.client.CanalConnector;
+import com.alibaba.otter.canal.client.CanalConnectors;
+import com.alibaba.otter.canal.client.adapter.OuterAdapter;
+import com.alibaba.otter.canal.client.impl.ClusterCanalConnector;
+import com.alibaba.otter.canal.client.impl.SimpleCanalConnector;
+import com.alibaba.otter.canal.protocol.Message;
+
+/**
+ * 原生canal-server对应的client适配器工作线程
+ *
+ * @author rewrema 2018-8-19 下午11:30:49
+ * @version 1.0.0
+ */
+public class CanalAdapterWorker extends AbstractCanalAdapterWorker {
+
+    private static final int BATCH_SIZE = 50;
+    private static final int SO_TIMEOUT = 0;
+
+    private CanalConnector   connector;
+
+    /**
+     * 单台client适配器worker的构造方法
+     *
+     * @param canalDestination canal实例名
+     * @param address canal-server地址
+     * @param canalOuterAdapters 外部适配器组
+     */
+    public CanalAdapterWorker(String canalDestination, SocketAddress address,
+                              List<List<OuterAdapter>> canalOuterAdapters){
+        super(canalOuterAdapters);
+        this.canalDestination = canalDestination;
+        connector = CanalConnectors.newSingleConnector(address, canalDestination, "", "");
+    }
+
+    /**
+     * HA模式下client适配器worker的构造方法
+     *
+     * @param canalDestination canal实例名
+     * @param zookeeperHosts zookeeper地址
+     * @param canalOuterAdapters 外部适配器组
+     */
+    public CanalAdapterWorker(String canalDestination, String zookeeperHosts,
+                              List<List<OuterAdapter>> canalOuterAdapters){
+        super(canalOuterAdapters);
+        this.canalDestination = canalDestination;
+        connector = CanalConnectors.newClusterConnector(zookeeperHosts, canalDestination, "", "");
+        ((ClusterCanalConnector) connector).setSoTimeout(SO_TIMEOUT);
+    }
+
+    @Override
+    protected void process() {
+        while (!running)
+            ; // waiting until running == true
+        while (running) {
+            try {
+                syncSwitch.get(canalDestination);
+
+                logger.info("=============> Start to connect destination: {} <=============", this.canalDestination);
+                connector.connect();
+                logger.info("=============> Start to subscribe destination: {} <=============", this.canalDestination);
+                connector.subscribe();
+                logger.info("=============> Subscribe destination: {} succeed <=============", this.canalDestination);
+                while (running) {
+                    try {
+                        syncSwitch.get(canalDestination, 1L, TimeUnit.MINUTES);
+                    } catch (TimeoutException e) {
+                        break;
+                    }
+                    if (!running) {
+                        break;
+                    }
+
+                    // server配置canal.instance.network.soTimeout(默认: 30s)
+                    // 范围内未与server交互，server将关闭本次socket连接
+                    Message message = connector.getWithoutAck(BATCH_SIZE); // 获取指定数量的数据
+                    long batchId = message.getId();
+                    try {
+                        int size = message.getEntries().size();
+                        if (batchId == -1 || size == 0) {
+                            Thread.sleep(1000);
+                        } else {
+                            if (logger.isDebugEnabled()) {
+                                logger.debug("destination: {} batchId: {} batchSize: {} ",
+                                    this.canalDestination,
+                                    batchId,
+                                    size);
+                            }
+                            long begin = System.currentTimeMillis();
+                            writeOut(message);
+                            if (logger.isDebugEnabled()) {
+                                logger.debug("destination: {} batchId: {} elapsed time: {} ms",
+                                    this.canalDestination,
+                                    batchId,
+                                    System.currentTimeMillis() - begin);
+                            }
+                        }
+                        connector.ack(batchId); // 提交确认
+                    } catch (Exception e) {
+                        connector.rollback(batchId); // 处理失败, 回滚数据
+                        logger.error("sync error!", e);
+                        Thread.sleep(500);
+                    }
+                }
+
+            } catch (Exception e) {
+                logger.error("process error!", e);
+            } finally {
+                connector.disconnect();
+                logger.info("=============> Disconnect destination: {} <=============", this.canalDestination);
+            }
+
+            if (running) { // is reconnect
+                try {
+                    Thread.sleep(1000);
+                } catch (InterruptedException e) {
+                    // ignore
+                }
+            }
+        }
+    }
+
+    @Override
+    public void stop() {
+        try {
+            if (!running) {
+                return;
+            }
+
+            if (connector instanceof ClusterCanalConnector) {
+                ((ClusterCanalConnector) connector).stopRunning();
+            } else if (connector instanceof SimpleCanalConnector) {
+                ((SimpleCanalConnector) connector).stopRunning();
+            }
+
+            running = false;
+
+            syncSwitch.release(canalDestination);
+
+            logger.info("destination {} is waiting for adapters' worker thread die!", canalDestination);
+            if (thread != null) {
+                try {
+                    thread.join();
+                } catch (InterruptedException e) {
+                    // ignore
+                }
+            }
+            groupInnerExecutorService.shutdown();
+            logger.info("destination {} adapters worker thread dead!", canalDestination);
+            canalOuterAdapters.forEach(outerAdapters -> outerAdapters.forEach(OuterAdapter::destroy));
+            logger.info("destination {} all adapters destroyed!", canalDestination);
+        } catch (Exception e) {
+            logger.error(e.getMessage(), e);
+        }
+    }
+}
@@ -0,0 +1,172 @@
+package com.alibaba.otter.canal.adapter.launcher.rest;
+
+import java.util.*;
+
+import javax.annotation.PostConstruct;
+import javax.annotation.Resource;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.web.bind.annotation.*;
+
+import com.alibaba.otter.canal.adapter.launcher.common.EtlLock;
+import com.alibaba.otter.canal.adapter.launcher.common.SyncSwitch;
+import com.alibaba.otter.canal.adapter.launcher.config.AdapterCanalConfig;
+import com.alibaba.otter.canal.client.adapter.OuterAdapter;
+import com.alibaba.otter.canal.client.adapter.support.EtlResult;
+import com.alibaba.otter.canal.client.adapter.support.ExtensionLoader;
+import com.alibaba.otter.canal.client.adapter.support.Result;
+
+/**
+ * 适配器操作Rest
+ *
+ * @author rewerma @ 2018-10-20
+ * @version 1.0.0
+ */
+@RestController
+public class CommonRest {
+
+    private static Logger                 logger           = LoggerFactory.getLogger(CommonRest.class);
+
+    private static final String           ETL_LOCK_ZK_NODE = "/sync-etl/";
+
+    private ExtensionLoader<OuterAdapter> loader;
+
+    @Resource
+    private SyncSwitch                    syncSwitch;
+    @Resource
+    private EtlLock                       etlLock;
+
+    @Resource
+    private AdapterCanalConfig            adapterCanalConfig;
+
+    @PostConstruct
+    public void init() {
+        loader = ExtensionLoader.getExtensionLoader(OuterAdapter.class);
+    }
+
+    /**
+     * ETL curl http://127.0.0.1:8081/etl/hbase/mytest_person2.yml -X POST
+     * 
+     * @param type 类型 hbase, es
+     * @param task 任务名对应配置文件名 mytest_person2.yml
+     * @param params etl where条件参数, 为空全部导入
+     * @return
+     */
+    @PostMapping("/etl/{type}/{task}")
+    public EtlResult etl(@PathVariable String type, @PathVariable String task,
+                         @RequestParam(name = "params", required = false) String params) {
+
+        boolean locked = etlLock.tryLock(ETL_LOCK_ZK_NODE + type + "-" + task);
+        if (!locked) {
+            EtlResult result = new EtlResult();
+            result.setSucceeded(false);
+            result.setErrorMessage(task + " 有其他进程正在导入中, 请稍后再试");
+            return result;
+        }
+        try {
+            OuterAdapter adapter = loader.getExtension(type);
+            String destination = adapter.getDestination(task);
+            Boolean oriSwithcStatus = null;
+            if (destination != null) {
+                oriSwithcStatus = syncSwitch.status(destination);
+                syncSwitch.off(destination);
+            }
+            try {
+                List<String> paramArr = null;
+                if (params != null) {
+                    String[] parmaArray = params.trim().split(";");
+                    paramArr = Arrays.asList(parmaArray);
+                }
+                return adapter.etl(task, paramArr);
+            } finally {
+                if (destination != null && oriSwithcStatus != null && oriSwithcStatus) {
+                    syncSwitch.on(destination);
+                }
+            }
+        } finally {
+            etlLock.unlock(ETL_LOCK_ZK_NODE + type + "-" + task);
+        }
+    }
+
+    /**
+     * 统计总数 curl http://127.0.0.1:8081/count/hbase/mytest_person2.yml
+     * 
+     * @param type 类型 hbase, es
+     * @param task 任务名对应配置文件名 mytest_person2.yml
+     * @return
+     */
+    @GetMapping("/count/{type}/{task}")
+    public Map<String, Object> count(@PathVariable String type, @PathVariable String task) {
+        OuterAdapter adapter = loader.getExtension(type);
+        return adapter.count(task);
+    }
+
+    /**
+     * 返回所有实例 curl http://127.0.0.1:8081/destinations
+     */
+    @GetMapping("/destinations")
+    public List<Map<String, String>> destinations() {
+        List<Map<String, String>> result = new ArrayList<>();
+        Set<String> destinations = adapterCanalConfig.DESTINATIONS;
+        for (String destination : destinations) {
+            Map<String, String> resMap = new LinkedHashMap<>();
+            Boolean status = syncSwitch.status(destination);
+            String resStatus = "none";
+            if (status != null && status) {
+                resStatus = "on";
+            } else if (status != null && !status) {
+                resStatus = "off";
+            }
+            resMap.put("destination", destination);
+            resMap.put("status", resStatus);
+            result.add(resMap);
+        }
+        return result;
+    }
+
+    /**
+     * 实例同步开关 curl http://127.0.0.1:8081/syncSwitch/example/off -X PUT
+     * 
+     * @param destination 实例名称
+     * @param status 开关状态: off on
+     * @return
+     */
+    @PutMapping("/syncSwitch/{destination}/{status}")
+    public Result etl(@PathVariable String destination, @PathVariable String status) {
+        if (status.equals("on")) {
+            syncSwitch.on(destination);
+            logger.info("#Destination: {} sync on", destination);
+            return Result.createSuccess("实例: " + destination + " 开启同步成功");
+        } else if (status.equals("off")) {
+            syncSwitch.off(destination);
+            logger.info("#Destination: {} sync off", destination);
+            return Result.createSuccess("实例: " + destination + " 关闭同步成功");
+        } else {
+            Result result = new Result();
+            result.setCode(50000);
+            result.setMessage("实例: " + destination + " 操作失败");
+            return result;
+        }
+    }
+
+    /**
+     * 获取实例开关状态 curl http://127.0.0.1:8081/syncSwitch/example
+     * 
+     * @param destination 实例名称
+     * @return
+     */
+    @GetMapping("/syncSwitch/{destination}")
+    public Map<String, String> etl(@PathVariable String destination) {
+        Boolean status = syncSwitch.status(destination);
+        String resStatus = "none";
+        if (status != null && status) {
+            resStatus = "on";
+        } else if (status != null && !status) {
+            resStatus = "off";
+        }
+        Map<String, String> res = new LinkedHashMap<>();
+        res.put("stauts", resStatus);
+        return res;
+    }
+}
@@ -0,0 +1,54 @@
+server:
+  port: 8081
+logging:
+  level:
+    com.alibaba.otter.canal.client.adapter.hbase: DEBUG
+spring:
+  jackson:
+    date-format: yyyy-MM-dd HH:mm:ss
+    time-zone: GMT+8
+    default-property-inclusion: non_null
+
+
+hbasezookeeper.quorum: 127.0.0.1
+hbase.zookeeper.property.clientPort: 2181
+hbase.zookeeper.znode.parent: /hbase
+
+canal.conf:
+  canalServerHost: 127.0.0.1:11111
+#  zookeeperHosts: slave1:2181
+#  bootstrapServers: slave1:6667 #or rocketmq nameservers:host1:9876;host2:9876
+  flatMessage: true
+  canalInstances:
+  - instance: example
+    groups:
+    - outAdapters:
+      - name: logger
+#      - name: hbase
+#        properties:
+#          hbase.zookeeper.quorum: ${hbase.zookeeper.quorum}
+#          hbase.zookeeper.property.clientPort: ${hbase.zookeeper.property.clientPort}
+#          zookeeper.znode.parent: ${hbase.zookeeper.znode.parent}
+#  mqTopics:
+#  - mqMode: kafka
+#    topic: example
+#    groups:
+#    - groupId: g2
+#      outAdapters:
+#      - name: logger
+#  mqTopics:
+#  - mqMode: rocketmq
+#    topic: example
+#    groups:
+#    - groupId: g2
+#      outAdapters:
+#      - name: logger
+
+#adapter.conf:
+#  datasourceConfigs:
+#    defaultDS:
+#      url: jdbc:mysql://127.0.0.1:3306/mytest?useUnicode=true
+#      username: root
+#      password: 121212
+#  adapterConfigs:
+#  - hbase/mytest_person2.yml
@@ -0,0 +1,46 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <parent>
+        <artifactId>canal.client-adapter</artifactId>
+        <groupId>com.alibaba.otter</groupId>
+        <version>1.1.1</version>
+    </parent>
+    <modelVersion>4.0.0</modelVersion>
+    <groupId>com.alibaba.otter</groupId>
+    <artifactId>client-adapter.logger</artifactId>
+    <packaging>jar</packaging>
+    <name>canal client adapter logger example module for otter ${project.version}</name>
+    <dependencies>
+        <dependency>
+            <groupId>com.alibaba.otter</groupId>
+            <artifactId>client-adapter.common</artifactId>
+            <version>${project.version}</version>
+            <scope>provided</scope>
+        </dependency>
+    </dependencies>
+
+
+    <build>
+        <plugins>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-assembly-plugin</artifactId>
+                <version>2.4</version>
+                <configuration>
+                    <descriptorRefs>
+                        <descriptorRef>jar-with-dependencies</descriptorRef>
+                    </descriptorRefs>
+                </configuration>
+                <executions>
+                    <execution>
+                        <id>make-assembly</id>
+                        <phase>package</phase>
+                        <goals>
+                            <goal>single</goal>
+                        </goals>
+                    </execution>
+                </executions>
+            </plugin>
+        </plugins>
+    </build>
+</project>
@@ -0,0 +1,37 @@
+package com.alibaba.otter.canal.client.adapter.logger;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.alibaba.otter.canal.client.adapter.OuterAdapter;
+import com.alibaba.otter.canal.client.adapter.support.OuterAdapterConfig;
+import com.alibaba.otter.canal.client.adapter.support.Dml;
+import com.alibaba.otter.canal.client.adapter.support.SPI;
+
+/**
+ * 外部适配器示例
+ *
+ * @author machengyuan 2018-8-19 下午11:45:38
+ * @version 1.0.0
+ */
+@SPI("logger")
+// logger参数对应CanalOuterAdapterConfiguration配置中的name
+public class LoggerAdapterExample implements OuterAdapter {
+
+    private Logger logger = LoggerFactory.getLogger(this.getClass());
+
+    @Override
+    public void init(OuterAdapterConfig configuration) {
+
+    }
+
+    @Override
+    public void sync(Dml dml) {
+        logger.info(dml.toString());
+    }
+
+    @Override
+    public void destroy() {
+
+    }
+}
@@ -0,0 +1 @@
+logger=com.alibaba.otter.canal.client.adapter.logger.LoggerAdapterExample
@@ -0,0 +1,41 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <groupId>com.alibaba.otter</groupId>
+    <artifactId>canal.client-adapter</artifactId>
+    <version>1.1.1</version>
+    <packaging>pom</packaging>
+    <name>canal client adapter module for otter ${project.version}</name>
+
+    <properties>
+        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+        <maven.test.skip>true</maven.test.skip>
+        <downloadSources>true</downloadSources>
+        <java_source_version>1.8</java_source_version>
+        <java_target_version>1.8</java_target_version>
+        <file_encoding>UTF-8</file_encoding>
+        <canal_version>1.1.1-SNAPSHOT</canal_version>
+    </properties>
+
+    <modules>
+        <module>common</module>
+        <module>logger</module>
+        <module>hbase</module>
+        <module>launcher</module>
+    </modules>
+
+    <build>
+        <plugins>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-compiler-plugin</artifactId>
+                <version>3.7.0</version>
+                <configuration>
+                    <source>${java_source_version}</source>
+                    <target>${java_target_version}</target>
+                    <encoding>${file_encoding}</encoding>
+                </configuration>
+            </plugin>
+        </plugins>
+    </build>
+</project>
@@ -3,7 +3,7 @@
 	<parent>
 		<groupId>com.alibaba.otter</groupId>
 		<artifactId>canal</artifactId>
-		<version>1.0.26-SNAPSHOT</version>
+		<version>1.1.1</version>
 		<relativePath>../pom.xml</relativePath>
 	</parent>
 	<groupId>com.alibaba.otter</groupId>
@@ -15,15 +15,180 @@
 			<groupId>com.alibaba.otter</groupId>
 			<artifactId>canal.protocol</artifactId>
 			<version>${project.version}</version>
+			<optional>true</optional>
+		</dependency>
+
+		<dependency>
+			<groupId>com.google.protobuf</groupId>
+			<artifactId>protobuf-java</artifactId>
+		</dependency>
+
+		<!-- https://mvnrepository.com/artifact/io.netty/netty-all -->
+		<dependency>
+			<groupId>io.netty</groupId>
+			<artifactId>netty-all</artifactId>
+		</dependency>
+		<!-- zk -->
+		<dependency>
+			<groupId>org.apache.zookeeper</groupId>
+			<artifactId>zookeeper</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>com.101tec</groupId>
+			<artifactId>zkclient</artifactId>
+		</dependency>
+		<!-- external -->
+		<dependency>
+			<groupId>commons-io</groupId>
+			<artifactId>commons-io</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>commons-lang</groupId>
+			<artifactId>commons-lang</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>commons-codec</groupId>
+			<artifactId>commons-codec</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>com.alibaba</groupId>
+			<artifactId>fastjson</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>com.google.guava</groupId>
+			<artifactId>guava</artifactId>
+		</dependency>
+		<!-- log -->
+		<dependency>
+			<groupId>ch.qos.logback</groupId>
+			<artifactId>logback-core</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>ch.qos.logback</groupId>
+			<artifactId>logback-classic</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>org.slf4j</groupId>
+			<artifactId>jcl-over-slf4j</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>org.slf4j</groupId>
+			<artifactId>slf4j-api</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>org.springframework</groupId>
+			<artifactId>spring-core</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>org.springframework</groupId>
+			<artifactId>spring-aop</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>org.springframework</groupId>
+			<artifactId>spring-context</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>org.springframework</groupId>
+			<artifactId>spring-jdbc</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>org.springframework</groupId>
+			<artifactId>spring-orm</artifactId>
+		</dependency>
+		<dependency>
+			<groupId>org.springframework</groupId>
+			<artifactId>spring-test</artifactId>
+			<version>${spring_version}</version>
+			<scope>test</scope>
+		</dependency>
+        <!-- 客户端要使用请单独引入rocketmq-client依赖 -->
+		<dependency>
+			<groupId>org.apache.rocketmq</groupId>
+			<artifactId>rocketmq-client</artifactId>
+			<version>4.3.0</version>
+            <scope>provided</scope>
+		</dependency>
+		<!-- 客户端要使用请单独引入kafka-clients依赖 -->
+		<dependency>
+			<groupId>org.apache.kafka</groupId>
+			<artifactId>kafka-clients</artifactId>
+			<version>1.1.1</version>
+			<scope>provided</scope>
 		</dependency>

 		<!-- junit -->
 		<dependency>
 			<groupId>junit</groupId>
 			<artifactId>junit</artifactId>
+			<scope>test</scope>
 		</dependency>
 	</dependencies>
-	
+
+	<build>
+		<plugins>
+			<plugin>
+				<groupId>org.apache.maven.plugins</groupId>
+				<artifactId>maven-shade-plugin</artifactId>
+				<version>2.4.1</version>
+				<configuration>
+					<createDependencyReducedPom>false</createDependencyReducedPom>
+					<artifactSet>
+						<excludes>
+							<exclude>org.apache.zookeeper:zookeeper</exclude>
+							<exclude>com.101tec:zkclient</exclude>
+							<exclude>commons-io:commons-io</exclude>
+							<exclude>commons-lang:commons-lang</exclude>
+							<exclude>commons-codec:commons-codec</exclude>
+							<exclude>com.alibaba:fastjson</exclude>
+							<exclude>ch.qos.logback:logback-core</exclude>
+							<exclude>ch.qos.logback:logback-classic</exclude>
+							<exclude>org.slf4j:jcl-over-slf4j</exclude>
+							<exclude>org.slf4j:slf4j-api</exclude>
+							<exclude>org.springframework:*</exclude>
+							<exclude>org.jboss.netty:netty</exclude>
+							<exclude>commons-logging:commons-logging</exclude>
+							<exclude>aopalliance:aopalliance</exclude>
+							<exclude>com.google.protobuf:protobuf-java</exclude>
+							<exclude>io.netty:*</exclude>
+							<exclude>junit:junit</exclude>
+						</excludes>
+					</artifactSet>
+				</configuration>
+				<executions>
+					<execution>
+						<phase>package</phase>
+						<goals>
+							<goal>shade</goal>
+						</goals>
+						<configuration>
+							<relocations>
+								<relocation>
+									<pattern>com.google.guava</pattern>
+									<shadedPattern>com.alibaba.google.guava</shadedPattern>
+								</relocation>
+								<relocation>
+									<pattern>org.joda</pattern>
+									<shadedPattern>com.alibaba.google.joda</shadedPattern>
+								</relocation>
+								<relocation>
+									<pattern>com.google.common</pattern>
+									<shadedPattern>com.alibaba.google.common</shadedPattern>
+								</relocation>
+								<relocation>
+									<pattern>com.google.thirdparty</pattern>
+									<shadedPattern>com.alibaba.google.thirdparty</shadedPattern>
+								</relocation>
+							</relocations>
+							<transformers>
+								<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer" />
+							</transformers>
+						</configuration>
+					</execution>
+				</executions>
+			</plugin>
+		</plugins>
+	</build>
+
 	<profiles>
 		<profile>
 			<id>dev</id>
@@ -91,7 +256,7 @@
 							<content>${project.build.directory}/apidocs</content>
 							<skipDeletedFiles>true</skipDeletedFiles>
 							<pubScmUrl>scm:git:git@github.com:alibaba/canal.git</pubScmUrl>
-							<scmBranch>gh-pages</scmBranch> 
+							<scmBranch>gh-pages</scmBranch>
 						</configuration>
 					</plugin>
 				</plugins>
@@ -149,11 +149,4 @@ public interface CanalConnector {
     * @throws CanalClientException
     */
    void rollback() throws CanalClientException;
-
-    /**
-     * 中断的阻塞，用于优雅停止client
-     * 
-     * @throws CanalClientException
-     */
-    void stopRunning() throws CanalClientException;
 }
@@ -0,0 +1,91 @@
+package com.alibaba.otter.canal.client;
+
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+import com.alibaba.otter.canal.protocol.FlatMessage;
+import com.alibaba.otter.canal.protocol.Message;
+import com.alibaba.otter.canal.protocol.exception.CanalClientException;
+
+/**
+ * canal MQ数据操作客户端
+ * 
+ * <pre>
+ * 1. canal server写入MQ消息，考虑性能会合并多条数据写入为一个MQ消息，一个Message对应一个MQ消息
+ * 2. canal client消费MQ消息，因为client性能会弱于server的写入，MQ数据获取时会拿到堆积的多条MQ消息，会拿到List<Message>
+ * 3. client的ack/rollback，都是和MQ直接交互，不存在对应的batchId概念
+ * </pre>
+ * 
+ * @author agapple 2018年10月28日 下午6:42:27
+ * @since 1.1.1
+ */
+public interface CanalMQConnector extends CanalConnector {
+
+    /**
+     * 获取数据，自动进行确认，设置timeout时间直到拿到数据为止
+     * 
+     * <pre>
+     * 该方法返回的条件：
+     *  a. 如果timeout=0，有多少取多少，不会阻塞等待
+     *  b. 如果timeout不为0，尝试阻塞对应的超时时间，直到拿到数据就返回
+     * </pre>
+     * 
+     * @return
+     * @throws CanalClientException
+     */
+    List<Message> getList(Long timeout, TimeUnit unit) throws CanalClientException;
+
+    /**
+     * 获取数据，设置timeout时间直到拿到数据为止
+     * 
+     * <pre>
+     * 该方法返回的条件：
+     *  a. 如果timeout=0，有多少取多少，不会阻塞等待
+     *  b. 如果timeout不为0，尝试阻塞对应的超时时间，直到拿到数据就返回
+     * </pre>
+     * 
+     * @throws CanalClientException
+     */
+    List<Message> getListWithoutAck(Long timeout, TimeUnit unit) throws CanalClientException;
+
+    /**
+     * 获取数据，自动进行确认，设置timeout时间直到拿到数据为止
+     * 
+     * <pre>
+     * 该方法返回的条件：
+     *  a. 如果timeout=0，有多少取多少，不会阻塞等待
+     *  b. 如果timeout不为0，尝试阻塞对应的超时时间，直到拿到数据就返回
+     * </pre>
+     * 
+     * @return
+     * @throws CanalClientException
+     */
+    List<FlatMessage> getFlatList(Long timeout, TimeUnit unit) throws CanalClientException;
+
+    /**
+     * 获取数据，设置timeout时间直到拿到数据为止
+     * 
+     * <pre>
+     * 该方法返回的条件：
+     *  a. 如果timeout=0，有多少取多少，不会阻塞等待
+     *  b. 如果timeout不为0，尝试阻塞对应的超时时间，直到拿到数据就返回
+     * </pre>
+     * 
+     * @throws CanalClientException
+     */
+    List<FlatMessage> getFlatListWithoutAck(Long timeout, TimeUnit unit) throws CanalClientException;
+
+    /**
+     * 消费确认。
+     * 
+     * @throws CanalClientException
+     */
+    void ack() throws CanalClientException;
+
+    /**
+     * 回滚到未进行 {@link #ack} 的地方，下次fetch的时候，可以从最后一个没有 {@link #ack} 的地方开始拿
+     * 
+     * @throws CanalClientException
+     */
+    void rollback() throws CanalClientException;
+}
@@ -0,0 +1,55 @@
+package com.alibaba.otter.canal.client;
+
+import com.alibaba.otter.canal.protocol.CanalEntry;
+import com.alibaba.otter.canal.protocol.CanalPacket;
+import com.alibaba.otter.canal.protocol.CanalPacket.Ack;
+import com.alibaba.otter.canal.protocol.CanalPacket.Compression;
+import com.alibaba.otter.canal.protocol.Message;
+import com.alibaba.otter.canal.protocol.exception.CanalClientException;
+import com.google.protobuf.ByteString;
+
+public class CanalMessageDeserializer {
+
+    public static Message deserializer(byte[] data) {
+        return deserializer(data, false);
+    }
+
+    public static Message deserializer(byte[] data, boolean lazyParseEntry) {
+        try {
+            if (data == null) {
+                return null;
+            } else {
+                CanalPacket.Packet p = CanalPacket.Packet.parseFrom(data);
+                switch (p.getType()) {
+                    case MESSAGES: {
+                        if (!p.getCompression().equals(Compression.NONE)
+                            && !p.getCompression().equals(Compression.COMPRESSIONCOMPATIBLEPROTO2)) {
+                            throw new CanalClientException("compression is not supported in this connector");
+                        }
+
+                        CanalPacket.Messages messages = CanalPacket.Messages.parseFrom(p.getBody());
+                        Message result = new Message(messages.getBatchId());
+                        if (lazyParseEntry) {
+                            // byteString
+                            result.setRawEntries(messages.getMessagesList());
+                        } else {
+                            for (ByteString byteString : messages.getMessagesList()) {
+                                result.addEntry(CanalEntry.Entry.parseFrom(byteString));
+                            }
+                        }
+                        return result;
+                    }
+                    case ACK: {
+                        Ack ack = Ack.parseFrom(p.getBody());
+                        throw new CanalClientException("something goes wrong with reason: " + ack.getErrorMessage());
+                    }
+                    default: {
+                        throw new CanalClientException("unexpected packet type: " + p.getType());
+                    }
+                }
+            }
+        } catch (Exception e) {
+            throw new CanalClientException("deserializer failed", e);
+        }
+    }
+}
@@ -218,8 +218,7 @@ public class ClusterCanalConnector implements CanalConnector {
                return;
            } catch (Throwable t) {
                logger.warn(String.format("something goes wrong when rollbacking data from server:%s",
-                    currentConnector != null ? currentConnector.getAddress() : "null"),
-                    t);
+                    currentConnector != null ? currentConnector.getAddress() : "null"), t);
                times++;
                restart();
                logger.info("restart the connector for next round retry.");
@@ -236,8 +235,7 @@ public class ClusterCanalConnector implements CanalConnector {
                return;
            } catch (Throwable t) {
                logger.warn(String.format("something goes wrong when rollbacking data from server:%s",
-                    currentConnector != null ? currentConnector.getAddress() : "null"),
-                    t);
+                    currentConnector != null ? currentConnector.getAddress() : "null"), t);
                times++;
                restart();
                logger.info("restart the connector for next round retry.");
@@ -255,8 +253,7 @@ public class ClusterCanalConnector implements CanalConnector {
                return;
            } catch (Throwable t) {
                logger.warn(String.format("something goes wrong when acking data from server:%s",
-                    currentConnector != null ? currentConnector.getAddress() : "null"),
-                    t);
+                    currentConnector != null ? currentConnector.getAddress() : "null"), t);
                times++;
                restart();
                logger.info("restart the connector for next round retry.");
@@ -18,13 +18,13 @@ import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;

 import com.alibaba.otter.canal.client.CanalConnector;
+import com.alibaba.otter.canal.client.CanalMessageDeserializer;
 import com.alibaba.otter.canal.client.impl.running.ClientRunningData;
 import com.alibaba.otter.canal.client.impl.running.ClientRunningListener;
 import com.alibaba.otter.canal.client.impl.running.ClientRunningMonitor;
 import com.alibaba.otter.canal.common.utils.AddressUtils;
 import com.alibaba.otter.canal.common.utils.BooleanMutex;
 import com.alibaba.otter.canal.common.zookeeper.ZkClientx;
-import com.alibaba.otter.canal.protocol.CanalEntry.Entry;
 import com.alibaba.otter.canal.protocol.CanalPacket.Ack;
 import com.alibaba.otter.canal.protocol.CanalPacket.ClientAck;
 import com.alibaba.otter.canal.protocol.CanalPacket.ClientAuth;
@@ -32,7 +32,6 @@ import com.alibaba.otter.canal.protocol.CanalPacket.ClientRollback;
 import com.alibaba.otter.canal.protocol.CanalPacket.Compression;
 import com.alibaba.otter.canal.protocol.CanalPacket.Get;
 import com.alibaba.otter.canal.protocol.CanalPacket.Handshake;
-import com.alibaba.otter.canal.protocol.CanalPacket.Messages;
 import com.alibaba.otter.canal.protocol.CanalPacket.Packet;
 import com.alibaba.otter.canal.protocol.CanalPacket.PacketType;
 import com.alibaba.otter.canal.protocol.CanalPacket.Sub;
@@ -159,7 +158,7 @@ public class SimpleCanalConnector implements CanalConnector {
            }
            //
            Handshake handshake = Handshake.parseFrom(p.getBody());
-            supportedCompressions.addAll(handshake.getSupportedCompressionsList());
+            supportedCompressions.add(handshake.getSupportedCompressions());
            //
            ClientAuth ca = ClientAuth.newBuilder()
                .setUsername(username != null ? username : "")
@@ -320,33 +319,7 @@ public class SimpleCanalConnector implements CanalConnector {

    private Message receiveMessages() throws IOException {
        byte[] data = readNextPacket();
-        Packet p = Packet.parseFrom(data);
-        switch (p.getType()) {
-            case MESSAGES: {
-                if (!p.getCompression().equals(Compression.NONE)) {
-                    throw new CanalClientException("compression is not supported in this connector");
-                }
-
-                Messages messages = Messages.parseFrom(p.getBody());
-                Message result = new Message(messages.getBatchId());
-                if (lazyParseEntry) {
-                    // byteString
-                    result.setRawEntries(messages.getMessagesList());
-                } else {
-                    for (ByteString byteString : messages.getMessagesList()) {
-                        result.addEntry(Entry.parseFrom(byteString));
-                    }
-                }
-                return result;
-            }
-            case ACK: {
-                Ack ack = Ack.parseFrom(p.getBody());
-                throw new CanalClientException("something goes wrong with reason: " + ack.getErrorMessage());
-            }
-            default: {
-                throw new CanalClientException("unexpected packet type: " + p.getType());
-            }
-        }
+        return CanalMessageDeserializer.deserializer(data, lazyParseEntry);
    }

    public void ack(long batchId) throws CanalClientException {
@@ -98,7 +98,7 @@ public class ClientRunningMonitor extends AbstractCanalLifeCycle {
        String path = ZookeeperPathUtils.getDestinationClientRunning(this.destination, clientData.getClientId());
        zkClient.unsubscribeDataChanges(path, dataListener);
        releaseRunning(); // 尝试一下release
-        //Fix issue #697
+        // Fix issue #697
        if (delayExector != null) {
            delayExector.shutdown();
        }
@@ -0,0 +1,288 @@
+package com.alibaba.otter.canal.client.kafka;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Properties;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.kafka.clients.consumer.ConsumerRecord;
+import org.apache.kafka.clients.consumer.ConsumerRecords;
+import org.apache.kafka.clients.consumer.KafkaConsumer;
+import org.apache.kafka.common.TopicPartition;
+import org.apache.kafka.common.serialization.StringDeserializer;
+
+import com.alibaba.fastjson.JSON;
+import com.alibaba.otter.canal.client.CanalMQConnector;
+import com.alibaba.otter.canal.client.impl.SimpleCanalConnector;
+import com.alibaba.otter.canal.protocol.FlatMessage;
+import com.alibaba.otter.canal.protocol.Message;
+import com.alibaba.otter.canal.protocol.exception.CanalClientException;
+import com.google.common.collect.Lists;
+
+/**
+ * canal kafka 数据操作客户端
+ * 
+ * <pre>
+ * 注意点：
+ * 1. 相比于canal {@linkplain SimpleCanalConnector}, 这里get和ack操作不能有并发, 必须是一个线程执行get后，内存里执行完毕ack后再取下一个get
+ * </pre>
+ *
+ * @author machengyuan @ 2018-6-12
+ * @version 1.1.1
+ */
+public class KafkaCanalConnector implements CanalMQConnector {
+
+    private KafkaConsumer<String, Message> kafkaConsumer;
+    private KafkaConsumer<String, String>  kafkaConsumer2;   // 用于扁平message的数据消费
+    private String                         topic;
+    private Integer                        partition;
+    private Properties                     properties;
+    private volatile boolean               connected = false;
+    private volatile boolean               running   = false;
+    private boolean                        flatMessage;
+
+    public KafkaCanalConnector(String servers, String topic, Integer partition, String groupId, boolean flatMessage){
+        this.topic = topic;
+        this.partition = partition;
+        this.flatMessage = flatMessage;
+
+        properties = new Properties();
+        properties.put("bootstrap.servers", servers);
+        properties.put("group.id", groupId);
+        properties.put("enable.auto.commit", false);
+        properties.put("auto.commit.interval.ms", "1000");
+        properties.put("auto.offset.reset", "latest"); // 如果没有offset则从最后的offset开始读
+        properties.put("request.timeout.ms", "40000"); // 必须大于session.timeout.ms的设置
+        properties.put("session.timeout.ms", "30000"); // 默认为30秒
+        properties.put("max.poll.records", "100");
+        properties.put("key.deserializer", StringDeserializer.class.getName());
+        if (!flatMessage) {
+            properties.put("value.deserializer", MessageDeserializer.class.getName());
+        } else {
+            properties.put("value.deserializer", StringDeserializer.class.getName());
+        }
+    }
+
+    /**
+     * 打开连接
+     */
+    public void connect() {
+        if (connected) {
+            return;
+        }
+
+        connected = true;
+        if (kafkaConsumer == null && !flatMessage) {
+            kafkaConsumer = new KafkaConsumer<String, Message>(properties);
+        }
+        if (kafkaConsumer2 == null && flatMessage) {
+            kafkaConsumer2 = new KafkaConsumer<String, String>(properties);
+        }
+    }
+
+    /**
+     * 关闭链接
+     */
+    public void disconnect() {
+        if (kafkaConsumer != null) {
+            kafkaConsumer.close();
+            kafkaConsumer = null;
+        }
+        if (kafkaConsumer2 != null) {
+            kafkaConsumer2.close();
+            kafkaConsumer2 = null;
+        }
+
+        connected = false;
+    }
+
+    private void waitClientRunning() {
+        running = true;
+    }
+
+    public boolean checkValid() {
+        return true;// 默认都放过
+    }
+
+    /**
+     * 订阅topic
+     */
+    public void subscribe() {
+        waitClientRunning();
+        if (!running) {
+            return;
+        }
+
+        if (partition == null) {
+            if (kafkaConsumer != null) {
+                kafkaConsumer.subscribe(Collections.singletonList(topic));
+            }
+            if (kafkaConsumer2 != null) {
+                kafkaConsumer2.subscribe(Collections.singletonList(topic));
+            }
+        } else {
+            TopicPartition topicPartition = new TopicPartition(topic, partition);
+            if (kafkaConsumer != null) {
+                kafkaConsumer.assign(Collections.singletonList(topicPartition));
+            }
+            if (kafkaConsumer2 != null) {
+                kafkaConsumer2.assign(Collections.singletonList(topicPartition));
+            }
+        }
+    }
+
+    /**
+     * 取消订阅
+     */
+    public void unsubscribe() {
+        waitClientRunning();
+        if (!running) {
+            return;
+        }
+
+        if (kafkaConsumer != null) {
+            kafkaConsumer.unsubscribe();
+        }
+        if (kafkaConsumer2 != null) {
+            kafkaConsumer2.unsubscribe();
+        }
+    }
+
+    @Override
+    public List<Message> getList(Long timeout, TimeUnit unit) throws CanalClientException {
+        waitClientRunning();
+        if (!running) {
+            return Lists.newArrayList();
+        }
+
+        List<Message> messages = getListWithoutAck(timeout, unit);
+        if (messages != null && !messages.isEmpty()) {
+            this.ack();
+        }
+        return messages;
+    }
+
+    @Override
+    public List<Message> getListWithoutAck(Long timeout, TimeUnit unit) throws CanalClientException {
+        waitClientRunning();
+        if (!running) {
+            return Lists.newArrayList();
+        }
+
+        ConsumerRecords<String, Message> records = kafkaConsumer.poll(unit.toMillis(timeout)); // 基于配置，最多只能poll到一条数据
+
+        if (!records.isEmpty()) {
+            List<Message> messages = new ArrayList<>();
+            for (ConsumerRecord<String, Message> record : records) {
+                messages.add(record.value());
+            }
+            return messages;
+        }
+        return Lists.newArrayList();
+    }
+
+    @Override
+    public List<FlatMessage> getFlatList(Long timeout, TimeUnit unit) throws CanalClientException {
+        waitClientRunning();
+        if (!running) {
+            return Lists.newArrayList();
+        }
+
+        List<FlatMessage> messages = getFlatListWithoutAck(timeout, unit);
+        if (messages != null && !messages.isEmpty()) {
+            this.ack();
+        }
+        return messages;
+    }
+
+    @Override
+    public List<FlatMessage> getFlatListWithoutAck(Long timeout, TimeUnit unit) throws CanalClientException {
+        waitClientRunning();
+        if (!running) {
+            return Lists.newArrayList();
+        }
+
+        ConsumerRecords<String, String> records = kafkaConsumer2.poll(unit.toMillis(timeout));
+        if (!records.isEmpty()) {
+            List<FlatMessage> flatMessages = new ArrayList<>();
+            for (ConsumerRecord<String, String> record : records) {
+                String flatMessageJson = record.value();
+                FlatMessage flatMessage = JSON.parseObject(flatMessageJson, FlatMessage.class);
+                flatMessages.add(flatMessage);
+            }
+
+            return flatMessages;
+        }
+        return Lists.newArrayList();
+    }
+
+    @Override
+    public void rollback() throws CanalClientException {
+    }
+
+    /**
+     * 提交offset，如果超过 session.timeout.ms 设置的时间没有ack则会抛出异常，ack失败
+     */
+    public void ack() {
+        waitClientRunning();
+        if (!running) {
+            return;
+        }
+
+        if (kafkaConsumer != null) {
+            kafkaConsumer.commitSync();
+        }
+        if (kafkaConsumer2 != null) {
+            kafkaConsumer2.commitSync();
+        }
+    }
+
+    @Override
+    public void subscribe(String filter) throws CanalClientException {
+        throw new CanalClientException("mq not support this method");
+    }
+
+    @Override
+    public Message get(int batchSize) throws CanalClientException {
+        throw new CanalClientException("mq not support this method");
+    }
+
+    @Override
+    public Message get(int batchSize, Long timeout, TimeUnit unit) throws CanalClientException {
+        throw new CanalClientException("mq not support this method");
+    }
+
+    @Override
+    public Message getWithoutAck(int batchSize) throws CanalClientException {
+        throw new CanalClientException("mq not support this method");
+    }
+
+    @Override
+    public Message getWithoutAck(int batchSize, Long timeout, TimeUnit unit) throws CanalClientException {
+        throw new CanalClientException("mq not support this method");
+    }
+
+    @Override
+    public void ack(long batchId) throws CanalClientException {
+        throw new CanalClientException("mq not support this method");
+    }
+
+    @Override
+    public void rollback(long batchId) throws CanalClientException {
+        throw new CanalClientException("mq not support this method");
+    }
+
+    /**
+     * 重新设置sessionTime
+     *
+     * @param timeout
+     * @param unit
+     */
+    public void setSessionTimeout(Long timeout, TimeUnit unit) {
+        long t = unit.toMillis(timeout);
+        properties.put("request.timeout.ms", String.valueOf(t + 60000));
+        properties.put("session.timeout.ms", String.valueOf(t));
+    }
+
+}
@@ -1,4 +1,4 @@
-package com.alibaba.otter.canal.kafka.client;
+package com.alibaba.otter.canal.client.kafka;

 /**
 * canal kafka connectors创建工具类
@@ -18,7 +18,7 @@ public class KafkaCanalConnectors {
     * @return
     */
    public static KafkaCanalConnector newKafkaConnector(String servers, String topic, Integer partition, String groupId) {
-        return new KafkaCanalConnector(null, servers, topic, partition, groupId);
+        return new KafkaCanalConnector(servers, topic, partition, groupId, false);
    }

    /**
@@ -30,33 +30,21 @@ public class KafkaCanalConnectors {
     * @return
     */
    public static KafkaCanalConnector newKafkaConnector(String servers, String topic, String groupId) {
-        return new KafkaCanalConnector(null, servers, topic, null, groupId);
+        return new KafkaCanalConnector(servers, topic, null, groupId, false);
    }

    /**
     * 创建kafka客户端链接
     *
-     * @param zkServers
     * @param servers
     * @param topic
     * @param partition
     * @param groupId
+     * @param flatMessage
     * @return
     */
-    public static KafkaCanalConnector newKafkaConnector(String zkServers, String servers, String topic, Integer partition, String groupId) {
-        return new KafkaCanalConnector(zkServers, servers, topic, partition, groupId);
-    }
-
-    /**
-     * 创建kafka客户端链接
-     *
-     * @param zkServers
-     * @param servers
-     * @param topic
-     * @param groupId
-     * @return
-     */
-    public static KafkaCanalConnector newKafkaConnector(String zkServers, String servers, String topic, String groupId) {
-        return new KafkaCanalConnector(zkServers, servers, topic, null, groupId);
+    public static KafkaCanalConnector newKafkaConnector(String servers, String topic, Integer partition,
+                                                        String groupId, boolean flatMessage) {
+        return new KafkaCanalConnector(servers, topic, partition, groupId, flatMessage);
    }
 }
@@ -0,0 +1,31 @@
+package com.alibaba.otter.canal.client.kafka;
+
+import java.util.Map;
+
+import org.apache.kafka.common.serialization.Deserializer;
+
+import com.alibaba.otter.canal.client.CanalMessageDeserializer;
+import com.alibaba.otter.canal.protocol.Message;
+
+/**
+ * Kafka Message类的反序列化
+ *
+ * @author machengyuan @ 2018-6-12
+ * @version 1.0.0
+ */
+public class MessageDeserializer implements Deserializer<Message> {
+
+    @Override
+    public void configure(Map<String, ?> configs, boolean isKey) {
+    }
+
+    @Override
+    public Message deserialize(String topic1, byte[] data) {
+        return CanalMessageDeserializer.deserializer(data);
+    }
+
+    @Override
+    public void close() {
+        // nothing to do
+    }
+}
@@ -0,0 +1,48 @@
+package com.alibaba.otter.canal.client.rocketmq;
+
+import java.util.List;
+import java.util.concurrent.CountDownLatch;
+import java.util.concurrent.TimeUnit;
+
+public class ConsumerBatchMessage<T> {
+
+    private final List<T>  data;
+    private CountDownLatch         latch;
+    private boolean                hasFailure = false;
+
+    public ConsumerBatchMessage(List<T> data){
+        this.data = data;
+        latch = new CountDownLatch(1);
+    }
+
+    public boolean waitFinish(long timeout) throws InterruptedException {
+        return latch.await(timeout, TimeUnit.MILLISECONDS);
+    }
+
+    public boolean isSuccess() {
+        return !hasFailure;
+    }
+
+    public List<T> getData() {
+        return data;
+    }
+
+    /**
+     * Countdown if the sub message is successful.
+     */
+    public void ack() {
+        latch.countDown();
+    }
+
+    /**
+     * Countdown and fail-fast if the sub message is failed.
+     */
+    public void fail() {
+        hasFailure = true;
+        // fail fast
+        long count = latch.getCount();
+        for (int i = 0; i < count; i++) {
+            latch.countDown();
+        }
+    }
+}
@@ -0,0 +1,263 @@
+package com.alibaba.otter.canal.client.rocketmq;
+
+import java.util.List;
+import java.util.concurrent.BlockingQueue;
+import java.util.concurrent.LinkedBlockingQueue;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.commons.lang.StringUtils;
+import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
+import org.apache.rocketmq.client.consumer.listener.ConsumeOrderlyContext;
+import org.apache.rocketmq.client.consumer.listener.ConsumeOrderlyStatus;
+import org.apache.rocketmq.client.consumer.listener.MessageListenerOrderly;
+import org.apache.rocketmq.client.exception.MQClientException;
+import org.apache.rocketmq.common.message.MessageExt;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.alibaba.fastjson.JSON;
+import com.alibaba.otter.canal.client.CanalMQConnector;
+import com.alibaba.otter.canal.client.CanalMessageDeserializer;
+import com.alibaba.otter.canal.client.impl.SimpleCanalConnector;
+import com.alibaba.otter.canal.protocol.FlatMessage;
+import com.alibaba.otter.canal.protocol.Message;
+import com.alibaba.otter.canal.protocol.exception.CanalClientException;
+import com.google.common.collect.Lists;
+
+/**
+ * RocketMQ的连接
+ * 
+ * <pre>
+ * 注意点:
+ * 1. 相比于canal {@linkplain SimpleCanalConnector}, 这里get和ack操作不能有并发, 必须是一个线程执行get后，内存里执行完毕ack后再取下一个get
+ * </pre>
+ * 
+ * @since 1.1.1
+ */
+public class RocketMQCanalConnector implements CanalMQConnector {
+
+    private static final Logger                 logger              = LoggerFactory.getLogger(RocketMQCanalConnector.class);
+
+    private String                              nameServer;
+    private String                              topic;
+    private String                              groupName;
+    private volatile boolean                    connected           = false;
+    private DefaultMQPushConsumer               rocketMQConsumer;
+    private BlockingQueue<ConsumerBatchMessage> messageBlockingQueue;
+    private long                                batchProcessTimeout = 60 * 1000;
+    private boolean                             flatMessage;
+    private volatile ConsumerBatchMessage       lastGetBatchMessage = null;
+
+    public RocketMQCanalConnector(String nameServer, String topic, String groupName, boolean flatMessage){
+        this.nameServer = nameServer;
+        this.topic = topic;
+        this.groupName = groupName;
+        this.flatMessage = flatMessage;
+        this.messageBlockingQueue = new LinkedBlockingQueue<>(1024);
+    }
+
+    public void connect() throws CanalClientException {
+        rocketMQConsumer = new DefaultMQPushConsumer(groupName);
+        if (!StringUtils.isBlank(nameServer)) {
+            rocketMQConsumer.setNamesrvAddr(nameServer);
+        }
+    }
+
+    public void disconnect() throws CanalClientException {
+        rocketMQConsumer.shutdown();
+        connected = false;
+    }
+
+    public boolean checkValid() throws CanalClientException {
+        return connected;
+    }
+
+    public synchronized void subscribe(String filter) throws CanalClientException {
+        if (connected) {
+            return;
+        }
+        try {
+            if (rocketMQConsumer == null) {
+                this.connect();
+            }
+            rocketMQConsumer.subscribe(this.topic, "*");
+            rocketMQConsumer.registerMessageListener(new MessageListenerOrderly() {
+
+                @Override
+                public ConsumeOrderlyStatus consumeMessage(List<MessageExt> messageExts, ConsumeOrderlyContext context) {
+                    context.setAutoCommit(true);
+                    boolean isSuccess = process(messageExts);
+                    if (isSuccess) {
+                        return ConsumeOrderlyStatus.SUCCESS;
+                    } else {
+                        return ConsumeOrderlyStatus.SUSPEND_CURRENT_QUEUE_A_MOMENT;
+                    }
+                }
+            });
+            rocketMQConsumer.start();
+        } catch (MQClientException ex) {
+            connected = false;
+            logger.error("Start RocketMQ consumer error", ex);
+        }
+        connected = true;
+    }
+
+    private boolean process(List<MessageExt> messageExts) {
+        logger.info("Get Message:{}", messageExts);
+        List messageList = Lists.newArrayList();
+        for (MessageExt messageExt : messageExts) {
+            byte[] data = messageExt.getBody();
+            if (data != null) {
+                try {
+                    if (!flatMessage) {
+                        Message message = CanalMessageDeserializer.deserializer(data);
+                        messageList.add(message);
+                    } else {
+                        FlatMessage flatMessage = JSON.parseObject(data, FlatMessage.class);
+                        messageList.add(flatMessage);
+                    }
+                } catch (Exception ex) {
+                    logger.error("Add message error", ex);
+                    throw new CanalClientException(ex);
+                }
+            } else {
+                logger.warn("Received message data is null");
+            }
+        }
+        ConsumerBatchMessage batchMessage;
+        if (!flatMessage) {
+            batchMessage = new ConsumerBatchMessage<Message>(messageList);
+        } else {
+            batchMessage = new ConsumerBatchMessage<FlatMessage>(messageList);
+        }
+        try {
+            messageBlockingQueue.put(batchMessage);
+        } catch (InterruptedException e) {
+            logger.error("Put message to queue error", e);
+            throw new RuntimeException(e);
+        }
+        boolean isCompleted;
+        try {
+            isCompleted = batchMessage.waitFinish(batchProcessTimeout);
+        } catch (InterruptedException e) {
+            logger.error("Interrupted when waiting messages to be finished.", e);
+            throw new RuntimeException(e);
+        }
+        boolean isSuccess = batchMessage.isSuccess();
+        return isCompleted && isSuccess;
+    }
+
+    public void subscribe() throws CanalClientException {
+        this.subscribe(null);
+    }
+
+    public void unsubscribe() throws CanalClientException {
+        this.rocketMQConsumer.unsubscribe(this.topic);
+    }
+
+    @Override
+    public List<Message> getList(Long timeout, TimeUnit unit) throws CanalClientException {
+        List<Message> messages = getListWithoutAck(timeout, unit);
+        if (messages != null && !messages.isEmpty()) {
+            ack();
+        }
+        return messages;
+    }
+
+    @Override
+    public List<Message> getListWithoutAck(Long timeout, TimeUnit unit) throws CanalClientException {
+        try {
+            if (this.lastGetBatchMessage != null) {
+                throw new CanalClientException("mq get/ack not support concurrent & async ack");
+            }
+
+            ConsumerBatchMessage batchMessage = messageBlockingQueue.poll(timeout, unit);
+            if (batchMessage != null) {
+                this.lastGetBatchMessage = batchMessage;
+                return batchMessage.getData();
+            }
+        } catch (InterruptedException ex) {
+            logger.warn("Get message timeout", ex);
+            throw new CanalClientException("Failed to fetch the data after: " + timeout);
+        }
+        return Lists.newArrayList();
+    }
+
+    @Override
+    public List<FlatMessage> getFlatList(Long timeout, TimeUnit unit) throws CanalClientException {
+        List<FlatMessage> messages = getFlatListWithoutAck(timeout, unit);
+        if (messages != null && !messages.isEmpty()) {
+            ack();
+        }
+        return messages;
+    }
+
+    @Override
+    public List<FlatMessage> getFlatListWithoutAck(Long timeout, TimeUnit unit) throws CanalClientException {
+        try {
+            if (this.lastGetBatchMessage != null) {
+                throw new CanalClientException("mq get/ack not support concurrent & async ack");
+            }
+
+            ConsumerBatchMessage batchMessage = messageBlockingQueue.poll(timeout, unit);
+            if (batchMessage != null) {
+                this.lastGetBatchMessage = batchMessage;
+                return batchMessage.getData();
+            }
+        } catch (InterruptedException ex) {
+            logger.warn("Get message timeout", ex);
+            throw new CanalClientException("Failed to fetch the data after: " + timeout);
+        }
+        return Lists.newArrayList();
+    }
+
+    @Override
+    public void ack() throws CanalClientException {
+        try {
+            this.lastGetBatchMessage.ack();
+        } catch (Throwable e) {
+            this.lastGetBatchMessage.fail();
+        } finally {
+            this.lastGetBatchMessage = null;
+        }
+    }
+
+    @Override
+    public void rollback() throws CanalClientException {
+        try {
+            this.lastGetBatchMessage.fail();
+        } finally {
+            this.lastGetBatchMessage = null;
+        }
+    }
+
+    public Message get(int batchSize) throws CanalClientException {
+        throw new CanalClientException("mq not support this method");
+    }
+
+    @Override
+    public Message get(int batchSize, Long timeout, TimeUnit unit) throws CanalClientException {
+        throw new CanalClientException("mq not support this method");
+    }
+
+    @Override
+    public Message getWithoutAck(int batchSize) throws CanalClientException {
+        throw new CanalClientException("mq not support this method");
+    }
+
+    @Override
+    public Message getWithoutAck(int batchSize, Long timeout, TimeUnit unit) throws CanalClientException {
+        throw new CanalClientException("mq not support this method");
+    }
+
+    @Override
+    public void ack(long batchId) throws CanalClientException {
+        throw new CanalClientException("mq not support this method");
+    }
+
+    @Override
+    public void rollback(long batchId) throws CanalClientException {
+        throw new CanalClientException("mq not support this method");
+    }
+
+}
@@ -0,0 +1,24 @@
+package com.alibaba.otter.canal.client.rocketmq;
+
+/**
+ * RocketMQ connector provider.
+ */
+public class RocketMQCanalConnectors {
+
+    /**
+     * Create RocketMQ connector
+     *
+     * @param nameServers name servers for RocketMQ
+     * @param topic
+     * @param groupId
+     * @return {@link RocketMQCanalConnector}
+     */
+    public static RocketMQCanalConnector newRocketMQConnector(String nameServers, String topic, String groupId) {
+        return new RocketMQCanalConnector(nameServers, topic, groupId, false);
+    }
+
+    public static RocketMQCanalConnector newRocketMQConnector(String nameServers, String topic, String groupId,
+                                                              boolean flatMessage) {
+        return new RocketMQCanalConnector(nameServers, topic, groupId, flatMessage);
+    }
+}
@@ -1,4 +1,4 @@
-package com.alibaba.otter.canal.kafka.client.running;
+package com.alibaba.otter.canal.client.running.kafka;

 import org.junit.Assert;

@@ -1,5 +1,6 @@
-package com.alibaba.otter.canal.kafka.client.running;
+package com.alibaba.otter.canal.client.running.kafka;

+import java.util.List;
 import java.util.concurrent.TimeUnit;

 import org.apache.kafka.common.errors.WakeupException;
@@ -7,8 +8,8 @@ import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.util.Assert;

-import com.alibaba.otter.canal.kafka.client.KafkaCanalConnector;
-import com.alibaba.otter.canal.kafka.client.KafkaCanalConnectors;
+import com.alibaba.otter.canal.client.kafka.KafkaCanalConnector;
+import com.alibaba.otter.canal.client.kafka.KafkaCanalConnectors;
 import com.alibaba.otter.canal.protocol.Message;

 /**
@@ -19,32 +20,32 @@ import com.alibaba.otter.canal.protocol.Message;
 */
 public class CanalKafkaClientExample {

-    protected final static Logger logger = LoggerFactory.getLogger(CanalKafkaClientExample.class);
+    protected final static Logger           logger  = LoggerFactory.getLogger(CanalKafkaClientExample.class);

-    private KafkaCanalConnector connector;
+    private KafkaCanalConnector             connector;

-    private static volatile boolean running = false;
+    private static volatile boolean         running = false;

-    private Thread thread = null;
+    private Thread                          thread  = null;

    private Thread.UncaughtExceptionHandler handler = new Thread.UncaughtExceptionHandler() {

-        public void uncaughtException(Thread t, Throwable e) {
-            logger.error("parse events has an error", e);
-        }
-    };
+                                                        public void uncaughtException(Thread t, Throwable e) {
+                                                            logger.error("parse events has an error", e);
+                                                        }
+                                                    };

-    public CanalKafkaClientExample(String zkServers, String servers, String topic, Integer partition, String groupId) {
-        connector = KafkaCanalConnectors.newKafkaConnector(zkServers, servers, topic, partition, groupId);
+    public CanalKafkaClientExample(String zkServers, String servers, String topic, Integer partition, String groupId){
+        connector = KafkaCanalConnectors.newKafkaConnector(servers, topic, partition, groupId, false);
    }

    public static void main(String[] args) {
        try {
            final CanalKafkaClientExample kafkaCanalClientExample = new CanalKafkaClientExample(AbstractKafkaTest.zkServers,
-                    AbstractKafkaTest.servers,
-                    AbstractKafkaTest.topic,
-                    AbstractKafkaTest.partition,
-                    AbstractKafkaTest.groupId);
+                AbstractKafkaTest.servers,
+                AbstractKafkaTest.topic,
+                AbstractKafkaTest.partition,
+                AbstractKafkaTest.groupId);
            logger.info("## start the kafka consumer: {}-{}", AbstractKafkaTest.topic, AbstractKafkaTest.groupId);
            kafkaCanalClientExample.start();
            logger.info("## the canal kafka consumer is running now ......");
@@ -87,7 +88,6 @@ public class CanalKafkaClientExample {
        if (!running) {
            return;
        }
-        connector.stopRunning();
        running = false;
        if (thread != null) {
            try {
@@ -107,21 +107,23 @@ public class CanalKafkaClientExample {
                connector.subscribe();
                while (running) {
                    try {
-                        Message message = connector.getWithoutAck(1L, TimeUnit.SECONDS); // 获取message
-                        if (message == null) {
+                        List<Message> messages = connector.getListWithoutAck(100L, TimeUnit.MILLISECONDS); // 获取message
+                        if (messages == null) {
                            continue;
                        }
-                        long batchId = message.getId();
-                        int size = message.getEntries().size();
-                        if (batchId == -1 || size == 0) {
-                            // try {
-                            // Thread.sleep(1000);
-                            // } catch (InterruptedException e) {
-                            // }
-                        } else {
-                            // printSummary(message, batchId, size);
-                            // printEntry(message.getEntries());
-                            logger.info(message.toString());
+                        for (Message message : messages) {
+                            long batchId = message.getId();
+                            int size = message.getEntries().size();
+                            if (batchId == -1 || size == 0) {
+                                // try {
+                                // Thread.sleep(1000);
+                                // } catch (InterruptedException e) {
+                                // }
+                            } else {
+                                // printSummary(message, batchId, size);
+                                // printEntry(message.getEntries());
+                                logger.info(message.toString());
+                            }
                        }

                        connector.ack(); // 提交确认
@@ -139,6 +141,6 @@ public class CanalKafkaClientExample {
        } catch (WakeupException e) {
            // No-op. Continue process
        }
-        connector.disconnnect();
+        connector.disconnect();
    }
 }
@@ -1,5 +1,6 @@
-package com.alibaba.otter.canal.kafka.client.running;
+package com.alibaba.otter.canal.client.running.kafka;

+import java.util.List;
 import java.util.concurrent.ExecutorService;
 import java.util.concurrent.Executors;
 import java.util.concurrent.TimeUnit;
@@ -9,8 +10,8 @@ import org.junit.Test;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;

-import com.alibaba.otter.canal.kafka.client.KafkaCanalConnector;
-import com.alibaba.otter.canal.kafka.client.KafkaCanalConnectors;
+import com.alibaba.otter.canal.client.kafka.KafkaCanalConnector;
+import com.alibaba.otter.canal.client.kafka.KafkaCanalConnectors;
 import com.alibaba.otter.canal.protocol.Message;

 /**
@@ -39,9 +40,9 @@ public class KafkaClientRunningTest extends AbstractKafkaTest {
                connector.subscribe();
                while (running) {
                    try {
-                        Message message = connector.getWithoutAck(3L, TimeUnit.SECONDS);
-                        if (message != null) {
-                            System.out.println(message);
+                        List<Message> messages = connector.getList(3L, TimeUnit.SECONDS);
+                        if (messages != null) {
+                            System.out.println(messages);
                        }
                        connector.ack();
                    } catch (WakeupException e) {
@@ -49,7 +50,7 @@ public class KafkaClientRunningTest extends AbstractKafkaTest {
                    }
                }
                connector.unsubscribe();
-                connector.disconnnect();
+                connector.disconnect();
            }
        });

@@ -0,0 +1,9 @@
+package com.alibaba.otter.canal.client.running.rocketmq;
+
+public class AbstractRocektMQTest {
+
+    public static String topic       = "example";
+    public static String groupId     = "group";
+    public static String nameServers = "localhost:9876";
+
+}
@@ -0,0 +1,138 @@
+package com.alibaba.otter.canal.client.running.rocketmq;
+
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+
+import org.apache.kafka.common.errors.WakeupException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.util.Assert;
+
+import com.alibaba.otter.canal.client.rocketmq.RocketMQCanalConnector;
+import com.alibaba.otter.canal.client.rocketmq.RocketMQCanalConnectors;
+import com.alibaba.otter.canal.client.running.kafka.AbstractKafkaTest;
+import com.alibaba.otter.canal.protocol.Message;
+
+/**
+ * Kafka client example
+ *
+ * @author machengyuan @ 2018-6-12
+ * @version 1.0.0
+ */
+public class CanalRocketMQClientExample extends AbstractRocektMQTest {
+
+    protected final static Logger           logger  = LoggerFactory.getLogger(CanalRocketMQClientExample.class);
+
+    private RocketMQCanalConnector          connector;
+
+    private static volatile boolean         running = false;
+
+    private Thread                          thread  = null;
+
+    private Thread.UncaughtExceptionHandler handler = new Thread.UncaughtExceptionHandler() {
+
+                                                        public void uncaughtException(Thread t, Throwable e) {
+                                                            logger.error("parse events has an error", e);
+                                                        }
+                                                    };
+
+    public CanalRocketMQClientExample(String nameServers, String topic, String groupId){
+        connector = RocketMQCanalConnectors.newRocketMQConnector(nameServers, topic, groupId);
+    }
+
+    public static void main(String[] args) {
+        try {
+            final CanalRocketMQClientExample rocketMQClientExample = new CanalRocketMQClientExample(nameServers,
+                topic,
+                groupId);
+            logger.info("## Start the rocketmq consumer: {}-{}", AbstractKafkaTest.topic, AbstractKafkaTest.groupId);
+            rocketMQClientExample.start();
+            logger.info("## The canal rocketmq consumer is running now ......");
+            Runtime.getRuntime().addShutdownHook(new Thread() {
+
+                public void run() {
+                    try {
+                        logger.info("## Stop the rocketmq consumer");
+                        rocketMQClientExample.stop();
+                    } catch (Throwable e) {
+                        logger.warn("## Something goes wrong when stopping rocketmq consumer:", e);
+                    } finally {
+                        logger.info("## Rocketmq consumer is down.");
+                    }
+                }
+
+            });
+            while (running)
+                ;
+        } catch (Throwable e) {
+            logger.error("## Something going wrong when starting up the rocketmq consumer:", e);
+            System.exit(0);
+        }
+    }
+
+    public void start() {
+        Assert.notNull(connector, "connector is null");
+        thread = new Thread(new Runnable() {
+
+            public void run() {
+                process();
+            }
+        });
+        thread.setUncaughtExceptionHandler(handler);
+        thread.start();
+        running = true;
+    }
+
+    public void stop() {
+        if (!running) {
+            return;
+        }
+        running = false;
+        if (thread != null) {
+            try {
+                thread.join();
+            } catch (InterruptedException e) {
+                // ignore
+            }
+        }
+    }
+
+    private void process() {
+        while (!running)
+            ;
+        while (running) {
+            try {
+                connector.connect();
+                connector.subscribe();
+                while (running) {
+                    List<Message> messages = connector.getListWithoutAck(100L, TimeUnit.MILLISECONDS); // 获取message
+                    for (Message message : messages) {
+                        long batchId = message.getId();
+                        int size = message.getEntries().size();
+                        if (batchId == -1 || size == 0) {
+                            // try {
+                            // Thread.sleep(1000);
+                            // } catch (InterruptedException e) {
+                            // }
+                        } else {
+                            // printSummary(message, batchId, size);
+                            // printEntry(message.getEntries());
+                            logger.info(message.toString());
+                        }
+                    }
+
+                    connector.ack(); // 提交确认
+                }
+            } catch (Exception e) {
+                logger.error(e.getMessage(), e);
+            }
+        }
+
+        try {
+            connector.unsubscribe();
+        } catch (WakeupException e) {
+            // No-op. Continue process
+        }
+//        connector.stopRunning();
+    }
+}
@@ -3,7 +3,7 @@
 	<parent>
 		<groupId>com.alibaba.otter</groupId>
 		<artifactId>canal</artifactId>
-		<version>1.0.26-SNAPSHOT</version>
+		<version>1.1.1</version>
 		<relativePath>../pom.xml</relativePath>
 	</parent>
 	<artifactId>canal.common</artifactId>
@@ -83,6 +83,10 @@
 			<groupId>org.springframework</groupId>
 			<artifactId>spring-orm</artifactId>
 		</dependency>
+		<dependency>
+			<groupId>com.google.code.findbugs</groupId>
+			<artifactId>jsr305</artifactId>
+		</dependency>
 		<dependency>
 			<groupId>org.springframework</groupId>
 			<artifactId>spring-test</artifactId>
@@ -5,7 +5,7 @@ import java.util.concurrent.TimeoutException;
 import java.util.concurrent.locks.AbstractQueuedSynchronizer;

 /**
- * 实现一个互斥实现，基于Cocurrent中的{@linkplain AbstractQueuedSynchronizer}实现了自己的sync <br/>
+ * 实现一个互斥实现，基于Cocurrent中的AbstractQueuedSynchronizer实现了自己的sync <br/>
 * 应用场景：系统初始化/授权控制，没权限时阻塞等待。有权限时所有线程都可以快速通过
 * 
 * <pre>
@@ -3,7 +3,7 @@
 	<parent>
 		<groupId>com.alibaba.otter</groupId>
 		<artifactId>canal</artifactId>
-		<version>1.0.26-SNAPSHOT</version>
+		<version>1.1.1</version>
 		<relativePath>../pom.xml</relativePath>
 	</parent>
 	<groupId>com.alibaba.otter</groupId>
@@ -11,6 +11,11 @@
 	<packaging>jar</packaging>
 	<name>canal dbsync module for otter ${project.version}</name>
 	<dependencies>
+		<dependency>
+			<groupId>com.alibaba.otter</groupId>
+			<artifactId>canal.parse.driver</artifactId>
+			<version>${project.version}</version>
+		</dependency>
 		<!-- log -->
 		<dependency>
 			<groupId>ch.qos.logback</groupId>
@@ -168,7 +168,9 @@ public final class FileLogFetcher extends LogFetcher {
     * @see com.taobao.tddl.dbsync.binlog.LogFetcher#close()
     */
    public void close() throws IOException {
-        if (fin != null) fin.close();
+        if (fin != null) {
+            fin.close();
+        }

        fin = null;
    }
@@ -3,7 +3,9 @@ package com.taobao.tddl.dbsync.binlog;
 import java.util.HashMap;
 import java.util.Map;

+import com.alibaba.otter.canal.parse.driver.mysql.packets.GTIDSet;
 import com.taobao.tddl.dbsync.binlog.event.FormatDescriptionLogEvent;
+import com.taobao.tddl.dbsync.binlog.event.GtidLogEvent;
 import com.taobao.tddl.dbsync.binlog.event.TableMapLogEvent;

 /**
@@ -20,6 +22,10 @@ public final class LogContext {

    private LogPosition                       logPosition;

+    private GTIDSet                           gtidSet;
+
+    private GtidLogEvent                      gtidLogEvent; // save current gtid log event
+
    public LogContext(){
        this.formatDescription = FormatDescriptionLogEvent.FORMAT_DESCRIPTION_EVENT_5_x;
    }
@@ -60,4 +66,20 @@ public final class LogContext {
        formatDescription = FormatDescriptionLogEvent.FORMAT_DESCRIPTION_EVENT_5_x;
        mapOfTable.clear();
    }
+
+    public GTIDSet getGtidSet() {
+        return gtidSet;
+    }
+
+    public void setGtidSet(GTIDSet gtidSet) {
+        this.gtidSet = gtidSet;
+    }
+
+    public GtidLogEvent getGtidLogEvent() {
+        return gtidLogEvent;
+    }
+
+    public void setGtidLogEvent(GtidLogEvent gtidLogEvent) {
+        this.gtidLogEvent = gtidLogEvent;
+    }
 }
@@ -6,6 +6,7 @@ import java.util.BitSet;
 import org.apache.commons.logging.Log;
 import org.apache.commons.logging.LogFactory;

+import com.alibaba.otter.canal.parse.driver.mysql.packets.GTIDSet;
 import com.taobao.tddl.dbsync.binlog.event.AppendBlockLogEvent;
 import com.taobao.tddl.dbsync.binlog.event.BeginLoadQueryLogEvent;
 import com.taobao.tddl.dbsync.binlog.event.CreateFileLogEvent;
@@ -46,7 +47,7 @@ import com.taobao.tddl.dbsync.binlog.event.mariadb.StartEncryptionLogEvent;

 /**
 * Implements a binary-log decoder.
- * 
+ *
 * <pre>
 * LogDecoder decoder = new LogDecoder();
 * decoder.handle(...);
@@ -61,7 +62,7 @@ import com.taobao.tddl.dbsync.binlog.event.mariadb.StartEncryptionLogEvent;
 * while (event != null);
 * // no more events in buffer.
 * </pre>
- * 
+ *
 * @author <a href="mailto:changyuan.lh@taobao.com">Changyuan.lh</a>
 * @version 1.0
 */
@@ -88,7 +89,7 @@ public final class LogDecoder {

    /**
     * Decoding an event from binary-log buffer.
-     * 
+     *
     * @return <code>UknownLogEvent</code> if event type is unknown or skipped,
     * <code>null</code> if buffer is not including a full event.
     */
@@ -140,7 +141,7 @@ public final class LogDecoder {

    /**
     * Deserialize an event from buffer.
-     * 
+     *
     * @return <code>UknownLogEvent</code> if event type is unknown or skipped.
     */
    public static LogEvent decode(LogBuffer buffer, LogHeader header, LogContext context) throws IOException {
@@ -159,18 +160,21 @@ public final class LogDecoder {
            // remove checksum bytes
            buffer.limit(header.getEventLen() - LogEvent.BINLOG_CHECKSUM_LEN);
        }
-
+        GTIDSet gtidSet = context.getGtidSet();
+        GtidLogEvent gtidLogEvent = context.getGtidLogEvent();
        switch (header.getType()) {
            case LogEvent.QUERY_EVENT: {
                QueryLogEvent event = new QueryLogEvent(header, buffer, descriptionEvent);
                /* updating position in context */
                logPosition.position = header.getLogPos();
+                header.putGtid(context.getGtidSet(), gtidLogEvent);
                return event;
            }
            case LogEvent.XID_EVENT: {
                XidLogEvent event = new XidLogEvent(header, buffer, descriptionEvent);
                /* updating position in context */
                logPosition.position = header.getLogPos();
+                header.putGtid(context.getGtidSet(), gtidLogEvent);
                return event;
            }
            case LogEvent.TABLE_MAP_EVENT: {
@@ -185,6 +189,7 @@ public final class LogDecoder {
                /* updating position in context */
                logPosition.position = header.getLogPos();
                event.fillTable(context);
+                header.putGtid(context.getGtidSet(), gtidLogEvent);
                return event;
            }
            case LogEvent.UPDATE_ROWS_EVENT_V1: {
@@ -192,6 +197,7 @@ public final class LogDecoder {
                /* updating position in context */
                logPosition.position = header.getLogPos();
                event.fillTable(context);
+                header.putGtid(context.getGtidSet(), gtidLogEvent);
                return event;
            }
            case LogEvent.DELETE_ROWS_EVENT_V1: {
@@ -199,6 +205,7 @@ public final class LogDecoder {
                /* updating position in context */
                logPosition.position = header.getLogPos();
                event.fillTable(context);
+                header.putGtid(context.getGtidSet(), gtidLogEvent);
                return event;
            }
            case LogEvent.ROTATE_EVENT: {
@@ -268,12 +275,14 @@ public final class LogDecoder {
                RandLogEvent event = new RandLogEvent(header, buffer, descriptionEvent);
                /* updating position in context */
                logPosition.position = header.getLogPos();
+                header.putGtid(context.getGtidSet(), gtidLogEvent);
                return event;
            }
            case LogEvent.USER_VAR_EVENT: {
                UserVarLogEvent event = new UserVarLogEvent(header, buffer, descriptionEvent);
                /* updating position in context */
                logPosition.position = header.getLogPos();
+                header.putGtid(context.getGtidSet(), gtidLogEvent);
                return event;
            }
            case LogEvent.FORMAT_DESCRIPTION_EVENT: {
@@ -336,6 +345,7 @@ public final class LogDecoder {
                RowsQueryLogEvent event = new RowsQueryLogEvent(header, buffer, descriptionEvent);
                /* updating position in context */
                logPosition.position = header.getLogPos();
+                header.putGtid(context.getGtidSet(), gtidLogEvent);
                return event;
            }
            case LogEvent.WRITE_ROWS_EVENT: {
@@ -343,6 +353,7 @@ public final class LogDecoder {
                /* updating position in context */
                logPosition.position = header.getLogPos();
                event.fillTable(context);
+                header.putGtid(context.getGtidSet(), gtidLogEvent);
                return event;
            }
            case LogEvent.UPDATE_ROWS_EVENT: {
@@ -350,6 +361,7 @@ public final class LogDecoder {
                /* updating position in context */
                logPosition.position = header.getLogPos();
                event.fillTable(context);
+                header.putGtid(context.getGtidSet(), gtidLogEvent);
                return event;
            }
            case LogEvent.DELETE_ROWS_EVENT: {
@@ -357,6 +369,7 @@ public final class LogDecoder {
                /* updating position in context */
                logPosition.position = header.getLogPos();
                event.fillTable(context);
+                header.putGtid(context.getGtidSet(), gtidLogEvent);
                return event;
            }
            case LogEvent.GTID_LOG_EVENT:
@@ -364,6 +377,13 @@ public final class LogDecoder {
                GtidLogEvent event = new GtidLogEvent(header, buffer, descriptionEvent);
                /* updating position in context */
                logPosition.position = header.getLogPos();
+                if (gtidSet != null) {
+                    gtidSet.update(event.getGtidStr());
+                    // update latest gtid
+                    header.putGtid(gtidSet, event);
+                }
+                // update current gtid event to context
+                context.setGtidLogEvent(event);
                return event;
            }
            case LogEvent.PREVIOUS_GTIDS_LOG_EVENT: {
@@ -394,6 +414,7 @@ public final class LogDecoder {
                AnnotateRowsEvent event = new AnnotateRowsEvent(header, buffer, descriptionEvent);
                /* updating position in context */
                logPosition.position = header.getLogPos();
+                header.putGtid(context.getGtidSet(), gtidLogEvent);
                return event;
            }
            case LogEvent.BINLOG_CHECKPOINT_EVENT: {
@@ -14,10 +14,10 @@ import com.taobao.tddl.dbsync.binlog.LogEvent;
 public class GtidLogEvent extends LogEvent {

    // / Length of the commit_flag in event encoding
-    public static final int ENCODED_FLAG_LENGTH = 1;
+    public static final int ENCODED_FLAG_LENGTH         = 1;
    // / Length of SID in event encoding
-    public static final int ENCODED_SID_LENGTH  = 16;
-    public static final int LOGICAL_TIMESTAMP_TYPE_CODE  = 2;
+    public static final int ENCODED_SID_LENGTH          = 16;
+    public static final int LOGICAL_TIMESTAMP_TYPE_CODE = 2;

    private boolean         commitFlag;
    private UUID            sid;
@@ -50,8 +50,6 @@ public class GtidLogEvent extends LogEvent {
            sequenceNumber = buffer.getLong64();
        }

-
-
        // ignore gtid info read
        // sid.copy_from((uchar *)ptr_buffer);
        // ptr_buffer+= ENCODED_SID_LENGTH;
@@ -82,4 +80,11 @@ public class GtidLogEvent extends LogEvent {
    public Long getSequenceNumber() {
        return sequenceNumber;
    }
+
+    public String getGtidStr() {
+        StringBuilder sb = new StringBuilder();
+        sb.append(sid.toString()).append(":");
+        sb.append(gno);
+        return sb.toString();
+    }
 }
@@ -1,8 +1,12 @@
 package com.taobao.tddl.dbsync.binlog.event;

+import com.alibaba.otter.canal.parse.driver.mysql.packets.GTIDSet;
 import com.taobao.tddl.dbsync.binlog.LogBuffer;
 import com.taobao.tddl.dbsync.binlog.LogEvent;

+import java.util.HashMap;
+import java.util.Map;
+
 /**
 * The Common-Header, documented in the table @ref Table_common_header "below",
 * always has the same form and length within one version of MySQL. Each event
@@ -120,6 +124,13 @@ public final class LogHeader {
     */
    protected String    logFileName;

+    protected Map<String, String> gtidMap = new HashMap<>();
+
+    private static final String CURRENT_GTID_STRING = "curt_gtid";
+    private static final String CURRENT_GTID_SN = "curt_gtid_sn";
+    private static final String CURRENT_GTID_LAST_COMMIT = "curt_gtid_lct";
+    private static final String GTID_SET_STRING = "gtid_str";
+
    /* for Start_event_v3 */
    public LogHeader(final int type){
        this.type = type;
@@ -288,4 +299,31 @@ public final class LogHeader {
            crc = buffer.getUint32(eventLen - LogEvent.BINLOG_CHECKSUM_LEN);
        }
    }
+
+    public String getGtidSetStr() {
+        return gtidMap.get(GTID_SET_STRING);
+    }
+
+    public String getCurrentGtid() {
+        return gtidMap.get(CURRENT_GTID_STRING);
+    }
+
+    public String getCurrentGtidSn() {
+        return gtidMap.get(CURRENT_GTID_SN);
+    }
+
+    public String getCurrentGtidLastCommit() {
+        return gtidMap.get(CURRENT_GTID_LAST_COMMIT);
+    }
+
+    public void putGtid(GTIDSet gtidSet, GtidLogEvent event) {
+        if (gtidSet != null) {
+            gtidMap.put(GTID_SET_STRING, gtidSet.toString());
+            if (event != null) {
+                gtidMap.put(CURRENT_GTID_STRING, event.getGtidStr());
+                gtidMap.put(CURRENT_GTID_SN, String.valueOf(event.getSequenceNumber()));
+                gtidMap.put(CURRENT_GTID_LAST_COMMIT, String.valueOf(event.getLastCommitted()));
+            }
+        }
+    }
 }
@@ -367,7 +367,7 @@ public final class RowsLogBuffer {
                    // 转化为unsign long
                    switch (len) {
                        case 1:
-                            value = buffer.getInt8();
+                            value = buffer.getUint8();
                            break;
                        case 2:
                            value = buffer.getBeUint16();
@@ -859,7 +859,7 @@ public final class RowsLogBuffer {
                    // 转化为unsign long
                    switch (len) {
                        case 1:
-                            value = buffer.getInt8();
+                            value = buffer.getUint8();
                            break;
                        case 2:
                            value = buffer.getUint16();
@@ -1082,10 +1082,10 @@ public final class RowsLogBuffer {
                buffer.fillBytes(binary, 0, len);

                /* Warning unsupport cloumn type */
-                logger.warn(String.format("!! Unsupport column type MYSQL_TYPE_GEOMETRY: meta=%d (%04X), len = %d",
-                    meta,
-                    meta,
-                    len));
+                // logger.warn(String.format("!! Unsupport column type MYSQL_TYPE_GEOMETRY: meta=%d (%04X), len = %d",
+                // meta,
+                // meta,
+                // len));
                javaType = Types.BINARY;
                value = binary;
                length = len;
@@ -15,9 +15,7 @@ public class FetcherPerformanceTest {
        DirectLogFetcher fetcher = new DirectLogFetcher();
        try {
            Class.forName("com.mysql.jdbc.Driver");
-            Connection connection = DriverManager.getConnection("jdbc:mysql://127.0.0.1:3306",
-                "root",
-                "hello");
+            Connection connection = DriverManager.getConnection("jdbc:mysql://127.0.0.1:3306", "root", "hello");
            Statement statement = connection.createStatement();
            statement.execute("SET @master_binlog_checksum='@@global.binlog_checksum'");
            statement.execute("SET @mariadb_slave_capability='" + LogEvent.MARIA_SLAVE_CAPABILITY_MINE + "'");
@@ -8,6 +8,15 @@ import java.sql.Statement;
 import org.junit.Assert;
 import org.junit.Test;

+import com.taobao.tddl.dbsync.binlog.event.DeleteRowsLogEvent;
+import com.taobao.tddl.dbsync.binlog.event.QueryLogEvent;
+import com.taobao.tddl.dbsync.binlog.event.RotateLogEvent;
+import com.taobao.tddl.dbsync.binlog.event.RowsQueryLogEvent;
+import com.taobao.tddl.dbsync.binlog.event.UpdateRowsLogEvent;
+import com.taobao.tddl.dbsync.binlog.event.WriteRowsLogEvent;
+import com.taobao.tddl.dbsync.binlog.event.XidLogEvent;
+import com.taobao.tddl.dbsync.binlog.event.mariadb.AnnotateRowsEvent;
+
 public class DirectLogFetcherTest extends BaseLogFetcherTest {

    @Test
@@ -15,54 +24,49 @@ public class DirectLogFetcherTest extends BaseLogFetcherTest {
        DirectLogFetcher fecther = new DirectLogFetcher();
        try {
            Class.forName("com.mysql.jdbc.Driver");
-            Connection connection = DriverManager.getConnection("jdbc:mysql://100.81.154.142:3306", "root", "hello");
+            Connection connection = DriverManager.getConnection("jdbc:mysql://127.0.0.1:3306", "root", "hello");
            Statement statement = connection.createStatement();
            statement.execute("SET @master_binlog_checksum='@@global.binlog_checksum'");
            statement.execute("SET @mariadb_slave_capability='" + LogEvent.MARIA_SLAVE_CAPABILITY_MINE + "'");

-            fecther.open(connection, "mysql-bin.000006", 120L, 2);
+            fecther.open(connection, "mysql-bin.000007", 89797036L, 2);

-            LogDecoder decoder = new LogDecoder(LogEvent.UNKNOWN_EVENT, LogEvent.UNKNOWN_EVENT);
+            LogDecoder decoder = new LogDecoder(LogEvent.UNKNOWN_EVENT, LogEvent.ENUM_END_EVENT);
            LogContext context = new LogContext();
            while (fecther.fetch()) {
-                decoder.decode(fecther, context);
-                continue;
-                // if (event == null) {
-                // continue;
-                // }
-                //
-                // int eventType = event.getHeader().getType();
-                // switch (eventType) {
-                // case LogEvent.ROTATE_EVENT:
-                // binlogFileName = ((RotateLogEvent) event).getFilename();
-                // break;
-                // case LogEvent.WRITE_ROWS_EVENT_V1:
-                // case LogEvent.WRITE_ROWS_EVENT:
-                // parseRowsEvent((WriteRowsLogEvent) event);
-                // break;
-                // case LogEvent.UPDATE_ROWS_EVENT_V1:
-                // case LogEvent.UPDATE_ROWS_EVENT:
-                // parseRowsEvent((UpdateRowsLogEvent) event);
-                // break;
-                // case LogEvent.DELETE_ROWS_EVENT_V1:
-                // case LogEvent.DELETE_ROWS_EVENT:
-                // parseRowsEvent((DeleteRowsLogEvent) event);
-                // break;
-                // case LogEvent.QUERY_EVENT:
-                // parseQueryEvent((QueryLogEvent) event);
-                // break;
-                // case LogEvent.ROWS_QUERY_LOG_EVENT:
-                // parseRowsQueryEvent((RowsQueryLogEvent) event);
-                // break;
-                // case LogEvent.ANNOTATE_ROWS_EVENT:
-                // parseAnnotateRowsEvent((AnnotateRowsEvent) event);
-                // break;
-                // case LogEvent.XID_EVENT:
-                // parseXidEvent((XidLogEvent) event);
-                // break;
-                // default:
-                // break;
-                // }
+                LogEvent event = decoder.decode(fecther, context);
+                int eventType = event.getHeader().getType();
+                switch (eventType) {
+                    case LogEvent.ROTATE_EVENT:
+                        binlogFileName = ((RotateLogEvent) event).getFilename();
+                        break;
+                    case LogEvent.WRITE_ROWS_EVENT_V1:
+                    case LogEvent.WRITE_ROWS_EVENT:
+                        parseRowsEvent((WriteRowsLogEvent) event);
+                        break;
+                    case LogEvent.UPDATE_ROWS_EVENT_V1:
+                    case LogEvent.UPDATE_ROWS_EVENT:
+                        parseRowsEvent((UpdateRowsLogEvent) event);
+                        break;
+                    case LogEvent.DELETE_ROWS_EVENT_V1:
+                    case LogEvent.DELETE_ROWS_EVENT:
+                        parseRowsEvent((DeleteRowsLogEvent) event);
+                        break;
+                    case LogEvent.QUERY_EVENT:
+                        parseQueryEvent((QueryLogEvent) event);
+                        break;
+                    case LogEvent.ROWS_QUERY_LOG_EVENT:
+                        parseRowsQueryEvent((RowsQueryLogEvent) event);
+                        break;
+                    case LogEvent.ANNOTATE_ROWS_EVENT:
+                        parseAnnotateRowsEvent((AnnotateRowsEvent) event);
+                        break;
+                    case LogEvent.XID_EVENT:
+                        parseXidEvent((XidLogEvent) event);
+                        break;
+                    default:
+                        break;
+                }
            }
        } catch (Exception e) {
            e.printStackTrace();
@@ -3,7 +3,7 @@
 	<parent>
 		<groupId>com.alibaba.otter</groupId>
 		<artifactId>canal</artifactId>
-		<version>1.0.26-SNAPSHOT</version>
+		<version>1.1.1</version>
 		<relativePath>../pom.xml</relativePath>
 	</parent>
 	<groupId>com.alibaba.otter</groupId>
@@ -18,12 +18,12 @@
 		</dependency>

 		<!-- 这里指定runtime的metrics provider-->
-		<!--<dependency>-->
-			<!--<groupId>com.alibaba.otter</groupId>-->
-			<!--<artifactId>canal.prometheus</artifactId>-->
-			<!--<version>${project.version}</version>-->
-			<!--<scope>runtime</scope>-->
-		<!--</dependency>-->
+		<dependency>
+			<groupId>com.alibaba.otter</groupId>
+			<artifactId>canal.prometheus</artifactId>
+			<version>${project.version}</version>
+			<scope>runtime</scope>
+		</dependency>
 	</dependencies>
 	
 	<build>
@@ -40,6 +40,7 @@
 						<exclude>**/canal.properties</exclude>
 						<exclude>**/spring/**</exclude>
 						<exclude>**/example/**</exclude>
+						<exclude>**/mq.yml</exclude>
 					</excludes>
 				</configuration>
 			</plugin>
@@ -1,15 +0,0 @@
-#!/bin/bash
-# Additional line arg for current prometheus solution
-case "`uname`" in
-Linux)
-    bin_abs_path=$(readlink -f $(dirname $0))
-	;;
-*)
-	bin_abs_path=`cd $(dirname $0); pwd`
-	;;
-esac
-base=${bin_abs_path}/..
-if [ $(ls $base/lib/aspectjweaver*.jar | wc -l) -eq 1 ]; then
-    WEAVER=$(ls $base/lib/aspectjweaver*.jar)
-    METRICS_OPTS=" -javaagent:"${WEAVER}" "
-fi
@@ -94,12 +94,7 @@ then
 	echo LOG CONFIGURATION : $logback_configurationFile
 	echo canal conf : $canal_conf 
 	echo CLASSPATH :$CLASSPATH
-#   metrics support options
-#	if [ -x $base/bin/metrics_env.sh ]; then
-#	    . $base/bin/metrics_env.sh
-#	    echo METRICS_OPTS $METRICS_OPTS
-#	fi
-	$JAVA $JAVA_OPTS $METRICS_OPTS $JAVA_DEBUG_OPT $CANAL_OPTS -classpath .:$CLASSPATH com.alibaba.otter.canal.deployer.CanalLauncher 1>>$base/logs/canal/canal.log 2>&1 &
+	$JAVA $JAVA_OPTS $JAVA_DEBUG_OPT $CANAL_OPTS -classpath .:$CLASSPATH com.alibaba.otter.canal.deployer.CanalLauncher 1>>$base/logs/canal/canal.log 2>&1 &
 	echo $! > $base/bin/canal.pid 
 	
 	echo "cd to $current_path for continue"
@@ -15,6 +15,7 @@ public class CanalConstants {
    public static final String CANAL_ID                          = ROOT + "." + "id";
    public static final String CANAL_IP                          = ROOT + "." + "ip";
    public static final String CANAL_PORT                        = ROOT + "." + "port";
+    public static final String CANAL_METRICS_PULL_PORT           = ROOT + "." + "metrics.pull.port";
    public static final String CANAL_ZKSERVERS                   = ROOT + "." + "zkServers";
    public static final String CANAL_WITHOUT_NETTY               = ROOT + "." + "withoutNetty";

@@ -22,6 +23,7 @@ public class CanalConstants {
    public static final String CANAL_AUTO_SCAN                   = ROOT + "." + "auto.scan";
    public static final String CANAL_AUTO_SCAN_INTERVAL          = ROOT + "." + "auto.scan.interval";
    public static final String CANAL_CONF_DIR                    = ROOT + "." + "conf.dir";
+    public static final String CANAL_SERVER_MODE                 = ROOT + "." + "serverMode";

    public static final String CANAL_DESTINATION_SPLIT           = ",";
    public static final String GLOBAL_NAME                       = "global";
@@ -98,6 +98,14 @@ public class CanalController {
        port = Integer.valueOf(getProperty(properties, CanalConstants.CANAL_PORT));
        embededCanalServer = CanalServerWithEmbedded.instance();
        embededCanalServer.setCanalInstanceGenerator(instanceGenerator);// 设置自定义的instanceGenerator
+        try {
+            int metricsPort = Integer.valueOf(getProperty(properties, CanalConstants.CANAL_METRICS_PULL_PORT));
+            embededCanalServer.setMetricsPort(metricsPort);
+        } catch (NumberFormatException e) {
+            logger.info("No valid metrics server port found, use default 11112.");
+            embededCanalServer.setMetricsPort(11112);
+        }
+        
        String canalWithoutNetty = getProperty(properties, CanalConstants.CANAL_WITHOUT_NETTY);
        if (canalWithoutNetty == null || "false".equals(canalWithoutNetty)) {
            canalServer = CanalServerWithNetty.instance();
@@ -379,7 +387,7 @@ public class CanalController {
        return config;
    }

-    private String getProperty(Properties properties, String key) {
+    public static String getProperty(Properties properties, String key) {
        key = StringUtils.trim(key);
        String value = System.getProperty(key);

@@ -7,9 +7,14 @@ import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;

+import com.alibaba.otter.canal.kafka.CanalKafkaProducer;
+import com.alibaba.otter.canal.rocketmq.CanalRocketMQProducer;
+import com.alibaba.otter.canal.server.CanalMQStarter;
+import com.alibaba.otter.canal.spi.CanalMQProducer;
+
 /**
 * canal独立版本启动的入口类
- * 
+ *
 * @author jianghang 2012-11-6 下午05:20:49
 * @version 1.0.0
 */
@@ -33,6 +38,19 @@ public class CanalLauncher {
                properties.load(new FileInputStream(conf));
            }

+            CanalMQProducer canalMQProducer = null;
+            String serverMode = CanalController.getProperty(properties, CanalConstants.CANAL_SERVER_MODE);
+            if (serverMode.equalsIgnoreCase("kafka")) {
+                canalMQProducer = new CanalKafkaProducer();
+            } else if (serverMode.equalsIgnoreCase("rocketmq")) {
+                canalMQProducer = new CanalRocketMQProducer();
+            }
+
+            if (canalMQProducer != null) {
+                // disable netty
+                System.setProperty(CanalConstants.CANAL_WITHOUT_NETTY, "true");
+            }
+
            logger.info("## start the canal server.");
            final CanalController controller = new CanalController(properties);
            controller.start();
@@ -51,6 +69,13 @@ public class CanalLauncher {
                }

            });
+
+            if (canalMQProducer != null) {
+                CanalMQStarter canalServerStarter = new CanalMQStarter(canalMQProducer);
+                if (canalServerStarter != null) {
+                    canalServerStarter.init();
+                }
+            }
        } catch (Throwable e) {
            logger.error("## Something goes wrong when starting up the canal Server:", e);
            System.exit(0);
@@ -4,10 +4,13 @@
 canal.id= 1
 canal.ip=
 canal.port=11111
+canal.metrics.pull.port=11112
 canal.zkServers=
 # flush data to zk
 canal.zookeeper.flush.period = 1000
 canal.withoutNetty = false
+# tcp, kafka, RocketMQ
+canal.serverMode = tcp
 # flush meta cursor/parse position to file
 canal.file.data.dir = ${canal.conf.dir}
 canal.file.flush.period = 1000
@@ -17,6 +20,7 @@ canal.instance.memory.buffer.size = 16384
 canal.instance.memory.buffer.memunit = 1024 
 ## meory store gets mode used MEMSIZE or ITEMSIZE
 canal.instance.memory.batch.mode = MEMSIZE
+canal.instance.memory.rawEntry = true

 ## detecing config
 canal.instance.detecting.enable = false
@@ -65,6 +69,14 @@ canal.instance.tsdb.dir=${canal.file.data.dir:../conf}/${canal.instance.destinat
 canal.instance.tsdb.url=jdbc:h2:${canal.instance.tsdb.dir}/h2;CACHE_SIZE=1000;MODE=MYSQL;
 canal.instance.tsdb.dbUsername=canal
 canal.instance.tsdb.dbPassword=canal
+# dump snapshot interval, default 24 hour
+canal.instance.tsdb.snapshot.interval=24
+# purge snapshot expire , default 360 hour(15 days)
+canal.instance.tsdb.snapshot.expire=360
+
+# rds oss binlog account
+canal.instance.rds.accesskey =
+canal.instance.rds.secretkey =

 #################################################
 ######### 		destinations		############# 
@@ -82,7 +94,6 @@ canal.instance.tsdb.spring.xml=classpath:spring/tsdb/h2-tsdb.xml
 canal.instance.global.mode = spring 
 canal.instance.global.lazy = false
 #canal.instance.global.manager.address = 127.0.0.1:1099
-#canal.instance.global.spring.xml = classpath:spring/local-instance.xml
 #canal.instance.global.spring.xml = classpath:spring/memory-instance.xml
 canal.instance.global.spring.xml = classpath:spring/file-instance.xml
 #canal.instance.global.spring.xml = classpath:spring/default-instance.xml
@@ -12,6 +12,11 @@ canal.instance.master.position=
 canal.instance.master.timestamp=
 canal.instance.master.gtid=

+# rds oss binlog
+canal.instance.rds.accesskey=
+canal.instance.rds.secretkey=
+canal.instance.rds.instanceId=
+
 # table meta tsdb info
 canal.instance.tsdb.enable=true
 #canal.instance.tsdb.url=jdbc:mysql://127.0.0.1:3306/canal_tsdb
@@ -20,14 +25,18 @@ canal.instance.tsdb.enable=true

 #canal.instance.standby.address =
 #canal.instance.standby.journal.name =
-#canal.instance.standby.position = 
+#canal.instance.standby.position =
 #canal.instance.standby.timestamp =
 #canal.instance.standby.gtid=

 # username/password
 canal.instance.dbUsername=canal
-canal.instance.dbPassword=canal
-canal.instance.connectionCharset=UTF-8
+canal.instance.dbPassword=cZozNf1mzW6EQLGO2q9u99619xbZLO0fbua3EX08r4BWNXb8lAt1aHrTEOBttd6UY8Vnuc0easlVXZDdLtt8BQ==
+canal.instance.pwdPublicKey=MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBALK4BUxdDltRRE5/zXpVEVPUgunvscYFtEip3pmLlhrWpacX7y7GCMo2/JM6LeHmiiNdH1FWgGCpUfircSwlWKUCAwEAAQ==
+canal.instance.connectionCharset = UTF-8
+canal.instance.defaultDatabaseName =test
+# enable druid Decrypt database password
+canal.instance.enableDruid=true

 # table regex
 canal.instance.filter.regex=.*\\..*
@@ -1,32 +0,0 @@
-#################################################
-# rds openapi binlog
-canal.instance.rds.open.url=https://rds.aliyuncs.com/
-canal.instance.rds.open.accesskey=
-canal.instance.rds.open.secretkey=
-canal.instance.rds.instanceId=
-canal.instance.rds.startTime=
-canal.instance.rds.endTime=
-
-# local binlog dir
-canal.instance.parser.directory=${canal.file.data.dir:../conf}/${canal.instance.destination:}/binlog
-# position info
-canal.instance.master.address=127.0.0.1:3306
-
-# table meta tsdb info
-canal.instance.tsdb.enable=true
-canal.instance.tsdb.dir=${canal.file.data.dir:../conf}/${canal.instance.destination:}
-canal.instance.tsdb.url=jdbc:h2:${canal.instance.tsdb.dir}/h2;CACHE_SIZE=1000;MODE=MYSQL;
-#canal.instance.tsdb.url=jdbc:mysql://127.0.0.1:3306/canal_tsdb
-#canal.instance.tsdb.dbUsername=canal
-#canal.instance.tsdb.dbPassword=canal
-
-# username/password
-canal.instance.dbUsername=canal
-canal.instance.dbPassword=canal
-canal.instance.defaultDatabaseName=test
-canal.instance.connectionCharset=UTF-8
-# table regex
-canal.instance.filter.regex=test\\..*
-# table black regex
-canal.instance.filter.black.regex=
-#################################################
@@ -6,7 +6,7 @@
 			</pattern>
 		</encoder>
 	</appender>
-	
+
 	<appender name="CANAL-ROOT" class="ch.qos.logback.classic.sift.SiftingAppender">
 		<discriminator>
 			<Key>destination</Key>
@@ -16,7 +16,7 @@
 			<appender name="FILE-${destination}" class="ch.qos.logback.core.rolling.RollingFileAppender">
 				<File>../logs/${destination}/${destination}.log</File>
 				<rollingPolicy
-					class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
+						class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
 					<!-- rollover daily -->
 					<fileNamePattern>../logs/${destination}/%d{yyyy-MM-dd}/${destination}-%d{yyyy-MM-dd}-%i.log.gz</fileNamePattern>
 					<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
@@ -33,7 +33,7 @@
 			</appender>
 		</sift>
 	</appender>
-	
+
 	<appender name="CANAL-META" class="ch.qos.logback.classic.sift.SiftingAppender">
 		<discriminator>
 			<Key>destination</Key>
@@ -43,7 +43,7 @@
 			<appender name="META-FILE-${destination}" class="ch.qos.logback.core.rolling.RollingFileAppender">
 				<File>../logs/${destination}/meta.log</File>
 				<rollingPolicy
-					class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
+						class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
 					<!-- rollover daily -->
 					<fileNamePattern>../logs/${destination}/%d{yyyy-MM-dd}/meta-%d{yyyy-MM-dd}-%i.log.gz</fileNamePattern>
 					<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
@@ -60,24 +60,24 @@
 			</appender>
 		</sift>
 	</appender>
-	
-    <logger name="com.alibaba.otter.canal.instance" additivity="false">  
-        <level value="INFO" />  
-        <appender-ref ref="CANAL-ROOT" />
-    </logger>
-    <logger name="com.alibaba.otter.canal.deployer" additivity="false">  
-        <level value="INFO" />  
-        <appender-ref ref="CANAL-ROOT" />
-    </logger>
-    <logger name="com.alibaba.otter.canal.meta.FileMixedMetaManager" additivity="false">  
-        <level value="INFO" />
-        <appender-ref ref="CANAL-META" />
-    </logger>
+
+	<logger name="com.alibaba.otter.canal.instance" additivity="false">
+		<level value="INFO" />
+		<appender-ref ref="CANAL-ROOT" />
+	</logger>
+	<logger name="com.alibaba.otter.canal.deployer" additivity="false">
+		<level value="INFO" />
+		<appender-ref ref="CANAL-ROOT" />
+	</logger>
+	<logger name="com.alibaba.otter.canal.meta.FileMixedMetaManager" additivity="false">
+		<level value="INFO" />
+		<appender-ref ref="CANAL-META" />
+	</logger>
 	<logger name="com.alibaba.otter.canal.kafka" additivity="false">
 		<level value="INFO" />
 		<appender-ref ref="CANAL-ROOT" />
 	</logger>
-    
+
 	<root level="WARN">
 		<appender-ref ref="STDOUT"/>
 		<appender-ref ref="CANAL-ROOT" />
@@ -0,0 +1,23 @@
+servers: localhost:9876 #for rocketmq: means the nameserver
+retries: 0
+batchSize: 16384
+lingerMs: 1
+bufferMemory: 33554432
+
+# Canal的batch size, 默认50K, 由于kafka最大消息体限制请勿超过1M(900K以下)
+canalBatchSize: 50
+# Canal get数据的超时时间, 单位: 毫秒, 0为不限超时
+canalGetTimeout: 100
+flatMessage: true
+
+canalDestinations:
+  - canalDestination: example
+    topic: example
+    partition: 1
+#  #对应topic分区数量
+#  partitionsNum: 3
+#  partitionHash:
+#    #库名.表名: 唯一主键
+#    mytest.person: id
+
+
@@ -0,0 +1,39 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<beans xmlns="http://www.springframework.org/schema/beans"
+	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:tx="http://www.springframework.org/schema/tx"
+	xmlns:aop="http://www.springframework.org/schema/aop" xmlns:lang="http://www.springframework.org/schema/lang"
+	xmlns:context="http://www.springframework.org/schema/context"
+	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
+           http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.0.xsd
+           http://www.springframework.org/schema/lang http://www.springframework.org/schema/lang/spring-lang-2.0.xsd
+           http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.0.xsd
+           http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd"
+	default-autowire="byName">
+
+	<!-- properties -->
+	<bean class="com.alibaba.otter.canal.instance.spring.support.PropertyPlaceholderConfigurer" lazy-init="false">
+		<property name="ignoreResourceNotFound" value="true" />
+		<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE"/><!-- 允许system覆盖 -->
+		<property name="locationNames">
+			<list>
+				<value>classpath:canal.properties</value>
+				<value>classpath:${canal.instance.destination:}/instance.properties</value>
+			</list>
+		</property>
+	</bean>
+	
+	<bean id="socketAddressEditor" class="com.alibaba.otter.canal.instance.spring.support.SocketAddressEditor" />
+	<bean class="org.springframework.beans.factory.config.CustomEditorConfigurer"> 
+		<property name="propertyEditorRegistrars">
+			<list>
+				<ref bean="socketAddressEditor" />
+			</list>
+		</property>
+	</bean>
+	
+	<bean id="baseEventParser" class="com.alibaba.otter.canal.parse.inbound.mysql.rds.RdsBinlogEventParserProxy" abstract="true">
+		<property name="accesskey" value="${canal.instance.rds.accesskey:}" />
+		<property name="secretkey" value="${canal.instance.rds.secretkey:}" />
+		<property name="instanceId" value="${canal.instance.rds.instanceId:}" />
+	</bean>
+</beans>
@@ -10,26 +10,7 @@
           http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd"
 	default-autowire="byName">

-	<!-- properties -->
-	<bean class="com.alibaba.otter.canal.instance.spring.support.PropertyPlaceholderConfigurer" lazy-init="false">
-		<property name="ignoreResourceNotFound" value="true" />
-		<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE"/><!-- 允许system覆盖 -->
-		<property name="locationNames">
-			<list>
-				<value>classpath:canal.properties</value>
-				<value>classpath:${canal.instance.destination:}/instance.properties</value>
-			</list>
-		</property>
-	</bean>
-	
-	<bean id="socketAddressEditor" class="com.alibaba.otter.canal.instance.spring.support.SocketAddressEditor" />
-	<bean class="org.springframework.beans.factory.config.CustomEditorConfigurer"> 
-		<property name="propertyEditorRegistrars">
-			<list>
-				<ref bean="socketAddressEditor" />
-			</list>
-		</property>
-	</bean>
+	<import resource="classpath:spring/base-instance.xml" />
 	
 	<bean id="instance" class="com.alibaba.otter.canal.instance.spring.CanalInstanceWithSpring">
 		<property name="destination" value="${canal.instance.destination}" />
@@ -77,6 +58,7 @@
 		<property name="bufferMemUnit" value="${canal.instance.memory.buffer.memunit:1024}" />
 		<property name="batchMode" value="${canal.instance.memory.batch.mode:MEMSIZE}" />
 		<property name="ddlIsolation" value="${canal.instance.get.ddl.isolation:false}" />
+		<property name="raw" value="${canal.instance.memory.rawEntry:true}" />
 	</bean>
 	
 	<bean id="eventSink" class="com.alibaba.otter.canal.sink.entry.EntryEventSink">
@@ -84,7 +66,7 @@
 		<property name="filterTransactionEntry" value="${canal.instance.filter.transaction.entry:false}"/>
 	</bean>

-	<bean id="eventParser" class="com.alibaba.otter.canal.parse.inbound.mysql.MysqlEventParser">
+	<bean id="eventParser" parent="baseEventParser" >
 		<property name="destination" value="${canal.instance.destination}" />
 		<property name="slaveId" value="${canal.instance.mysql.slaveId:0}" />
 		<!-- 心跳配置 -->
@@ -145,18 +127,22 @@
 		
 		<!-- 解析数据库信息 -->
 		<property name="masterInfo">
-			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo">
+			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo" init-method="initPwd">
 				<property name="address" value="${canal.instance.master.address}" />
 				<property name="username" value="${canal.instance.dbUsername:retl}" />
 				<property name="password" value="${canal.instance.dbPassword:retl}" />
+				<property name="pwdPublicKey" value="${canal.instance.pwdPublicKey:retl}" />
+				<property name="enableDruid" value="${canal.instance.enableDruid:false}" />
 				<property name="defaultDatabaseName" value="${canal.instance.defaultDatabaseName:test}" />
 			</bean>
 		</property>
 		<property name="standbyInfo">
-			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo">
+			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo" init-method="initPwd">
 				<property name="address" value="${canal.instance.standby.address}" />
 				<property name="username" value="${canal.instance.dbUsername:retl}" />
 				<property name="password" value="${canal.instance.dbPassword:retl}" />
+				<property name="pwdPublicKey" value="${canal.instance.pwdPublicKey:retl}" />
+				<property name="enableDruid" value="${canal.instance.enableDruid:false}" />
 				<property name="defaultDatabaseName" value="${canal.instance.defaultDatabaseName:test}" />
 			</bean>
 		</property>
@@ -190,6 +176,8 @@
 		<!--表结构相关-->
 		<property name="enableTsdb" value="${canal.instance.tsdb.enable:true}"/>
 		<property name="tsdbSpringXml" value="${canal.instance.tsdb.spring.xml:}"/>
+		<property name="tsdbSnapshotInterval" value="${canal.instance.tsdb.snapshot.interval:24}" />
+		<property name="tsdbSnapshotExpire" value="${canal.instance.tsdb.snapshot.expire:360}" />
 		
 		<!--是否启用GTID模式-->
 		<property name="isGTIDMode" value="${canal.instance.gtidon:false}"/>
@@ -10,26 +10,7 @@
           http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd"
 	default-autowire="byName">

-	<!-- properties -->
-	<bean class="com.alibaba.otter.canal.instance.spring.support.PropertyPlaceholderConfigurer" lazy-init="false">
-		<property name="ignoreResourceNotFound" value="true" />
-		<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE"/><!-- 允许system覆盖 -->
-		<property name="locationNames">
-			<list>
-				<value>classpath:canal.properties</value>
-				<value>classpath:${canal.instance.destination:}/instance.properties</value>
-			</list>
-		</property>
-	</bean>
-	
-	<bean id="socketAddressEditor" class="com.alibaba.otter.canal.instance.spring.support.SocketAddressEditor" />
-	<bean class="org.springframework.beans.factory.config.CustomEditorConfigurer"> 
-		<property name="propertyEditorRegistrars">
-			<list>
-				<ref bean="socketAddressEditor" />
-			</list>
-		</property>
-	</bean>
+	<import resource="classpath:spring/base-instance.xml" />
 	
 	<bean id="instance" class="com.alibaba.otter.canal.instance.spring.CanalInstanceWithSpring">
 		<property name="destination" value="${canal.instance.destination}" />
@@ -63,6 +44,7 @@
 		<property name="bufferMemUnit" value="${canal.instance.memory.buffer.memunit:1024}" />
 		<property name="batchMode" value="${canal.instance.memory.batch.mode:MEMSIZE}" />
 		<property name="ddlIsolation" value="${canal.instance.get.ddl.isolation:false}" />
+		<property name="raw" value="${canal.instance.memory.rawEntry:true}" />
 	</bean>
 	
 	<bean id="eventSink" class="com.alibaba.otter.canal.sink.entry.EntryEventSink">
@@ -70,7 +52,7 @@
 		<property name="filterTransactionEntry" value="${canal.instance.filter.transaction.entry:false}"/>
 	</bean>

-	<bean id="eventParser" class="com.alibaba.otter.canal.parse.inbound.mysql.MysqlEventParser">
+	<bean id="eventParser" parent="baseEventParser">
 		<property name="destination" value="${canal.instance.destination}" />
 		<property name="slaveId" value="${canal.instance.mysql.slaveId:0}" />
 		<!-- 心跳配置 -->
@@ -130,22 +112,26 @@
 		
 		<!-- 解析数据库信息 -->
 		<property name="masterInfo">
-			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo">
+			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo" init-method="initPwd">
 				<property name="address" value="${canal.instance.master.address}" />
 				<property name="username" value="${canal.instance.dbUsername:retl}" />
 				<property name="password" value="${canal.instance.dbPassword:retl}" />
+				<property name="pwdPublicKey" value="${canal.instance.pwdPublicKey:retl}" />
+				<property name="enableDruid" value="${canal.instance.enableDruid:false}" />
 				<property name="defaultDatabaseName" value="${canal.instance.defaultDatabaseName:test}" />
 			</bean>
 		</property>
 		<property name="standbyInfo">
-			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo">
+			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo" init-method="initPwd">
 				<property name="address" value="${canal.instance.standby.address}" />
 				<property name="username" value="${canal.instance.dbUsername:retl}" />
 				<property name="password" value="${canal.instance.dbPassword:retl}" />
+				<property name="pwdPublicKey" value="${canal.instance.pwdPublicKey:retl}" />
+				<property name="enableDruid" value="${canal.instance.enableDruid:false}" />
 				<property name="defaultDatabaseName" value="${canal.instance.defaultDatabaseName:test}" />
 			</bean>
 		</property>
-		
+
 		<!-- 解析起始位点 -->
 		<property name="masterPosition">
 			<bean class="com.alibaba.otter.canal.protocol.position.EntryPosition">
@@ -175,6 +161,8 @@
 		<!--表结构相关-->
 		<property name="enableTsdb" value="${canal.instance.tsdb.enable:true}"/>
 		<property name="tsdbSpringXml" value="${canal.instance.tsdb.spring.xml:}"/>
+		<property name="tsdbSnapshotInterval" value="${canal.instance.tsdb.snapshot.interval:24}" />
+		<property name="tsdbSnapshotExpire" value="${canal.instance.tsdb.snapshot.expire:360}" />

 		<!--是否启用GTID模式-->
 		<property name="isGTIDMode" value="${canal.instance.gtidon:false}"/>
@@ -10,26 +10,7 @@
           http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd"
 	default-autowire="byName">
 	
-	<!-- properties -->
-	<bean class="com.alibaba.otter.canal.instance.spring.support.PropertyPlaceholderConfigurer" lazy-init="false">
-		<property name="ignoreResourceNotFound" value="true" />
-		<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE"/><!-- 允许system覆盖 -->
-		<property name="locationNames">
-			<list>
-				<value>classpath:canal.properties</value>
-				<value>classpath:${canal.instance.destination:}/instance.properties</value>
-			</list>
-		</property>
-	</bean>
-	
-	<bean id="socketAddressEditor" class="com.alibaba.otter.canal.instance.spring.support.SocketAddressEditor" />
-	<bean class="org.springframework.beans.factory.config.CustomEditorConfigurer"> 
-		<property name="propertyEditorRegistrars">
-			<list>
-				<ref bean="socketAddressEditor" />
-			</list>
-		</property>
-	</bean>
+	<import resource="classpath:spring/base-instance.xml" />

 	<bean id="instance" class="com.alibaba.otter.canal.instance.spring.CanalInstanceWithSpring">
 		<property name="destination" value="${canal.instance.destination}" />
@@ -60,6 +41,7 @@
 		<property name="bufferMemUnit" value="${canal.instance.memory.buffer.memunit:1024}" />
 		<property name="batchMode" value="${canal.instance.memory.batch.mode:MEMSIZE}" />
 		<property name="ddlIsolation" value="${canal.instance.get.ddl.isolation:false}" />
+		<property name="raw" value="${canal.instance.memory.rawEntry:true}" />
 	</bean>
 	
 	<bean id="eventSink" class="com.alibaba.otter.canal.sink.entry.EntryEventSink">
@@ -76,7 +58,7 @@
 		</property>
 	</bean>

-	<bean id="eventParser1" class="com.alibaba.otter.canal.parse.inbound.mysql.MysqlEventParser">
+	<bean id="eventParser1" parent="baseEventParser">
 		<property name="destination" value="${canal.instance.destination}" />
 		<property name="slaveId" value="${canal.instance.mysql.slaveId:0}" />
 		<!-- 心跳配置 -->
@@ -127,18 +109,22 @@
 		
 		<!-- 解析数据库信息 -->
 		<property name="masterInfo">
-			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo">
+			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo" init-method="initPwd">
 				<property name="address" value="${canal.instance.master1.address}" />
 				<property name="username" value="${canal.instance.dbUsername:retl}" />
 				<property name="password" value="${canal.instance.dbPassword:retl}" />
+				<property name="pwdPublicKey" value="${canal.instance.pwdPublicKey:retl}" />
+				<property name="enableDruid" value="${canal.instance.enableDruid:false}" />
 				<property name="defaultDatabaseName" value="${canal.instance.defaultDatabaseName:test}" />
 			</bean>
 		</property>
 		<property name="standbyInfo">
-			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo">
+			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo" init-method="initPwd">
 				<property name="address" value="${canal.instance.standby1.address}" />
 				<property name="username" value="${canal.instance.dbUsername:retl}" />
 				<property name="password" value="${canal.instance.dbPassword:retl}" />
+				<property name="pwdPublicKey" value="${canal.instance.pwdPublicKey:retl}" />
+				<property name="enableDruid" value="${canal.instance.enableDruid:false}" />
 				<property name="defaultDatabaseName" value="${canal.instance.defaultDatabaseName:test}" />
 			</bean>
 		</property>
@@ -174,7 +160,7 @@
 		<property name="parallelBufferSize" value="${canal.instance.parser.parallelBufferSize:256}" />
 	</bean>
 	
-	<bean id="eventParser2" class="com.alibaba.otter.canal.parse.inbound.mysql.MysqlEventParser">
+	<bean id="eventParser2" parent="baseEventParser">
 		<property name="destination" value="${canal.instance.destination}" />
 		<property name="slaveId" value="${canal.instance.mysql.slaveId:0}" />
 		<!-- 心跳配置 -->
@@ -225,18 +211,22 @@
 		
 		<!-- 解析数据库信息 -->
 		<property name="masterInfo">
-			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo">
+			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo" init-method="initPwd">
 				<property name="address" value="${canal.instance.master2.address}" />
 				<property name="username" value="${canal.instance.dbUsername:retl}" />
 				<property name="password" value="${canal.instance.dbPassword:retl}" />
+				<property name="pwdPublicKey" value="${canal.instance.pwdPublicKey:retl}" />
+				<property name="enableDruid" value="${canal.instance.enableDruid:false}" />
 				<property name="defaultDatabaseName" value="${canal.instance.defaultDatabaseName:test}" />
 			</bean>
 		</property>
 		<property name="standbyInfo">
-			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo">
+			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo" init-method="initPwd">
 				<property name="address" value="${canal.instance.standby2.address}" />
 				<property name="username" value="${canal.instance.dbUsername:retl}" />
 				<property name="password" value="${canal.instance.dbPassword:retl}" />
+				<property name="pwdPublicKey" value="${canal.instance.pwdPublicKey:retl}" />
+				<property name="enableDruid" value="${canal.instance.enableDruid:false}" />
 				<property name="defaultDatabaseName" value="${canal.instance.defaultDatabaseName:test}" />
 			</bean>
 		</property>
@@ -1,147 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<beans xmlns="http://www.springframework.org/schema/beans"
-	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:tx="http://www.springframework.org/schema/tx"
-	xmlns:aop="http://www.springframework.org/schema/aop" xmlns:lang="http://www.springframework.org/schema/lang"
-	xmlns:context="http://www.springframework.org/schema/context"
-	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
-           http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.0.xsd
-           http://www.springframework.org/schema/lang http://www.springframework.org/schema/lang/spring-lang-2.0.xsd
-           http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.0.xsd
-           http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd"
-	default-autowire="byName">
-
-	<!-- properties -->
-	<bean class="com.alibaba.otter.canal.instance.spring.support.PropertyPlaceholderConfigurer" lazy-init="false">
-		<property name="ignoreResourceNotFound" value="true" />
-		<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE"/><!-- 允许system覆盖 -->
-		<property name="locationNames">
-			<list>
-				<value>classpath:canal.properties</value>
-				<value>classpath:${canal.instance.destination:}/instance.properties</value>
-			</list>
-		</property>
-	</bean>
-	
-	<bean id="socketAddressEditor" class="com.alibaba.otter.canal.instance.spring.support.SocketAddressEditor" />
-	<bean class="org.springframework.beans.factory.config.CustomEditorConfigurer"> 
-		<property name="propertyEditorRegistrars">
-			<list>
-				<ref bean="socketAddressEditor" />
-			</list>
-		</property>
-	</bean>
-	
-	<bean id="instance" class="com.alibaba.otter.canal.instance.spring.CanalInstanceWithSpring">
-		<property name="destination" value="${canal.instance.destination}" />
-		<property name="eventParser">
-			<ref local="eventParser" />
-		</property>
-		<property name="eventSink">
-			<ref local="eventSink" />
-		</property>
-		<property name="eventStore">
-			<ref local="eventStore" />
-		</property>
-		<property name="metaManager">
-			<ref local="metaManager" />
-		</property>
-		<property name="alarmHandler">
-			<ref local="alarmHandler" />
-		</property>
-	</bean>
-	
-	<!-- 报警处理类 -->
-	<bean id="alarmHandler" class="com.alibaba.otter.canal.common.alarm.LogAlarmHandler" />
-	
-	<bean id="metaManager" class="com.alibaba.otter.canal.meta.FileMixedMetaManager">
-		<property name="dataDir" value="${canal.file.data.dir:../conf}" />
-		<property name="period" value="${canal.file.flush.period:1000}" />
-	</bean>
-	
-	<bean id="eventStore" class="com.alibaba.otter.canal.store.memory.MemoryEventStoreWithBuffer">
-		<property name="bufferSize" value="${canal.instance.memory.buffer.size:16384}" />
-		<property name="bufferMemUnit" value="${canal.instance.memory.buffer.memunit:1024}" />
-		<property name="batchMode" value="${canal.instance.memory.batch.mode:MEMSIZE}" />
-		<property name="ddlIsolation" value="${canal.instance.get.ddl.isolation:false}" />
-	</bean>
-	
-	<bean id="eventSink" class="com.alibaba.otter.canal.sink.entry.EntryEventSink">
-		<property name="eventStore" ref="eventStore" />
-		<property name="filterTransactionEntry" value="${canal.instance.filter.transaction.entry:false}"/>
-	</bean>
-
-	<bean id="eventParser" class="com.alibaba.otter.canal.parse.inbound.mysql.rds.RdsLocalBinlogEventParser">
-		<property name="destination" value="${canal.instance.destination}" />			
-		<property name="alarmHandler" ref="alarmHandler" />
-
-		<!-- 解析过滤处理 -->
-		<property name="eventFilter">
-			<bean class="com.alibaba.otter.canal.filter.aviater.AviaterRegexFilter" >
-				<constructor-arg index="0" value="${canal.instance.filter.regex:.*\..*}" />
-			</bean>
-		</property>
-		
-		<property name="eventBlackFilter">
-			<bean class="com.alibaba.otter.canal.filter.aviater.AviaterRegexFilter" >
-				<constructor-arg index="0" value="${canal.instance.filter.black.regex:}" />
-				<constructor-arg index="1" value="false" />
-			</bean>
-		</property>
-		
-		<!-- 最大事务解析大小，超过该大小后事务将被切分为多个事务投递 -->
-		<property name="transactionSize" value="${canal.instance.transaction.size:1024}" />
-			
-		<!-- 解析编码 -->
-		<property name="connectionCharset" value="${canal.instance.connectionCharset:UTF-8}" />
-	
-		<!-- 解析位点记录 -->
-		<property name="logPositionManager">
-			<bean class="com.alibaba.otter.canal.parse.index.FailbackLogPositionManager">
-				<constructor-arg>
-					<bean class="com.alibaba.otter.canal.parse.index.MemoryLogPositionManager" />
-				</constructor-arg>
-				<constructor-arg>
-					<bean class="com.alibaba.otter.canal.parse.index.MetaLogPositionManager">
-						<constructor-arg ref="metaManager"/>
-					</bean>
-				</constructor-arg>
-			</bean>
-		</property>
-		
-		<!-- 解析数据库信息 -->
-		<property name="masterInfo">
-			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo">
-				<property name="address" value="${canal.instance.master.address}" />
-				<property name="username" value="${canal.instance.dbUsername:retl}" />
-				<property name="password" value="${canal.instance.dbPassword:retl}" />
-				<property name="defaultDatabaseName" value="${canal.instance.defaultDatabaseName:test}" />
-			</bean>
-		</property>
-		
-		<property name="filterQueryDml" value="${canal.instance.filter.query.dml:false}" />
-		<property name="filterQueryDcl" value="${canal.instance.filter.query.dcl:false}" />
-		<property name="filterQueryDdl" value="${canal.instance.filter.query.ddl:false}" />
-		<property name="useDruidDdlFilter" value="${canal.instance.filter.druid.ddl:true}" />
-		<property name="filterRows" value="${canal.instance.filter.rows:false}" />
-		<property name="filterTableError" value="${canal.instance.filter.table.error:false}" />
-		<property name="needWait" value="${canal.instance.parser.needWait:false}"/>
-		<property name="directory" value="${canal.instance.parser.directory:}"/>
-		
-		<!-- rds相关 -->
-		<property name="url" value="${canal.instance.rds.open.url:}"/>
-		<property name="accesskey" value="${canal.instance.rds.open.accesskey:}"/>
-		<property name="secretkey" value="${canal.instance.rds.open.secretkey:}"/>
-		<property name="instanceId" value="${canal.instance.rds.instanceId:}"/>
-		<property name="startTime" value="${canal.instance.rds.startTime:}"/>
-		<property name="endTime" value="${canal.instance.rds.endTime:}"/>
-		
-		<!--表结构相关-->
-		<property name="enableTsdb" value="${canal.instance.tsdb.enable:true}"/>
-		<property name="tsdbSpringXml" value="${canal.instance.tsdb.spring.xml:}"/>
-		
-		<!-- parallel parser -->
-		<property name="parallel" value="${canal.instance.parser.parallel:true}" />
-		<property name="parallelThreadSize" value="${canal.instance.parser.parallelThreadSize}" />
-		<property name="parallelBufferSize" value="${canal.instance.parser.parallelBufferSize:256}" />
-	</bean>
-</beans>
@@ -10,26 +10,7 @@
           http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-2.5.xsd"
 	default-autowire="byName">
 	
-	<!-- properties -->
-	<bean class="com.alibaba.otter.canal.instance.spring.support.PropertyPlaceholderConfigurer" lazy-init="false">
-		<property name="ignoreResourceNotFound" value="true" />
-		<property name="systemPropertiesModeName" value="SYSTEM_PROPERTIES_MODE_OVERRIDE"/><!-- 允许system覆盖 -->
-		<property name="locationNames">
-			<list>
-				<value>classpath:canal.properties</value>
-				<value>classpath:${canal.instance.destination:}/instance.properties</value>
-			</list>
-		</property>
-	</bean>
-	
-	<bean id="socketAddressEditor" class="com.alibaba.otter.canal.instance.spring.support.SocketAddressEditor" />
-	<bean class="org.springframework.beans.factory.config.CustomEditorConfigurer"> 
-		<property name="propertyEditorRegistrars">
-			<list>
-				<ref bean="socketAddressEditor" />
-			</list>
-		</property>
-	</bean>
+	<import resource="classpath:spring/base-instance.xml" />

 	<bean id="instance" class="com.alibaba.otter.canal.instance.spring.CanalInstanceWithSpring">
 		<property name="destination" value="${canal.instance.destination}" />
@@ -60,6 +41,7 @@
 		<property name="bufferMemUnit" value="${canal.instance.memory.buffer.memunit:1024}" />
 		<property name="batchMode" value="${canal.instance.memory.batch.mode:MEMSIZE}" />
 		<property name="ddlIsolation" value="${canal.instance.get.ddl.isolation:false}" />
+		<property name="raw" value="${canal.instance.memory.rawEntry:true}" />
 	</bean>
 	
 	<bean id="eventSink" class="com.alibaba.otter.canal.sink.entry.EntryEventSink">
@@ -67,7 +49,7 @@
 		<property name="filterTransactionEntry" value="${canal.instance.filter.transaction.entry:false}"/>
 	</bean>

-	<bean id="eventParser" class="com.alibaba.otter.canal.parse.inbound.mysql.MysqlEventParser">
+	<bean id="eventParser" parent="baseEventParser">
 		<property name="destination" value="${canal.instance.destination}" />
 		<property name="slaveId" value="${canal.instance.mysql.slaveId:0}" />
 		<!-- 心跳配置 -->
@@ -115,21 +97,25 @@
 		
 		<!-- failover切换时回退的时间 -->
 		<property name="fallbackIntervalInSeconds" value="${canal.instance.fallbackIntervalInSeconds:60}" />
-		
+
 		<!-- 解析数据库信息 -->
 		<property name="masterInfo">
-			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo">
+			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo" init-method="initPwd">
 				<property name="address" value="${canal.instance.master.address}" />
 				<property name="username" value="${canal.instance.dbUsername:retl}" />
 				<property name="password" value="${canal.instance.dbPassword:retl}" />
+				<property name="pwdPublicKey" value="${canal.instance.pwdPublicKey:retl}" />
+				<property name="enableDruid" value="${canal.instance.enableDruid:false}" />
 				<property name="defaultDatabaseName" value="${canal.instance.defaultDatabaseName:test}" />
 			</bean>
 		</property>
 		<property name="standbyInfo">
-			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo">
+			<bean class="com.alibaba.otter.canal.parse.support.AuthenticationInfo" init-method="initPwd">
 				<property name="address" value="${canal.instance.standby.address}" />
 				<property name="username" value="${canal.instance.dbUsername:retl}" />
 				<property name="password" value="${canal.instance.dbPassword:retl}" />
+				<property name="pwdPublicKey" value="${canal.instance.pwdPublicKey:retl}" />
+				<property name="enableDruid" value="${canal.instance.enableDruid:false}" />
 				<property name="defaultDatabaseName" value="${canal.instance.defaultDatabaseName:test}" />
 			</bean>
 		</property>
@@ -163,6 +149,8 @@
 		<!--表结构相关-->
 		<property name="enableTsdb" value="${canal.instance.tsdb.enable:false}"/>
 		<property name="tsdbSpringXml" value="${canal.instance.tsdb.spring.xml:}"/>
+		<property name="tsdbSnapshotInterval" value="${canal.instance.tsdb.snapshot.interval:24}" />
+		<property name="tsdbSnapshotExpire" value="${canal.instance.tsdb.snapshot.expire:360}" />
 		
 		<!--是否启用GTID模式-->
 		<property name="isGTIDMode" value="${canal.instance.gtidon:false}"/>
@@ -36,10 +36,10 @@
    </delete>


-    <delete id="deleteByGmtModified" parameterClass="java.util.Map">
+    <delete id="deleteByTimestamp" parameterClass="java.util.Map">
        <![CDATA[
 		delete from meta_history
-		where gmt_modified < timestamp(#timestamp#)
+		where destination=#destination# and binlog_timestamp < #timestamp#
        ]]>
    </delete>
 </sqlMap>
@@ -42,10 +42,10 @@
        where destination=#destination#
    </delete>

-    <delete id="deleteByGmtModified" parameterClass="java.util.Map">
+    <delete id="deleteByTimestamp" parameterClass="java.util.Map">
        <![CDATA[
 		delete from meta_snapshot
-		where gmt_modified < timestamp(#timestamp#)
+		where destination=#destination# and binlog_timestamp < #timestamp# and binlog_timestamp > 0
        ]]>
    </delete>
 </sqlMap>
--- a/Show More
+++ b/Show More
				`@@ -0,0 +1 @@`
				`hbase=com.alibaba.otter.canal.client.adapter.hbase.HbaseAdapter`
				`@@ -0,0 +1 @@`
				`logger=com.alibaba.otter.canal.client.adapter.logger.LoggerAdapterExample`