Compare commits

...
Author SHA1 Message Date
Calvin b69612923f [refactor] optimize code 2025-08-23 00:57:19 +08:00
Calvin 22897e4360 [refactor] optimize code 2025-08-23 00:38:52 +08:00
Calvin 8a705c9ced [refactor] update class and method comments 2025-08-20 02:14:50 +08:00
CalvinandCopilot 2276937b72 Update hertzbeat-manager/src/main/java/org/apache/hertzbeat/manager/pojo/JobCache.java
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Calvin <zhengqiwei@apache.org>
2025-08-20 00:11:22 +08:00
tomsun28 553bc1edae Merge branch 'master' into refactor_manager 2025-08-19 23:44:29 +08:00
Calvinandtomsun28 344cb25229 [doc] add japanese i18n in app-redis_cluster.yml (#3672)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-19 23:30:11 +08:00
DeleiGuo 74d21a7acd [feature] add user password monitoring metrics in oracle monitor (#3674) 2025-08-19 22:08:05 +08:00
Calvin 2d95df374a [refactor] optimize code structure about collector and job within Manager 2025-08-18 21:21:14 +08:00
f315bf9607 [fix] Fixed Grafana visualization integration display issue (#3666)
Co-authored-by: Calvin <zhengqiwei@apache.org>
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-17 14:51:14 +08:00
810d54bef5 [improve] Optimize the scheduling logic for batch flush tasks (#3660)
Signed-off-by: Cyanty <153884653+Cyanty@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Sherlock Yin <sherlock.yin1994@gmail.com>
Co-authored-by: Calvin <zhengqiwei@apache.org>
2025-08-17 14:24:48 +08:00
DeleiGuoandCalvin 48eafa1382 [Feature] add Apache DolphinScheduler monitoring support (#3656)
Co-authored-by: Calvin <zhengqiwei@apache.org>
2025-08-17 10:58:46 +08:00
Calvinandtomsun28 4b6c1e0f08 [doc] add japanese i18n in app-redis.yml (#3669)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-17 09:59:11 +08:00
edfd857030 [GSOC] MCP server setup, authorization, and basic tool support (#3610)
Signed-off-by: Sarthak Arora <f20200060@pilani.bits-pilani.ac.in>
Co-authored-by: Calvin <zhengqiwei@apache.org>
Co-authored-by: Jast <shenghang@apache.org>
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-16 19:08:13 +08:00
tomsun28andLogic 408a06d3eb [doc] update contribution doc (#3667)
Co-authored-by: Logic <zqr10159@dromara.org>
2025-08-16 16:07:41 +08:00
Calvinandtomsun28 fd3a2f0569 [doc] add japanese i18n and remove chinese in script folder (#3665)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-16 15:43:24 +08:00
7d8ba72717 [doc] add japanese i18n in app-redhat.yml (#3655)
Co-authored-by: shown <yuluo08290126@gmail.com>
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-16 15:25:39 +08:00
Calvinandtomsun28 f936b50ce9 [doc] add japanese i18n in app-spring_gateway.yml, app-sqlserver.yml (#3661)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-15 20:25:01 +08:00
Calvinandtomsun28 93f5da2d02 [doc] add japanese i18n in app-process.yml (#3654)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-15 20:17:14 +08:00
Calvinandshown bc06c268ca [doc] add japanese i18n in app-prestodb.yml, app-spark.yml (#3651)
Co-authored-by: shown <yuluo08290126@gmail.com>
2025-08-15 20:12:07 +08:00
DeleiGuoandCalvin 9e6f2762c0 [doc] improve documentation formatting and readability (#3659)
Co-authored-by: Calvin <zhengqiwei@apache.org>
2025-08-13 00:40:37 +08:00
Duansg 6ac8a42bd3 [doc] add japanese i18n in app-springboot3.yml (#3658) 2025-08-13 00:37:49 +08:00
Calvinandtomsun28 89647d96c7 [doc] add japanese i18n in app-tidb.yml, app-windows_script.yml (#3650)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-11 00:57:21 +08:00
tomsun28 0b0fac4a41 [bugfix] fix the collector aes decode with secret error (#3652)
Signed-off-by: tomsun28 <tomsun28@outlook.com>
2025-08-10 22:34:10 +08:00
Duansg f59e15bbbe [fix] Fixed an issue where template configuration was overwritten (#3649) 2025-08-09 23:37:10 +08:00
b7398b1bf9 [feature] Support HertzBeat self monitor (#3641)
Co-authored-by: Calvin <zhengqiwei@apache.org>
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-09 17:14:17 +08:00
Calvinandtomsun28 0082e724bc [doc] add japanese i18n in app-s7.yml, app-tplink_switch.yml, app-shenyu.yml (#3648)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-09 14:25:46 +08:00
57a2daf30d [doc] add japanese i18n in app-prometheus.yml, app-pulsar.yml and app-registry.yml (#3639)
Co-authored-by: aias00 <liuhongyu@apache.org>
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-09 14:23:39 +08:00
Calvinandtomsun28 db90f29cd0 [doc] add japanese i18n in app-qq_mailbox.yml and app-redfish.yml (#3635)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-09 14:21:28 +08:00
DeleiGuoandtomsun28 00fd8fa1da [bugfix] fixed jexl keyword issue in springboot2 monitoring template (#3632)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-09 14:18:46 +08:00
60100eb980 [doc] add English version of linux operating system practice usecase (#3631)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
Co-authored-by: aias00 <liuhongyu@apache.org>
2025-08-09 14:16:09 +08:00
DuansgandLogic a6eb6be1ab [feat] Monitoring templates support JSON nested parsing. (#3645)
Co-authored-by: Logic <zqr10159@dromara.org>
2025-08-08 21:54:38 +08:00
DeleiGuoandLogic 5eff094af6 [bugfix] fix potential StringIndexOutOfBoundsException (#3642)
Co-authored-by: Logic <zqr10159@dromara.org>
2025-08-08 21:35:25 +08:00
CalvinandLogic 3b126f53d2 [doc] add japanese i18n in app-smtp.yml, app-ssl_cert.yml, app-storm.yml, app-udp_port.yml, app-uniview.yml, app-website.yml, app-websocket.yml (#3643)
Co-authored-by: Logic <zqr10159@dromara.org>
2025-08-08 21:15:37 +08:00
Calvinandaias00 1a5b722994 [doc] add japanese i18n in app-openai.yml and app-opengauss.yml (#3617)
Co-authored-by: aias00 <liuhongyu@apache.org>
2025-08-08 20:56:45 +08:00
Duansg 5fe20dff05 [fix] remove optional metrics (#3636) 2025-08-07 00:44:35 +08:00
04cfdae1e5 [fix] Fixed issue where jdbc url was replaced (#3625)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
Co-authored-by: 铁甲小宝 <2922035952@qq.com>
2025-08-05 09:57:04 +08:00
8776f98cf5 [Feature] add customized labels to VM metrics (#3622)
Co-authored-by: zhochi <zhochi@cisco.com>
Co-authored-by: Sherlock Yin <sherlock.yin1994@gmail.com>
Co-authored-by: tomsun28 <tomsun28@outlook.com>
Co-authored-by: Logic <zqr10159@dromara.org>
2025-08-05 09:37:12 +08:00
Duansgandtomsun28 da484aef26 [fix] Fixed springboot3 template yaml has the jexl inner conflict keyword(#3629)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-04 23:32:02 +08:00
DeleiGuoandtomsun28 e8ebc65053 [doc] add linux operating system practice usecase (#3628)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-04 23:12:12 +08:00
Calvinandtomsun28 bd08de41d5 [doc] add japanese i18n in app-postgresql.yml (#3624)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-04 23:09:05 +08:00
Calvinandtomsun28 18f8a541f0 [doc] add japanese i18n in app-oracle.yml (#3621)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-04 23:06:40 +08:00
Calvinandtomsun28 007ecd2db1 [doc] add japanese i18n in app-opensuse.yml (#3619)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-08-04 23:04:33 +08:00
VampireAchaoandCalvin 89aaad2401 [doc] Optimize the home website navbar layout UI for medium-sized screens (#3616)
Co-authored-by: Calvin <zhengqiwei@apache.org>
2025-08-04 22:47:13 +08:00
17d836752e [feat] Http collect supports jsonpath parsing of numeric type (#3612)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
Co-authored-by: Sherlock Yin <sherlock.yin1994@gmail.com>
2025-08-01 19:05:08 +08:00
VampireAchaoandCalvin 3949b73751 [doc] add japanese i18n in app-oceanbase.yml (#3614)
Signed-off-by: VampireAchao <achao@apache.org>
Co-authored-by: Calvin <zhengqiwei@apache.org>
2025-08-01 00:05:48 +08:00
Calvinandtomsun28 2731504278 [doc] add japanese i18n in app-netease_mailbox.yml and app-ntp.yml (#3607)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-07-30 22:02:09 +08:00
babbfaea68 [bugfix] the tagValue may be null in the determineNewLabels method (#3606)
Co-authored-by: Calvin <zhengqiwei@apache.org>
Co-authored-by: Yang Chen <1597081640@qq.com>
2025-07-30 22:00:19 +08:00
9f7b9994b8 [doc] add japanese i18n in app-nebula_graph_cluster.yml and update japanese of client (#3603)
Co-authored-by: shown <yuluo08290126@gmail.com>
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-07-29 09:26:51 +08:00
d156c10aed [doc] update japanese i18n in app-nacos_sd.yml, app-nebula_graph.yml, app-zookeeper.yml (#3602)
Co-authored-by: shown <yuluo08290126@gmail.com>
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-07-29 09:22:32 +08:00
Calvinandtomsun28 9f3329c85f [doc] add japanese i18n in app-nvidia.yml, app-ping.yml, app-pop3.yml, app-port.yml and app-push.yml (#3611)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-07-29 09:20:14 +08:00
cto-huhangandCalvin fd1c58f68d [doc] add japanese i18n in app-nacos.yml (#3599)
Co-authored-by: Calvin <zhengqiwei@apache.org>
2025-07-28 12:04:27 +08:00
DeleiGuo 6d1aa503df [doc] fix image reference issues in documents (#3608) 2025-07-28 01:54:05 +08:00
a58ad35c8b [fix] Fix collector offline without sending offline notification (#3601)
Co-authored-by: Calvin <zhengqiwei@apache.org>
Co-authored-by: shown <yuluo08290126@gmail.com>
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-07-25 21:53:15 +08:00
Calvinandshown 64cd5f9658 [doc] update japanese i18n in app-mqtt.yml and app-mysql.yml (#3597)
Co-authored-by: shown <yuluo08290126@gmail.com>
2025-07-25 08:19:08 +08:00
Yang Chenandshown a13dd89fd2 [feat] Auto-generate AES key if not configured (#3604)
Co-authored-by: shown <yuluo08290126@gmail.com>
2025-07-24 21:06:45 +08:00
2f5cfa9d5d [improve] VM write request sets up gzip compression and adds saveData unit tests (#3595)
Co-authored-by: Calvin <zhengqiwei@apache.org>
Co-authored-by: Sherlock Yin <sherlock.yin1994@gmail.com>
2025-07-24 19:18:17 +08:00
LunaRain_079andCalvin a9dc7bf7f3 [doc] add japanese i18n in app-rocketmq.yml and app-rabbitmq.yml (#3598)
Co-authored-by: Calvin <zhengqiwei@apache.org>
2025-07-23 10:59:41 +08:00
DuansgandCalvin c1492deaa0 [doc] add japanese i18n in app-nginx.yml (#3600)
Co-authored-by: Calvin <zhengqiwei@apache.org>
2025-07-23 00:18:28 +08:00
CyantyandCalvin 2252b2a769 [doc] add japanese i18n in app-zookeeper.yml and app-zookeeper_sd.yml (#3596)
Co-authored-by: Calvin <zhengqiwei@apache.org>
2025-07-22 00:32:41 +08:00
cto-huhang 73f71503f5 [doc] add japanese i18n in app-mysql.yml (#3594) 2025-07-20 21:17:21 +08:00
Calvinandtomsun28 2e224ce626 [doc] add japanese i18n in app-mongodb_atlas.yml (#3593)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-07-20 09:38:27 +08:00
e5406d57c6 [fix] Fix collector online metadata filling (#3579)
Co-authored-by: Calvin <zhengqiwei@apache.org>
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-07-20 09:35:14 +08:00
433a83303f [refactor] Refactor the comparison logic (#3574)
Co-authored-by: Calvin <zhengqiwei@apache.org>
Co-authored-by: aias00 <liuhongyu@apache.org>
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-07-20 01:34:03 +08:00
Calvinandtomsun28 f309d5eafd [doc] add japanese i18n in app-mongodb.yml (#3590)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-07-19 18:20:36 +08:00
986103395b [feature] System time zone optimization. (#3588)
Co-authored-by: Sherlock Yin <sherlock.yin1994@gmail.com>
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-07-19 18:18:25 +08:00
Calvinandtomsun28 d569721e01 [doc] add japanese i18n in app-modbus.yml (#3580)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-07-19 00:44:03 +08:00
Calvinandtomsun28 08cac3c6cc [doc] add japanese i18n in app-memcached.yml (#3578)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-07-19 00:42:16 +08:00
Calvinandtomsun28 f80d066851 [doc] add japanese i18n in app-mariadb.yml (#3577)
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-07-19 00:39:02 +08:00
d6584093f5 [doc] add japanese i18n in app-linux_script.yml (#3575)
Co-authored-by: aias00 <liuhongyu@apache.org>
Co-authored-by: tomsun28 <tomsun28@outlook.com>
2025-07-19 00:37:03 +08:00
DuansgandCalvin fb03a57725 [feature] Support i18n of collector online notification (#3585)
Co-authored-by: Calvin <zhengqiwei@apache.org>
2025-07-18 19:51:08 +08:00
tomsun28 779272d65f [infra] upgrade openjdk image to fix Debian source failed (#3587)
Signed-off-by: tomsun28 <tomsun28@outlook.com>
2025-07-18 15:32:34 +08:00
Logic c21a2952a5 [doc]: add FAQs about task limits (#3581) 2025-07-17 21:52:18 +08:00
222 changed files with 13134 additions and 4909 deletions
+19
View File
@@ -6,6 +6,7 @@
"files": [
"README.md",
"README_CN.md",
"README_JP.md"
"home/src/pages/team/index.jsx"
],
"imageSize": 100,
@@ -2504,6 +2505,24 @@
"code",
"test"
]
},
{
"login": "RainBondsongyg",
"name": "songyg",
"avatar_url": "https://avatars.githubusercontent.com/u/94501396?v=4",
"profile": "https://github.com/RainBondsongyg",
"contributions": [
"doc"
]
},
{
"login": "lx1229",
"name": "Liuxin",
"avatar_url": "https://avatars.githubusercontent.com/u/44620005?v=4",
"profile": "https://github.com/lx1229",
"contributions": [
"code"
]
}
],
"contributorsPerLine": 7,
-9
View File
@@ -48,15 +48,6 @@ jobs:
- name: Build with Maven
run: mvnd clean -B package -Prelease -Dmaven.test.skip=false --file pom.xml
- name: Upload test reports
if: failure()
uses: actions/upload-artifact@v4
with:
name: test-reports-${{ github.run_id }}
path: |
**/target/surefire-reports
**/target/failsafe-reports
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v4.0.1
with:
+2
View File
@@ -527,6 +527,8 @@ Thanks to these wonderful people, welcome to join us:
</tr>
<tr>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/xiaomizhou2"><img src="https://avatars.githubusercontent.com/u/47807926?v=4?s=100" width="100px;" alt="zhangyaxi"/><br /><sub><b>zhangyaxi</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=xiaomizhou2" title="Code">💻</a> <a href="https://github.com/apache/hertzbeat/commits?author=xiaomizhou2" title="Tests">⚠️</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/RainBondsongyg"><img src="https://avatars.githubusercontent.com/u/94501396?v=4?s=100" width="100px;" alt="songyg"/><br /><sub><b>songyg</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=RainBondsongyg" title="Documentation">📖</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/lx1229"><img src="https://avatars.githubusercontent.com/u/44620005?v=4?s=100" width="100px;" alt="Liuxin"/><br /><sub><b>Liuxin</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=lx1229" title="Code">💻</a></td>
</tr>
</tbody>
</table>
+2
View File
@@ -525,6 +525,8 @@ Thanks these wonderful people, welcome to join us:
</tr>
<tr>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/xiaomizhou2"><img src="https://avatars.githubusercontent.com/u/47807926?v=4?s=100" width="100px;" alt="zhangyaxi"/><br /><sub><b>zhangyaxi</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=xiaomizhou2" title="Code">💻</a> <a href="https://github.com/apache/hertzbeat/commits?author=xiaomizhou2" title="Tests">⚠️</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/RainBondsongyg"><img src="https://avatars.githubusercontent.com/u/94501396?v=4?s=100" width="100px;" alt="songyg"/><br /><sub><b>songyg</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=RainBondsongyg" title="Documentation">📖</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/lx1229"><img src="https://avatars.githubusercontent.com/u/44620005?v=4?s=100" width="100px;" alt="Liuxin"/><br /><sub><b>Liuxin</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=lx1229" title="Code">💻</a></td>
</tr>
</tbody>
</table>
+6
View File
@@ -525,6 +525,12 @@ Thanks these wonderful people, welcome to join us:
<td align="center" valign="top" width="14.28%"><a href="https://github.com/boyucjz"><img src="https://avatars.githubusercontent.com/u/18730041?v=4?s=100" width="100px;" alt="boyucjz"/><br /><sub><b>boyucjz</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=boyucjz" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Cyanty"><img src="https://avatars.githubusercontent.com/u/153884653?v=4?s=100" width="100px;" alt="Cyanty"/><br /><sub><b>Cyanty</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Cyanty" title="Code">💻</a> <a href="https://github.com/apache/hertzbeat/commits?author=Cyanty" title="Documentation">📖</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/KevinLLF"><img src="https://avatars.githubusercontent.com/u/85452733?v=4?s=100" width="100px;" alt="Jay丿167"/><br /><sub><b>Jay丿167</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=KevinLLF" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Duansg"><img src="https://avatars.githubusercontent.com/u/112607719?v=4?s=100" width="100px;" alt="Duansg"/><br /><sub><b>Duansg</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Duansg" title="Documentation">📖</a></td>
</tr>
<tr>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/xiaomizhou2"><img src="https://avatars.githubusercontent.com/u/47807926?v=4?s=100" width="100px;" alt="zhangyaxi"/><br /><sub><b>zhangyaxi</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=xiaomizhou2" title="Code">💻</a> <a href="https://github.com/apache/hertzbeat/commits?author=xiaomizhou2" title="Tests">⚠️</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/RainBondsongyg"><img src="https://avatars.githubusercontent.com/u/94501396?v=4?s=100" width="100px;" alt="songyg"/><br /><sub><b>songyg</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=RainBondsongyg" title="Documentation">📖</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/lx1229"><img src="https://avatars.githubusercontent.com/u/44620005?v=4?s=100" width="100px;" alt="Liuxin"/><br /><sub><b>Liuxin</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=lx1229" title="Code">💻</a></td>
</tr>
</tbody>
</table>
+85
View File
@@ -0,0 +1,85 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.hertzbeat</groupId>
<artifactId>hertzbeat</artifactId>
<version>2.0-SNAPSHOT</version>
</parent>
<artifactId>hertzbeat-ai-agent</artifactId>
<version>${hertzbeat.version}</version>
<properties>
<spring-ai.version>1.0.1</spring-ai.version>
<java.version>17</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-mcp-server-webmvc</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-launcher</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>
<dependency>
<groupId>org.apache.hertzbeat</groupId>
<artifactId>hertzbeat-common</artifactId>
</dependency>
<dependency>
<groupId>com.usthe.sureness</groupId>
<artifactId>spring-boot3-starter-sureness</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-bom</artifactId>
<version>${spring-ai.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
</plugins>
</build>
</project>
@@ -0,0 +1,41 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.adapters;
import org.springframework.data.domain.Page;
import org.apache.hertzbeat.common.entity.manager.Monitor;
import java.util.List;
/**
* Interface that provides access to monitor information by retrieving monitor data
* through the underlying monitor service.
*/
public interface MonitorServiceAdapter {
Page<Monitor> getMonitors(
List<Long> ids,
String app,
String search,
Byte status,
String sort,
String order,
Integer pageIndex,
Integer pageSize,
String labels
);
}
@@ -0,0 +1,100 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.adapters.impl;
import com.usthe.sureness.subject.SubjectSum;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.ai.agent.adapters.MonitorServiceAdapter;
import org.apache.hertzbeat.ai.agent.config.McpContextHolder;
import org.springframework.data.domain.Page;
import org.apache.hertzbeat.common.entity.manager.Monitor;
import org.apache.hertzbeat.common.support.SpringContextHolder;
import org.springframework.stereotype.Component;
import java.lang.reflect.Method;
import java.util.List;
/**
* Implementation of the MonitorServiceAdapter interface that provides access to monitor information
* through reflection by invoking the underlying monitor service implementation.
*/
@Slf4j
@Component
public class MonitorServiceAdapterImpl implements MonitorServiceAdapter {
@Override
public Page<Monitor> getMonitors(
List<Long> ids,
String app,
String search,
Byte status,
String sort,
String order,
Integer pageIndex,
Integer pageSize,
String labels) {
try {
// Provide default values for all nullable parameters
if (sort == null || sort.trim().isEmpty()) {
sort = "gmtCreate";
}
if (order == null || order.trim().isEmpty()) {
order = "desc";
}
if (pageIndex == null) {
pageIndex = 0;
}
if (pageSize == null) {
pageSize = 8;
}
Object monitorService = null;
SubjectSum subjectSum = McpContextHolder.getSubject();
log.debug("Current security subject: {}", subjectSum);
try {
monitorService = SpringContextHolder.getBean("monitorServiceImpl");
} catch (Exception e) {
log.debug("Could not find bean by name 'monitorServiceImpl', trying by class name");
}
assert monitorService != null;
log.debug("MonitorService bean found: {}", monitorService.getClass().getSimpleName());
Method method = monitorService.getClass().getMethod(
"getMonitors",
List.class, String.class, String.class, Byte.class,
String.class, String.class, int.class, int.class, String.class);
@SuppressWarnings("unchecked")
Page<Monitor> result = (Page<Monitor>) method.invoke(
monitorService,
ids, app, search, status, sort, order, pageIndex, pageSize, labels);
log.debug("MonitorServiceAdapter.getMonitors result: {}", result.getContent());
return result;
} catch (NoSuchMethodException e) {
throw new RuntimeException("Method not found: getMonitors", e);
} catch (Exception e) {
log.debug("Failed to invoke getMonitors via adapter", e);
throw new RuntimeException("Failed to invoke getMonitors via adapter", e);
}
}
}
@@ -0,0 +1,246 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.config;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.usthe.sureness.mgt.SurenessSecurityManager;
import com.usthe.sureness.subject.SubjectSum;
import io.modelcontextprotocol.spec.McpError;
import io.modelcontextprotocol.spec.McpSchema;
import io.modelcontextprotocol.spec.McpServerSession;
import io.modelcontextprotocol.spec.McpServerTransport;
import io.modelcontextprotocol.spec.McpServerTransportProvider;
import io.modelcontextprotocol.util.Assert;
import java.io.IOException;
import java.time.Duration;
import java.util.HashMap;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;
import jakarta.servlet.http.HttpServletRequest;
import lombok.Getter;
import lombok.Setter;
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.HttpStatus;
import org.springframework.web.servlet.function.RouterFunction;
import org.springframework.web.servlet.function.RouterFunctions;
import org.springframework.web.servlet.function.ServerRequest;
import org.springframework.web.servlet.function.ServerResponse;
import reactor.core.publisher.Flux;
import reactor.core.publisher.Mono;
/**
* Custom Server-Sent Events transport provider for Model Context Protocol.
*/
@Slf4j
public class CustomSseServerTransport implements McpServerTransportProvider {
private final ObjectMapper objectMapper;
private final String messageEndpoint;
private final String sseEndpoint;
private final String baseUrl;
@Getter
private final RouterFunction<ServerResponse> routerFunction;
@Setter
private McpServerSession.Factory sessionFactory;
private final Map<String, Object> sessionRequest = new HashMap<>();
private final ConcurrentHashMap<String, McpServerSession> sessions;
private volatile boolean isClosing;
public CustomSseServerTransport(ObjectMapper objectMapper, String messageEndpoint) {
this(objectMapper, messageEndpoint, "/sse");
}
public CustomSseServerTransport(ObjectMapper objectMapper, String messageEndpoint, String sseEndpoint) {
this(objectMapper, "", messageEndpoint, sseEndpoint);
}
public CustomSseServerTransport(ObjectMapper objectMapper, String baseUrl, String messageEndpoint, String sseEndpoint) {
this.sessions = new ConcurrentHashMap();
this.isClosing = false;
Assert.notNull(objectMapper, "ObjectMapper must not be null");
Assert.notNull(baseUrl, "Message base URL must not be null");
Assert.notNull(messageEndpoint, "Message endpoint must not be null");
Assert.notNull(sseEndpoint, "SSE endpoint must not be null");
this.objectMapper = objectMapper;
this.baseUrl = baseUrl;
this.messageEndpoint = messageEndpoint;
this.sseEndpoint = sseEndpoint;
this.routerFunction = RouterFunctions.route().GET(this.sseEndpoint, this::handleSseConnection).POST(this.messageEndpoint, this::handleMessage).build();
}
public Mono<Void> notifyClients(String method, Object params) {
if (this.sessions.isEmpty()) {
log.debug("No active sessions to broadcast message to");
return Mono.empty();
} else {
log.debug("Attempting to broadcast message to {} active sessions", this.sessions.size());
return Flux.fromIterable(this.sessions.values())
.flatMap((session) -> session.sendNotification(method, params)
.doOnError((e) -> log.error("Failed to send message to session {}: {}", session.getId(), e.getMessage()))
.onErrorComplete())
.then();
}
}
public Mono<Void> closeGracefully() {
return Flux.fromIterable(this.sessions.values()).doFirst(() -> {
this.isClosing = true;
log.debug("Initiating graceful shutdown with {} active sessions", this.sessions.size());
}).flatMap(McpServerSession::closeGracefully).then().doOnSuccess((v) -> log.debug("Graceful shutdown completed"));
}
private ServerResponse handleSseConnection(ServerRequest request) {
log.debug("Handling SSE connection for request: {}", request);
HttpServletRequest servletRequest = request.servletRequest();
try {
log.debug("Processing SSE connection for servlet request: {}", servletRequest);
log.debug("Authorization header: {}", servletRequest.getHeader("Authorization"));
} catch (Exception e) {
log.error("Authentication failed for SSE connection: {}", e.getMessage());
return ServerResponse.status(HttpStatus.UNAUTHORIZED).body("Unauthorized: " + e.getMessage());
}
if (this.isClosing) {
return ServerResponse.status(HttpStatus.SERVICE_UNAVAILABLE).body("Server is shutting down");
} else {
String sessionId = UUID.randomUUID().toString();
log.debug("Generated session ID for SSE connection: {}", sessionId);
log.debug("Creating new SSE connection for session: {}", sessionId);
return ServerResponse.sse((sseBuilder) -> {
sseBuilder.onComplete(() -> {
log.debug("SSE connection completed for session: {}", sessionId);
this.sessions.remove(sessionId);
});
sseBuilder.onTimeout(() -> {
log.debug("SSE connection timed out for session: {}", sessionId);
this.sessions.remove(sessionId);
});
CustomSseServerTransport.WebMvcMcpSessionTransport sessionTransport = new CustomSseServerTransport.WebMvcMcpSessionTransport(sessionId, sseBuilder);
McpServerSession session = this.sessionFactory.create(sessionTransport);
this.sessionRequest.put(sessionId, request.servletRequest());
this.sessions.put(sessionId, session);
try {
sseBuilder.id(sessionId).event("endpoint").data(this.baseUrl + this.messageEndpoint + "?sessionId=" + sessionId);
} catch (Exception e) {
log.error("Failed to send initial endpoint event: {}", e.getMessage());
sseBuilder.error(e);
}
}, Duration.ZERO);
}
}
private ServerResponse handleMessage(ServerRequest request) {
if (this.isClosing) {
return ServerResponse.status(HttpStatus.SERVICE_UNAVAILABLE).body("Server is shutting down");
} else if (request.param("sessionId").isEmpty()) {
return ServerResponse.badRequest().body(new McpError("Session ID missing in message endpoint"));
} else {
String sessionId = (String) request.param("sessionId").get();
McpServerSession session = (McpServerSession) this.sessions.get(sessionId);
log.debug("Authorization header for message request: {}", request.servletRequest().getHeader("Authorization"));
SubjectSum subject = SurenessSecurityManager.getInstance().checkIn(sessionRequest.get(sessionId));
McpContextHolder.setSubject(subject);
if (session == null) {
return ServerResponse.status(HttpStatus.NOT_FOUND).body(new McpError("Session not found: " + sessionId));
} else {
try {
String body = request.body(String.class);
McpSchema.JSONRPCMessage message = McpSchema.deserializeJsonRpcMessage(this.objectMapper, body);
session.handle(message).block();
return ServerResponse.ok().build();
} catch (IOException | IllegalArgumentException e) {
log.error("Failed to deserialize message: {}", ((Exception) e).getMessage());
return ServerResponse.badRequest().body(new McpError("Invalid message format"));
} catch (Exception e) {
log.error("Error handling message: {}", e.getMessage());
return ServerResponse.status(HttpStatus.INTERNAL_SERVER_ERROR).body(new McpError(e.getMessage()));
}
}
}
}
private class WebMvcMcpSessionTransport implements McpServerTransport {
private final String sessionId;
private final ServerResponse.SseBuilder sseBuilder;
WebMvcMcpSessionTransport(String sessionId, ServerResponse.SseBuilder sseBuilder) {
this.sessionId = sessionId;
this.sseBuilder = sseBuilder;
log.debug("Session transport {} initialized with SSE builder", sessionId);
}
public Mono<Void> sendMessage(McpSchema.JSONRPCMessage message) {
return Mono.fromRunnable(() -> {
try {
String jsonText = CustomSseServerTransport.this.objectMapper.writeValueAsString(message);
this.sseBuilder.id(this.sessionId).event("message").data(jsonText);
log.debug("Message sent to session {}", this.sessionId);
} catch (Exception e) {
log.error("Failed to send message to session {}: {}", this.sessionId, e.getMessage());
this.sseBuilder.error(e);
}
});
}
public <T> T unmarshalFrom(Object data, TypeReference<T> typeRef) {
return (T) CustomSseServerTransport.this.objectMapper.convertValue(data, typeRef);
}
public Mono<Void> closeGracefully() {
return Mono.fromRunnable(() -> {
log.debug("Closing session transport: {}", this.sessionId);
try {
this.sseBuilder.complete();
log.debug("Successfully completed SSE builder for session {}", this.sessionId);
} catch (Exception e) {
log.warn("Failed to complete SSE builder for session {}: {}", this.sessionId, e.getMessage());
}
});
}
public void close() {
try {
this.sseBuilder.complete();
log.debug("Successfully completed SSE builder for session {}", this.sessionId);
} catch (Exception e) {
log.warn("Failed to complete SSE builder for session {}: {}", this.sessionId, e.getMessage());
}
}
}
}
@@ -0,0 +1,37 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.config;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.openai.OpenAiChatModel;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
/**
* Configuration class for Large Language Model (LLM) settings.
*/
@Configuration
public class LlmConfig {
@Bean
public ChatClient openAiChatClient(OpenAiChatModel chatModel) {
return ChatClient.create(chatModel);
}
}
@@ -0,0 +1,53 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.config;
import com.usthe.sureness.subject.SubjectSum;
import org.springframework.core.NamedInheritableThreadLocal;
/**
* Context holder for AI agent security context.
*/
public final class McpContextHolder {
private static final ThreadLocal<SubjectSum> subjectHolder =
new NamedInheritableThreadLocal<>("MCP Security and User Identification Context");
private McpContextHolder() {}
/**
* Attaches the user's context to the current thread.
*/
public static void setSubject(SubjectSum subject) {
subjectHolder.set(subject);
}
/**
* Retrieves the context from the current thread.
*/
public static SubjectSum getSubject() {
return subjectHolder.get();
}
/**
* Clears the context from the thread to prevent memory leaks.
*/
public static void clear() {
subjectHolder.remove();
}
}
@@ -0,0 +1,53 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.config;
import org.springframework.stereotype.Component;
/**
* Provider for system prompts used in the AI agent
*/
@Component
public class PromptProvider {
/**
* Static version of the HertzBeat monitoring prompt
*/
public static final String HERTZBEAT_MONITORING_PROMPT = """
You are an AI assistant specialized in monitoring infrastructure and applications with HertzBeat.
Your role is to help users manage and analyze their monitoring data using the available tools.
You have access to the following HertzBeat monitoring tools:
- list_monitors: Query monitor information with flexible filtering and pagination
- add_monitor: Add a new monitor to the system
When users ask questions about their monitoring setup or data, identify which tool would be most helpful
and use it to provide relevant information. Always provide clear explanations of the monitoring data and
suggest next steps or insights based on the results.
For monitoring-related queries:
1. If users want to see their monitors, use list_monitors with appropriate filters
2. If users want to add a new monitor, use add_monitor with the necessary details
3. If the monitoring information shows potential issues, highlight them and suggest troubleshooting steps
For parameters that accept specific values:
- Monitor status values: 0 (no monitor), 1 (usable), 2 (disabled), 9 (all)
- Sort fields typically include: name, host, app, gmtCreate
- Sort order should be 'asc' or 'desc'
Keep responses focused on monitoring topics and HertzBeat capabilities.
If you're unsure about specific monitoring details, ask clarifying questions before using the tools.
""";
}
@@ -0,0 +1,70 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.controller;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.http.MediaType;
import org.springframework.web.servlet.mvc.method.annotation.SseEmitter;
import org.apache.hertzbeat.ai.agent.pojo.dto.ChatRequestContext;
import org.apache.hertzbeat.ai.agent.service.ChatClientProviderService;
/**
* Controller class for handling chat-related HTTP requests.
*/
@RestController
@RequestMapping("/api/chat")
public class ChatController {
private final ChatClientProviderService chatClientProviderService;
@Autowired
public ChatController(@Qualifier("openAiChatClient") ChatClient openAiChatClient,
ChatClientProviderService chatClientProviderService) {
this.chatClientProviderService = chatClientProviderService;
}
/**
* Send a message and get a streaming response
*
* @param context The chat request context containing message and optional
* conversationId
* @return SSE emitter for streaming response
*/
@PostMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public SseEmitter streamChat(@RequestBody ChatRequestContext context) {
SseEmitter emitter = new SseEmitter();
new Thread(() -> {
try {
String aiResponse = chatClientProviderService.streamChat(context);
emitter.send(aiResponse);
emitter.complete();
} catch (Exception e) {
emitter.completeWithError(e);
}
}).start();
return emitter;
}
}
@@ -0,0 +1,26 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.controller;
/**
* Controller for managing conversations.
*/
public class ConversationController {
}
@@ -0,0 +1,25 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.dao;
/**
* Data Access Object interface for Conversation entities.
*/
public interface ConversationDao {
}
@@ -0,0 +1,25 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.dao;
/**
* Data Access Object interface for Message entities.
*/
public interface MessageDao {
}
@@ -0,0 +1,25 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.dao;
/**
* Data Access Object interface for UserPreference entities.
*/
public interface UserPreferenceDao {
}
@@ -0,0 +1,40 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.pojo.dto;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
/**
* Chat request context for AI chat endpoint.
*/
@Data
@NoArgsConstructor
@AllArgsConstructor
public class ChatRequestContext {
/**
* The user's message (required)
*/
private String message;
/**
* Optional conversation ID for context
*/
private String conversationId;
}
@@ -0,0 +1,26 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.service;
/**
* Service interface for agent operations.
*/
public interface AgentService {
}
@@ -0,0 +1,31 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.service;
import org.apache.hertzbeat.ai.agent.pojo.dto.ChatRequestContext;
/**
* Service for interacting with LLM providers (like OpenAI, Anthropic, etc.)
*/
public interface ChatClientProviderService {
String complete(String message);
String streamChat(ChatRequestContext context);
}
@@ -0,0 +1,71 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.service;
import org.springframework.web.servlet.mvc.method.annotation.SseEmitter;
import java.util.List;
import java.util.Map;
/**
* Service for managing chat conversations and interactions with LLM providers.
*/
public interface ConversationService {
/**
* Send a message and receive a streaming response
*
* @param message The user's message
* @param conversationId Optional conversation ID for continuing a chat
* @return SseEmitter for streaming the response
*/
SseEmitter streamChat(String message, String conversationId);
/**
* Send a message and get a complete response
*
* @param message The user's message
* @param conversationId Optional conversation ID for continuing a chat
* @return Response object containing the AI's response and conversation metadata
*/
Map<String, Object> chat(String message, String conversationId);
/**
* Get conversation history for a specific conversation
*
* @param conversationId Conversation ID
* @return Conversation data including messages
*/
Map<String, Object> getConversation(String conversationId);
/**
* Get all conversations for the current user
*
* @return List of conversations
*/
List<Map<String, Object>> getAllConversations();
/**
* Delete a conversation
*
* @param conversationId Conversation ID to delete
*/
void deleteConversation(String conversationId);
}
@@ -0,0 +1,28 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.service;
import org.springframework.ai.tool.ToolCallbackProvider;
/**
* Service interface for MCP server operations.
*/
public interface McpServerService {
ToolCallbackProvider hertzbeatTools();
}
@@ -0,0 +1,30 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.service.impl;
import org.apache.hertzbeat.ai.agent.service.AgentService;
import org.springframework.stereotype.Service;
/**
* Implementation of the AgentService interface.
* This service provides functionality for handling AI agent operations.
*/
@Service
public class AgentServiceImpl implements AgentService {
}
@@ -0,0 +1,70 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.service.impl;
import org.apache.hertzbeat.ai.agent.config.PromptProvider;
import org.apache.hertzbeat.ai.agent.service.ChatClientProviderService;
import org.springframework.stereotype.Service;
import org.apache.hertzbeat.ai.agent.pojo.dto.ChatRequestContext;
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.tool.ToolCallbackProvider;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
/**
* Implementation of the {@link ChatClientProviderService}.
* Provides functionality to interact with the ChatClient for handling chat
* messages.
*/
@Service
public class ChatClientProviderServiceImpl implements ChatClientProviderService {
private final ChatClient chatClient;
@Qualifier("hertzbeatTools")
@Autowired
private ToolCallbackProvider toolCallbackProvider;
@Autowired
public ChatClientProviderServiceImpl(@Qualifier("openAiChatClient") ChatClient openAiChatClient) {
this.chatClient = openAiChatClient;
}
@Override
public String complete(String message) {
return this.chatClient.prompt()
.user(message)
.call()
.content();
}
@Override
public String streamChat(ChatRequestContext context) {
try {
return this.chatClient.prompt(PromptProvider.HERTZBEAT_MONITORING_PROMPT)
.user(context.getMessage())
.toolCallbacks(toolCallbackProvider)
.call()
.content();
} catch (Exception e) {
return "Error: " + e.getMessage();
}
}
}
@@ -0,0 +1,29 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.service.impl;
import org.springframework.stereotype.Service;
/**
* Implementation of the ConversationService interface for managing chat conversations.
*/
@Service
public class ConversationServiceImpl {
}
@@ -0,0 +1,82 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.service.impl;
import org.apache.hertzbeat.ai.agent.config.CustomSseServerTransport;
import org.apache.hertzbeat.ai.agent.service.McpServerService;
import org.springframework.ai.mcp.server.autoconfigure.McpServerProperties;
import org.springframework.context.annotation.Primary;
import org.springframework.stereotype.Service;
import org.apache.hertzbeat.ai.agent.tools.impl.MonitorToolsImpl;
import org.springframework.ai.tool.ToolCallbackProvider;
import org.springframework.ai.tool.method.MethodToolCallbackProvider;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.beans.factory.annotation.Autowired;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.web.servlet.function.RouterFunction;
import org.springframework.web.servlet.function.ServerResponse;
/**
* Implementation of the McpServerService interface.
* This service provides functionality for handling MCP server operations.
*/
@Service
@Configuration
public class McpServerServiceImpl implements McpServerService {
@Autowired
private MonitorToolsImpl monitorTools;
@Bean
public ToolCallbackProvider hertzbeatTools() {
return MethodToolCallbackProvider.builder().toolObjects(monitorTools).build();
}
/**
* Provides a custom SSE server transport for the MCP server.
*
* @param objectMapper the ObjectMapper instance for JSON serialization
* @param serverProperties the properties for the MCP server configuration
* @return a CustomSseServerTransport instance configured with the provided properties
*/
@Bean
public CustomSseServerTransport webMvcSseServerTransportProvider(
ObjectMapper objectMapper,
McpServerProperties serverProperties
) {
return new CustomSseServerTransport(
objectMapper,
serverProperties.getBaseUrl(),
serverProperties.getSseMessageEndpoint(),
serverProperties.getSseEndpoint()
);
}
/**
* Provides the MCP server transport bean.
*
* @param transport the custom SSE server transport
* @return the MCP server transport instance
*/
@Primary
@Bean
public RouterFunction<ServerResponse> mvcMcpRouterFunction(CustomSseServerTransport transport) {
return transport.getRouterFunction();
}
}
@@ -0,0 +1,25 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.tools;
/**
* Tools for alert operations
*/
public interface AlertTools {
}
@@ -0,0 +1,25 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.tools;
/**
* Tools for metrics operations
*/
public interface MetricsTools {
}
@@ -0,0 +1,50 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.tools;
import org.springframework.ai.chat.model.ToolContext;
import java.util.List;
/**
* Interface for Monitoring Tools
*/
public interface MonitorTools {
String addMonitor(String name, ToolContext context);
/**
* Query monitor information with flexible filtering and pagination.
* Supports filtering by monitor IDs, type, status, host, labels, sorting, and
* pagination.
* Returns results as plain JSON.
*/
String listMonitors(
List<Long> ids,
String app,
Byte status,
String search,
String labels,
String sort,
String order,
Integer pageIndex,
Integer pageSize,
ToolContext context);
}
@@ -0,0 +1,25 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.tools.impl;
/**
* Implementation of Alert Tools functionality
*/
public class AlertToolsImpl {
}
@@ -0,0 +1,25 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.tools.impl;
/**
* Implementation of Metrics Tools functionality
*/
public class MetricsToolsImpl {
}
@@ -0,0 +1,87 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.ai.agent.tools.impl;
import com.usthe.sureness.subject.SubjectSum;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.ai.agent.adapters.MonitorServiceAdapter;
import org.apache.hertzbeat.ai.agent.config.McpContextHolder;
import org.springframework.ai.chat.model.ToolContext;
import org.springframework.ai.tool.annotation.Tool;
import org.springframework.data.domain.Page;
import org.springframework.stereotype.Service;
import org.apache.hertzbeat.ai.agent.tools.MonitorTools;
import org.springframework.ai.tool.annotation.ToolParam;
import org.springframework.beans.factory.annotation.Autowired;
import org.apache.hertzbeat.common.entity.manager.Monitor;
import java.util.List;
/**
* Implementation of Monitoring Tools functionality
*/
@Slf4j
@Service
public class MonitorToolsImpl implements MonitorTools {
@Autowired
private MonitorServiceAdapter monitorServiceAdapter;
/**
* Tool to query monitor information with flexible filtering and pagination.
* Supports filtering by monitor IDs, type, status, host, labels, sorting, and
* pagination.
* Returns monitor names as string.
*/
@Override
@Tool(name = "list_monitors", returnDirect = true, description = """
Query monitor information with flexible filtering and pagination.
Supports filtering by monitor IDs, type, status, host, labels, sorting, and pagination.
Returns results as String. When no parameters are available, pass the default value as mentioned below. If the user doesn't provide any specific parameter, the default value will be used.
""")
public String listMonitors(
@ToolParam(description = "List of monitor IDs to filter (default: empty list)", required = false) List<Long> ids,
@ToolParam(description = "Monitor type, e.g., 'linux' (default: null)", required = false) String app,
@ToolParam(description = "Monitor status (0: no monitor, 1: usable, 2: disabled, 9: all) (default: null)", required = false) Byte status,
@ToolParam(description = "Fuzzy search for host or name (default: null)", required = false) String search,
@ToolParam(description = "Monitor labels, e.g., 'env:prod,instance:22' (default: null)", required = false) String labels,
@ToolParam(description = "Sort field, e.g., 'name' (default: gmtCreate)", required = false) String sort,
@ToolParam(description = "Sort order, 'asc' or 'desc' (default: desc)", required = false) String order,
@ToolParam(description = "Page index (default: 0)", required = false) Integer pageIndex,
@ToolParam(description = "Page size (default: 8)", required = false) Integer pageSize,
ToolContext context) {
try {
Page<Monitor> result = monitorServiceAdapter.getMonitors(ids, app, search, status, sort, order, pageIndex, pageSize, labels);
log.debug("MonitorServiceAdapter.getMonitors result: {}", result);
return result.getContent().stream().map(Monitor::getName).toList().toString();
} catch (Exception e) {
return "error is" + e.getMessage();
}
}
@Override
@Tool(name = "add_monitor", description = "Add a new monitor")
public String addMonitor(@ToolParam(description = "Name of the monitor") String name, ToolContext context) {
log.debug("Adding monitor with name: {}", name);
SubjectSum subjectSum = McpContextHolder.getSubject();
log.debug("Current subject in tool: {}", subjectSum);
return "Monitor added: " + name;
}
}
@@ -109,4 +109,9 @@ public class AlarmCacheManager {
private String getCustomKey(String fingerPrint) {
return CUSTOM_FIRING_ROW_KEY + fingerPrint;
}
public SingleAlert removeFiring(String fingerPrint) {
return this.firingAlertMap.remove(getCustomKey(fingerPrint), fingerPrint);
}
}
@@ -76,11 +76,12 @@ public class CollectorAlertHandler {
fingerPrints.put(KEY_COLLECTOR_VERSION, collector.getVersion());
fingerPrints.put(KEY_COLLECTOR_HOST, collector.getIp());
String fingerprint = AlertUtil.calculateFingerprint(fingerPrints);
SingleAlert firingAlert = alarmCacheManager.getFiring(fingerprint);
SingleAlert firingAlert = alarmCacheManager.removeFiring(fingerprint);
if (firingAlert != null) {
firingAlert.setTriggerTimes(1);
firingAlert.setEndAt(System.currentTimeMillis());
firingAlert.setStatus(CommonConstants.ALERT_STATUS_RESOLVED);
firingAlert.setContent(this.bundle.getString("alerter.availability.collector.recover"));
alarmCommonReduce.reduceAndSendAlarm(firingAlert.clone());
}
}
@@ -27,7 +27,7 @@ import org.antlr.v4.runtime.tree.AbstractParseTreeVisitor;
* of the available methods.
*
* @param <T> The return type of the visit operation. Use {@link Void} for
* operations with no return type.
* operations with no return type.
*/
@SuppressWarnings("CheckReturnValue")
public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> implements AlertExpressionVisitor<T> {
@@ -37,7 +37,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitExpression(AlertExpressionParser.ExpressionContext ctx) { return visitChildren(ctx); }
@Override
public T visitExpression(AlertExpressionParser.ExpressionContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -45,7 +48,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitAndExpr(AlertExpressionParser.AndExprContext ctx) { return visitChildren(ctx); }
@Override
public T visitAndExpr(AlertExpressionParser.AndExprContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -53,7 +59,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitComparisonExpr(AlertExpressionParser.ComparisonExprContext ctx) { return visitChildren(ctx); }
@Override
public T visitComparisonExpr(AlertExpressionParser.ComparisonExprContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -61,7 +70,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitUnlessExpr(AlertExpressionParser.UnlessExprContext ctx) { return visitChildren(ctx); }
@Override
public T visitUnlessExpr(AlertExpressionParser.UnlessExprContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -69,7 +81,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitSqlExpr(AlertExpressionParser.SqlExprContext ctx) { return visitChildren(ctx); }
@Override
public T visitSqlExpr(AlertExpressionParser.SqlExprContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -77,7 +92,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitSqlCallExpr(AlertExpressionParser.SqlCallExprContext ctx) { return visitChildren(ctx); }
@Override
public T visitSqlCallExpr(AlertExpressionParser.SqlCallExprContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -85,7 +103,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitLiteralExpr(AlertExpressionParser.LiteralExprContext ctx) { return visitChildren(ctx); }
@Override
public T visitLiteralExpr(AlertExpressionParser.LiteralExprContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -93,7 +114,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitParenExpr(AlertExpressionParser.ParenExprContext ctx) { return visitChildren(ctx); }
@Override
public T visitParenExpr(AlertExpressionParser.ParenExprContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -101,7 +125,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitPromqlCallExpr(AlertExpressionParser.PromqlCallExprContext ctx) { return visitChildren(ctx); }
@Override
public T visitPromqlCallExpr(AlertExpressionParser.PromqlCallExprContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -109,7 +136,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitPromqlExpr(AlertExpressionParser.PromqlExprContext ctx) { return visitChildren(ctx); }
@Override
public T visitPromqlExpr(AlertExpressionParser.PromqlExprContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -117,7 +147,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitOrExpr(AlertExpressionParser.OrExprContext ctx) { return visitChildren(ctx); }
@Override
public T visitOrExpr(AlertExpressionParser.OrExprContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -125,7 +158,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitFunctionCall(AlertExpressionParser.FunctionCallContext ctx) { return visitChildren(ctx); }
@Override
public T visitFunctionCall(AlertExpressionParser.FunctionCallContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -133,7 +169,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitParameterList(AlertExpressionParser.ParameterListContext ctx) { return visitChildren(ctx); }
@Override
public T visitParameterList(AlertExpressionParser.ParameterListContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -141,7 +180,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitParameter(AlertExpressionParser.ParameterContext ctx) { return visitChildren(ctx); }
@Override
public T visitParameter(AlertExpressionParser.ParameterContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -149,7 +191,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitNumber(AlertExpressionParser.NumberContext ctx) { return visitChildren(ctx); }
@Override
public T visitNumber(AlertExpressionParser.NumberContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -157,7 +202,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitString(AlertExpressionParser.StringContext ctx) { return visitChildren(ctx); }
@Override
public T visitString(AlertExpressionParser.StringContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -165,7 +213,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitDuration(AlertExpressionParser.DurationContext ctx) { return visitChildren(ctx); }
@Override
public T visitDuration(AlertExpressionParser.DurationContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -173,7 +224,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitFunctionName(AlertExpressionParser.FunctionNameContext ctx) { return visitChildren(ctx); }
@Override
public T visitFunctionName(AlertExpressionParser.FunctionNameContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -181,7 +235,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitSelectSql(AlertExpressionParser.SelectSqlContext ctx) { return visitChildren(ctx); }
@Override
public T visitSelectSql(AlertExpressionParser.SelectSqlContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -189,7 +246,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitSelectFieldList(AlertExpressionParser.SelectFieldListContext ctx) { return visitChildren(ctx); }
@Override
public T visitSelectFieldList(AlertExpressionParser.SelectFieldListContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -197,7 +257,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitSelectField(AlertExpressionParser.SelectFieldContext ctx) { return visitChildren(ctx); }
@Override
public T visitSelectField(AlertExpressionParser.SelectFieldContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -205,7 +268,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitGroupByList(AlertExpressionParser.GroupByListContext ctx) { return visitChildren(ctx); }
@Override
public T visitGroupByList(AlertExpressionParser.GroupByListContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -213,7 +279,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitOrderByList(AlertExpressionParser.OrderByListContext ctx) { return visitChildren(ctx); }
@Override
public T visitOrderByList(AlertExpressionParser.OrderByListContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -221,7 +290,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitOrderByField(AlertExpressionParser.OrderByFieldContext ctx) { return visitChildren(ctx); }
@Override
public T visitOrderByField(AlertExpressionParser.OrderByFieldContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -229,7 +301,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitLimitClause(AlertExpressionParser.LimitClauseContext ctx) { return visitChildren(ctx); }
@Override
public T visitLimitClause(AlertExpressionParser.LimitClauseContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -237,7 +312,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitRelList(AlertExpressionParser.RelListContext ctx) { return visitChildren(ctx); }
@Override
public T visitRelList(AlertExpressionParser.RelListContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -245,7 +323,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitRelation(AlertExpressionParser.RelationContext ctx) { return visitChildren(ctx); }
@Override
public T visitRelation(AlertExpressionParser.RelationContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -253,7 +334,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitConditionList(AlertExpressionParser.ConditionListContext ctx) { return visitChildren(ctx); }
@Override
public T visitConditionList(AlertExpressionParser.ConditionListContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -261,7 +345,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitCompOp(AlertExpressionParser.CompOpContext ctx) { return visitChildren(ctx); }
@Override
public T visitCompOp(AlertExpressionParser.CompOpContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -269,7 +356,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitCondition(AlertExpressionParser.ConditionContext ctx) { return visitChildren(ctx); }
@Override
public T visitCondition(AlertExpressionParser.ConditionContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -277,7 +367,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitConditionUnit(AlertExpressionParser.ConditionUnitContext ctx) { return visitChildren(ctx); }
@Override
public T visitConditionUnit(AlertExpressionParser.ConditionUnitContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -285,7 +378,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitPromql(AlertExpressionParser.PromqlContext ctx) { return visitChildren(ctx); }
@Override
public T visitPromql(AlertExpressionParser.PromqlContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -293,7 +389,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitMetricSelector(AlertExpressionParser.MetricSelectorContext ctx) { return visitChildren(ctx); }
@Override
public T visitMetricSelector(AlertExpressionParser.MetricSelectorContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -301,7 +400,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitLabelMatcherList(AlertExpressionParser.LabelMatcherListContext ctx) { return visitChildren(ctx); }
@Override
public T visitLabelMatcherList(AlertExpressionParser.LabelMatcherListContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -309,7 +411,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitLabelMatcherItem(AlertExpressionParser.LabelMatcherItemContext ctx) { return visitChildren(ctx); }
@Override
public T visitLabelMatcherItem(AlertExpressionParser.LabelMatcherItemContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -317,7 +422,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitLabelMatcherOp(AlertExpressionParser.LabelMatcherOpContext ctx) { return visitChildren(ctx); }
@Override
public T visitLabelMatcherOp(AlertExpressionParser.LabelMatcherOpContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -325,7 +433,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitLabelList(AlertExpressionParser.LabelListContext ctx) { return visitChildren(ctx); }
@Override
public T visitLabelList(AlertExpressionParser.LabelListContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -333,7 +444,10 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitInstantVectorOp(AlertExpressionParser.InstantVectorOpContext ctx) { return visitChildren(ctx); }
@Override
public T visitInstantVectorOp(AlertExpressionParser.InstantVectorOpContext ctx) {
return visitChildren(ctx);
}
/**
* {@inheritDoc}
@@ -341,13 +455,8 @@ public class AlertExpressionBaseVisitor<T> extends AbstractParseTreeVisitor<T> i
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitAggregationOperator(AlertExpressionParser.AggregationOperatorContext ctx) { return visitChildren(ctx); }
/**
* {@inheritDoc}
*
* <p>The default implementation returns the result of calling
* {@link #visitChildren} on {@code ctx}.</p>
*/
@Override public T visitBinaryOperator(AlertExpressionParser.BinaryOperatorContext ctx) { return visitChildren(ctx); }
@Override
public T visitAggregationOperator(AlertExpressionParser.AggregationOperatorContext ctx) {
return visitChildren(ctx);
}
}
@@ -18,11 +18,12 @@
package org.apache.hertzbeat.alert.expr;
import org.antlr.v4.runtime.CommonTokenStream;
import org.apache.commons.collections4.CollectionUtils;
import org.apache.hertzbeat.common.support.exception.ExpressionVisitorException;
import org.apache.hertzbeat.warehouse.db.QueryExecutor;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Set;
@@ -33,7 +34,7 @@ import java.util.stream.Collectors;
*/
public class AlertExpressionEvalVisitor extends AlertExpressionBaseVisitor<List<Map<String, Object>>> {
private static final String THRESHOLD = "__threshold__";
private static final String SCALAR = "__scalar__";
private static final String NAME = "__name__";
private static final String VALUE = "__value__";
private static final String TIMESTAMP = "__timestamp__";
@@ -60,28 +61,100 @@ public class AlertExpressionEvalVisitor extends AlertExpressionBaseVisitor<List<
public List<Map<String, Object>> visitComparisonExpr(AlertExpressionParser.ComparisonExprContext ctx) {
List<Map<String, Object>> leftResult = visit(ctx.left);
List<Map<String, Object>> rightResult = visit(ctx.right);
if (rightResult.size() == 1 && rightResult.get(0).containsKey(THRESHOLD)) {
double threshold = (double) rightResult.get(0).get(THRESHOLD);
String operator = ctx.op.getText();
int type = ctx.op.getType();
boolean boolModifier = ctx.BOOL() != null;
boolean leftIsScalar = isScalar(leftResult);
boolean rightIsScalar = isScalar(rightResult);
List<Map<String, Object>> results = new ArrayList<>();
List<Map<String, Object>> result = new ArrayList<>();
for (Map<String, Object> item : leftResult) {
Object queryValues = item.get(VALUE);
if (queryValues == null) {
// ignore the query result data is empty
// scalar and scalar
if (leftIsScalar && rightIsScalar) {
if (!boolModifier) {
// Between two scalars,
// the bool modifier must be provided and these operators result in another scalar that is either 0 (false) or 1 (true), depending on the comparison result.
return results;
}
Object leftVal = leftResult.get(0).get(SCALAR);
Object rightVal = rightResult.get(0).get(SCALAR);
Boolean match = compareOp(leftVal, type, rightVal);
// returns a result only if the comparison condition is met.
Map<String, Object> result = new HashMap<>();
result.put(VALUE, match ? 1 : 0);
results.add(result);
return results;
}
// scalar and vector
if (leftIsScalar) {
Object leftVal = leftResult.get(0).get(SCALAR);
for (Map<String, Object> rightItem : rightResult) {
Object rightVal = rightItem.getOrDefault(VALUE, null);
if (isValidValue(rightVal)) {
continue;
}
// queryValues may be a list of values, or a single value
Object matchValue = evaluateCondition(queryValues, operator, threshold);
Map<String, Object> resultMap = new HashMap(item);
resultMap.put(VALUE, matchValue);
// if matchValue is null, mean not match the threshold
// if not null, mean match the threshold
result.add(resultMap);
Boolean match = compareOp(leftVal, type, rightVal);
Map<String, Object> result = new HashMap<>(rightItem);
if (boolModifier) {
result.put(VALUE, match ? 1 : 0);
results.add(result);
} else {
result.put(VALUE, match ? rightVal : null);
results.add(result);
}
}
return result;
return results;
}
return new LinkedList<>();
// vector and scalar
if (rightIsScalar) {
Object rightVal = rightResult.get(0).get(SCALAR);
for (Map<String, Object> leftItem : leftResult) {
Object leftVal = leftItem.getOrDefault(VALUE, null);
if (isValidValue(leftVal)) {
continue;
}
Boolean match = compareOp(leftVal, type, rightVal);
Map<String, Object> result = new HashMap<>(leftItem);
if (boolModifier) {
result.put(VALUE, match ? 1 : 0);
results.add(result);
} else {
result.put(VALUE, match ? leftVal : null);
results.add(result);
}
}
return results;
}
// vector and vector
Map<String, Map<String, Object>> rightMap = rightResult.stream()
.filter(item -> item.get(VALUE) != null)
.collect(Collectors.toMap(this::labelKey, item -> item, (existing, replacement) -> existing));
for (Map<String, Object> leftItem : leftResult) {
Object leftVal = leftItem.getOrDefault(VALUE, null);
if (isValidValue(leftVal)) {
continue;
}
Map<String, Object> rightItem = rightMap.get(labelKey(leftItem));
if (rightItem == null) {
continue;
}
Object rightVal = rightItem.get(VALUE);
if (isValidValue(rightVal)) {
continue;
}
Boolean match = compareOp(leftVal, type, rightVal);
Map<String, Object> result = new HashMap<>(leftItem);
if (boolModifier) {
result.put(VALUE, match ? 1 : 0);
results.add(result);
} else {
result.put(VALUE, match ? leftVal : null);
results.add(result);
}
}
return results;
}
@Override
@@ -167,7 +240,7 @@ public class AlertExpressionEvalVisitor extends AlertExpressionBaseVisitor<List<
double value = Double.parseDouble(ctx.number().getText());
List<Map<String, Object>> numAsList = new ArrayList<>();
Map<String, Object> valueMap = new HashMap<>();
valueMap.put(THRESHOLD, value);
valueMap.put(SCALAR, value);
numAsList.add(valueMap);
return numAsList;
}
@@ -194,83 +267,7 @@ public class AlertExpressionEvalVisitor extends AlertExpressionBaseVisitor<List<
return callSqlOrPromql(tokens.getText(ctx.string()));
}
private Object evaluateCondition(Object value, String operator, Double threshold) {
// value may be a list of values, or a single value
switch (operator) {
case ">":
// if value is list, return the max value
if (value instanceof List<?> values) {
Double doubleValue = values.stream().map(v -> Double.valueOf(v.toString())).max(Double::compareTo).orElse(null);
if (doubleValue != null) {
return doubleValue > threshold ? doubleValue : null;
} else {
return null;
}
} else {
return Double.parseDouble(value.toString()) > threshold ? value : null;
}
case ">=":
if (value instanceof List<?> values) {
Double doubleValue = values.stream().map(v -> Double.valueOf(v.toString())).max(Double::compareTo).orElse(null);
if (doubleValue != null) {
return doubleValue >= threshold ? doubleValue : null;
} else {
return null;
}
} else {
return Double.parseDouble(value.toString()) >= threshold ? value : null;
}
case "<":
if (value instanceof List<?> values) {
Double doubleValue = values.stream().map(v -> Double.valueOf(v.toString())).min(Double::compareTo).orElse(null);
if (doubleValue != null) {
return doubleValue < threshold ? doubleValue : null;
} else {
return null;
}
} else {
return Double.parseDouble(value.toString()) < threshold ? value : null;
}
case "<=":
if (value instanceof List<?> values) {
Double doubleValue = values.stream().map(v -> Double.valueOf(v.toString())).min(Double::compareTo).orElse(null);
if (doubleValue != null) {
return doubleValue <= threshold ? doubleValue : null;
} else {
return null;
}
} else {
return Double.parseDouble(value.toString()) <= threshold ? value : null;
}
case "==":
if (value instanceof List<?> values) {
for (Object v : values) {
if (v.equals(threshold)) {
return v;
}
}
return null;
} else {
return value.equals(threshold) ? value : null;
}
case "!=":
if (value instanceof List<?> values) {
for (Object v : values) {
if (v.equals(threshold)) {
return null;
}
}
return value;
} else {
return value.equals(threshold) ? null : value;
}
default:
// unsupported operator todo add more operator
return null;
}
}
private List<Map<String, Object>> callSqlOrPromql(String text){
private List<Map<String, Object>> callSqlOrPromql(String text) {
String script = text.substring(1, text.length() - 1);
return executor.execute(script);
}
@@ -290,4 +287,36 @@ public class AlertExpressionEvalVisitor extends AlertExpressionBaseVisitor<List<
return key.isEmpty() ? "-" : key;
}
private boolean isScalar(List<Map<String, Object>> context) {
return CollectionUtils.isNotEmpty(context)
&& context.size() == 1
&& context.get(0).containsKey(SCALAR)
&& null != context.get(0).get(SCALAR);
}
private double parseStrToDouble(String text) {
try {
return Double.parseDouble(text);
} catch (NumberFormatException e) {
throw new ExpressionVisitorException("number format exception", e);
}
}
private boolean isValidValue(Object val) {
return val == null || val instanceof List<?>;
}
private Boolean compareOp(Object leftVal, int opType, Object rightVal) {
double left = parseStrToDouble(leftVal.toString());
double right = parseStrToDouble(rightVal.toString());
return switch (opType) {
case AlertExpressionParser.GT -> left > right;
case AlertExpressionParser.GE -> left >= right;
case AlertExpressionParser.LT -> left < right;
case AlertExpressionParser.LE -> left <= right;
case AlertExpressionParser.EQ -> left == right;
case AlertExpressionParser.NE -> left != right;
default -> false;
};
}
}
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -26,12 +26,13 @@ import org.antlr.v4.runtime.tree.ParseTreeVisitor;
* by {@link AlertExpressionParser}.
*
* @param <T> The return type of the visit operation. Use {@link Void} for
* operations with no return type.
* operations with no return type.
*/
public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#expression}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -40,6 +41,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by the {@code AndExpr}
* labeled alternative in {@link AlertExpressionParser#expr}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -48,6 +50,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by the {@code ComparisonExpr}
* labeled alternative in {@link AlertExpressionParser#expr}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -56,6 +59,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by the {@code UnlessExpr}
* labeled alternative in {@link AlertExpressionParser#expr}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -64,6 +68,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by the {@code SqlExpr}
* labeled alternative in {@link AlertExpressionParser#expr}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -72,6 +77,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by the {@code SqlCallExpr}
* labeled alternative in {@link AlertExpressionParser#expr}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -80,6 +86,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by the {@code LiteralExpr}
* labeled alternative in {@link AlertExpressionParser#expr}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -88,6 +95,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by the {@code ParenExpr}
* labeled alternative in {@link AlertExpressionParser#expr}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -96,6 +104,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by the {@code PromqlCallExpr}
* labeled alternative in {@link AlertExpressionParser#expr}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -104,6 +113,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by the {@code PromqlExpr}
* labeled alternative in {@link AlertExpressionParser#expr}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -112,6 +122,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by the {@code OrExpr}
* labeled alternative in {@link AlertExpressionParser#expr}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -119,6 +130,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#functionCall}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -126,6 +138,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#parameterList}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -133,6 +146,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#parameter}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -140,6 +154,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#number}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -147,6 +162,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#string}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -154,6 +170,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#duration}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -161,6 +178,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#functionName}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -168,6 +186,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#selectSql}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -175,6 +194,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#selectFieldList}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -182,6 +202,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#selectField}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -189,6 +210,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#groupByList}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -196,6 +218,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#orderByList}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -203,6 +226,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#orderByField}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -210,6 +234,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#limitClause}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -217,6 +242,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#relList}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -224,6 +250,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#relation}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -231,6 +258,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#conditionList}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -238,6 +266,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#compOp}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -245,6 +274,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#condition}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -252,6 +282,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#conditionUnit}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -259,6 +290,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#promql}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -266,6 +298,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#metricSelector}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -273,6 +306,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#labelMatcherList}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -280,6 +314,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#labelMatcherItem}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -287,6 +322,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#labelMatcherOp}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -294,6 +330,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#labelList}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -301,6 +338,7 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#instantVectorOp}.
*
* @param ctx the parse tree
* @return the visitor result
*/
@@ -308,15 +346,9 @@ public interface AlertExpressionVisitor<T> extends ParseTreeVisitor<T> {
/**
* Visit a parse tree produced by {@link AlertExpressionParser#aggregationOperator}.
*
* @param ctx the parse tree
* @return the visitor result
*/
T visitAggregationOperator(AlertExpressionParser.AggregationOperatorContext ctx);
/**
* Visit a parse tree produced by {@link AlertExpressionParser#binaryOperator}.
* @param ctx the parse tree
* @return the visitor result
*/
T visitBinaryOperator(AlertExpressionParser.BinaryOperatorContext ctx);
}
@@ -23,7 +23,7 @@ expression
expr
: LPAREN expr RPAREN # ParenExpr
| left=expr op=(GE|LE|GT|LT|EQ|NE) right=expr # ComparisonExpr
| left=expr op=(GE|LE|GT|LT|EQ|NE) BOOL? right=expr # ComparisonExpr
| left=expr AND right=expr # AndExpr
| left=expr UNLESS right=expr # UnlessExpr
| left=expr OR right=expr # OrExpr
@@ -191,11 +191,6 @@ aggregationOperator
: SUM | AVG | COUNT | MIN | MAX | STDDEV | STDVAR | TOPK | BOTTOMK | QUANTILE
;
binaryOperator
: EQ | NE | GT | LT | GE | LE
| AND | OR | UNLESS
;
// Lexer rules
// Boolean operators
@@ -259,6 +254,7 @@ LT : '<' ;
LE : '<=' ;
EQ : '==' | '=' ;
NE : '!=' ;
BOOL : 'bool';
// Delimiters
LPAREN : '(' ;
@@ -0,0 +1,147 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.alert.calculate;
import org.apache.hertzbeat.alert.dao.AlertCollectorDao;
import org.apache.hertzbeat.alert.dao.SingleAlertDao;
import org.apache.hertzbeat.alert.reduce.AlarmCommonReduce;
import org.apache.hertzbeat.alert.util.AlertUtil;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.entity.alerter.SingleAlert;
import org.apache.hertzbeat.common.entity.manager.Collector;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.ArgumentCaptor;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.atLeast;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
/**
* Collector alert handler test
*/
@ExtendWith(MockitoExtension.class)
public class CollectorAlertHandlerTest {
@Mock
private AlertCollectorDao alertCollectorDao;
@Mock
private SingleAlertDao singleAlertDao;
@Mock
private AlarmCommonReduce alarmCommonReduce;
private AlarmCacheManager alarmCacheManager;
private CollectorAlertHandler collectorAlertHandler;
@BeforeEach
void setUp() {
alarmCacheManager = Mockito.spy(new AlarmCacheManager(singleAlertDao));
collectorAlertHandler = new CollectorAlertHandler(alarmCommonReduce, alertCollectorDao, alarmCacheManager);
}
@Test
public void testOnline() {
String identity = "localhost-collector";
Collector collector = Collector.builder()
.name(identity)
.ip("127.0.0.1")
.version("1.0")
.build();
SingleAlert singleAlert = SingleAlert.builder().build();
when(alertCollectorDao.findCollectorByName(any(String.class))).thenReturn(collector);
when(alarmCacheManager.removeFiring(any(String.class))).thenReturn(singleAlert);
collectorAlertHandler.online(identity);
assertEquals(CommonConstants.ALERT_STATUS_RESOLVED, singleAlert.getStatus());
verify(alarmCommonReduce, times(1)).reduceAndSendAlarm(any(SingleAlert.class));
}
@Test
public void testOffline() {
String identity = "localhost-collector";
Collector collector = Collector.builder()
.name(identity)
.ip("127.0.0.1")
.version("1.0")
.build();
when(alertCollectorDao.findCollectorByName(any(String.class))).thenReturn(collector);
when(alarmCacheManager.getFiring(any(String.class))).thenReturn(null);
collectorAlertHandler.offline(identity);
verify(alarmCacheManager, times(1)).putFiring(any(String.class), any(SingleAlert.class));
verify(alarmCommonReduce, times(1)).reduceAndSendAlarm(any(SingleAlert.class));
}
@Test
void testOfflineAndOnlineLifecycle() {
String identity = "localhost-collector";
Collector collector = Collector.builder()
.name(identity)
.ip("127.0.0.1")
.version("1.0")
.build();
Map<String, String> fingerPrints = new HashMap<>(8);
fingerPrints.put("collectorName", collector.getName());
fingerPrints.put("collectorVersion", collector.getVersion());
fingerPrints.put("collectorHost", collector.getIp());
String fingerprint = AlertUtil.calculateFingerprint(fingerPrints);
when(alertCollectorDao.findCollectorByName(identity)).thenReturn(collector);
// step1. first time offline
collectorAlertHandler.offline(identity);
assertNotNull(alarmCacheManager.getFiring(fingerprint));
// step2. online
collectorAlertHandler.online(identity);
assertNull(alarmCacheManager.getFiring(fingerprint));
// step3. second time offline
collectorAlertHandler.offline(identity);
assertNotNull(alarmCacheManager.getFiring(fingerprint));
// Verify that the push status is correct.
ArgumentCaptor<SingleAlert> captor = ArgumentCaptor.forClass(SingleAlert.class);
verify(alarmCommonReduce, atLeast(3)).reduceAndSendAlarm(captor.capture());
List<SingleAlert> alerts = captor.getAllValues();
assertEquals(CommonConstants.ALERT_STATUS_FIRING, alerts.get(0).getStatus());
assertEquals(CommonConstants.ALERT_STATUS_RESOLVED, alerts.get(1).getStatus());
assertEquals(CommonConstants.ALERT_STATUS_FIRING, alerts.get(2).getStatus());
}
}
@@ -181,12 +181,10 @@ class AlertExpressionEvalVisitorTest {
List.of(new HashMap<>(Map.of("__value__", List.of(10.0, 20.0, 30.0)))));
// promql
List<Map<String, Object>> result = evaluate("multi_val > 25");
assertEquals(1, result.size());
assertEquals(30.0, result.get(0).get("__value__"));
assertEquals(0, result.size());
// sql
result = evaluate("(select values from multi_metrics where group_id = 'test_group') > 25");
assertEquals(1, result.size());
assertEquals(30.0, result.get(0).get("__value__"));
assertEquals(0, result.size());
}
@Test
@@ -196,12 +194,10 @@ class AlertExpressionEvalVisitorTest {
List.of(new HashMap<>(Map.of("__value__", List.of(10.0, 20.0, 30.0)))));
// promql
List<Map<String, Object>> result = evaluate("multi_val < 15");
assertEquals(1, result.size());
assertEquals(10.0, result.get(0).get("__value__"));
assertEquals(0, result.size());
// sql
result = evaluate("(select response_times from performance_data where service = 'api') < 15");
assertEquals(1, result.size());
assertEquals(10.0, result.get(0).get("__value__"));
assertEquals(0, result.size());
}
@Test
@@ -211,12 +207,10 @@ class AlertExpressionEvalVisitorTest {
List.of(new HashMap<>(Map.of("__value__", List.of()))));
// promql
List<Map<String, Object>> result = evaluate("empty_list > 50");
assertEquals(1, result.size());
assertNull(result.get(0).get("__value__"));
assertEquals(0, result.size());
// sql
result = evaluate("(select error_codes from error_log where date = '2024-01-01') > 50");
assertEquals(1, result.size());
assertNull(result.get(0).get("__value__"));
assertEquals(0, result.size());
}
@Test
@@ -757,6 +751,118 @@ class AlertExpressionEvalVisitorTest {
assertEquals(16, result.get(1).get("__value__"));
}
@Test
void testComparisonExpr() {
String promql = "3 > bool 2";
// scalar and scalar
List<Map<String, Object>> result = evaluate(promql);
assertEquals(1, result.size());
assertEquals(1, result.get(0).get("__value__"));
// scalar and vector
promql = "0 < http_server_requests_seconds_count";
Map<String, Object> vector = new HashMap<>() {
{
put("instance", "host.docker.internal:8989");
put("__value__", 1307);
put("method", "GET");
put("__name__", "http_server_requests_seconds_count");
put("__timestamp__", "1.750320922467E9");
put("uri", "/actuator/prometheus");
put("status", "200");
}
};
when(mockExecutor.execute("http_server_requests_seconds_count")).thenReturn(List.of(vector));
result = evaluate(promql);
assertEquals(1, result.size());
assertEquals(1307, result.get(0).get("__value__"));
// scalar and vector bool
promql = "0 < bool http_server_requests_seconds_count";
// true
result = evaluate(promql);
assertEquals(1, result.size());
assertEquals(1, result.get(0).get("__value__"));
// false
vector.put("__value__", -1);
result = evaluate(promql);
assertEquals(1, result.size());
assertEquals(0, result.get(0).get("__value__"));
// vector and scalar
promql = "http_server_requests_seconds_sum > 0";
vector = new HashMap<>() {
{
put("instance", "host.docker.internal:8989");
put("__value__", 13);
put("method", "GET");
put("__name__", "http_server_requests_seconds_count");
put("__timestamp__", "1.750320922467E9");
put("uri", "/actuator/prometheus");
put("status", "200");
}
};
when(mockExecutor.execute("http_server_requests_seconds_sum")).thenReturn(List.of(vector));
result = evaluate(promql);
assertEquals(1, result.size());
assertEquals(13, result.get(0).get("__value__"));
// vector and scalar bool
promql = "http_server_requests_seconds_sum > bool 0";
// true
result = evaluate(promql);
assertEquals(1, result.size());
assertEquals(1, result.get(0).get("__value__"));
// false
vector.put("__value__", -1);
result = evaluate(promql);
assertEquals(1, result.size());
assertEquals(0, result.get(0).get("__value__"));
// vector and vector
promql = "http_server_requests_seconds_count > http_server_requests_seconds_sum";
Map<String, Object> vectorCount = new HashMap<>() {
{
put("instance", "host.docker.internal:8989");
put("__value__", 13);
put("method", "GET");
put("__name__", "http_server_requests_seconds_count");
put("__timestamp__", "1.750320922467E9");
put("uri", "/actuator/prometheus");
put("status", "200");
}
};
Map<String, Object> vectorSum = new HashMap<>() {
{
put("instance", "host.docker.internal:8989");
put("__value__", 10);
put("method", "GET");
put("__name__", "http_server_requests_seconds_count");
put("__timestamp__", "1.750320922467E9");
put("uri", "/actuator/prometheus");
put("status", "200");
}
};
when(mockExecutor.execute("http_server_requests_seconds_count")).thenReturn(List.of(vectorCount));
when(mockExecutor.execute("http_server_requests_seconds_sum")).thenReturn(List.of(vectorSum));
result = evaluate(promql);
assertEquals(1, result.size());
assertEquals(13, result.get(0).get("__value__"));
// vector and vector bool
promql = "http_server_requests_seconds_count > bool http_server_requests_seconds_sum";
// true
result = evaluate(promql);
assertEquals(1, result.size());
assertEquals(1, result.get(0).get("__value__"));
// false
vectorCount.put("__value__", 1);
result = evaluate(promql);
assertEquals(1, result.size());
assertEquals(0, result.get(0).get("__value__"));
}
private List<Map<String, Object>> evaluate(String expression) {
AlertExpressionLexer lexer = new AlertExpressionLexer(CharStreams.fromString(expression));
@@ -471,19 +471,19 @@ public class JdbcCommonCollect extends AbstractCollect {
// remove special characters
String cleanedUrl = jdbcProtocol.getUrl().replaceAll("[\\x00-\\x1F\\x7F\\xA0]", "");
String url = recursiveDecode(cleanedUrl);
url = url.toLowerCase();
String urlLowerCase = url.toLowerCase();
// url format check
if (!url.matches("^jdbc:[a-zA-Z0-9]+:([^\\s;]+)(;[^\\s;]+)*$")) {
if (!urlLowerCase.matches("^jdbc:[a-zA-Z0-9]+:([^\\s;]+)(;[^\\s;]+)*$")) {
throw new IllegalArgumentException("Invalid JDBC URL format");
}
// backlist check
for (String keyword : BLACK_LIST) {
if (url.contains(keyword.toLowerCase())) {
if (urlLowerCase.contains(keyword.toLowerCase())) {
throw new IllegalArgumentException("Invalid JDBC URL: contains potentially malicious parameter: " + keyword);
}
}
// universal detection
String normalizedUrl = url.replaceAll("[\\x00-\\x1F\\x7F\\xA0]", " ").toLowerCase();
String normalizedUrl = urlLowerCase.replaceAll("[\\x00-\\x1F\\x7F\\xA0]", " ");
// universal detection of JDBC injection and deserialization attacks
if (normalizedUrl.matches(".*jndi\\s*[:=].*")
|| normalizedUrl.matches(".*ldap\\s*[:=].*")
@@ -514,7 +514,7 @@ public class JdbcCommonCollect extends AbstractCollect {
}
}
}
return normalizedUrl;
return url;
}
assert jdbcProtocol.getPlatform() != null;
return switch (jdbcProtocol.getPlatform()) {
@@ -604,6 +604,18 @@ public class HttpCollectImpl extends AbstractCollect {
}
}
builder.addValueRow(valueRowBuilder.build());
} else if (objectValue instanceof Number numberValue) {
CollectRep.ValueRow.Builder valueRowBuilder = CollectRep.ValueRow.newBuilder();
for (String alias : aliasFields) {
if (NetworkConstants.RESPONSE_TIME.equalsIgnoreCase(alias)) {
valueRowBuilder.addColumn(responseTime.toString());
} else if (CollectorConstants.KEYWORD.equalsIgnoreCase(alias)) {
valueRowBuilder.addColumn(Integer.toString(keywordNum));
} else {
valueRowBuilder.addColumn(numberValue.toString());
}
}
builder.addValueRow(valueRowBuilder.build());
}
}
}
@@ -17,11 +17,6 @@
package org.apache.hertzbeat.collector.collect.database;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
import org.apache.hertzbeat.collector.dispatch.DispatchConstants;
import org.apache.hertzbeat.common.entity.job.Metrics;
import org.apache.hertzbeat.common.entity.job.protocol.JdbcProtocol;
@@ -29,6 +24,14 @@ import org.apache.hertzbeat.common.entity.message.CollectRep;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
/**
* Test case for {@link JdbcCommonCollect}
*/
@@ -128,4 +131,81 @@ class JdbcCommonCollectTest {
String protocol = jdbcCommonCollect.supportProtocol();
assertEquals(DispatchConstants.PROTOCOL_JDBC, protocol);
}
@Test
void testUrlPassThrough() {
String[] testUrls = {
"jdbc:mysql://localhost:3306/test?allowPublicKeyRetrieval=true&useSSL=false",
"jdbc:mysql://localhost:3306/test?usessl=false&verifyServerCertificate=true",
"jdbc:mysql://localhost:3306/test?serverTimezone=UTC&autoReconnect=false"
};
JdbcCommonCollect jdbcCollect = new JdbcCommonCollect();
for (String originalUrl : testUrls) {
try {
JdbcProtocol jdbcProtocol = JdbcProtocol.builder()
.host("localhost")
.port("3306")
.platform("mysql")
.username("root")
.password("root")
.database("test")
.url(originalUrl)
.build();
// Use reflection to call constructDatabaseUrl method
Method constructMethod = JdbcCommonCollect.class.getDeclaredMethod("constructDatabaseUrl", JdbcProtocol.class, String.class, String.class);
constructMethod.setAccessible(true);
String processedUrl = (String) constructMethod.invoke(jdbcCollect, jdbcProtocol, "localhost", "3306");
// Verify that the processed URL is the same as the original URL
assertEquals(originalUrl, processedUrl,
"URL should be passed through without modification: " + originalUrl);
} catch (Exception e) {
System.out.println("URL rejected by security validation: " + originalUrl + ", reason: " + e.getMessage());
}
}
}
@Test
void testConstructDatabaseUrlSecurityInterception() {
JdbcCommonCollect jdbcCollect = new JdbcCommonCollect();
String[] maliciousUrls = {
// URL length limit test
"jdbc:mysql://localhost:3306/test?" + "a".repeat(2050) + "=value",
// url format check
"jdbca:mysql://localhost:3306/test?allowLoadLocalInfile=true",
// backlist check
"jdbc:mysql://localhost:3306/test?allowLoadLocalInfile=true",
// universal detection of JDBC injection and deserialization attacks
"jdbc:mysql://localhost:3306/test?jndi:ldap://duansg.com/exploit",
// universal detection of bypass
"jdbc:mysql://localhost:3306/test?param=create\\trigger",
// database platform specific bypass detection
"jdbc:mysql://localhost:3306/test?allow\\nload\\nlocal\\ninfile=true"
};
// Test malicious URLs - should throw exceptions
for (String maliciousUrl : maliciousUrls) {
JdbcProtocol jdbcProtocol = JdbcProtocol.builder()
.host("localhost")
.port("3306")
.platform("mysql")
.username("root")
.password("root")
.database("test")
.url(maliciousUrl)
.build();
assertThrows(Exception.class, () -> {
try {
Method constructMethod = JdbcCommonCollect.class.getDeclaredMethod("constructDatabaseUrl", JdbcProtocol.class, String.class, String.class);
constructMethod.setAccessible(true);
constructMethod.invoke(jdbcCollect, jdbcProtocol, "localhost", "3306");
} catch (InvocationTargetException e) {
throw e.getCause();
}
}, "Malicious URL should be blocked: " + maliciousUrl);
}
}
}
@@ -17,15 +17,6 @@
package org.apache.hertzbeat.collector.collect.http;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertEquals;
import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.List;
import java.util.ArrayList;
import com.google.common.collect.Lists;
import org.apache.hertzbeat.collector.dispatch.DispatchConstants;
import org.apache.hertzbeat.common.entity.job.Metrics;
@@ -34,6 +25,15 @@ import org.apache.hertzbeat.common.entity.message.CollectRep;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
/**
* Test case for {@link HttpCollectImpl}
*/
@@ -186,4 +186,107 @@ class HttpCollectImplTest {
assertEquals("0.0", secondRow.getColumns(2), "Second server CPU should be 0.0");
assertEquals("0", secondRow.getColumns(3), "Second server memory should be 0");
}
@Test
void parseResponseByJsonPath() throws Exception {
String jsonResponse = "{"
+ " \"name\": \"jvm.memory.used\","
+ " \"description\": \"The amount of used memory\","
+ " \"baseUnit\": \"bytes\","
+ " \"measurements\": ["
+ " {"
+ " \"statistic\": \"VALUE\","
+ " \"value\": 90282296"
+ " }"
+ " ],"
+ " \"availableTags\": ["
+ " {"
+ " \"tag\": \"area\","
+ " \"values\": ["
+ " \"heap\","
+ " \"nonheap\""
+ " ]"
+ " },"
+ " {"
+ " \"tag\": \"id\","
+ " \"values\": ["
+ " \"G1 Survivor Space\","
+ " \"G1 Eden Space\""
+ " ]"
+ " }"
+ " ]"
+ "}";
HttpProtocol http = HttpProtocol.builder()
.parseType(DispatchConstants.PARSE_JSON_PATH)
.parseScript("$.availableTags[?(@.tag == \"id\")].values[*]")
.build();
List<CollectRep.ValueRow> capturedRows = new ArrayList<>();
CollectRep.MetricsData.Builder builder = new CollectRep.MetricsData.Builder() {
@Override
public CollectRep.MetricsData.Builder addValueRow(CollectRep.ValueRow valueRow) {
capturedRows.add(valueRow);
return super.addValueRow(valueRow);
}
};
Method parseMethod = HttpCollectImpl.class.getDeclaredMethod(
"parseResponseByJsonPath",
String.class,
List.class,
HttpProtocol.class,
CollectRep.MetricsData.Builder.class,
Long.class);
parseMethod.setAccessible(true);
// Call the method
parseMethod.invoke(httpCollectImpl, jsonResponse, Lists.newArrayList("id"), http, builder, 100L);
// Verify the results
assertEquals(2, capturedRows.size());
CollectRep.ValueRow firstRow = capturedRows.get(0);
assertEquals("G1 Survivor Space", firstRow.getColumns(0));
CollectRep.ValueRow secondRow = capturedRows.get(1);
assertEquals("G1 Eden Space", secondRow.getColumns(0));
// number
String numberJson = "{"
+ " \"name\": \"system.cpu.usage\","
+ " \"description\": \"The \\\"recent cpu usage\\\" of the system the application is running in\","
+ " \"measurements\": ["
+ " {"
+ " \"statistic\": \"VALUE\","
+ " \"value\": 0.268751364291017"
+ " }"
+ " ],"
+ " \"availableTags\": []"
+ "}";
http = HttpProtocol.builder()
.parseType(DispatchConstants.PARSE_JSON_PATH)
.parseScript("$.measurements[?(@.statistic == \"VALUE\")].value")
.build();
capturedRows.clear();
builder = new CollectRep.MetricsData.Builder() {
@Override
public CollectRep.MetricsData.Builder addValueRow(CollectRep.ValueRow valueRow) {
capturedRows.add(valueRow);
return super.addValueRow(valueRow);
}
};
parseMethod = HttpCollectImpl.class.getDeclaredMethod(
"parseResponseByJsonPath",
String.class,
List.class,
HttpProtocol.class,
CollectRep.MetricsData.Builder.class,
Long.class);
parseMethod.setAccessible(true);
// Call the method
parseMethod.invoke(httpCollectImpl, numberJson, Lists.newArrayList("usage"), http, builder, 100L);
// Verify the results
assertEquals(1, capturedRows.size());
firstRow = capturedRows.get(0);
assertEquals("0.268751364291017", firstRow.getColumns(0));
}
}
@@ -70,14 +70,11 @@
<version>${hertzbeat.version}</version>
</dependency>
<!-- spring -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
@@ -87,7 +84,15 @@
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
</dependency>
<!-- metrics -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
</dependencies>
<build>
@@ -23,6 +23,7 @@ import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.collector.dispatch.entrance.internal.CollectJobService;
import org.apache.hertzbeat.collector.metrics.HertzBeatMetricsCollector;
import org.apache.hertzbeat.common.timer.Timeout;
import org.apache.hertzbeat.collector.timer.TimerDispatch;
import org.apache.hertzbeat.collector.timer.WheelTimerTask;
@@ -33,6 +34,7 @@ import org.apache.hertzbeat.common.entity.job.Job;
import org.apache.hertzbeat.common.entity.job.Metrics;
import org.apache.hertzbeat.common.entity.message.CollectRep;
import org.apache.hertzbeat.common.queue.CommonDataQueue;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import java.util.HashMap;
@@ -90,6 +92,9 @@ public class CommonDispatcher implements MetricsTaskDispatch, CollectDataDispatc
private final String collectorIdentity;
@Autowired
private HertzBeatMetricsCollector metricsCollector;
public CommonDispatcher(MetricsCollectorQueue jobRequestQueue,
TimerDispatch timerDispatch,
CommonDataQueue commonDataQueue,
@@ -153,12 +158,23 @@ public class CommonDispatcher implements MetricsTaskDispatch, CollectDataDispatc
for (Map.Entry<String, MetricsTime> entry : metricsTimeoutMonitorMap.entrySet()) {
MetricsTime metricsTime = entry.getValue();
if (metricsTime.getStartTime() < deadline) {
// Metrics collection timeout
// Metrics collection timeout
MetricsTime removedMetricsTime = metricsTimeoutMonitorMap.remove(entry.getKey());
if (removedMetricsTime == null) {
continue;
}
WheelTimerTask timerJob = (WheelTimerTask) metricsTime.getTimeout().task();
Job job = timerJob.getJob();
// timeout metrics
if (metricsCollector != null) {
long duration = System.currentTimeMillis() - removedMetricsTime.getStartTime();
metricsCollector.recordCollectMetrics(job, duration, "timeout");
}
CollectRep.MetricsData metricsData = CollectRep.MetricsData.newBuilder()
.setId(timerJob.getJob().getMonitorId())
.setTenantId(timerJob.getJob().getTenantId())
.setApp(timerJob.getJob().getApp())
.setId(job.getMonitorId())
.setTenantId(job.getTenantId())
.setApp(job.getApp())
.setMetrics(metricsTime.getMetrics().getName())
.setPriority(metricsTime.getMetrics().getPriority())
.setTime(System.currentTimeMillis())
@@ -167,7 +183,6 @@ public class CommonDispatcher implements MetricsTaskDispatch, CollectDataDispatc
if (metricsData.getPriority() == 0) {
dispatchCollectData(metricsTime.timeout, metricsTime.getMetrics(), metricsData);
}
metricsTimeoutMonitorMap.remove(entry.getKey());
}
}
} catch (Exception e) {
@@ -177,7 +192,7 @@ public class CommonDispatcher implements MetricsTaskDispatch, CollectDataDispatc
@Override
public void dispatchMetricsTask(Timeout timeout) {
// Divide the collection task of a single application into corresponding collection tasks of the metrics according to the metrics under it.
// Divide the collection task of a single application into corresponding collection tasks of the metrics under it.
// Put each collect task into the thread pool for scheduling
WheelTimerTask timerTask = (WheelTimerTask) timeout.task();
Job job = timerTask.getJob();
@@ -201,16 +216,27 @@ public class CommonDispatcher implements MetricsTaskDispatch, CollectDataDispatc
public void dispatchCollectData(Timeout timeout, Metrics metrics, CollectRep.MetricsData metricsData) {
WheelTimerTask timerJob = (WheelTimerTask) timeout.task();
Job job = timerJob.getJob();
String monitorKey;
if (metrics.isHasSubTask()) {
monitorKey = job.getId() + "-" + metrics.getName() + "-sub-" + metrics.getSubTaskId();
} else {
monitorKey = job.getId() + "-" + metrics.getName();
}
MetricsTime metricsTime = metricsTimeoutMonitorMap.remove(monitorKey);
// job completed metrics
if (metricsTime != null && metricsCollector != null) {
long duration = System.currentTimeMillis() - metricsTime.getStartTime();
String status = metricsData.getCode() == CollectRep.Code.SUCCESS ? "success" : "fail";
metricsCollector.recordCollectMetrics(job, duration, status);
}
if (metrics.isHasSubTask()) {
metricsTimeoutMonitorMap.remove(job.getId() + "-" + metrics.getName() + "-sub-" + metrics.getSubTaskId());
boolean isLastTask = metrics.consumeSubTaskResponse(metricsData);
if (isLastTask) {
metricsData = metrics.getSubTaskDataRef().get().build();
} else {
return;
}
} else {
metricsTimeoutMonitorMap.remove(job.getId() + "-" + metrics.getName());
}
Set<Metrics> metricsSet = job.getNextCollectMetrics(metrics, false);
if (job.isCyclic()) {
@@ -322,7 +348,13 @@ public class CommonDispatcher implements MetricsTaskDispatch, CollectDataDispatc
public void dispatchCollectData(Timeout timeout, Metrics metrics, List<CollectRep.MetricsData> metricsDataList) {
WheelTimerTask timerJob = (WheelTimerTask) timeout.task();
Job job = timerJob.getJob();
metricsTimeoutMonitorMap.remove(String.valueOf(job.getId()));
MetricsTime metricsTime = metricsTimeoutMonitorMap.remove(String.valueOf(job.getId()));
if (metricsTime != null && metricsCollector != null) {
long duration = System.currentTimeMillis() - metricsTime.getStartTime();
// For a list, we consider it a success if at least one item is successful.
boolean isSuccess = metricsDataList.stream().anyMatch(item -> item.getCode() == CollectRep.Code.SUCCESS);
metricsCollector.recordCollectMetrics(job, duration, isSuccess ? "success" : "fail");
}
if (job.isCyclic()) {
// The collection and execution of all task of this job are completed.
// The periodic task pushes the task to the time wheel again.
@@ -340,7 +372,6 @@ public class CommonDispatcher implements MetricsTaskDispatch, CollectDataDispatc
// and the result listener is notified of the combination of all metrics data
timerDispatch.responseSyncJobData(job.getId(), metricsDataList);
}
}
@@ -354,4 +385,4 @@ public class CommonDispatcher implements MetricsTaskDispatch, CollectDataDispatc
private Metrics metrics;
private Timeout timeout;
}
}
}
@@ -422,6 +422,9 @@ public class MetricsCollect implements Runnable, Comparable<MetricsCollect> {
*/
private Object[] transformCal(String cal, Map<String, String> fieldAliasMap) {
int splitIndex = cal.indexOf("=");
if (splitIndex < 0) {
return null;
}
String field = cal.substring(0, splitIndex).trim();
String expressionStr = cal.substring(splitIndex + 1).trim().replace("\\#", "#");
JexlExpression expression;
@@ -0,0 +1,89 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.apache.hertzbeat.collector.metrics;
import io.micrometer.core.instrument.Counter;
import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.core.instrument.Timer;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.common.entity.job.Job;
import org.springframework.stereotype.Service;
import java.time.Duration;
import java.util.Map;
/**
* Service for managing and recording Micrometer metrics.
* This service uses Micrometer which integrates natively with Spring Boot Actuator.
*/
@Service
@Slf4j
public class HertzBeatMetricsCollector {
private final MeterRegistry meterRegistry;
public HertzBeatMetricsCollector(MeterRegistry meterRegistry) {
this.meterRegistry = meterRegistry;
log.info("MetricsService initialized with MeterRegistry: {}", meterRegistry.getClass().getSimpleName());
}
/**
* Records the metrics for a completed collection sub-task.
*
* @param job The parent job containing monitor info.
* @param durationMillis The duration of the collection task in milliseconds.
* @param status The final status of the collection ("success", "fail", "timeout").
*/
public void recordCollectMetrics(Job job, long durationMillis, String status) {
if (job == null) {
return;
}
Map<String, String> metadata = job.getMetadata();
String monitorName = metadata != null ? metadata.get("instancename") : "unknown";
String monitorTarget = metadata != null ? metadata.get("instancehost") : "unknown";
// Record collection count
Counter.builder("hertzbeat.collect.total")
.description("The total number of collection tasks executed")
.tag("status", status)
.tag("monitor_type", job.getApp())
.tag("monitor_id", String.valueOf(job.getMonitorId()))
.tag("monitor_name", monitorName)
.tag("monitor_target", monitorTarget)
.register(meterRegistry)
.increment();
// Record collection duration
Timer.builder("hertzbeat.collect.duration")
.description("The duration of collection task executions")
.tag("status", status)
.tag("monitor_type", job.getApp())
.tag("monitor_id", String.valueOf(job.getMonitorId()))
.tag("monitor_name", monitorName)
.tag("monitor_target", monitorTarget)
.register(meterRegistry)
.record(Duration.ofMillis(durationMillis));
if (log.isDebugEnabled()) {
log.debug("Recorded metrics for monitor [{}] ({}): status={}, duration={}ms",
monitorName, job.getMonitorId(), status, durationMillis);
}
}
}
@@ -30,7 +30,23 @@ spring:
# need to disable spring boot mongodb auto config, or default mongodb connection tried and failed...
autoconfigure:
exclude: org.springframework.boot.autoconfigure.mongo.MongoAutoConfiguration, org.springframework.boot.autoconfigure.data.mongo.MongoDataAutoConfiguration, org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration, org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration
management:
endpoints:
web:
exposure:
include:
- 'prometheus'
endpoint:
prometheus:
access: read_only
metrics:
tags:
application: ${spring.application.name}
environment: ${spring.profiles.active}
prometheus:
metrics:
export:
enabled: true
---
spring:
config:
@@ -22,8 +22,11 @@ import io.netty.channel.ChannelHandlerContext;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.collector.timer.TimerDispatch;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.entity.dto.ServerInfo;
import org.apache.hertzbeat.common.entity.message.ClusterMsg;
import org.apache.hertzbeat.common.support.SpringContextHolder;
import org.apache.hertzbeat.common.util.AesUtil;
import org.apache.hertzbeat.common.util.JsonUtil;
import org.apache.hertzbeat.remoting.netty.NettyRemotingProcessor;
/**
@@ -40,6 +43,16 @@ public class GoOnlineProcessor implements NettyRemotingProcessor {
if (this.timerDispatch == null) {
this.timerDispatch = SpringContextHolder.getBean(TimerDispatch.class);
}
if (message.getMsg().isEmpty()) {
log.warn("The message that server response to collector is empty, please upgrade server");
} else {
ServerInfo serverInfo = JsonUtil.fromJson(message.getMsg().toStringUtf8(), ServerInfo.class);
if (serverInfo == null || serverInfo.getAesSecret() == null) {
log.warn("The message that server response to collector has not secret empty, please check");
} else {
AesUtil.setDefaultSecretKey(serverInfo.getAesSecret());
}
}
timerDispatch.goOnline();
log.info("receive online message and handle success");
return ClusterMsg.Message.newBuilder()
@@ -19,7 +19,6 @@ package org.apache.hertzbeat.common.config;
import org.apache.hertzbeat.common.constants.ConfigConstants;
import org.apache.hertzbeat.common.constants.SignConstants;
import org.apache.hertzbeat.common.util.AesUtil;
import org.springframework.boot.autoconfigure.AutoConfiguration;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.ComponentScan;
@@ -34,10 +33,4 @@ import org.springframework.context.annotation.ComponentScan;
+ ConfigConstants.FunctionModuleConstants.COMMON)
@EnableConfigurationProperties(CommonProperties.class)
public class CommonConfig {
public CommonConfig(CommonProperties commonProperties) {
if (commonProperties != null && commonProperties.getSecret() != null) {
AesUtil.setDefaultSecretKey(commonProperties.getSecret());
}
}
}
@@ -0,0 +1,25 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.common.constants;
/**
* Enum representing the possible statuses of a collector.
*/
public enum CollectorStatus {
ONLINE, OFFLINE;
}
@@ -372,4 +372,9 @@ public interface CommonConstants {
* status page incident state resolved
*/
byte MONITOR_TYPE_DISCOVERY_AUTO_CREATE = 2;
/**
* JEXL custom function `json`
*/
String JEXL_CUSTOM_JSON_FUNCTION = "json";
}
@@ -0,0 +1,62 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.common.constants;
import java.util.Arrays;
/**
* Jexl keywords enum
*/
public enum JexlKeywordsEnum {
SIZE("size"),
EMPTY("empty"),
NEW("new"),
VAR("var"),
RETURN("return"),
IF("if"),
ELSE("else"),
ELSEIF("elseif"),
WHILE("while"),
DO("do"),
FOR("for"),
CONTINUE("continue"),
BREAK("break"),
TRUE("true"),
FALSE("false"),
NULL("null"),
UNDEFINED("undefined");
private final String keyword;
JexlKeywordsEnum(String keyword) {
this.keyword = keyword;
}
public String getKeyword() {
return keyword;
}
public static boolean match(String word) {
if (word == null || word.trim().isEmpty()) {
return false;
}
return Arrays.stream(values()).anyMatch(t -> t.keyword.equals(word));
}
}
@@ -0,0 +1,40 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.common.entity.dto;
import io.swagger.v3.oas.annotations.media.Schema;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;
/**
* collector info
*/
@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
@Schema(description = "manager server info")
public class ServerInfo {
/**
* the aes secret key value for aes encode decode
*/
private String aesSecret;
}
@@ -0,0 +1,29 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.common.support.exception;
/**
* Expression visitor exception
*/
public class ExpressionVisitorException extends RuntimeException {
public ExpressionVisitorException(String message, Throwable cause) {
super(message, cause);
}
}
@@ -37,7 +37,7 @@ public final class AesUtil {
* Default encryption key The AES encryption key is 16 bits by default.
* If the AES encryption key is larger than or smaller than 16 bits, an error message is displayed
*/
private static final String ENCODE_RULES = "tomSun28HaHaHaHa";
public static final String DEFAULT_ENCODE_RULES = "tomSun28HaHaHaHa";
/**
* Default algorithm
@@ -50,13 +50,17 @@ public final class AesUtil {
* Encryption key The AES encryption key is 16 bits.
* If the AES encryption key is larger than 16 bits, an error message is displayed
*/
private static String secretKey = ENCODE_RULES;
private static String secretKey = DEFAULT_ENCODE_RULES;
private AesUtil() {}
public static void setDefaultSecretKey(String secretKeyNow) {
secretKey = secretKeyNow;
}
public static String getDefaultSecretKey() {
return secretKey;
}
public static String aesEncode(String content) {
return aesEncode(content, secretKey);
@@ -109,9 +113,9 @@ public final class AesUtil {
byte[] byteDecode = getBytes(content, decryptKey);
return new String(byteDecode, StandardCharsets.UTF_8);
} catch (BadPaddingException e) {
if (!ENCODE_RULES.equals(decryptKey)) {
if (!DEFAULT_ENCODE_RULES.equals(decryptKey)) {
log.warn("There has default encode secret encode content, try to decode with default secret key");
return aesDecode(content, ENCODE_RULES);
return aesDecode(content, DEFAULT_ENCODE_RULES);
}
log.error("aes decode content error: {}, please config right common secret key", e.getMessage());
return content;
@@ -19,15 +19,19 @@
package org.apache.hertzbeat.common.util;
import com.fasterxml.jackson.core.type.TypeReference;
import com.google.common.collect.Maps;
import java.nio.charset.StandardCharsets;
import java.util.Map;
import org.apache.commons.jexl3.JexlBuilder;
import org.apache.commons.jexl3.JexlContext;
import org.apache.commons.jexl3.JexlEngine;
import org.apache.commons.jexl3.JexlExpression;
import org.apache.commons.jexl3.JexlFeatures;
import org.apache.commons.jexl3.MapContext;
import org.apache.hertzbeat.common.constants.CommonConstants;
import java.nio.charset.StandardCharsets;
import java.util.Map;
import java.util.function.Function;
/**
* jexl express runner
@@ -41,6 +45,7 @@ public class JexlExpressionRunner {
Map<String, Object> functions = Maps.newLinkedHashMap();
// set the root namespace function
functions.put(null, new JexlCommonFunction());
functions.put(CommonConstants.JEXL_CUSTOM_JSON_FUNCTION, jsonFunction());
ClassLoader classLoader = new ClassLoader() {
@Override
public String getName() {
@@ -78,5 +83,17 @@ public class JexlExpressionRunner {
public static JexlExpression compile(String expression) {
return jexlEngine.createExpression(expression);
}
/**
* custom function `json:apply(xx)`
*
* @return java.util.function.Function
*/
private static Function<String, Map<String, Object>> jsonFunction() {
return json -> {
TypeReference<Map<String, Object>> typeReference = new TypeReference<>() {
};
return JsonUtil.fromJson(json, typeReference);
};
}
}
@@ -90,4 +90,15 @@ class AesUtilTest {
assertFalse(isCiphertext(encryptedText, invalidKey));
}
@Test
void testDefaultKeyCompatibility() {
// Test with default key
String originalText = "This is a secret message";
// encode use default secret key
String encryptedText = aesEncode(originalText, AesUtil.DEFAULT_ENCODE_RULES);
// decode use new secret key
String decryptedText = aesDecode(encryptedText, "newkey1234567890");
// old data can decode with default secret key
assertEquals(originalText, decryptedText);
}
}
@@ -21,6 +21,7 @@ package org.apache.hertzbeat.common.util;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import java.util.HashMap;
@@ -69,4 +70,21 @@ public class JexlExpressionRunnerTest {
contains("Abc", "a")
"""));
}
@Test
void testJsonFunction() {
Map<String, Object> context = new HashMap<String, Object>();
context.put("$.info", "{\"current\":537132,\"grantTotal\":63129,\"instanceCode\":\"CSDZMC\",\"timeMillis\":1754579874357}");
String expression = "json:apply($.info).current";
JexlExpression expObj = JexlExpressionRunner.compile(expression);
assertEquals(537132, JexlExpressionRunner.evaluate(expObj, context));
context.put("$.info", "{\"grantTotal\":63129,\"instanceCode\":\"CSDZMC\",\"timeMillis\":1754579874357}");
assertNull(JexlExpressionRunner.evaluate(expObj, context));
context.clear();
assertNull(JexlExpressionRunner.evaluate(expObj, context));
}
}
@@ -128,11 +128,7 @@ public class DashboardService {
? GrafanaConstants.generateUseDatasource(currentDatasourceName) : "";
String relativeDashboardUrl = grafanaDashboard.getUrl();
if (relativeDashboardUrl != null && grafanaProperties.getUrl() != null && relativeDashboardUrl.startsWith(grafanaProperties.getUrl())) {
relativeDashboardUrl = relativeDashboardUrl.substring(grafanaProperties.getUrl().length());
}
String fullDashboardUrl = grafanaProperties.exposeUrl().replaceAll("/$", "")
+ (relativeDashboardUrl != null ? relativeDashboardUrl.replaceAll("^/", "") : "");
String fullDashboardUrl = grafanaProperties.exposeUrl().replaceAll("/$", "") + relativeDashboardUrl;
grafanaDashboard.setUrl(fullDashboardUrl + KIOSK + REFRESH + INSTANCE + monitorId + useDatasource);
@@ -0,0 +1,141 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.grafana.service;
import org.apache.hertzbeat.common.entity.grafana.GrafanaDashboard;
import org.apache.hertzbeat.grafana.config.GrafanaProperties;
import org.apache.hertzbeat.grafana.dao.DashboardDao;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.ArgumentCaptor;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.client.RestTemplate;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
/**
* Test case for {@link DashboardService}
*/
@ExtendWith(MockitoExtension.class)
public class DashboardServiceTest {
@Mock
private ServiceAccountService serviceAccountService;
@Mock
private DashboardDao dashboardDao;
@Mock
private GrafanaProperties grafanaProperties;
@Mock
private RestTemplate restTemplate;
@Mock
private DatasourceService datasourceService;
@InjectMocks
private DashboardService dashboardService;
static final String GRAFANA_API_RESULT = "{"
+ " \"folderUid\": \"\","
+ " \"id\": 3,"
+ " \"slug\": \"prometheus-dashboard\","
+ " \"status\": \"success\","
+ " \"uid\": \"5d7d89b0-b273-40fe-bb30-d652b82f47eb\","
+ " \"url\": \"/d/5d7d89b0-b273-40fe-bb30-d652b82f47eb/prometheus-dashboard\","
+ " \"version\": 3"
+ "}";
@BeforeEach
void setUp() {
when(datasourceService.getCurrentDatasourceName()).thenReturn("hertzbeat-vm-localhost-8428");
when(grafanaProperties.enabled()).thenReturn(Boolean.TRUE);
when(grafanaProperties.getPrefix()).thenReturn("");
when(grafanaProperties.getUrl()).thenReturn("http://127.0.0.1:3000");
when(grafanaProperties.exposeUrl()).thenReturn("http://127.0.0.1:3000");
when(serviceAccountService.getToken()).thenReturn("test-token");
}
@Test
void testCreateOrUpdateDashboard() {
ResponseEntity<String> responseEntity = new ResponseEntity<>(GRAFANA_API_RESULT, HttpStatus.OK);
when(restTemplate.postForEntity(
eq("http://127.0.0.1:3000/api/dashboards/db"), any(HttpEntity.class), eq(String.class)
)).thenReturn(responseEntity);
ArgumentCaptor<GrafanaDashboard> dashboardCaptor = ArgumentCaptor.forClass(GrafanaDashboard.class);
dashboardService.createOrUpdateDashboard("{\"id\":11}", 1L);
verify(dashboardDao).save(dashboardCaptor.capture());
GrafanaDashboard savedDashboard = dashboardCaptor.getValue();
assertNotNull(savedDashboard);
assertNotNull(savedDashboard.getUrl());
String expectedBaseUrl = "http://127.0.0.1:3000/d/5d7d89b0-b273-40fe-bb30-d652b82f47eb";
// Verify that the URL begins with the expected base URL.
assertTrue(savedDashboard.getUrl().startsWith(expectedBaseUrl), "URL should start with: " + expectedBaseUrl + ", but was: " + savedDashboard.getUrl());
assertTrue(savedDashboard.getUrl().contains("kiosk=tv"), "URL should contain kiosk parameter");
assertTrue(savedDashboard.getUrl().contains("refresh=15s"), "URL should contain refresh parameter");
assertTrue(savedDashboard.getUrl().contains("var-instance=1"), "URL should contain instance parameter");
}
@Test
void testCreateOrUpdateDashboardWithTrailingSlash() {
when(grafanaProperties.exposeUrl()).thenReturn("http://127.0.0.1:3000/");
ResponseEntity<String> responseEntity = new ResponseEntity<>(GRAFANA_API_RESULT, HttpStatus.OK);
when(restTemplate.postForEntity(
eq("http://127.0.0.1:3000/api/dashboards/db"), any(HttpEntity.class), eq(String.class)
)).thenReturn(responseEntity);
ArgumentCaptor<GrafanaDashboard> dashboardCaptor = ArgumentCaptor.forClass(GrafanaDashboard.class);
dashboardService.createOrUpdateDashboard("{\"id\":11}", 1L);
verify(dashboardDao).save(dashboardCaptor.capture());
GrafanaDashboard savedDashboard = dashboardCaptor.getValue();
assertNotNull(savedDashboard);
assertNotNull(savedDashboard.getUrl());
String expectedBaseUrl = "http://127.0.0.1:3000/d/5d7d89b0-b273-40fe-bb30-d652b82f47eb";
// Verify that the URL begins with the expected base URL.
assertTrue(savedDashboard.getUrl().startsWith(expectedBaseUrl), "URL should start with: " + expectedBaseUrl + ", but was: " + savedDashboard.getUrl());
assertTrue(savedDashboard.getUrl().contains("kiosk=tv"), "URL should contain kiosk parameter");
assertTrue(savedDashboard.getUrl().contains("refresh=15s"), "URL should contain refresh parameter");
assertTrue(savedDashboard.getUrl().contains("var-instance=1"), "URL should contain instance parameter");
}
}
+9
View File
@@ -210,6 +210,15 @@
<groupId>org.apache.arrow</groupId>
<artifactId>arrow-memory-netty</artifactId>
</dependency>
<dependency>
<groupId>org.apache.hertzbeat</groupId>
<artifactId>hertzbeat-ai-agent</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
</dependencies>
<build>
@@ -18,6 +18,8 @@
package org.apache.hertzbeat.manager;
import javax.annotation.PostConstruct;
import org.apache.hertzbeat.common.constants.ConfigConstants;
import org.apache.hertzbeat.manager.nativex.HertzbeatRuntimeHintsRegistrar;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
@@ -36,10 +38,10 @@ import org.springframework.scheduling.annotation.EnableScheduling;
@SpringBootApplication
@EnableJpaAuditing
@EnableJpaRepositories(basePackages = {"org.apache.hertzbeat"})
@EntityScan(basePackages = {"org.apache.hertzbeat"})
@ComponentScan(basePackages = {"org.apache.hertzbeat"})
@ConfigurationPropertiesScan(basePackages = {"org.apache.hertzbeat"})
@EnableJpaRepositories(basePackages = {ConfigConstants.PkgConstant.PKG})
@EntityScan(basePackages = {ConfigConstants.PkgConstant.PKG})
@ComponentScan(basePackages = {ConfigConstants.PkgConstant.PKG})
@ConfigurationPropertiesScan(basePackages = {ConfigConstants.PkgConstant.PKG})
@ImportRuntimeHints(HertzbeatRuntimeHintsRegistrar.class)
@EnableAsync
@EnableScheduling
@@ -52,4 +54,4 @@ public class Manager {
public void init() {
System.setProperty("jdk.jndi.object.factoriesFilter", "!com.zaxxer.hikari.HikariJNDIFactory");
}
}
}
@@ -40,7 +40,7 @@ import org.apache.hertzbeat.common.entity.manager.Monitor;
import org.apache.hertzbeat.common.entity.manager.StatusPageComponent;
import org.apache.hertzbeat.common.entity.manager.StatusPageHistory;
import org.apache.hertzbeat.common.entity.manager.StatusPageOrg;
import org.apache.hertzbeat.manager.config.StatusProperties;
import org.apache.hertzbeat.manager.properties.StatusProperties;
import org.apache.hertzbeat.manager.dao.MonitorDao;
import org.apache.hertzbeat.manager.dao.StatusPageComponentDao;
import org.apache.hertzbeat.manager.dao.StatusPageHistoryDao;
@@ -20,14 +20,15 @@ package org.apache.hertzbeat.manager.config;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.usthe.sureness.util.JsonWebTokenUtil;
import jakarta.annotation.Resource;
import java.security.SecureRandom;
import java.text.SimpleDateFormat;
import java.util.Locale;
import java.util.Random;
import java.util.TimeZone;
import lombok.SneakyThrows;
import org.apache.commons.lang3.StringUtils;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.entity.manager.GeneralConfig;
import org.apache.hertzbeat.common.util.AesUtil;
import org.apache.hertzbeat.common.util.TimeZoneUtil;
import org.apache.hertzbeat.base.dao.GeneralConfigDao;
import org.apache.hertzbeat.manager.pojo.dto.MuteConfig;
@@ -62,6 +63,9 @@ public class ConfigInitializer implements SmartLifecycle {
@Value("${sureness.jwt.secret:" + DEFAULT_JWT_SECRET + "}")
private String currentJwtSecret;
@Value("${common.secret:" + AesUtil.DEFAULT_ENCODE_RULES + "}")
private String currentAesSecret;
@Resource
private SystemGeneralConfigServiceImpl systemGeneralConfigService;
@@ -110,30 +114,41 @@ public class ConfigInitializer implements SmartLifecycle {
// for template config, flush the template config in db to memory
TemplateConfig templateConfig = templateConfigService.getConfig();
appService.updateCustomTemplateConfig(templateConfig);
// for system secrets
// for system jwt secrets and aes secret
boolean needUpdate = false;
SystemSecret.SystemSecretBuilder systemSecretBuilder = SystemSecret.builder();
SystemSecret systemSecret = systemSecretService.getConfig();
if (systemSecret != null) {
systemSecretBuilder.jwtSecret(systemSecret.getJwtSecret());
systemSecretBuilder.aesSecret(systemSecret.getAesSecret());
}
if (DEFAULT_JWT_SECRET.equals(currentJwtSecret)) {
// use the random jwt secret
SystemSecret systemSecret = systemSecretService.getConfig();
if (systemSecret == null || StringUtils.isBlank(systemSecret.getJwtSecret())) {
char[] chars = DEFAULT_JWT_SECRET.toCharArray();
Random rand = new Random();
for (int i = 0; i < chars.length; i++) {
int index = rand.nextInt(chars.length);
char temp = chars[i];
chars[i] = chars[index];
chars[index] = temp;
}
currentJwtSecret = new String(chars);
systemSecret = SystemSecret.builder().jwtSecret(currentJwtSecret).build();
systemSecretService.saveConfig(systemSecret);
currentJwtSecret = randomizeSecret(DEFAULT_JWT_SECRET);
systemSecretBuilder.jwtSecret(currentJwtSecret);
needUpdate = true;
} else {
currentJwtSecret = systemSecret.getJwtSecret();
}
}
// else use the user custom jwt secret
// set the jwt secret token in util
// else use the user custom jwt secret, set the jwt secret token in util
JsonWebTokenUtil.setDefaultSecretKey(currentJwtSecret);
// Aes secret config
if (AesUtil.DEFAULT_ENCODE_RULES.equals(currentAesSecret)) {
// use the random aes secret
if (systemSecret == null || StringUtils.isBlank(systemSecret.getAesSecret())) {
currentAesSecret = randomizeSecret(AesUtil.DEFAULT_ENCODE_RULES);
systemSecretBuilder.aesSecret(currentAesSecret);
needUpdate = true;
} else {
currentAesSecret = systemSecret.getAesSecret();
}
}
AesUtil.setDefaultSecretKey(currentAesSecret);
if (needUpdate) {
systemSecretService.saveConfig(systemSecretBuilder.build());
}
// init web-app mute config
MuteConfig muteConfig = muteGeneralConfigService.getConfig();
if (muteConfig == null) {
@@ -162,4 +177,18 @@ public class ConfigInitializer implements SmartLifecycle {
public int getPhase() {
return Ordered.HIGHEST_PRECEDENCE;
}
private String randomizeSecret(String secret) {
SecureRandom random = new SecureRandom();
StringBuilder sb = new StringBuilder(secret.length());
for (int i = 0; i < secret.length(); i++) {
char ch;
do {
int codePoint = random.nextInt('z' - '0' + 1) + '0';
ch = (char) codePoint;
} while (!Character.isLetterOrDigit(ch));
sb.append(ch);
}
return sb.toString();
}
}
@@ -18,8 +18,8 @@
package org.apache.hertzbeat.manager.config;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.manager.scheduler.ConsistentHash;
import org.apache.hertzbeat.manager.scheduler.SchedulerProperties;
import org.apache.hertzbeat.manager.scheduler.ConsistentHashCollectorKeeper;
import org.apache.hertzbeat.manager.properties.SchedulerProperties;
import org.springframework.boot.autoconfigure.AutoConfigureAfter;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@@ -33,8 +33,8 @@ import org.springframework.context.annotation.Configuration;
public class SchedulerConfig {
@Bean
public ConsistentHash consistentHasInstance() {
return new ConsistentHash();
public ConsistentHashCollectorKeeper consistentHasInstance() {
return new ConsistentHashCollectorKeeper();
}
}
@@ -17,7 +17,6 @@
package org.apache.hertzbeat.manager.controller;
import static org.springframework.http.MediaType.APPLICATION_JSON_VALUE;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.tags.Tag;
@@ -25,6 +24,7 @@ import jakarta.annotation.Resource;
import jakarta.validation.constraints.NotNull;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.common.entity.dto.Message;
import org.apache.hertzbeat.common.util.CommonUtil;
import org.apache.hertzbeat.common.util.ResponseUtil;
import org.apache.hertzbeat.manager.pojo.dto.TemplateConfig;
import org.apache.hertzbeat.manager.service.ConfigService;
@@ -37,6 +37,19 @@ import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.time.format.TextStyle;
import java.util.Comparator;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.stream.Collectors;
import static org.springframework.http.MediaType.APPLICATION_JSON_VALUE;
/**
* Alert sender Configuration API
*/
@@ -45,6 +58,9 @@ import org.springframework.web.bind.annotation.RestController;
@Tag(name = "Alert sender Configuration API")
@Slf4j
public class GeneralConfigController {
private static final Set<String> ZONE_IDS = ZoneId.getAvailableZoneIds();
@Resource
private ConfigService configService;
@@ -74,4 +90,28 @@ public class GeneralConfigController {
@RequestBody TemplateConfig.AppTemplate template) {
return ResponseUtil.handle(() -> configService.updateTemplateAppConfig(app, template));
}
@GetMapping(path = "/timezones")
@Operation(summary = "Get all available timezones and their current UTC offset", description = "Get all available timezones and their current UTC offset")
public ResponseEntity<Message<List<Map<String, String>>>> getTimezones() {
List<Map<String, String>> timezones = ZONE_IDS.stream()
.map(id -> {
try {
ZoneId zoneId = ZoneId.of(id);
ZonedDateTime now = ZonedDateTime.now(zoneId);
int totalSeconds = now.getOffset().getTotalSeconds();
String offset = String.format("UTC%+03d:%02d", totalSeconds / 3600, Math.abs((totalSeconds / 60) % 60));
String displayName = zoneId.getDisplayName(TextStyle.FULL, Locale.getDefault());
return Map.of("zoneId", id, "offset", offset, "displayName", displayName);
} catch (Exception e) {
String errorMsg = CommonUtil.getMessageFromThrowable(e);
log.warn("Query Timezone failed. {} ", errorMsg);
return null;
}
})
.filter(t -> Objects.nonNull(t) && Objects.nonNull(t.get("zoneId")))
.sorted(Comparator.comparing(m -> m.get("zoneId")))
.collect(Collectors.toList());
return ResponseEntity.ok(Message.success(timezones));
}
}
@@ -19,7 +19,6 @@
package org.apache.hertzbeat.manager.nativex;
import java.lang.reflect.Constructor;
import java.util.Set;
import org.apache.sshd.common.channel.ChannelListener;
import org.apache.sshd.common.forward.PortForwardingEventListener;
@@ -28,7 +27,6 @@ import org.apache.sshd.common.io.nio2.Nio2ServiceFactoryFactory;
import org.apache.sshd.common.session.SessionListener;
import org.apache.sshd.common.util.security.bouncycastle.BouncyCastleSecurityProviderRegistrar;
import org.apache.sshd.common.util.security.eddsa.EdDSASecurityProviderRegistrar;
import org.springframework.aot.hint.ExecutableMode;
import org.springframework.aot.hint.MemberCategory;
import org.springframework.aot.hint.RuntimeHints;
import org.springframework.aot.hint.RuntimeHintsRegistrar;
@@ -60,11 +58,4 @@ public class HertzbeatRuntimeHintsRegistrar implements RuntimeHintsRegistrar {
TypeReference.of(PortForwardingEventListener.class), TypeReference.of(SessionListener.class));
}
}
private void registerConstructor(RuntimeHints hints, Class<?> clazz) {
Constructor<?>[] declaredConstructors = clazz.getDeclaredConstructors();
for (Constructor<?> declaredConstructor : declaredConstructors) {
hints.reflection().registerConstructor(declaredConstructor, ExecutableMode.INVOKE);
}
}
}
@@ -0,0 +1,156 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.manager.pojo;
import lombok.Data;
import org.apache.hertzbeat.common.constants.CollectorStatus;
import org.apache.hertzbeat.manager.scheduler.AssignJobs;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
/**
* Collector Node
*/
@Data
public class CollectorNode {
/**
* Default number of VM nodes
*/
private static final byte VIRTUAL_NODE_DEFAULT_SIZE = 10;
/**
* collector identity
*/
private final String identity;
/**
* collector mode: public or private
*/
private String mode;
/**
* ip
*/
private String ip;
/**
* collector On-line time stamp
*/
private long uptime;
/**
* collector's own performance service quality score 0 - 127
* The number of virtual nodes will be calculated based on this service quality score
*
*/
private Byte quality;
private CollectorStatus collectorStatus;
/**
* use this collector's collect job ID list
* jobId,jobVersion
*/
private AssignJobs assignJobs;
/**
* the collection task ID list mapped by each virtual node corresponding to this node
* Long[] [0]-jobId, [1]-dispatchHash
*/
private Map<Integer, Set<Long[]>> virtualNodeMap;
public CollectorNode(String identity, String mode, String ip, long uptime, Byte quality) {
this.identity = identity;
this.mode = mode;
this.ip = ip;
this.uptime = uptime;
this.quality = quality;
assignJobs = new AssignJobs();
virtualNodeMap = new ConcurrentHashMap<>(VIRTUAL_NODE_DEFAULT_SIZE);
}
public synchronized void addJob(Integer virtualNodeKey, Integer dispatchHash, Long jobId, boolean isFlushed) {
if (virtualNodeMap == null) {
virtualNodeMap = new ConcurrentHashMap<>(VIRTUAL_NODE_DEFAULT_SIZE);
}
if (assignJobs == null) {
assignJobs = new AssignJobs();
}
Set<Long[]> virtualNodeJob = virtualNodeMap.computeIfAbsent(virtualNodeKey, k -> ConcurrentHashMap.newKeySet(16));
virtualNodeJob.add(new Long[]{jobId, dispatchHash.longValue()});
if (isFlushed) {
assignJobs.addAssignJob(jobId);
} else {
assignJobs.addAddingJob(jobId);
}
}
/**
* obtain the collection task routed by the specified virtual node according to virtualNodeKey
* @param virtualNodeKey virtualNodeKey
* @return collection task
*/
public Set<Long[]> clearVirtualNodeJobs(Integer virtualNodeKey) {
if (virtualNodeMap == null || virtualNodeMap.isEmpty()) {
return null;
}
Set<Long[]> virtualNodeJobs = virtualNodeMap.remove(virtualNodeKey);
virtualNodeMap.put(virtualNodeKey, ConcurrentHashMap.newKeySet(16));
return virtualNodeJobs;
}
public void addVirtualNodeJobs(Integer virtualHashKey, Set<Long[]> reDispatchJobs) {
if (reDispatchJobs == null) {
return;
}
if (virtualNodeMap == null) {
virtualNodeMap = new ConcurrentHashMap<>(16);
}
virtualNodeMap.computeIfPresent(virtualHashKey, (k, v) -> {
reDispatchJobs.addAll(v);
return v;
});
virtualNodeMap.put(virtualHashKey, reDispatchJobs);
}
public void removeVirtualNodeJob(Long jobId) {
if (jobId == null || virtualNodeMap == null) {
return;
}
for (Set<Long[]> jobSet : virtualNodeMap.values()) {
Optional<Long[]> optional = jobSet.stream().filter(item -> Objects.equals(item[0], jobId)).findFirst();
if (optional.isPresent()) {
jobSet.remove(optional.get());
break;
}
}
}
public void destroy() {
if (assignJobs != null) {
assignJobs.clear();
}
if (virtualNodeMap != null) {
virtualNodeMap.clear();
}
}
}
@@ -0,0 +1,53 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.manager.pojo;
import org.apache.hertzbeat.common.entity.job.Job;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
/**
* Utility class for caching {@link Job} objects in memory.
* <p>
* This class provides static methods to store, retrieve, and remove {@code Job} instances
* using a thread-safe {@link ConcurrentHashMap}. It is intended to be used as a simple
* in-memory cache for job data within the manager component.
* <p>
* Usage:
* <pre>
* JobCache.put(job);
* Job job = JobCache.get(jobId);
* JobCache.remove(jobId);
* </pre>
*/
public class JobCache {
private static final Map<Long, Job> jobContentCache = new ConcurrentHashMap<>(16);
public static Job get(Long jobId) {
return jobContentCache.get(jobId);
}
public static void put(Job job) {
jobContentCache.put(job.getId(), job);
}
public static void remove(Long jobId) {
jobContentCache.remove(jobId);
}
}
@@ -35,4 +35,9 @@ public class SystemSecret {
* secret key for jwt
*/
private String jwtSecret;
/**
* secret key for aes
*/
private String aesSecret;
}
@@ -15,7 +15,7 @@
* limitations under the License.
*/
package org.apache.hertzbeat.manager.scheduler;
package org.apache.hertzbeat.manager.properties;
import lombok.Getter;
import lombok.Setter;
@@ -15,7 +15,7 @@
* limitations under the License.
*/
package org.apache.hertzbeat.manager.config;
package org.apache.hertzbeat.manager.properties;
import lombok.Getter;
import lombok.Setter;
@@ -29,6 +29,7 @@ import java.util.concurrent.ConcurrentHashMap;
@Slf4j
@Data
public class AssignJobs {
private static final Integer DEFAULT_CAPACITY = 16;
/**
* current assign jobIds
@@ -51,10 +52,10 @@ public class AssignJobs {
private Set<Long> pinnedJobs;
public AssignJobs() {
jobs = ConcurrentHashMap.newKeySet(16);
addingJobs = ConcurrentHashMap.newKeySet(16);
removingJobs = ConcurrentHashMap.newKeySet(16);
pinnedJobs = ConcurrentHashMap.newKeySet(16);
jobs = ConcurrentHashMap.newKeySet(DEFAULT_CAPACITY);
addingJobs = ConcurrentHashMap.newKeySet(DEFAULT_CAPACITY);
removingJobs = ConcurrentHashMap.newKeySet(DEFAULT_CAPACITY);
pinnedJobs = ConcurrentHashMap.newKeySet(DEFAULT_CAPACITY);
}
public void addAssignJob(Long jobId) {
@@ -29,13 +29,17 @@ import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
import lombok.Setter;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.collections4.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.hertzbeat.collector.dispatch.entrance.internal.CollectJobService;
import org.apache.hertzbeat.collector.dispatch.entrance.internal.CollectResponseEventListener;
import org.apache.hertzbeat.common.constants.CollectorStatus;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.entity.dto.CollectorInfo;
import org.apache.hertzbeat.common.entity.dto.ServerInfo;
import org.apache.hertzbeat.common.entity.job.Configmap;
import org.apache.hertzbeat.common.entity.job.Job;
import org.apache.hertzbeat.common.entity.manager.Collector;
@@ -45,12 +49,17 @@ import org.apache.hertzbeat.common.entity.manager.Param;
import org.apache.hertzbeat.common.entity.manager.ParamDefine;
import org.apache.hertzbeat.common.entity.message.ClusterMsg;
import org.apache.hertzbeat.common.entity.message.CollectRep;
import org.apache.hertzbeat.common.util.AesUtil;
import org.apache.hertzbeat.common.util.JsonUtil;
import org.apache.hertzbeat.common.util.SnowFlakeIdGenerator;
import org.apache.hertzbeat.manager.dao.CollectorDao;
import org.apache.hertzbeat.manager.dao.CollectorMonitorBindDao;
import org.apache.hertzbeat.manager.dao.MonitorDao;
import org.apache.hertzbeat.manager.dao.ParamDao;
import org.apache.hertzbeat.manager.pojo.CollectorNode;
import org.apache.hertzbeat.manager.pojo.JobCache;
import org.apache.hertzbeat.manager.properties.SchedulerProperties;
import org.apache.hertzbeat.manager.scheduler.collector.CollectorKeeper;
import org.apache.hertzbeat.manager.scheduler.netty.ManageServer;
import org.apache.hertzbeat.manager.service.AppService;
import org.springframework.beans.factory.annotation.Autowired;
@@ -63,9 +72,7 @@ import org.springframework.stereotype.Component;
@Component
@AutoConfigureAfter(value = {SchedulerProperties.class})
@Slf4j
public class CollectorJobScheduler implements CollectorScheduling, CollectJobScheduling {
private final Map<Long, Job> jobContentCache = new ConcurrentHashMap<>(16);
public class CollectorJobScheduler implements CollectorOperation, CollectorOperationReceiver, JobOperation {
private final Map<Long, CollectResponseEventListener> eventListeners = new ConcurrentHashMap<>(16);
@@ -75,9 +82,6 @@ public class CollectorJobScheduler implements CollectorScheduling, CollectJobSch
@Autowired
private CollectorMonitorBindDao collectorMonitorBindDao;
@Autowired
private ConsistentHash consistentHash;
@Autowired
private CollectJobService collectJobService;
@@ -90,6 +94,10 @@ public class CollectorJobScheduler implements CollectorScheduling, CollectJobSch
@Autowired
private ParamDao paramDao;
@Autowired
private CollectorKeeper collectorKeeper;
@Setter
private ManageServer manageServer;
@Override
@@ -123,15 +131,18 @@ public class CollectorJobScheduler implements CollectorScheduling, CollectJobSch
.build();
}
collectorDao.save(collector);
ConsistentHash.Node node = new ConsistentHash.Node(identity, collector.getMode(),
collector.getIp(), System.currentTimeMillis(), null);
consistentHash.addNode(node);
reBalanceCollectorAssignJobs();
CollectorNode node = new CollectorNode(identity, collector.getMode(), collector.getIp(), System.currentTimeMillis(), null);
collectorKeeper.addNode(node);
collectorKeeper.changeStatus(identity, CollectorStatus.ONLINE);
collectorKeeper.rebalanceJobs(this::doRebalanceJobs);
// Read database The fixed collection tasks at this collector are delivered
List<CollectorMonitorBind> binds = collectorMonitorBindDao.findCollectorMonitorBindsByCollector(identity);
if (CollectionUtils.isEmpty(binds)){
return;
}
List<Monitor> monitors = monitorDao.findMonitorsByIdIn(binds.stream().map(CollectorMonitorBind::getMonitorId).collect(Collectors.toSet()));
for (Monitor monitor : monitors) {
if (Objects.isNull(monitor) || monitor.getStatus() == CommonConstants.MONITOR_PAUSED_CODE) {
@@ -147,6 +158,9 @@ public class CollectorJobScheduler implements CollectorScheduling, CollectJobSch
appDefine.setDefaultInterval(monitor.getIntervals());
appDefine.setCyclic(true);
appDefine.setTimestamp(System.currentTimeMillis());
Map<String, String> metadata = Map.of(CommonConstants.LABEL_INSTANCE_NAME, monitor.getName(),
CommonConstants.LABEL_INSTANCE_HOST, monitor.getHost());
appDefine.setMetadata(metadata);
List<Param> params = paramDao.findParamsByMonitorId(monitor.getId());
List<Configmap> configmaps = params.stream()
.map(param -> Configmap.builder()
@@ -184,56 +198,10 @@ public class CollectorJobScheduler implements CollectorScheduling, CollectJobSch
}
collector.setStatus(CommonConstants.COLLECTOR_STATUS_OFFLINE);
collectorDao.save(collector);
consistentHash.removeNode(identity);
reBalanceCollectorAssignJobs();
log.info("the collector: {} go offline success.", identity);
}
@Override
public void reBalanceCollectorAssignJobs() {
consistentHash.getAllNodes().entrySet().parallelStream().forEach(entry -> {
String collectorName = entry.getKey();
AssignJobs assignJobs = entry.getValue().getAssignJobs();
if (StringUtils.isBlank(collectorName) || Objects.isNull(assignJobs)) {
return;
}
if (CollectionUtils.isNotEmpty(assignJobs.getAddingJobs())) {
Set<Long> addedJobIds = new HashSet<>(8);
for (Long addingJobId : assignJobs.getAddingJobs()) {
Job job = jobContentCache.get(addingJobId);
if (Objects.isNull(job)) {
log.error("assigning job {} content is null.", addingJobId);
continue;
}
addedJobIds.add(addingJobId);
if (CommonConstants.MAIN_COLLECTOR_NODE.equals(collectorName)) {
collectJobService.addAsyncCollectJob(job);
} else {
ClusterMsg.Message message = ClusterMsg.Message.newBuilder()
.setDirection(ClusterMsg.Direction.REQUEST)
.setType(ClusterMsg.MessageType.ISSUE_CYCLIC_TASK)
.setMsg(ByteString.copyFromUtf8(JsonUtil.toJson(job)))
.build();
this.manageServer.sendMsg(collectorName, message);
}
}
assignJobs.addAssignJobs(addedJobIds);
assignJobs.removeAddingJobs(addedJobIds);
}
if (CollectionUtils.isNotEmpty(assignJobs.getRemovingJobs())) {
if (CommonConstants.MAIN_COLLECTOR_NODE.equals(collectorName)) {
assignJobs.getRemovingJobs().forEach(jobId -> collectJobService.cancelAsyncCollectJob(jobId));
} else {
ClusterMsg.Message message = ClusterMsg.Message.newBuilder()
.setDirection(ClusterMsg.Direction.REQUEST)
.setType(ClusterMsg.MessageType.DELETE_CYCLIC_TASK)
.setMsg(ByteString.copyFromUtf8(JsonUtil.toJson(assignJobs.getRemovingJobs())))
.build();
this.manageServer.sendMsg(collectorName, message);
}
assignJobs.clearRemovingJobs();
}
});
collectorKeeper.changeStatus(identity, CollectorStatus.OFFLINE);
collectorKeeper.rebalanceJobs(this::doRebalanceJobs);
log.info("the collector: {} go offline success.", identity);
}
@Override
@@ -258,9 +226,11 @@ public class CollectorJobScheduler implements CollectorScheduling, CollectJobSch
if (Objects.isNull(collector)) {
return false;
}
ServerInfo serverInfo = ServerInfo.builder().aesSecret(AesUtil.getDefaultSecretKey()).build();
ClusterMsg.Message message = ClusterMsg.Message.newBuilder()
.setType(ClusterMsg.MessageType.GO_ONLINE)
.setDirection(ClusterMsg.Direction.REQUEST)
.setMsg(ByteString.copyFromUtf8(JsonUtil.toJson(serverInfo)))
.setIdentity(identity)
.build();
ClusterMsg.Message response = this.manageServer.sendMsgSync(identity, message);
@@ -278,56 +248,11 @@ public class CollectorJobScheduler implements CollectorScheduling, CollectJobSch
return true;
}
@Override
public List<CollectRep.MetricsData> collectSyncJobData(Job job) {
// todo dispatchKey ip+port or id
String dispatchKey = String.valueOf(job.getMonitorId());
ConsistentHash.Node node = consistentHash.preDispatchJob(dispatchKey);
if (Objects.isNull(node)) {
log.error("there is no collector online to assign job.");
CollectRep.MetricsData metricsData = CollectRep.MetricsData.newBuilder()
.setCode(CollectRep.Code.FAIL)
.setMsg("no collector online to assign job")
.build();
return Collections.singletonList(metricsData);
}
if (CommonConstants.MAIN_COLLECTOR_NODE.equals(node.getIdentity())) {
return collectJobService.collectSyncJobData(job);
} else {
List<CollectRep.MetricsData> metricsData = new LinkedList<>();
CountDownLatch countDownLatch = new CountDownLatch(1);
ClusterMsg.Message message = ClusterMsg.Message.newBuilder()
.setType(ClusterMsg.MessageType.ISSUE_ONE_TIME_TASK)
.setDirection(ClusterMsg.Direction.REQUEST)
.setMsg(ByteString.copyFromUtf8(JsonUtil.toJson(job)))
.build();
boolean result = this.manageServer.sendMsg(node.getIdentity(), message);
if (result) {
CollectResponseEventListener listener = new CollectResponseEventListener() {
@Override
public void response(List<CollectRep.MetricsData> responseMetrics) {
if (responseMetrics != null) {
metricsData.addAll(responseMetrics);
}
countDownLatch.countDown();
}
};
eventListeners.put(job.getMonitorId(), listener);
}
try {
countDownLatch.await(120, TimeUnit.SECONDS);
} catch (Exception e) {
log.info("The sync task runs for 120 seconds with no response and returns");
}
return metricsData;
}
}
@Override
public List<CollectRep.MetricsData> collectSyncJobData(Job job, String collector) {
ConsistentHash.Node node = consistentHash.getNode(collector);
CollectorNode node = StringUtils.isBlank(collector)
? collectorKeeper.determineNode(job.getMonitorId())
: collectorKeeper.getNode(collector);
if (Objects.isNull(node)) {
log.error("there is no collector online to assign job.");
CollectRep.MetricsData metricsData = CollectRep.MetricsData.newBuilder()
@@ -336,9 +261,11 @@ public class CollectorJobScheduler implements CollectorScheduling, CollectJobSch
.build();
return Collections.singletonList(metricsData);
}
if (CommonConstants.MAIN_COLLECTOR_NODE.equals(node.getIdentity())) {
return collectJobService.collectSyncJobData(job);
}
List<CollectRep.MetricsData> metricsData = new LinkedList<>();
ClusterMsg.Message message = ClusterMsg.Message.newBuilder()
.setType(ClusterMsg.MessageType.ISSUE_ONE_TIME_TASK)
@@ -371,25 +298,10 @@ public class CollectorJobScheduler implements CollectorScheduling, CollectJobSch
public long addAsyncCollectJob(Job job, String collector) {
long jobId = SnowFlakeIdGenerator.generateId();
job.setId(jobId);
jobContentCache.put(jobId, job);
ConsistentHash.Node node;
if (StringUtils.isBlank(collector)) {
// todo dispatchKey ip+port or id
String dispatchKey = String.valueOf(job.getMonitorId());
node = consistentHash.dispatchJob(dispatchKey, jobId);
if (node == null) {
log.error("there is no collector online to assign job.");
return jobId;
}
} else {
node = consistentHash.getNode(collector);
if (node == null) {
log.error("there is no collector name: {} online to assign job.", collector);
return jobId;
}
node.getAssignJobs().addPinnedJob(jobId);
}
if (CommonConstants.MAIN_COLLECTOR_NODE.equals(node.getIdentity())) {
CollectorNode collectorNode = collectorKeeper.addJob(job, collector);
if (CommonConstants.MAIN_COLLECTOR_NODE.equals(collectorNode.getIdentity())) {
collectJobService.addAsyncCollectJob(job);
} else {
ClusterMsg.Message message = ClusterMsg.Message.newBuilder()
@@ -397,27 +309,16 @@ public class CollectorJobScheduler implements CollectorScheduling, CollectJobSch
.setDirection(ClusterMsg.Direction.REQUEST)
.setMsg(ByteString.copyFromUtf8(JsonUtil.toJson(job)))
.build();
this.manageServer.sendMsg(node.getIdentity(), message);
this.manageServer.sendMsg(collectorNode.getIdentity(), message);
}
return jobId;
}
@Override
public long updateAsyncCollectJob(Job modifyJob) {
// delete and add
long preJobId = modifyJob.getId();
long newJobId = addAsyncCollectJob(modifyJob, null);
jobContentCache.remove(preJobId);
cancelAsyncCollectJob(preJobId);
return newJobId;
}
@Override
public long updateAsyncCollectJob(Job modifyJob, String collector) {
// delete and add
long preJobId = modifyJob.getId();
long newJobId = addAsyncCollectJob(modifyJob, collector);
jobContentCache.remove(preJobId);
cancelAsyncCollectJob(preJobId);
return newJobId;
}
@@ -427,24 +328,21 @@ public class CollectorJobScheduler implements CollectorScheduling, CollectJobSch
if (jobId == null) {
return;
}
jobContentCache.remove(jobId);
for (ConsistentHash.Node node : consistentHash.getAllNodes().values()) {
AssignJobs assignJobs = node.getAssignJobs();
if (assignJobs.getPinnedJobs().remove(jobId)
|| assignJobs.getJobs().remove(jobId) || assignJobs.getAddingJobs().remove(jobId)) {
node.removeVirtualNodeJob(jobId);
if (CommonConstants.MAIN_COLLECTOR_NODE.equals(node.getIdentity())) {
collectJobService.cancelAsyncCollectJob(jobId);
} else {
ClusterMsg.Message deleteMessage = ClusterMsg.Message.newBuilder()
.setType(ClusterMsg.MessageType.DELETE_CYCLIC_TASK)
.setDirection(ClusterMsg.Direction.REQUEST)
.setMsg(ByteString.copyFromUtf8(JsonUtil.toJson(List.of(jobId))))
.build();
this.manageServer.sendMsg(node.getIdentity(), deleteMessage);
}
// break; if is there jod exist in multi collector?
}
CollectorNode collectorNode = collectorKeeper.removeJob(jobId);
if (collectorNode == null) {
return;
}
if (CommonConstants.MAIN_COLLECTOR_NODE.equals(collectorNode.getIdentity())) {
collectJobService.cancelAsyncCollectJob(jobId);
} else {
ClusterMsg.Message deleteMessage = ClusterMsg.Message.newBuilder()
.setType(ClusterMsg.MessageType.DELETE_CYCLIC_TASK)
.setDirection(ClusterMsg.Direction.REQUEST)
.setMsg(ByteString.copyFromUtf8(JsonUtil.toJson(List.of(jobId))))
.build();
this.manageServer.sendMsg(collectorNode.getIdentity(), deleteMessage);
}
}
@@ -461,7 +359,55 @@ public class CollectorJobScheduler implements CollectorScheduling, CollectJobSch
}
}
public void setManageServer(ManageServer manageServer) {
this.manageServer = manageServer;
private void doRebalanceJobs(AssignJobs assignJobs, String collectorName) {
handleAddingJobs(assignJobs, collectorName);
handleRemovingJobs(assignJobs, collectorName);
}
private void handleAddingJobs(AssignJobs assignJobs, String collectorName) {
if (CollectionUtils.isEmpty(assignJobs.getAddingJobs())) {
return;
}
Set<Long> addedJobIds = new HashSet<>(8);
for (Long addingJobId : assignJobs.getAddingJobs()) {
Job job = JobCache.get(addingJobId);
if (Objects.isNull(job)) {
log.error("assigning job {} content is null.", addingJobId);
continue;
}
addedJobIds.add(addingJobId);
if (CommonConstants.MAIN_COLLECTOR_NODE.equals(collectorName)) {
collectJobService.addAsyncCollectJob(job);
} else {
ClusterMsg.Message message = ClusterMsg.Message.newBuilder()
.setDirection(ClusterMsg.Direction.REQUEST)
.setType(ClusterMsg.MessageType.ISSUE_CYCLIC_TASK)
.setMsg(ByteString.copyFromUtf8(JsonUtil.toJson(job)))
.build();
this.manageServer.sendMsg(collectorName, message);
}
}
assignJobs.addAssignJobs(addedJobIds);
assignJobs.removeAddingJobs(addedJobIds);
}
private void handleRemovingJobs(AssignJobs assignJobs, String collectorName) {
if (CollectionUtils.isEmpty(assignJobs.getRemovingJobs())) {
return;
}
if (CommonConstants.MAIN_COLLECTOR_NODE.equals(collectorName)) {
assignJobs.getRemovingJobs().forEach(jobId -> collectJobService.cancelAsyncCollectJob(jobId));
} else {
ClusterMsg.Message message = ClusterMsg.Message.newBuilder()
.setDirection(ClusterMsg.Direction.REQUEST)
.setType(ClusterMsg.MessageType.DELETE_CYCLIC_TASK)
.setMsg(ByteString.copyFromUtf8(JsonUtil.toJson(assignJobs.getRemovingJobs())))
.build();
this.manageServer.sendMsg(collectorName, message);
}
assignJobs.clearRemovingJobs();
}
}
@@ -0,0 +1,46 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.manager.scheduler;
/**
* Interface defining operations for managing collector.
* Implementations of this interface provide functionality to control the operation state
* of collectors in the system.
*/
public interface CollectorOperation {
/**
* Takes a collector offline by stopping its collection operations.
* This is typically used for maintenance, updates, or when the collector is no longer needed.
*
* @param identity The unique identifier of the collector to be taken offline
* @return true if the collector was successfully taken offline,
* false if the operation failed or the collector wasn't found
*/
boolean offlineCollector(String identity);
/**
* Brings a collector online by starting its collection operations.
* This is used to activate a collector that was previously offline.
*
* @param identity The unique identifier of the collector to be brought online
* @return true if the collector was successfully brought online,
* false if the operation failed or the collector wasn't found
*/
boolean onlineCollector(String identity);
}
@@ -20,39 +20,25 @@ package org.apache.hertzbeat.manager.scheduler;
import org.apache.hertzbeat.common.entity.dto.CollectorInfo;
/**
* slave collector service
* Interface defining operations for receiving collector status updates from remote collectors.
* This interface serves as a callback mechanism for handling collector online/offline events.
*/
public interface CollectorScheduling {
public interface CollectorOperationReceiver {
/**
* register collector go online
* @param identity collector identity name
* @param collectorInfo collector information
* Notifies the system when a collector comes online.
* This method should be called when a collector establishes connection and becomes available.
*
* @param identity The unique identifier of the collector (e.g., hostname, IP, or custom ID)
* @param collectorInfo Detailed information about the collector including capabilities,
* configuration, and status metadata
*/
void collectorGoOnline(String identity, CollectorInfo collectorInfo);
/**
* register collector go offline
* @param identity collector identity name
*/
void collectorGoOffline(String identity);
/**
* reBalance dispatch monitoring jobs when collector go online or offline or timeout
*/
void reBalanceCollectorAssignJobs();
/**
* offline collector(stop collector collect operation)
* @param identity collector identity name
* @return true/false
*/
boolean offlineCollector(String identity);
/**
* online collector(start collector collect operation)
* @param identity collector identity name
* @return true/false
* Notifies the system when a collector goes offline.
* This method should be called when a collector disconnects or becomes unavailable.
*
* @param identity The unique identifier of the collector to be marked as offline
*/
boolean onlineCollector(String identity);
void collectorGoOffline(String identity);
}
@@ -23,46 +23,149 @@ import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.BiConsumer;
import java.util.stream.Collectors;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.hertzbeat.common.constants.CollectorStatus;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.entity.job.Job;
import org.apache.hertzbeat.manager.pojo.CollectorNode;
import org.apache.hertzbeat.manager.pojo.JobCache;
import org.apache.hertzbeat.manager.scheduler.collector.CollectorKeeper;
/**
* Collector and task mapping scheduling implemented by consistent hashing
*/
@Slf4j
public class ConsistentHash {
public class ConsistentHashCollectorKeeper implements CollectorKeeper {
/**
* consistent hash circle
*/
private final ConcurrentTreeMap<Integer, Node> hashCircle;
private final ConcurrentTreeMap<Integer, CollectorNode> hashCircle = new ConcurrentTreeMap<>();
/**
* collector node
*/
private final Map<String, Node> existNodeMap;
private final Map<String, CollectorNode> existNodeMap = new ConcurrentHashMap<>(16);
/**
* not dispatched job cache
* not dispatched job cache, in order to obtain the cached collection scheduling task
*/
private final List<DispatchJob> dispatchJobCache;
@Getter
private final List<DispatchJob> dispatchJobCache = Collections.synchronizedList(new LinkedList<>());
/**
* Default number of VM nodes
*/
private static final byte VIRTUAL_NODE_DEFAULT_SIZE = 10;
public ConsistentHash() {
hashCircle = new ConcurrentTreeMap<>();
existNodeMap = new ConcurrentHashMap<>(16);
dispatchJobCache = Collections.synchronizedList(new LinkedList<>());
/**
* add collector node
* @param newNode node
*/
@Override
public void addNode(CollectorNode newNode) {
// when mode is cluster public, need reBalance dispatch jobs. else not when is cloud-edge private
if (!CommonConstants.MODE_PRIVATE.equals(newNode.getMode())) {
byte virtualNodeNum = newNode.getQuality() == null ? VIRTUAL_NODE_DEFAULT_SIZE : newNode.getQuality();
for (byte i = 0; i < virtualNodeNum; i++) {
addVirtualNode(newNode, newNode.getIdentity() + i);
}
}
existNodeMap.put(newNode.getIdentity(), newNode);
dispatchJobInCache();
}
@Override
public CollectorNode addJob(Job job, String collectorId) {
JobCache.put(job);
CollectorNode collectorNode;
if (StringUtils.isBlank(collectorId)) {
// todo dispatchKey ip+port or id
String dispatchKey = String.valueOf(job.getMonitorId());
collectorNode = this.dispatchJob(dispatchKey, job.getId());
if (collectorNode == null) {
log.error("there is no collector online to assign job.");
}
} else {
collectorNode = getNode(collectorId);
if (collectorNode == null) {
log.error("there is no collector name: {} online to assign job.", collectorId);
return null;
}
collectorNode.getAssignJobs().addPinnedJob(job.getId());
}
return collectorNode;
}
/**
* get node
* @param collectorName collector name
* @return node
*/
@Override
public CollectorNode getNode(String collectorName) {
return existNodeMap.get(collectorName);
}
@Override
public CollectorNode determineNode(Long jobId) {
String dispatchKey = String.valueOf(jobId);
if (dispatchKey == null || StringUtils.isBlank(dispatchKey)) {
log.error("The dispatch key can not null.");
return null;
}
int dispatchHash = hash(dispatchKey);
return preDispatchJob(dispatchHash);
}
@Override
public void changeStatus(String collectorId, CollectorStatus collectorStatus) {
switch (collectorStatus) {
case ONLINE -> this.getNode(collectorId).setCollectorStatus(collectorStatus);
case OFFLINE -> this.removeNode(collectorId);
default -> {}
}
}
@Override
public void rebalanceJobs(BiConsumer<AssignJobs, String> assignJobCollectorConsumer) {
existNodeMap.entrySet().parallelStream().forEach(entry -> {
String collectorName = entry.getKey();
AssignJobs assignJobs = entry.getValue().getAssignJobs();
if (StringUtils.isBlank(collectorName) || Objects.isNull(assignJobs)) {
return;
}
assignJobCollectorConsumer.accept(assignJobs, collectorName);
});
}
@Override
public CollectorNode removeJob(Long jobId) {
JobCache.remove(jobId);
for (CollectorNode node : existNodeMap.values()) {
AssignJobs assignJobs = node.getAssignJobs();
if (assignJobs.getPinnedJobs().remove(jobId)
|| assignJobs.getJobs().remove(jobId) || assignJobs.getAddingJobs().remove(jobId)) {
node.removeVirtualNodeJob(jobId);
return node;
// break; if is there jod exist in multi collector?
}
}
return null;
}
/**
@@ -70,15 +173,15 @@ public class ConsistentHash {
* @param newNode node
* @param identity virtual node identity
*/
public synchronized void addVirtualNode(Node newNode, String identity){
private synchronized void addVirtualNode(CollectorNode newNode, String identity){
int virtualHashKey = hash(identity);
hashCircle.put(virtualHashKey, newNode);
newNode.addVirtualNodeJobs(virtualHashKey, ConcurrentHashMap.newKeySet(16));
Map.Entry<Integer, Node> higherVirtualNode = hashCircle.higherOrFirstEntry(virtualHashKey);
Map.Entry<Integer, CollectorNode> higherVirtualNode = hashCircle.higherOrFirstEntry(virtualHashKey);
// Reassign tasks that are routed to the higherVirtualNode virtual node
// Tasks are either on the original virtual node or on the new virtual node
Integer higherVirtualNodeKey = higherVirtualNode.getKey();
Node higherNode = higherVirtualNode.getValue();
CollectorNode higherNode = higherVirtualNode.getValue();
Set<Long[]> dispatchJobs = higherNode.clearVirtualNodeJobs(higherVirtualNodeKey);
if (dispatchJobs != null && !dispatchJobs.isEmpty()) {
Set<Long[]> reDispatchJobs = ConcurrentHashMap.newKeySet(dispatchJobs.size());
@@ -91,53 +194,37 @@ public class ConsistentHash {
iterator.remove();
}
}
higherNode.virtualNodeMap.put(higherVirtualNodeKey, dispatchJobs);
higherNode.getVirtualNodeMap().put(higherVirtualNodeKey, dispatchJobs);
Set<Long> jobIds = reDispatchJobs.stream().map(item -> item[0]).collect(Collectors.toSet());
newNode.addVirtualNodeJobs(virtualHashKey, reDispatchJobs);
if (higherNode != newNode) {
higherNode.assignJobs.removeAssignJobs(jobIds);
higherNode.assignJobs.addRemovingJobs(jobIds);
newNode.assignJobs.addAddingJobs(jobIds);
higherNode.getAssignJobs().removeAssignJobs(jobIds);
higherNode.getAssignJobs().addRemovingJobs(jobIds);
newNode.getAssignJobs().addAddingJobs(jobIds);
}
}
}
/**
* add collector node
* @param newNode node
*/
public void addNode(Node newNode) {
// when mode is cluster public, need reBalance dispatch jobs. else not when is cloud-edge private
if (!CommonConstants.MODE_PRIVATE.equals(newNode.mode)) {
byte virtualNodeNum = newNode.quality == null ? VIRTUAL_NODE_DEFAULT_SIZE : newNode.quality;
for (byte i = 0; i < virtualNodeNum; i++) {
addVirtualNode(newNode, newNode.identity + i);
}
}
existNodeMap.put(newNode.identity, newNode);
dispatchJobInCache();
}
/**
* remove virtual node
* @param deletedNode node
* @param virtualNodeHash virtual node hash key
*/
public synchronized void removeVirtualNode(Node deletedNode, Integer virtualNodeHash) {
Set<Long[]> removeJobHashSet = deletedNode.virtualNodeMap.get(virtualNodeHash);
private synchronized void removeVirtualNode(CollectorNode deletedNode, Integer virtualNodeHash) {
Set<Long[]> removeJobHashSet = deletedNode.getVirtualNodeMap().get(virtualNodeHash);
// Migrate the virtualNodeEntry collection task to the nearest virtual node that is larger than it
hashCircle.remove(virtualNodeHash);
if (removeJobHashSet == null || removeJobHashSet.isEmpty()) {
return;
}
Map.Entry<Integer, Node> higherVirtualEntry = hashCircle.higherOrFirstEntry(virtualNodeHash);
Map.Entry<Integer, CollectorNode> higherVirtualEntry = hashCircle.higherOrFirstEntry(virtualNodeHash);
if (higherVirtualEntry == null || higherVirtualEntry.getValue() == deletedNode) {
higherVirtualEntry = null;
}
// jobId
Set<Long> removeJobIds = removeJobHashSet.stream().map(item -> item[0]).collect(Collectors.toSet());
deletedNode.assignJobs.removeAssignJobs(removeJobIds);
deletedNode.assignJobs.addRemovingJobs(removeJobIds);
deletedNode.getAssignJobs().removeAssignJobs(removeJobIds);
deletedNode.getAssignJobs().addRemovingJobs(removeJobIds);
if (higherVirtualEntry == null) {
// jobId-dispatchHash
removeJobHashSet.forEach(value -> {
@@ -150,9 +237,9 @@ public class ConsistentHash {
}
});
} else {
Node higherVirtualNode = higherVirtualEntry.getValue();
CollectorNode higherVirtualNode = higherVirtualEntry.getValue();
higherVirtualNode.addVirtualNodeJobs(higherVirtualEntry.getKey(), removeJobHashSet);
higherVirtualNode.assignJobs.addAddingJobs(removeJobIds);
higherVirtualNode.getAssignJobs().addAddingJobs(removeJobIds);
}
}
@@ -160,20 +247,19 @@ public class ConsistentHash {
* deleted collector node
* @param name collector name
*/
public Node removeNode(String name) {
Node deletedNode = existNodeMap.remove(name);
private void removeNode(String name) {
CollectorNode deletedNode = existNodeMap.remove(name);
if (deletedNode == null) {
return null;
}
for (Integer virtualNodeHash : deletedNode.virtualNodeMap.keySet()) {
removeVirtualNode(deletedNode, virtualNodeHash);
return;
}
deletedNode.getVirtualNodeMap().keySet()
.forEach(virtualNodeHash -> removeVirtualNode(deletedNode, virtualNodeHash));
deletedNode.destroy();
dispatchJobInCache();
return deletedNode;
}
public synchronized void dispatchJobInCache() {
private synchronized void dispatchJobInCache() {
if (!dispatchJobCache.isEmpty()) {
int size = dispatchJobCache.size();
for (int index = 0; index < size; index++) {
@@ -183,31 +269,6 @@ public class ConsistentHash {
}
}
/**
* get all collector nodes
* @return nodes
*/
public Map<String, Node> getAllNodes() {
return existNodeMap;
}
/**
* get node
* @param collectorName collector name
* @return node
*/
public Node getNode(String collectorName) {
return existNodeMap.get(collectorName);
}
/**
* Obtain the cached collection scheduling task
* @return cache task
*/
public List<DispatchJob> getDispatchJobCache() {
return dispatchJobCache;
}
/**
* obtain the collector node according to the collection task information
*
@@ -215,7 +276,7 @@ public class ConsistentHash {
* @param jobId jobId
* @return collector node
*/
public Node dispatchJob(String dispatchKey, Long jobId) {
private CollectorNode dispatchJob(String dispatchKey, Long jobId) {
if (dispatchKey == null || StringUtils.isBlank(dispatchKey)) {
log.error("The dispatch key can not null.");
return null;
@@ -224,38 +285,23 @@ public class ConsistentHash {
return dispatchJob(dispatchHash, jobId, true);
}
/**
* The collector node to which the collector is assigned is obtained in advance based on the collection task information
*
* @param dispatchKey collector task route key: ip+appId
* @return collector node
*/
public Node preDispatchJob(String dispatchKey) {
if (dispatchKey == null || StringUtils.isBlank(dispatchKey)) {
log.error("The dispatch key can not null.");
return null;
}
int dispatchHash = hash(dispatchKey);
return preDispatchJob(dispatchHash);
}
/**
* Obtain the collector node to which the collector is assigned based on the collection task information
*
* @param dispatchHash The task route hash is collected
* @param jobId jobId
* @param isFlushed is has flush this job or wait to dispatch
* @param isFlushed if it has flushed this job or wait to dispatch
* @return collector node
*/
public Node dispatchJob(Integer dispatchHash, Long jobId, boolean isFlushed) {
private CollectorNode dispatchJob(Integer dispatchHash, Long jobId, boolean isFlushed) {
if (dispatchHash == null || hashCircle == null || hashCircle.isEmpty()) {
log.warn("There is no available collector registered. Cache the job {}.", jobId);
dispatchJobCache.add(new DispatchJob(dispatchHash, jobId));
return null;
}
Map.Entry<Integer, Node> ceilEntry = hashCircle.ceilingOrFirstEntry(dispatchHash);
Map.Entry<Integer, CollectorNode> ceilEntry = hashCircle.ceilingOrFirstEntry(dispatchHash);
int virtualKey = ceilEntry.getKey();
Node curNode = ceilEntry.getValue();
CollectorNode curNode = ceilEntry.getValue();
curNode.addJob(virtualKey, dispatchHash, jobId, isFlushed);
return curNode;
@@ -267,25 +313,15 @@ public class ConsistentHash {
* @param dispatchHash The task route hash is collected
* @return collector node
*/
public Node preDispatchJob(Integer dispatchHash) {
private CollectorNode preDispatchJob(Integer dispatchHash) {
if (dispatchHash == null || hashCircle == null || hashCircle.isEmpty()) {
log.warn("There is no available collector registered.");
return null;
}
Map.Entry<Integer, Node> ceilEntry = hashCircle.ceilingOrFirstEntry(dispatchHash);
Map.Entry<Integer, CollectorNode> ceilEntry = hashCircle.ceilingOrFirstEntry(dispatchHash);
return ceilEntry.getValue();
}
/**
* hash long
* @param key long value
* @return hash value
*/
private int hash(long key) {
String keyStr = String.valueOf(key);
return hash(keyStr);
}
/**
* FNV1_32_HASH algorithm
* @param key the key
@@ -314,7 +350,7 @@ public class ConsistentHash {
* dispatch job summary
*/
@AllArgsConstructor
public static class DispatchJob {
private static class DispatchJob {
/**
* dispatch task route key
@@ -327,130 +363,4 @@ public class ConsistentHash {
@Getter
private Long jobId;
}
/**
* collector node machine address
*/
public static class Node {
/**
* collector identity
*/
@Getter
private final String identity;
/**
* collector mode: public or private
*/
private final String mode;
/**
* ip
*/
private final String ip;
/**
* collector On-line time stamp
*/
private final long uptime;
/**
* collector's own performance service quality score 0 - 127
* The number of virtual nodes will be calculated based on this service quality score
*
*/
private final Byte quality;
/**
* use this collector's collect job ID list
* jobId,jobVersion
*/
private AssignJobs assignJobs;
/**
* the collection task ID list mapped by each virtual node corresponding to this node
* Long[] [0]-jobId, [1]-dispatchHash
*/
private Map<Integer, Set<Long[]>> virtualNodeMap;
public Node(String identity, String mode, String ip, long uptime, Byte quality) {
this.identity = identity;
this.mode = mode;
this.ip = ip;
this.uptime = uptime;
this.quality = quality;
assignJobs = new AssignJobs();
virtualNodeMap = new ConcurrentHashMap<>(VIRTUAL_NODE_DEFAULT_SIZE);
}
private synchronized void addJob(Integer virtualNodeKey, Integer dispatchHash, Long jobId, boolean isFlushed) {
if (virtualNodeMap == null) {
virtualNodeMap = new ConcurrentHashMap<>(VIRTUAL_NODE_DEFAULT_SIZE);
}
if (assignJobs == null) {
assignJobs = new AssignJobs();
}
Set<Long[]> virtualNodeJob = virtualNodeMap.computeIfAbsent(virtualNodeKey, k -> ConcurrentHashMap.newKeySet(16));
virtualNodeJob.add(new Long[]{jobId, dispatchHash.longValue()});
if (isFlushed) {
assignJobs.addAssignJob(jobId);
} else {
assignJobs.addAddingJob(jobId);
}
}
/**
* obtain the collection task routed by the specified virtual node according to virtualNodeKey
* @param virtualNodeKey virtualNodeKey
* @return collection task
*/
private Set<Long[]> clearVirtualNodeJobs(Integer virtualNodeKey) {
if (virtualNodeMap == null || virtualNodeMap.isEmpty()) {
return null;
}
Set<Long[]> virtualNodeJobs = virtualNodeMap.remove(virtualNodeKey);
virtualNodeMap.put(virtualNodeKey, ConcurrentHashMap.newKeySet(16));
return virtualNodeJobs;
}
private void addVirtualNodeJobs(Integer virtualHashKey, Set<Long[]> reDispatchJobs) {
if (reDispatchJobs == null) {
return;
}
if (virtualNodeMap == null) {
virtualNodeMap = new ConcurrentHashMap<>(16);
}
virtualNodeMap.computeIfPresent(virtualHashKey, (k, v) -> {
reDispatchJobs.addAll(v);
return v;
});
virtualNodeMap.put(virtualHashKey, reDispatchJobs);
}
public void removeVirtualNodeJob(Long jobId) {
if (jobId == null || virtualNodeMap == null) {
return;
}
for (Set<Long[]> jobSet : virtualNodeMap.values()) {
Optional<Long[]> optional = jobSet.stream().filter(item -> Objects.equals(item[0], jobId)).findFirst();
if (optional.isPresent()) {
jobSet.remove(optional.get());
break;
}
}
}
public AssignJobs getAssignJobs() {
return assignJobs;
}
public void destroy() {
if (assignJobs != null) {
assignJobs.clear();
}
if (virtualNodeMap != null) {
virtualNodeMap.clear();
}
}
}
}
@@ -24,15 +24,8 @@ import org.apache.hertzbeat.common.entity.message.CollectRep;
/**
* Collection job management provides api interface
*/
public interface CollectJobScheduling {
public interface JobOperation {
/**
* Execute a one-time collection task and get the collected data response
* @param job Collect task details
* @return Collection results
*/
List<CollectRep.MetricsData> collectSyncJobData(Job job);
/**
* Execute a one-time collection task and get the collected data response
* @param job Collect task details
@@ -49,13 +42,6 @@ public interface CollectJobScheduling {
*/
long addAsyncCollectJob(Job job, String collector);
/**
* Update the periodic asynchronous collection tasks that have been delivered
* @param modifyJob Collect task details
* @return long Job ID
*/
long updateAsyncCollectJob(Job modifyJob);
/**
* Update the periodic asynchronous collection tasks that have been delivered
* @param modifyJob Collect task details
@@ -52,10 +52,10 @@ import org.springframework.util.StringUtils;
public class SchedulerInit implements CommandLineRunner {
@Autowired
private CollectorScheduling collectorScheduling;
private CollectorOperationReceiver collectorOperationReceiver;
@Autowired
private CollectJobScheduling collectJobScheduling;
private JobOperation jobOperation;
private static final String MAIN_COLLECTOR_NODE_IP = "127.0.0.1";
private static final String DEFAULT_COLLECTOR_VERSION = "DEBUG";
@@ -91,7 +91,7 @@ public class SchedulerInit implements CommandLineRunner {
.ip(MAIN_COLLECTOR_NODE_IP)
.version(DEFAULT_COLLECTOR_VERSION)
.build();
collectorScheduling.collectorGoOnline(CommonConstants.MAIN_COLLECTOR_NODE, collectorInfo);
collectorOperationReceiver.collectorGoOnline(CommonConstants.MAIN_COLLECTOR_NODE, collectorInfo);
// init jobs
List<Monitor> monitors = monitorDao.findMonitorsByStatusNotInAndJobIdNotNull(List.of(CommonConstants.MONITOR_PAUSED_CODE));
List<CollectorMonitorBind> monitorBinds = collectorMonitorBindDao.findAll();
@@ -136,7 +136,7 @@ public class SchedulerInit implements CommandLineRunner {
});
appDefine.setConfigmap(configmaps);
String collector = monitorIdCollectorMap.get(monitor.getId());
long jobId = collectJobScheduling.addAsyncCollectJob(appDefine, collector);
long jobId = jobOperation.addAsyncCollectJob(appDefine, collector);
monitor.setJobId(jobId);
monitorDao.save(monitor);
} catch (Exception e) {
@@ -0,0 +1,82 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.manager.scheduler.collector;
import org.apache.hertzbeat.common.constants.CollectorStatus;
import org.apache.hertzbeat.common.entity.job.Job;
import org.apache.hertzbeat.manager.pojo.CollectorNode;
import org.apache.hertzbeat.manager.scheduler.AssignJobs;
import java.util.function.BiConsumer;
/**
* Interface for managing collector nodes and their associated jobs.
* Maintains all collector information and provides operations for managing collectors and job assignments.
*/
public interface CollectorKeeper {
/**
* Adds a new collector node to the keeper's management pool.
* @param newNode The collector node to be added to the management system
*/
void addNode(CollectorNode newNode);
/**
* Assigns a monitoring job to a specific collector node.
* @param job The monitoring job to be assigned
* @param collectorId The unique identifier of the target collector node
* @return The collector node that received the job assignment
*/
CollectorNode addJob(Job job, String collectorId);
/**
* Retrieves a collector node by its unique identifier.
* @param collectorId The unique identifier of the collector node
* @return The collector node matching the given ID, or null if not found
*/
CollectorNode getNode(String collectorId);
/**
* Determines the most appropriate collector node for a given job based on scheduling logic.
* @param jobId The unique identifier of the job to be assigned
* @return The collector node selected to handle this job
*/
CollectorNode determineNode(Long jobId);
/**
* Updates the operational status of a collector node.
* @param collectorId The unique identifier of the collector node
* @param collectorStatus The new status to assign to the collector
*/
void changeStatus(String collectorId, CollectorStatus collectorStatus);
/**
* Rebalances job assignments across collector nodes, typically triggered by status changes.
* Uses a callback mechanism to handle job reassignments.
* @param assignJobCollectorConsumer A biconsumer that handles the job reassignment process,
* taking the job assignment logic and collector ID as parameters
*/
void rebalanceJobs(BiConsumer<AssignJobs, String> assignJobCollectorConsumer);
/**
* Removes a job from whichever collector node it is currently assigned to.
* @param jobId The unique identifier of the job to be removed
* @return The collector node from which the job was removed, or null if job wasn't found
*/
CollectorNode removeJob(Long jobId);
}
@@ -28,7 +28,7 @@ import org.apache.hertzbeat.alert.calculate.CollectorAlertHandler;
import org.apache.hertzbeat.common.entity.message.ClusterMsg;
import org.apache.hertzbeat.common.support.CommonThreadPool;
import org.apache.hertzbeat.manager.scheduler.CollectorJobScheduler;
import org.apache.hertzbeat.manager.scheduler.SchedulerProperties;
import org.apache.hertzbeat.manager.properties.SchedulerProperties;
import org.apache.hertzbeat.manager.scheduler.netty.process.CollectCyclicDataResponseProcessor;
import org.apache.hertzbeat.manager.scheduler.netty.process.CollectCyclicServiceDiscoveryDataResponseProcessor;
import org.apache.hertzbeat.manager.scheduler.netty.process.CollectOneTimeDataResponseProcessor;
@@ -45,6 +45,8 @@ import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;
import javax.annotation.PreDestroy;
/**
* manage server
*/
@@ -112,6 +114,7 @@ public class ManageServer implements CommandLineRunner {
}, 10, 3, TimeUnit.SECONDS);
}
@PreDestroy
public void shutdown() {
this.remotingServer.shutdown();
@@ -17,12 +17,15 @@
package org.apache.hertzbeat.manager.scheduler.netty.process;
import com.google.protobuf.ByteString;
import io.netty.channel.ChannelHandlerContext;
import java.net.InetSocketAddress;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.hertzbeat.common.entity.dto.CollectorInfo;
import org.apache.hertzbeat.common.entity.dto.ServerInfo;
import org.apache.hertzbeat.common.entity.message.ClusterMsg;
import org.apache.hertzbeat.common.util.AesUtil;
import org.apache.hertzbeat.common.util.JsonUtil;
import org.apache.hertzbeat.manager.scheduler.netty.ManageServer;
import org.apache.hertzbeat.remoting.netty.NettyRemotingProcessor;
@@ -52,6 +55,12 @@ public class CollectorOnlineProcessor implements NettyRemotingProcessor {
}
this.manageServer.addChannel(collector, ctx.channel());
this.manageServer.getCollectorAndJobScheduler().collectorGoOnline(collector, collectorInfo);
return null;
ServerInfo serverInfo = ServerInfo.builder().aesSecret(AesUtil.getDefaultSecretKey()).build();
return ClusterMsg.Message.newBuilder()
.setIdentity(message.getIdentity())
.setDirection(ClusterMsg.Direction.RESPONSE)
.setMsg(ByteString.copyFromUtf8(JsonUtil.toJson(serverInfo)))
.setType(ClusterMsg.MessageType.GO_ONLINE)
.build();
}
}
@@ -41,6 +41,7 @@ import org.apache.commons.io.IOUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.hertzbeat.collector.dispatch.DispatchConstants;
import org.apache.hertzbeat.collector.util.CollectUtil;
import org.apache.hertzbeat.common.constants.JexlKeywordsEnum;
import org.apache.hertzbeat.common.entity.job.Configmap;
import org.apache.hertzbeat.common.entity.job.Job;
import org.apache.hertzbeat.common.entity.job.Metrics;
@@ -458,6 +459,10 @@ public class AppServiceImpl implements AppService, InitializingBean {
throw new IllegalArgumentException(app.getApp() + " " + metrics.getName() + " "
+ field.getField() + " can not duplicated.");
}
if (JexlKeywordsEnum.match(field.getField())) {
throw new IllegalArgumentException(app.getApp() + " " + metrics.getName() + " "
+ field.getField() + " prohibited keywords.");
}
fieldsSet.add(field.getField());
}
}
@@ -518,16 +523,15 @@ public class AppServiceImpl implements AppService, InitializingBean {
} else {
if (objectStoreConfig.getType() == ObjectStoreDTO.Type.OBS) {
appDefineStore = new ObjectStoreAppDefineStoreImpl();
} else if (objectStoreConfig.getType() == ObjectStoreDTO.Type.DATABASE){
} else if (objectStoreConfig.getType() == ObjectStoreDTO.Type.DATABASE) {
appDefineStore = new DatabaseAppDefineStoreImpl();
} else {
appDefineStore = new LocalFileAppDefineStoreImpl();
}
}
var success = appDefineStore.loadAppDefines();
if (!success) {
new JarAppDefineStoreImpl().loadAppDefines();
}
jarAppDefineStore.loadAppDefines();
// merge define yml files inside jars
appDefineStore.loadAppDefines();
}
private interface AppDefineStore {
@@ -709,7 +713,7 @@ public class AppServiceImpl implements AppService, InitializingBean {
}
}
});
return false;
return true;
}
@Override
@@ -769,8 +773,7 @@ public class AppServiceImpl implements AppService, InitializingBean {
appDefines.put(define.getApp().toLowerCase(), app);
}
}
// merge define yml files inside jars
return false;
return true;
}
@Override
@@ -31,8 +31,9 @@ import org.apache.hertzbeat.common.support.exception.CommonException;
import org.apache.hertzbeat.common.util.IpDomainUtil;
import org.apache.hertzbeat.manager.dao.CollectorDao;
import org.apache.hertzbeat.manager.dao.CollectorMonitorBindDao;
import org.apache.hertzbeat.manager.pojo.CollectorNode;
import org.apache.hertzbeat.manager.scheduler.AssignJobs;
import org.apache.hertzbeat.manager.scheduler.ConsistentHash;
import org.apache.hertzbeat.manager.scheduler.ConsistentHashCollectorKeeper;
import org.apache.hertzbeat.manager.scheduler.netty.ManageServer;
import org.apache.hertzbeat.manager.service.CollectorService;
import org.springframework.beans.factory.annotation.Autowired;
@@ -57,7 +58,7 @@ public class CollectorServiceImpl implements CollectorService {
private CollectorMonitorBindDao collectorMonitorBindDao;
@Autowired
private ConsistentHash consistentHash;
private ConsistentHashCollectorKeeper consistentHashCollectorKeeper;
@Autowired(required = false)
private ManageServer manageServer;
@@ -81,7 +82,7 @@ public class CollectorServiceImpl implements CollectorService {
List<CollectorSummary> collectorSummaryList = new LinkedList<>();
for (Collector collector : collectors.getContent()) {
CollectorSummary.CollectorSummaryBuilder summaryBuilder = CollectorSummary.builder().collector(collector);
ConsistentHash.Node node = consistentHash.getNode(collector.getName());
CollectorNode node = consistentHashCollectorKeeper.getNode(collector.getName());
if (node != null && node.getAssignJobs() != null) {
AssignJobs assignJobs = node.getAssignJobs();
summaryBuilder.pinMonitorNum(assignJobs.getPinnedJobs().size());
@@ -67,7 +67,7 @@ public class LabelServiceImpl implements LabelService {
public void modifyLabel(Label label) {
Optional<Label> optional = labelDao.findById(label.getId());
if (optional.isPresent()) {
Optional<Label> existOptional = labelDao.findLabelByNameAndTagValue(label.getName(), label.getTagValue());
if (existOptional.isPresent() && !existOptional.get().getId().equals(label.getId())) {
throw new IllegalArgumentException("The label with same key and value already exists.");
@@ -116,33 +116,38 @@ public class LabelServiceImpl implements LabelService {
@Override
public void deleteLabels(HashSet<Long> ids) {
if (CollectionUtils.isEmpty(ids)){
if (CollectionUtils.isEmpty(ids)) {
return;
}
labelDao.deleteLabelsByIdIn(ids);
}
public List<Label> determineNewLabels(Set<Map.Entry<String, String>> originLabels){
@Override
public List<Label> determineNewLabels(Set<Map.Entry<String, String>> originLabels) {
if (originLabels == null || originLabels.isEmpty()) return List.of();
// Get all labels from the database
Set<Map.Entry<String, String>> allLabels = labelDao.findAll().stream()
.map(label -> Map.entry(label.getName(), label.getTagValue()))
Set<Label> allLabels = labelDao.findAll().stream()
.map(label -> Label.builder()
.name(label.getName())
.tagValue(label.getTagValue())
.build())
.collect(Collectors.toSet());
// If the bound label (key:value) does not exist, then add it
Set<Map.Entry<String, String>> addLabelsKv = originLabels.stream()
return originLabels.stream()
.map(entry -> Label.builder()
.name(entry.getKey())
.tagValue(entry.getValue())
.build())
.filter(label -> !allLabels.contains(label))
.collect(Collectors.toCollection(HashSet::new));
return addLabelsKv.stream().map(kv -> {
Label label = new Label();
label.setId(null);
label.setName(kv.getKey());
label.setTagValue(kv.getValue());
label.setType((byte) 0);
return label;
}).toList();
.map(label -> Label.builder()
.id(null)
.name(label.getName())
.tagValue(label.getTagValue())
.type((byte) 0)
.build())
.toList();
}
}
@@ -27,6 +27,7 @@ import org.apache.hertzbeat.alert.dao.AlertDefineBindDao;
import org.apache.hertzbeat.collector.dispatch.DispatchConstants;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.constants.ExportFileConstants;
import org.apache.hertzbeat.common.constants.JexlKeywordsEnum;
import org.apache.hertzbeat.common.constants.NetworkConstants;
import org.apache.hertzbeat.common.constants.SignConstants;
import org.apache.hertzbeat.common.entity.grafana.GrafanaDashboard;
@@ -59,7 +60,7 @@ import org.apache.hertzbeat.manager.dao.MonitorDao;
import org.apache.hertzbeat.manager.dao.ParamDao;
import org.apache.hertzbeat.manager.pojo.dto.AppCount;
import org.apache.hertzbeat.manager.pojo.dto.MonitorDto;
import org.apache.hertzbeat.manager.scheduler.CollectJobScheduling;
import org.apache.hertzbeat.manager.scheduler.JobOperation;
import org.apache.hertzbeat.manager.service.AppService;
import org.apache.hertzbeat.manager.service.ImExportService;
import org.apache.hertzbeat.manager.service.LabelService;
@@ -112,7 +113,7 @@ public class MonitorServiceImpl implements MonitorService {
@Autowired
private AppService appService;
@Autowired
private CollectJobScheduling collectJobScheduling;
private JobOperation jobOperation;
@Autowired
private MonitorDao monitorDao;
@Autowired
@@ -192,11 +193,10 @@ public class MonitorServiceImpl implements MonitorService {
return new Configmap(param.getField(), param.getParamValue(), param.getType());
}).collect(Collectors.toList());
appDefine.setConfigmap(configmaps);
long jobId = collector == null ? collectJobScheduling.addAsyncCollectJob(appDefine, null) :
collectJobScheduling.addAsyncCollectJob(appDefine, collector);
try {
detectMonitor(monitor, params, collector);
} catch (Exception ignored) {}
long jobId = jobOperation.addAsyncCollectJob(appDefine, collector);
detectMonitorSafely(monitor, params, collector);
try {
if (collector != null) {
@@ -216,7 +216,7 @@ public class MonitorServiceImpl implements MonitorService {
paramDao.saveAll(params);
} catch (Exception e) {
log.error("Error while adding monitor: {}", e.getMessage(), e);
collectJobScheduling.cancelAsyncCollectJob(jobId);
jobOperation.cancelAsyncCollectJob(jobId);
throw new MonitorDatabaseException(e.getMessage());
}
}
@@ -436,6 +436,27 @@ public class MonitorServiceImpl implements MonitorService {
}
}
}
checkJobFields(monitorDto.getMonitor().getApp());
}
private void checkJobFields(String app) {
if (null == app || app.trim().isEmpty()) {
return;
}
Job job = appService.getAppDefine(app);
if (null != job && !CollectionUtils.isEmpty(job.getMetrics())) {
for (Metrics metrics : job.getMetrics()) {
if (null == metrics.getFields() || metrics.getFields().isEmpty()) {
continue;
}
for (Metrics.Field field : metrics.getFields()) {
if (JexlKeywordsEnum.match(field.getField())) {
throw new IllegalArgumentException(job.getApp() + " " + metrics.getName() + " "
+ field.getField() + " prohibited keywords, please modify the template information.");
}
}
}
}
}
@Override
@@ -488,18 +509,11 @@ public class MonitorServiceImpl implements MonitorService {
List<Configmap> configmaps = params.stream().map(param ->
new Configmap(param.getField(), param.getParamValue(), param.getType())).collect(Collectors.toList());
appDefine.setConfigmap(configmaps);
long newJobId;
if (collector == null) {
newJobId = collectJobScheduling.updateAsyncCollectJob(appDefine);
} else {
newJobId = collectJobScheduling.updateAsyncCollectJob(appDefine, collector);
}
long newJobId = jobOperation.updateAsyncCollectJob(appDefine, collector);
monitor.setJobId(newJobId);
// execute only in non paused status
try {
detectMonitor(monitor, params, collector);
} catch (Exception ignored) {}
detectMonitorSafely(monitor, params, collector);
}
// After the update is successfully released, refresh the database
@@ -526,7 +540,7 @@ public class MonitorServiceImpl implements MonitorService {
} catch (Exception e) {
log.error(e.getMessage(), e);
// Repository brushing abnormally cancels the previously delivered task
collectJobScheduling.cancelAsyncCollectJob(monitor.getJobId());
jobOperation.cancelAsyncCollectJob(monitor.getJobId());
throw new MonitorDatabaseException(e.getMessage());
}
}
@@ -556,7 +570,7 @@ public class MonitorServiceImpl implements MonitorService {
for (Monitor monitor : monitors) {
monitorBindDao.deleteByMonitorId(monitor.getId());
collectorMonitorBindDao.deleteCollectorMonitorBindsByMonitorId(monitor.getId());
collectJobScheduling.cancelAsyncCollectJob(monitor.getJobId());
jobOperation.cancelAsyncCollectJob(monitor.getJobId());
applicationContext.publishEvent(new MonitorDeletedEvent(applicationContext, monitor.getId()));
}
}
@@ -670,17 +684,17 @@ public class MonitorServiceImpl implements MonitorService {
// The jobId is not deleted, and the jobId is reused again after the management is started.
Set<Long> subMonitorIds = monitorBindDao.findMonitorBindsByBizIdIn(ids).stream().map(MonitorBind::getMonitorId).collect(Collectors.toSet());
ids.addAll(subMonitorIds);
List<Monitor> managedMonitors = monitorDao.findMonitorsByIdIn(ids)
.stream().filter(monitor ->
monitor.getStatus() != CommonConstants.MONITOR_PAUSED_CODE)
List<Monitor> managedMonitors = monitorDao.findMonitorsByIdIn(ids).stream()
.filter(monitor -> monitor.getStatus() != CommonConstants.MONITOR_PAUSED_CODE)
.peek(monitor -> monitor.setStatus(CommonConstants.MONITOR_PAUSED_CODE))
.collect(Collectors.toList());
if (!CollectionUtils.isEmpty(managedMonitors)) {
for (Monitor monitor : managedMonitors) {
collectJobScheduling.cancelAsyncCollectJob(monitor.getJobId());
}
monitorDao.saveAll(managedMonitors);
if (CollectionUtils.isEmpty(managedMonitors)) {
return;
}
managedMonitors.forEach(monitor -> jobOperation.cancelAsyncCollectJob(monitor.getJobId()));
monitorDao.saveAll(managedMonitors);
}
@Override
@@ -688,9 +702,8 @@ public class MonitorServiceImpl implements MonitorService {
// Update monitoring status Add corresponding monitoring periodic task
Set<Long> subMonitorIds = monitorBindDao.findMonitorBindsByBizIdIn(ids).stream().map(MonitorBind::getMonitorId).collect(Collectors.toSet());
ids.addAll(subMonitorIds);
List<Monitor> unManagedMonitors = monitorDao.findMonitorsByIdIn(ids)
.stream().filter(monitor ->
monitor.getStatus() == CommonConstants.MONITOR_PAUSED_CODE)
List<Monitor> unManagedMonitors = monitorDao.findMonitorsByIdIn(ids).stream()
.filter(monitor -> monitor.getStatus() == CommonConstants.MONITOR_PAUSED_CODE)
.peek(monitor -> monitor.setStatus(CommonConstants.MONITOR_UP_CODE))
.collect(Collectors.toList());
if (unManagedMonitors.isEmpty()) {
@@ -735,13 +748,11 @@ public class MonitorServiceImpl implements MonitorService {
Optional<CollectorMonitorBind> bindOptional =
collectorMonitorBindDao.findCollectorMonitorBindByMonitorId(monitor.getId());
String collector = bindOptional.map(CollectorMonitorBind::getCollector).orElse(null);
long newJobId = collectJobScheduling.addAsyncCollectJob(appDefine, collector);
long newJobId = jobOperation.addAsyncCollectJob(appDefine, collector);
monitor.setJobId(newJobId);
applicationContext.publishEvent(new MonitorDeletedEvent(applicationContext, monitor.getId()));
try {
detectMonitor(monitor, params, collector);
} catch (Exception ignored) {
}
detectMonitorSafely(monitor, params, collector);
}
monitorDao.saveAll(unManagedMonitors);
}
@@ -830,7 +841,7 @@ public class MonitorServiceImpl implements MonitorService {
// if is pinned collector
String collector = monitorIdCollectorMap.get(monitor.getId());
// Delivering a collection task
long newJobId = collectJobScheduling.updateAsyncCollectJob(appDefine, collector);
long newJobId = jobOperation.updateAsyncCollectJob(appDefine, collector);
monitor.setJobId(newJobId);
monitorDao.save(monitor);
} catch (Exception e) {
@@ -910,12 +921,8 @@ public class MonitorServiceImpl implements MonitorService {
new Configmap(param.getField(), param.getParamValue(), param.getType())).collect(Collectors.toList());
appDefine.setConfigmap(configmaps);
appDefine.setSd(true);
List<CollectRep.MetricsData> collectRep;
if (collector != null) {
collectRep = collectJobScheduling.collectSyncJobData(appDefine, collector);
} else {
collectRep = collectJobScheduling.collectSyncJobData(appDefine);
}
List<CollectRep.MetricsData> collectRep = jobOperation.collectSyncJobData(appDefine, collector);
monitor.setStatus(CommonConstants.MONITOR_UP_CODE);
// If the detection result fails, a detection exception is thrown
if (collectRep == null || collectRep.isEmpty()) {
@@ -953,12 +960,7 @@ public class MonitorServiceImpl implements MonitorService {
List<Metrics> availableMetrics = appDefine.getMetrics().stream()
.filter(item -> item.getPriority() == 0).collect(Collectors.toList());
appDefine.setMetrics(availableMetrics);
List<CollectRep.MetricsData> collectRep;
if (collector != null) {
collectRep = collectJobScheduling.collectSyncJobData(appDefine, collector);
} else {
collectRep = collectJobScheduling.collectSyncJobData(appDefine);
}
List<CollectRep.MetricsData> collectRep = jobOperation.collectSyncJobData(appDefine, collector);
monitor.setStatus(CommonConstants.MONITOR_UP_CODE);
// If the detection result fails, a detection exception is thrown
@@ -972,4 +974,10 @@ public class MonitorServiceImpl implements MonitorService {
}
collectRep.forEach(CollectRep.MetricsData::close);
}
private void detectMonitorSafely(Monitor monitor, List<Param> params, String collector) {
try {
detectMonitor(monitor, params, collector);
} catch (Exception ignored) {}
}
}
@@ -22,6 +22,7 @@ spring:
url: jdbc:h2:./data/test;MODE=MYSQL
hikari:
max-lifetime: 120000
jpa:
show-sql: false
database-platform: org.eclipse.persistence.platform.database.MySQLPlatform
@@ -30,8 +31,10 @@ spring:
eclipselink:
logging:
level: SEVERE
flyway:
enabled: false
mail:
host: smtp.exmail.qq.com
username: example@tancloud.cn
@@ -55,26 +58,57 @@ warehouse:
jpa:
enabled: true
expire-time: 1h
max-history-record-num: 6000
victoria-metrics:
enabled: false
url: http://localhost:8428
username: root
password: root
insert:
buffer-size: 1000
buffer-size: 100
flush-interval: 3
compression:
enabled: false
cluster:
enabled: false
select:
url: http://localhost:8481
username: root
password: root
insert:
url: http://localhost:8480
username: root
password: root
buffer-size: 1000
flush-interval: 3
td-engine:
enabled: false
driver-class-name: com.taosdata.jdbc.rs.RestfulDriver
url: jdbc:TAOS-RS://127.0.0.1:6041/hertzbeat
username: root
password: taosdata
greptime:
enabled: false
grpc-endpoints: localhost:4001
http-endpoint: http://localhost:4000
database: public
username: greptime
password: greptime
iot-db:
enabled: false
host: 127.0.0.1
rpc-port: 6667
username: root
password: root
query-timeout-in-ms: -1
expire-time: '7776000000'
influxdb:
enabled: false
server-url: http://127.0.0.1:8086
username: root
password: root
expire-time: '30d'
replication: 1
# store real-time metrics data, enable only one below
real-time:
memory:
@@ -92,6 +126,58 @@ warehouse:
# redis db index, default: DB0
db: 0
alerter:
# custom console url
console-url: https://console.tancloud.io
# we work
we-work-webhook-url: https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=
# ding ding talk
ding-talk-webhook-url: https://oapi.dingtalk.com/robot/send?access_token=
# fei shu fly book
fly-book-webhook-url: https://open.feishu.cn/open-apis/bot/v2/hook/
# telegram
telegram-webhook-url: https://api.telegram.org/bot%s/sendMessage
# discord
discord-webhook-url: https://discord.com/api/v9/channels/%s/messages
# serverChan
server-chan-webhook-url: https://sctapi.ftqq.com/%s.send
# gotify
gotify-webhook-url: http://127.0.0.1/message?token=%s
# alert inhibit ttl unit ms, default 14400000(4 hours)
inhibit:
ttl: 14400000
sms:
enable: false
type: tencent
tencent:
secret-id:
secret-key:
app-id:
sign-name:
template-id:
alibaba:
access-key-id:
access-key-secret:
sign-name:
template-code:
unisms:
# auth-mode: simple or hmac
auth-mode: simple
access-key-id: YOUR_ACCESS_KEY_ID
# hmac mode need to fill in access-key-secret
access-key-secret: YOUR_ACCESS_KEY_SECRET
signature: YOUR_SMS_SIGNATURE
template-id: YOUR_TEMPLATE_ID
smslocal:
api-key: YOUR_API_KEY_HERE
aws:
access-key-id: YOUR_ACCESS_KEY_ID
access-key-secret: YOUR_ACCESS_KEY_SECRET
region: AWS_REGION_FOR_END_USER_MESSAGING
twilio:
account-sid: YOUR_ACCOUNT_SID
auth-token: YOUR_AUTH_TOKEN
twilio-phone-number: YOUR_TWILIO_PHONE_NUMBER
scheduler:
server:
enabled: true
@@ -100,13 +186,13 @@ scheduler:
grafana:
enabled: false
url: http://127.0.0.1:3000
expose-url: http://127.0.0.1:3000
username: admin
password: admin
# AI config
# See the documentation for details : https://hertzbeat.apache.org/zh-cn/docs/help/aiConfig
ai:
# AI Type:zhiPu、alibabaAi、kimiAi、sparkDesk
# AI Type:zhiPu、alibabaAi、kimiAi、sparkDesk、ollama、openRouter
type:
# Model nameglm-4、qwen-turboo、moonshot-v1-8k、generalv3.5
model:
@@ -114,3 +200,5 @@ ai:
api-key:
#At present, only IFLYTEK large model needs to be filled in
api-secret:
# The URL of the ollama AI service
api-url:
@@ -19,6 +19,33 @@ spring:
name: ${HOSTNAME:@hertzbeat@}${PID}
profiles:
active: prod
ai:
mcp:
server:
enabled: true
stdio: false
name: sse-mcp-server
version: 1.0.0
resource-change-notification: true
tool-change-notification: true
prompt-change-notification: true
sse-endpoint: /api/sse
sse-message-endpoint: /api/mcp/message
type: SYNC
capabilities:
tool: true
resource: true
prompt: true
completion: true
chat:
client:
enabled: false
openai:
api-key: OPENAI_API_KEY
chat:
options:
model: gpt-4.1-nano-2025-04-14
mvc:
static-path-pattern: /**
jackson:
@@ -38,6 +65,7 @@ spring:
max-file-size: 100MB
max-request-size: 100MB
management:
health:
mail:
@@ -48,7 +76,19 @@ management:
include:
- 'metrics'
- 'health'
- 'prometheus'
enabled-by-default: on
endpoint:
prometheus:
access: read_only
metrics:
tags:
application: ${spring.application.name}
environment: ${spring.profiles.active}
prometheus:
metrics:
export:
enabled: true
sureness:
container: jakarta_servlet
@@ -140,6 +180,8 @@ warehouse:
insert:
buffer-size: 100
flush-interval: 3
compression:
enabled: false
cluster:
enabled: false
select:
@@ -0,0 +1,558 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# The monitoring type categoryservice-application service monitoring db-database monitoring custom-custom monitoring os-operating system monitoring
category: bigdata
# The monitoring type eg: linux windows tomcat mysql aws...
app: dolphinscheduler
# The monitoring i18n name
name:
zh-CN: Apache DolphinScheduler
en-US: Apache DolphinScheduler
# The description and help of this monitoring type
help:
zh-CN: Hertzbeat 对 Apache DolphinScheduler (支持v3.3.0或更高版本)通用指标进行测量监控。<br>您可以点击 “<i>新建 Apache DolphinScheduler</i>” 并进行配置,或者选择“<i>更多操作</i>”,导入已有配置。
en-US: Hertzbeat measures and monitors the general metrics of Apache DolphinScheduler (support v3.3.0 or later).<br>You can click on “<i>Create New Apache DolphinScheduler</i>” to configure it, or select “<i>More Actions</i>” to import an existing configuration.
helpLink:
zh-CN: https://hertzbeat.apache.org/zh-cn/docs/help/dolphinscheduler
en-US: https://hertzbeat.apache.org/docs/help/dolphinscheduler
# Input params define for monitoring(render web ui by the definition)
params:
# field-param field key
- field: host
# name-param field display i18n name
name:
zh-CN: 目标Host
en-US: Target Host
# type-param field type(most mapping the html input type)
type: host
# required-true or false
required: true
# field-param field key
- field: port
# name-param field display i18n name
name:
zh-CN: 端口
en-US: Port
# type-param field type(most mapping the html input type)
type: number
# when type is number, range is required
range: '[0,65535]'
# required-true or false
required: true
# default value
defaultValue: 12345
# field-param field key
- field: timeout
# name-param field display i18n name
name:
zh-CN: 查询超时时间
en-US: Query Timeout
# type-param field type(most mapping the html input type)
type: number
# required-true or false
required: false
# hide param-true or false
hide: true
# default value
defaultValue: 6000
# field-param field key
- field: ssl
# name-param field display i18n name
name:
zh-CN: 启用HTTPS
en-US: SSL
# type-param field type(boolean mapping the html h tag)
type: boolean
# required-true or false
required: true
- field: token
# name-param field display i18n name
name:
zh-CN: 令牌
en-US: Token
type: text
limit: 100
required: true
# collect metrics config list
metrics:
- name: master
i18n:
zh-CN: Master
en-US: Master
# metrics scheduling priority(0->127)->(high->low), metrics with the same priority will be scheduled in parallel
# priority 0's metrics is availability metrics, it will be scheduled first, only availability metrics collect success will the scheduling continue
priority: 0
# collect metrics content
fields:
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
- field: host
type: 1
i18n:
zh-CN: 主机地址
en-US: Host
- field: port
type: 1
i18n:
zh-CN: 端口
en-US: Port
- field: serverStatus
type: 1
i18n:
zh-CN: 状态
en-US: Server Status
- field: processId
type: 1
i18n:
zh-CN: 进程 ID
en-US: Process Id
- field: runningTime
type: 0
i18n:
zh-CN: 运行时间
en-US: Running Time
- field: cpuUsage
type: 0
unit: '%'
i18n:
zh-CN: 处理器使用量
en-US: CPU Usage
- field: memoryUsage
type: 0
unit: '%'
i18n:
zh-CN: 内存使用量
en-US: Memory Usage
- field: diskUsage
type: 0
unit: '%'
i18n:
zh-CN: 磁盘使用量
en-US: Disk Usage
- field: jvmCpuUsage
type: 0
unit: '%'
i18n:
zh-CN: JVM CPU 使用量
en-US: JVM CPU Usage
- field: jvmMemoryUsage
type: 0
unit: '%'
i18n:
zh-CN: JVM 内存 使用量
en-US: JVM Memory Usage
- field: jvmHeapUsed
type: 0
i18n:
zh-CN: JVM 已使用的堆内存大小
en-US: JVM Heap Used
- field: jvmNonHeapUsed
type: 0
i18n:
zh-CN: JVM 已使用的非堆内存大小
en-US: JVM NonHeap Used
- field: jvmHeapMax
type: 0
i18n:
zh-CN: JVM 配置的最大堆内存大小
en-US: JVM Heap Max
- field: jvmNonHeapMax
type: 0
i18n:
zh-CN: JVM 配置的最大非堆内存大小
en-US: JVM NonHeap Max
# (optional)metrics field alias name, it is used as an alias field to map and convert the collected data and metrics field
aliasFields:
- $.heartBeatInfo
# mapping and conversion expressions, use these and aliasField above to calculate metrics value
# eg: cores=core1+core2, usage=usage, waitTime=allTime-runningTime
calculates:
- host=json:apply($.heartBeatInfo).host
- port=json:apply($.heartBeatInfo).port
- serverStatus=json:apply($.heartBeatInfo).serverStatus
- processId=json:apply($.heartBeatInfo).processId
- runningTime=(now()-json:apply($.heartBeatInfo).startupTime)/86400000
- cpuUsage=json:apply($.heartBeatInfo).cpuUsage * 100
- memoryUsage=json:apply($.heartBeatInfo).memoryUsage * 100
- diskUsage=json:apply($.heartBeatInfo).diskUsage * 100
- jvmCpuUsage=json:apply($.heartBeatInfo).jvmCpuUsage
- jvmMemoryUsage=json:apply($.heartBeatInfo).jvmMemoryUsage
- jvmHeapUsed=json:apply($.heartBeatInfo).jvmHeapUsed
- jvmNonHeapUsed=json:apply($.heartBeatInfo).jvmNonHeapUsed
- jvmHeapMax=json:apply($.heartBeatInfo).jvmHeapMax
- jvmNonHeapMax=json:apply($.heartBeatInfo).jvmNonHeapMax
# the protocol used for monitoring, eg: sql, ssh, http, telnet, wmi, snmp, sdk
protocol: http
# the config content when protocol is http
http:
# http host: ipv4 ipv6 domain
host: ^_^host^_^
# http port
port: ^_^port^_^
# http url
url: /dolphinscheduler/monitor/MASTER
# http method: GET POST PUT DELETE PATCH
method: GET
# if enabled https
ssl: ^_^ssl^_^
# http request header content
headers:
token: ^_^token^_^
# http response data parse type: default-hertzbeat rule, jsonpath-jsonpath script, website-for website monitoring, prometheus-prometheus exporter rule
parseType: jsonPath
parseScript: '$.data[*]'
- name: worker
i18n:
zh-CN: Worker
en-US: Worker
# metrics scheduling priority(0->127)->(high->low), metrics with the same priority will be scheduled in parallel
# priority 0's metrics is availability metrics, it will be scheduled first, only availability metrics collect success will the scheduling continue
priority: 1
# collect metrics content
fields:
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
- field: host
type: 1
i18n:
zh-CN: 主机地址
en-US: Host
- field: port
type: 1
i18n:
zh-CN: 端口
en-US: Port
- field: serverStatus
type: 1
i18n:
zh-CN: 状态
en-US: Server Status
- field: processId
type: 1
i18n:
zh-CN: 进程 ID
en-US: Process Id
- field: runningTime
type: 0
i18n:
zh-CN: 运行时间
en-US: Running Time
- field: cpuUsage
type: 0
unit: '%'
i18n:
zh-CN: 处理器使用量
en-US: CPU Usage
- field: memoryUsage
type: 0
unit: '%'
i18n:
zh-CN: 内存使用量
en-US: Memory Usage
- field: diskUsage
type: 0
unit: '%'
i18n:
zh-CN: 磁盘使用量
en-US: Disk Usage
- field: jvmCpuUsage
type: 0
unit: '%'
i18n:
zh-CN: JVM CPU 使用量
en-US: JVM CPU Usage
- field: jvmMemoryUsage
type: 0
unit: '%'
i18n:
zh-CN: JVM 内存 使用量
en-US: JVM Memory Usage
- field: jvmHeapUsed
type: 0
i18n:
zh-CN: JVM 已使用的堆内存大小
en-US: JVM Heap Used
- field: jvmNonHeapUsed
type: 0
i18n:
zh-CN: JVM 已使用的非堆内存大小
en-US: JVM NonHeap Used
- field: jvmHeapMax
type: 0
i18n:
zh-CN: JVM 配置的最大堆内存大小
en-US: JVM Heap Max
- field: jvmNonHeapMax
type: 0
i18n:
zh-CN: JVM 配置的最大非堆内存大小
en-US: JVM NonHeap Max
- field: workerHostWeight
type: 0
i18n:
zh-CN: 权重
en-US: Weight
- field: threadPoolUsage
type: 0
unit: '%'
i18n:
zh-CN: 线程池使用量
en-US: Thread Pool Usage
- field: workerGroup
type: 1
i18n:
zh-CN: Worker 组
en-US: Worker Group
# (optional)metrics field alias name, it is used as an alias field to map and convert the collected data and metrics field
aliasFields:
- $.heartBeatInfo
# mapping and conversion expressions, use these and aliasField above to calculate metrics value
# eg: cores=core1+core2, usage=usage, waitTime=allTime-runningTime
calculates:
- host=json:apply($.heartBeatInfo).host
- port=json:apply($.heartBeatInfo).port
- serverStatus=json:apply($.heartBeatInfo).serverStatus
- processId=json:apply($.heartBeatInfo).processId
- runningTime=(now()-json:apply($.heartBeatInfo).startupTime)/86400000
- cpuUsage=json:apply($.heartBeatInfo).cpuUsage * 100
- memoryUsage=json:apply($.heartBeatInfo).memoryUsage * 100
- diskUsage=json:apply($.heartBeatInfo).diskUsage * 100
- jvmCpuUsage=json:apply($.heartBeatInfo).jvmCpuUsage
- jvmMemoryUsage=json:apply($.heartBeatInfo).jvmMemoryUsage
- jvmHeapUsed=json:apply($.heartBeatInfo).jvmHeapUsed
- jvmNonHeapUsed=json:apply($.heartBeatInfo).jvmNonHeapUsed
- jvmHeapMax=json:apply($.heartBeatInfo).jvmHeapMax
- jvmNonHeapMax=json:apply($.heartBeatInfo).jvmNonHeapMax
- workerHostWeight=json:apply($.heartBeatInfo).workerHostWeight
- threadPoolUsage=json:apply($.heartBeatInfo).threadPoolUsage
- workerGroup=json:apply($.heartBeatInfo).workerGroup
# the protocol used for monitoring, eg: sql, ssh, http, telnet, wmi, snmp, sdk
protocol: http
# the config content when protocol is http
http:
# http host: ipv4 ipv6 domain
host: ^_^host^_^
# http port
port: ^_^port^_^
# http url
url: /dolphinscheduler/monitor/WORKER
# http method: GET POST PUT DELETE PATCH
method: GET
# if enabled https
ssl: ^_^ssl^_^
# http request header content
headers:
token: ^_^token^_^
# http response data parse type: default-hertzbeat rule, jsonpath-jsonpath script, website-for website monitoring, prometheus-prometheus exporter rule
parseType: jsonPath
parseScript: '$.data[*]'
- name: alert-server
i18n:
zh-CN: Alert Server
en-US: Alert Server
# metrics scheduling priority(0->127)->(high->low), metrics with the same priority will be scheduled in parallel
# priority 0's metrics is availability metrics, it will be scheduled first, only availability metrics collect success will the scheduling continue
priority: 0
# collect metrics content
fields:
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
- field: host
type: 1
i18n:
zh-CN: 主机地址
en-US: Host
- field: port
type: 1
i18n:
zh-CN: 端口
en-US: Port
- field: serverStatus
type: 1
i18n:
zh-CN: 状态
en-US: Server Status
- field: processId
type: 1
i18n:
zh-CN: 进程 ID
en-US: Process Id
- field: runningTime
type: 0
i18n:
zh-CN: 运行时间
en-US: Running Time
- field: cpuUsage
type: 0
unit: '%'
i18n:
zh-CN: 处理器使用量
en-US: CPU Usage
- field: memoryUsage
type: 0
unit: '%'
i18n:
zh-CN: 内存使用量
en-US: Memory Usage
- field: diskUsage
type: 0
unit: '%'
i18n:
zh-CN: 磁盘使用量
en-US: Disk Usage
- field: jvmCpuUsage
type: 0
unit: '%'
i18n:
zh-CN: JVM CPU 使用量
en-US: JVM CPU Usage
- field: jvmMemoryUsage
type: 0
unit: '%'
i18n:
zh-CN: JVM 内存 使用量
en-US: JVM Memory Usage
- field: jvmHeapUsed
type: 0
i18n:
zh-CN: JVM 已使用的堆内存大小
en-US: JVM Heap Used
- field: jvmNonHeapUsed
type: 0
i18n:
zh-CN: JVM 已使用的非堆内存大小
en-US: JVM NonHeap Used
- field: jvmHeapMax
type: 0
i18n:
zh-CN: JVM 配置的最大堆内存大小
en-US: JVM Heap Max
- field: jvmNonHeapMax
type: 0
i18n:
zh-CN: JVM 配置的最大非堆内存大小
en-US: JVM NonHeap Max
# (optional)metrics field alias name, it is used as an alias field to map and convert the collected data and metrics field
aliasFields:
- $.heartBeatInfo
# mapping and conversion expressions, use these and aliasField above to calculate metrics value
# eg: cores=core1+core2, usage=usage, waitTime=allTime-runningTime
calculates:
- host=json:apply($.heartBeatInfo).host
- port=json:apply($.heartBeatInfo).port
- serverStatus=json:apply($.heartBeatInfo).serverStatus
- processId=json:apply($.heartBeatInfo).processId
- runningTime=(now()-json:apply($.heartBeatInfo).startupTime)/86400000
- cpuUsage=json:apply($.heartBeatInfo).cpuUsage * 100
- memoryUsage=json:apply($.heartBeatInfo).memoryUsage * 100
- diskUsage=json:apply($.heartBeatInfo).diskUsage * 100
- jvmCpuUsage=json:apply($.heartBeatInfo).jvmCpuUsage
- jvmMemoryUsage=json:apply($.heartBeatInfo).jvmMemoryUsage
- jvmHeapUsed=json:apply($.heartBeatInfo).jvmHeapUsed
- jvmNonHeapUsed=json:apply($.heartBeatInfo).jvmNonHeapUsed
- jvmHeapMax=json:apply($.heartBeatInfo).jvmHeapMax
- jvmNonHeapMax=json:apply($.heartBeatInfo).jvmNonHeapMax
# the protocol used for monitoring, eg: sql, ssh, http, telnet, wmi, snmp, sdk
protocol: http
# the config content when protocol is http
http:
# http host: ipv4 ipv6 domain
host: ^_^host^_^
# http port
port: ^_^port^_^
# http url
url: /dolphinscheduler/monitor/ALERT_SERVER
# http method: GET POST PUT DELETE PATCH
method: GET
# if enabled https
ssl: ^_^ssl^_^
# http request header content
headers:
token: ^_^token^_^
# http response data parse type: default-hertzbeat rule, jsonpath-jsonpath script, website-for website monitoring, prometheus-prometheus exporter rule
parseType: jsonPath
parseScript: '$.data[*]'
- name: database
i18n:
zh-CN: 数据库
en-US: Database
# metrics scheduling priority(0->127)->(high->low), metrics with the same priority will be scheduled in parallel
# priority 0's metrics is availability metrics, it will be scheduled first, only availability metrics collect success will the scheduling continue
priority: 2
# collect metrics content
fields:
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
- field: dbType
type: 1
i18n:
zh-CN: 数据库类型
en-US: Database Type
- field: state
type: 1
i18n:
zh-CN: 状态
en-US: State
- field: maxConnections
type: 0
i18n:
zh-CN: 最大连接数
en-US: Max Connections
- field: threadsConnections
type: 0
i18n:
zh-CN: 当前连接数
en-US: Threads Connections
- field: threadsRunningConnections
type: 0
i18n:
zh-CN: 当前活跃连接数
en-US: Threads Running Connections
# (optional)metrics field alias name, it is used as an alias field to map and convert the collected data and metrics field
aliasFields:
- dbType
- state
- maxConnections
- threadsConnections
- threadsRunningConnections
# mapping and conversion expressions, use these and aliasField above to calculate metrics value
# eg: cores=core1+core2, usage=usage, waitTime=allTime-runningTime
calculates:
- dbType=dbType
- state=state
- maxConnections=maxConnections
- threadsConnections=threadsConnections
- threadsRunningConnections=threadsRunningConnections
# the protocol used for monitoring, eg: sql, ssh, http, telnet, wmi, snmp, sdk
protocol: http
# the config content when protocol is http
http:
# http host: ipv4 ipv6 domain
host: ^_^host^_^
# http port
port: ^_^port^_^
# http url
url: /dolphinscheduler/monitor/databases
# http method: GET POST PUT DELETE PATCH
method: GET
# if enabled https
ssl: ^_^ssl^_^
# http request header content
headers:
token: ^_^token^_^
# http response data parse type: default-hertzbeat rule, jsonpath-jsonpath script, website-for website monitoring, prometheus-prometheus exporter rule
parseType: jsonPath
parseScript: '$.data[0]'
@@ -667,7 +667,7 @@ metrics:
i18n:
zh-CN: 客户端命令统计
en-US: cmdstat client
ja-JP: クライントのコマンド
ja-JP: クライントのコマンド
- field: cmdstat_config
type: 1
i18n:
@@ -21,11 +21,13 @@ app: linux_script
name:
zh-CN: Linux 命令
en-US: Linux Script
ja-JP: Linux Script
# The description and help of this monitoring type
help:
zh-CN: Hertzbeat 使用采集器作为 agent 直接运行 <a class='help_module_content' href='https://hertzbeat.apache.org/docs/advanced/extend-script'> Shell 命令 </a> 对 Linux 操作系统的通用性能指标 (系统信息、CPU、内存、磁盘、网卡、文件系统、TOP资源进程等) 进行采集监控。<br>您可以点击“<i>新建 Linux Script</i>”进行添加。或者选择“<i>更多操作</i>”,导入已有配置。
en-US: Hertzbeat uses a collector as an agent to directly execute <a class='help_module_content' href='https://hertzbeat.apache.org/docs/advanced/extend-script'> Shell commands </a> to collect and monitor general performance metrics of the Linux operating system (system information, CPU, memory, disk, network card, file system, TOP resource processes, etc.).<br>You can click “<i>Create New Linux Script</i>” to add it. Or select “<i>More Actions</i>” to import an existing configuration.
zh-TW: Hertzbeat 使用采集器作為 agent 直接運行 <a class='help_module_content' href='https://hertzbeat.apache.org/docs/advanced/extend-script'> Shell 命令 </a> 對 Linux 操作系統的通用性能指標 (系統信息、CPU、內存、磁盤、網卡、文件系統、TOP資源進程等) 進行採集監控。<br>您可以點擊“<i>新建 Linux Script</i>”進行添加。或者選擇“<i>更多操作</i>”,導入已有配置。
ja-JP: Hertzbeat は agent として <a class='help_module_content' href='https://hertzbeat.apache.org/docs/advanced/extend-script'> Shell コマンドを介して </a> Linuxシステムの一般的なパフォーマンスのメトリクスを監視します。<br>「<i>新規 Linux Script</i>」をクリックしてホストなどのパラメタを設定した後、新規することができます。
helpLink:
zh-CN: https://hertzbeat.apache.org/zh-cn/docs/help/linux_script
en-US: https://hertzbeat.apache.org/docs/help/linux_script
@@ -37,6 +39,7 @@ params:
name:
zh-CN: 目标Host
en-US: Target Host
ja-JP: 目標ホスト
# type-param field type(most mapping the html input type)
type: host
# required-true or false
@@ -48,6 +51,7 @@ metrics:
i18n:
zh-CN: 系统基本信息
en-US: Basic Info
ja-JP: 基礎情報
# metrics scheduling priority(0->127)->(high->low), metrics with the same priority will be scheduled in parallel
# priority 0's metrics is availability metrics, it will be scheduled first, only availability metrics collect success will the scheduling continue
priority: 0
@@ -60,16 +64,19 @@ metrics:
i18n:
zh-CN: 主机名称
en-US: Host Name
ja-JP: ホスト名
- field: version
type: 1
i18n:
zh-CN: 操作系统版本
en-US: System Version
ja-JP: オーエスバージョン
- field: uptime
type: 1
i18n:
zh-CN: 启动时间
en-US: Uptime
ja-JP: アップタイム
# the protocol used for monitoring, eg: sql, ssh, http, telnet, wmi, snmp, sdk
protocol: script
# the config content when protocol is ssh
@@ -89,6 +96,7 @@ metrics:
i18n:
zh-CN: CPU 信息
en-US: CPU Info
ja-JP: CPU情報
priority: 1
fields:
- field: info
@@ -96,32 +104,38 @@ metrics:
i18n:
zh-CN: 型号
en-US: Info
ja-JP: バージョン
- field: cores
type: 1
i18n:
zh-CN: 核数
en-US: Cores
ja-JP: コア数
- field: interrupt
type: 0
i18n:
zh-CN: 中断数
en-US: Interrupt
ja-JP: 割り込み数
- field: load
type: 1
i18n:
zh-CN: 负载
en-US: Load
ja-JP: ロード
- field: context_switch
type: 0
i18n:
zh-CN: 上下文切换
en-US: Context Switch
ja-JP: コンテキストスイッチ
- field: usage
type: 0
unit: '%'
i18n:
zh-CN: 使用率
en-US: Usage
ja-JP: 使用率
# (optional)metrics field alias name, it is used as an alias field to map and convert the collected data and metrics field
aliasFields:
- info
@@ -152,6 +166,7 @@ metrics:
i18n:
zh-CN: 内存信息
en-US: Memory Info
ja-JP: メモリ情報
priority: 2
fields:
- field: total
@@ -160,36 +175,42 @@ metrics:
i18n:
zh-CN: 总内存容量
en-US: Total Memory
ja-JP: メモリ容量
- field: used
type: 0
unit: Mb
i18n:
zh-CN: 用户程序内存量
en-US: User Program Memory
ja-JP: ユーザープログラムメモリ
- field: free
type: 0
unit: Mb
i18n:
zh-CN: 空闲内存容量
en-US: Free Memory
ja-JP: 空きメモリ
- field: buff_cache
type: 0
unit: Mb
i18n:
zh-CN: 缓存占用内存
en-US: Buff Cache Memory
ja-JP: バッファメモリ
- field: available
type: 0
unit: Mb
i18n:
zh-CN: 剩余可用内存
en-US: Available Memory
ja-JP: 使用可能のメモリ
- field: usage
type: 0
unit: '%'
i18n:
zh-CN: 内存使用率
en-US: Memory Usage
ja-JP: メモリ使用率
aliasFields:
- total
- used
@@ -214,6 +235,7 @@ metrics:
i18n:
zh-CN: 磁盘信息
en-US: Disk Info
ja-JP: ディスク情報
priority: 3
fields:
- field: disk_num
@@ -221,27 +243,32 @@ metrics:
i18n:
zh-CN: 磁盘总数
en-US: Disk Num
ja-JP: ディスク番号
- field: partition_num
type: 1
i18n:
zh-CN: 分区总数
en-US: Partition Num
ja-JP: パーティション
- field: block_write
type: 0
i18n:
zh-CN: 写磁盘块数
en-US: Block Write
ja-JP: 書き込みディスクブロック数
- field: block_read
type: 0
i18n:
zh-CN: 读磁盘块数
en-US: Block Read
ja-JP: 読み取りブロック数
- field: write_rate
type: 0
unit: iops
i18n:
zh-CN: 磁盘写速率
en-US: Write Rate
ja-JP: ディスク書き込み速度
protocol: script
script:
scriptTool: bash
@@ -253,6 +280,7 @@ metrics:
i18n:
zh-CN: 网卡信息
en-US: Interface Info
ja-JP: ネットワークカード情報
priority: 4
fields:
- field: interface_name
@@ -261,18 +289,21 @@ metrics:
i18n:
zh-CN: 网卡名称
en-US: Interface Name
ja-JP: ネットワークカード名
- field: receive_bytes
type: 0
unit: Mb
i18n:
zh-CN: 入站数据流量
en-US: Receive Bytes
ja-JP: 受信されたバイト数
- field: transmit_bytes
type: 0
unit: Mb
i18n:
zh-CN: 出站数据流量
en-US: Transmit Bytes
ja-JP: 転送されたバイト数
units:
- receive_bytes=B->MB
- transmit_bytes=B->MB
@@ -287,6 +318,7 @@ metrics:
i18n:
zh-CN: 文件系统
en-US: Disk Free
ja-JP: ファイルシステム
priority: 5
fields:
- field: filesystem
@@ -294,30 +326,35 @@ metrics:
i18n:
zh-CN: 文件系统
en-US: Filesystem
ja-JP: ファイルシステム
- field: used
type: 0
unit: Mb
i18n:
zh-CN: 已使用量
en-US: Used
ja-JP: 使用済み
- field: available
type: 0
unit: Mb
i18n:
zh-CN: 可用量
en-US: Available
ja-JP: 使用可能
- field: usage
type: 0
unit: '%'
i18n:
zh-CN: 使用率
en-US: Usage
ja-JP: 使用率
- field: mounted
type: 1
label: true
i18n:
zh-CN: 挂载点
en-US: Mounted
ja-JP: マウント
protocol: script
script:
scriptTool: bash
@@ -329,6 +366,7 @@ metrics:
i18n:
zh-CN: Top10 CPU 进程
en-US: Top10 CPU Process
ja-JP: トップ10 CPUプロセス
priority: 6
fields:
- field: pid
@@ -337,23 +375,27 @@ metrics:
i18n:
zh-CN: 进程ID
en-US: PID
ja-JP: プロセスID
- field: cpu_usage
type: 0
unit: '%'
i18n:
zh-CN: CPU占用率
en-US: CPU Usage
ja-JP: CPU使用率
- field: mem_usage
type: 0
unit: '%'
i18n:
zh-CN: 内存占用率
en-US: Memory Usage
ja-JP: メモリ使用率
- field: command
type: 1
i18n:
zh-CN: 执行命令
en-US: Command
ja-JP: コマンド
protocol: script
script:
scriptTool: bash
@@ -365,6 +407,7 @@ metrics:
i18n:
zh-CN: Top10 内存进程
en-US: Top10 Memory Process
ja-JP: トップ10 メモリプロセス
priority: 7
fields:
- field: pid
@@ -373,23 +416,27 @@ metrics:
i18n:
zh-CN: 进程ID
en-US: PID
ja-JP: プロセスID
- field: mem_usage
type: 0
unit: '%'
i18n:
zh-CN: 内存占用率
en-US: Memory Usage
ja-JP: メモリ使用率
- field: cpu_usage
type: 0
unit: '%'
i18n:
zh-CN: CPU占用率
en-US: CPU Usage
ja-JP: CPU使用率
- field: command
type: 1
i18n:
zh-CN: 执行命令
en-US: Command
ja-JP: コマンド
protocol: script
script:
scriptTool: bash
@@ -21,11 +21,13 @@ app: mariadb
name:
zh-CN: MariaDB数据库
en-US: MariaDB
ja-JP: MariaDB データベース
# The description and help of this monitoring type
help:
zh-CN: HertzBeat 使用 <a class='help_module_content' href='https://hertzbeat.apache.org/docs/advanced/extend-jdbc'> JDBC 协议</a> 通过配置 SQL 对 MariaDB 数据库的通用性能指标(系统信息、性能状态、Innodb、缓存、事物、用户线程、慢SQL等)进行采集监控,支持版本为 MariaDB5+。<br>您可以点击“<i>新建 MariaDB 数据库</i>”并进行配置,或者选择“<i>更多操作</i>”,导入已有配置。
en-US: HertzBeat uses <a class='help_module_content' href='https://hertzbeat.apache.org/docs/advanced/extend-jdbc'> JDBC Protocol</a> to configure SQL for collecting general metrics of MariaDB database (system information, performance status, Innodb, cache, things, user threads, slow SQL, etc.). Supported version is MariaDB5+. <br>You can click "<i>New MariaDB Database</i>" and configure it, or select "<i>More Operations</i>" to import the existing configuration.
zh-TW: HertzBeat 使用 <a class='help_module_content' href='https://hertzbeat.apache.org/docs/advanced/extend-jdbc'> JDBC 協議</a> 通過配置 SQL 對 MariaDB 數據庫的通用性能指標(系統信息、性能狀態、Innodb、緩存、事物、用戶線程、慢SQL等)進行采集監控,支持版本爲 MariaDB5+。<br>您可以點擊“<i>新建 MariaDB 數據庫</i>”並進行配置,或者選擇“<i>更多操作</i>”,導入已有配置。
ja-JP: Hertzbeat は <a class='help_module_content' href='https://hertzbeat.apache.org/docs/advanced/extend-jdbc'> JDBCプロトコルを介して</a> MariaDB データベース(MariaDB5+)の一般的なパフォーマンスのメトリクスを監視します。<br>「<i>新規 MariaDB データベース</i>」をクリックしてパラメタを設定した後、新規することができます。
helpLink:
zh-CN: https://hertzbeat.apache.org/zh-cn/docs/help/mariadb
en-US: https://hertzbeat.apache.org/docs/help/mariadb
@@ -37,6 +39,7 @@ params:
name:
zh-CN: 目标Host
en-US: Target Host
ja-JP: 目標ホスト
# type-param field type(most mapping the html input type)
type: host
# required-true or false
@@ -47,6 +50,7 @@ params:
name:
zh-CN: 端口
en-US: Port
ja-JP: ポート
# type-param field type(most mapping the html input type)
type: number
# when type is number, range is required
@@ -59,6 +63,7 @@ params:
name:
zh-CN: 查询超时时间(ms)
en-US: Query Timeout(ms)
ja-JP: クエリタイムアウト(ms)
type: number
# when type is number, range is required
range: '[400,200000]'
@@ -69,12 +74,14 @@ params:
name:
zh-CN: 数据库名称
en-US: Database Name
ja-JP: データベース名
type: text
required: false
- field: username
name:
zh-CN: 用户名
en-US: Username
ja-JP: ユーザー名
type: text
limit: 50
required: false
@@ -82,12 +89,14 @@ params:
name:
zh-CN: 密码
en-US: Password
ja-JP: パスワード
type: password
required: false
- field: url
name:
zh-CN: URL
en-US: URL
ja-JP: URL
type: text
required: false
hide: true
@@ -100,8 +109,9 @@ metrics:
# priority 0's metrics is availability metrics, it will be scheduled first, only availability metrics collect success will the scheduling continue
priority: 0
i18n:
zh-CN: 基础 信息
zh-CN: 基础信息
en-US: Basic Info
ja-JP: 基礎情報
# collect metrics content
fields:
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
@@ -110,32 +120,38 @@ metrics:
i18n:
zh-CN: 版本
en-US: Version
ja-JP: バージョン
- field: port
type: 1
i18n:
zh-CN: 端口
en-US: Port
ja-JP: ポート
- field: datadir
type: 1
i18n:
zh-CN: 存储目录
en-US: DataDir
en-US: Data Directory
ja-JP: データディレクトリ
- field: max_connections
type: 0
i18n:
zh-CN: 最大连接数
en-US: Max Connections
ja-JP: 最大接続数
- field: thread_cache_size
type: 0
i18n:
zh-CN: 连接池大小
en-US: Thread Cache Size
ja-JP: キャッシュするスレッドの数
- field: innodb_buffer_pool_size
type: 0
unit: KB
i18n:
zh-CN: InnoDB缓冲池的大小
en-US: Innodb Buffer Pool Size
ja-JP: Innodbバッファープールのサイズ
# (optional)metrics field alias name, it is used as an alias field to map and convert the collected data and metrics field
aliasFields:
- version
@@ -188,8 +204,9 @@ metrics:
priority: 1
# collect metrics content
i18n:
zh-CN: 缓存 信息
zh-CN: 缓存信息
en-US: Cache Info
ja-JP: キャッシュ情報
fields:
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
- field: query_cache_hit_rate
@@ -198,27 +215,32 @@ metrics:
i18n:
zh-CN: 缓存命中率
en-US: Query Cache Hit Rate
ja-JP: キャッシュ命中率
- field: cache_hits
type: 0
i18n:
zh-CN: 缓存命中数
en-US: Cache Hits
ja-JP: キャッシュ命中数
- field: cache_inserts
type: 0
i18n:
zh-CN: 缓存插入数
en-US: Cache Inserts
ja-JP: キャッシュインサート数
- field: cache_free_blocks
type: 0
i18n:
zh-CN: 缓存空闲块数量
en-US: Cache Free Blocks
ja-JP: キャッシュの空きブロック数
- field: cache_free_memory
type: 0
unit: KB
i18n:
zh-CN: 空闲缓存大小
en-US: Cache Free Memory
ja-JP: キャッシュの利用可能メモリ
# (optional)metrics field alias name, it is used as an alias field to map and convert the collected data and metrics field
aliasFields:
- Qcache_hits
@@ -241,7 +263,6 @@ metrics:
# the config content when protocol is jdbc
jdbc:
# jdbc host
# jdbchost
host: ^_^host^_^
# port
port: ^_^port^_^
@@ -264,19 +285,22 @@ metrics:
- name: performance
priority: 2
i18n:
zh-CN: 性能 信息
zh-CN: 性能信息
en-US: Performance Info
ja-JP: パフォーマンス情報
fields:
- field: questions
type: 0
i18n:
zh-CN: 查询总数
en-US: Questions
ja-JP: クエリ総数
- field: qps
type: 0
i18n:
zh-CN: 每秒处理查询数
en-US: Queries Per Second
zh-CN: QPS
en-US: QPS
ja-JP: QPS
aliasFields:
- uptime
- questions
@@ -298,8 +322,9 @@ metrics:
- name: innodb
priority: 3
i18n:
zh-CN: Innodb 信息
zh-CN: Innodb信息
en-US: Innodb Info
ja-JP: Innodb情報
fields:
- field: innodb_data_reads
type: 0
@@ -307,30 +332,35 @@ metrics:
i18n:
zh-CN: 磁盘读取次数
en-US: Innodb Data Reads
ja-JP: Innodb読み取り回数
- field: innodb_data_writes
type: 0
unit: Times
i18n:
zh-CN: 磁盘写入次数
en-US: Innodb Data Writes
ja-JP: Innodb書き込み回数
- field: innodb_data_read
type: 0
unit: KB
i18n:
zh-CN: 磁盘读取数据量
en-US: Innodb Data Read
ja-JP: Innodb読み取り(KB
- field: innodb_data_written
type: 0
unit: KB
i18n:
zh-CN: 磁盘写入数据量
en-US: Innodb Data Written
ja-JP: Innodb書き込み(KB
- field: innodb_buffer_hit_rate
type: 0
unit: '%'
i18n:
zh-CN: Innodb 缓存命中数
en-US: Innodb Buffer Hit Rate
ja-JP: Innodbバッファーキャッシュ命中率
aliasFields:
- Innodb_data_reads
- Innodb_data_writes
@@ -364,8 +394,9 @@ metrics:
- name: status
priority: 4
i18n:
zh-CN: 状态 信息
zh-CN: 状态信息
en-US: Status Info
ja-JP: 状態情報
fields:
- field: uptime
type: 0
@@ -373,61 +404,73 @@ metrics:
i18n:
zh-CN: 运行时间
en-US: Uptime
ja-JP: アップタイム
- field: com_select
type: 0
i18n:
zh-CN: 查询次数
en-US: Com Select
ja-JP: セレクト回数
- field: com_insert
type: 0
i18n:
zh-CN: 插入次数
en-US: Com Insert
ja-JP: インサート回数
- field: com_update
type: 0
i18n:
zh-CN: 更新次数
en-US: Com Update
ja-JP: 更新回数
- field: com_delete
type: 0
i18n:
zh-CN: 删除次数
en-US: Com Delete
ja-JP: 削除回数
- field: com_commit
type: 0
i18n:
zh-CN: 事务提交次数
en-US: Com Commit
ja-JP: トランザクションのコミット数
- field: com_rollback
type: 0
i18n:
zh-CN: 事务回滚次数
en-US: Com Rollback
ja-JP: トランザクションのロールバック数
- field: threads_created
type: 0
i18n:
zh-CN: 线程创建数
en-US: Threads Created
ja-JP: スレッド作成数
- field: threads_connected
type: 0
i18n:
zh-CN: 连接线程数
en-US: Threads Connected
ja-JP: 接続スレッド数
- field: threads_cached
type: 0
i18n:
zh-CN: 缓存线程数
en-US: Threads Cached
ja-JP: キャッシュスレッド数
- field: threads_running
type: 0
i18n:
zh-CN: 活动线程数
en-US: Threads Running
ja-JP: 活動中のスレッド数
- field: qps
type: 0
i18n:
zh-CN: 每秒处理查询数
en-US: Com Insert
zh-CN: QPS
en-US: QPS
ja-JP: QPS
aliasFields:
- uptime
- com_select
@@ -459,69 +502,82 @@ metrics:
- name: handler
priority: 5
i18n:
zh-CN: 处理器 信息
zh-CN: 处理器信息
en-US: Handler Info
ja-JP: ハンドラ情報
fields:
- field: handler_commit
type: 0
i18n:
zh-CN: 处理的事务提交数
en-US: Handler Commit
ja-JP: ハンドラのコミット操作数
- field: handler_delete
type: 0
i18n:
zh-CN: 处理的删除操作数
en-US: Handler Delete
ja-JP: ハンドラの削除操作数
- field: handler_write
type: 0
i18n:
zh-CN: 处理的写入操作数
en-US: Handler Write
ja-JP: ハンドラの書き込み操作数
- field: handler_update
type: 0
i18n:
zh-CN: 处理的更新操作数
en-US: Handler Update
ja-JP: ハンドラの更新操作数
- field: handler_discover
type: 0
i18n:
zh-CN: 处理的发现操作数
en-US: Handler Discover
ja-JP: ハンドラのディスカバー操作数
- field: handler_prepare
type: 0
i18n:
zh-CN: 处理的准备操作数
en-US: Handler Prepare
ja-JP: ハンドラの準備操作数
- field: handler_read_first
type: 0
i18n:
zh-CN: 读取首行操作数
en-US: Handler Read First
ja-JP: ハンドラの最初の行を読む操作数
- field: handler_read_rnd
type: 0
i18n:
zh-CN: 随机读取操作数
en-US: Handler Read RND
ja-JP: ハンドラのランダムアクセス操作数
- field: handler_read_next
type: 0
i18n:
zh-CN: 读取下一行操作数
en-US: Handler Read Next
ja-JP: ハンドラの次の行を読む操作数
- field: handler_read_rnd_next
type: 0
i18n:
zh-CN: 随机读取下一行操作数
en-US: Handler Read RND Next
ja-JP: ハンドラの次の行をランダムに読む操作数
- field: handler_read_key
type: 0
i18n:
zh-CN: 按索引键读取操作数
en-US: Handler Read Key
ja-JP: ハンドラのインデックスを使用して読み取る操作数
- field: handler_external_lock
type: 0
i18n:
zh-CN: 外部锁定操作数
en-US: Handler External Key
ja-JP: ハンドラの外部のキーを使用して読み取る操作数
protocol: jdbc
jdbc:
host: ^_^host^_^
@@ -538,29 +594,34 @@ metrics:
- name: connection
priority: 6
i18n:
zh-CN: 连接 信息
zh-CN: 连接信息
en-US: Connection Info
ja-JP: 接続情報
fields:
- field: connections
type: 0
i18n:
zh-CN: 当前连接数
en-US: Connections
ja-JP: 現在接続数
- field: max_used_connections
type: 0
i18n:
zh-CN: 历史最大连接数
en-US: Max Used Connections
ja-JP: 最大接続数
- field: aborted_connects
type: 0
i18n:
zh-CN: 失败连接数
en-US: Aborted Connects
ja-JP: 失敗接続数
- field: aborted_clients
type: 0
i18n:
zh-CN: 客户端中断数
en-US: Aborted Clients
ja-JP: クライアントの割り込み数
protocol: jdbc
jdbc:
host: ^_^host^_^
@@ -577,29 +638,34 @@ metrics:
- name: thread
priority: 7
i18n:
zh-CN: 线程 信息
zh-CN: 线程信息
en-US: Thread Info
ja-JP: スレッド情報
fields:
- field: threads_created
type: 0
i18n:
zh-CN: 创建线程数
en-US: Threads Created
ja-JP: スレッド作成数
- field: threads_connected
type: 0
i18n:
zh-CN: 连接线程数
en-US: Threads Connected
ja-JP: 接続スレッド数
- field: threads_cached
type: 0
i18n:
zh-CN: 缓存线程数
en-US: CThreads Cached
ja-JP: キャッシュスレッド数
- field: threads_running
type: 0
i18n:
zh-CN: 运行线程数
en-US: Threads Running
ja-JP: 活動中のスレッド数
protocol: jdbc
jdbc:
host: ^_^host^_^
@@ -616,24 +682,28 @@ metrics:
- name: tmp
priority: 8
i18n:
zh-CN: 临时表 信息
zh-CN: 临时表信息
en-US: TMP Info
ja-JP: 一時テーブル情報
fields:
- field: created_tmp_tables
type: 0
i18n:
zh-CN: 创建的临时表数量
en-US: Created TMP Tables
ja-JP: 一時テーブル作成数
- field: created_tmp_disk_tables
type: 0
i18n:
zh-CN: 创建的临时磁盘表数量
en-US: Created TMP Disk Tables
ja-JP: 一時ディスクテーブル作成数
- field: created_tmp_files
type: 0
i18n:
zh-CN: 创建的临时文件数量
en-US: Created TMP Files
ja-JP: 一時ファイル作成数
protocol: jdbc
jdbc:
host: ^_^host^_^
@@ -650,34 +720,40 @@ metrics:
- name: select_type
priority: 9
i18n:
zh-CN: 查询类型 信息
zh-CN: 查询类型信息
en-US: Select Type Info
ja-JP: 検索タイプ情報
fields:
- field: select_full_join
type: 0
i18n:
zh-CN: 全表连接数
en-US: Select Full Join
ja-JP: 全件結合数
- field: select_full_range_join
type: 0
i18n:
zh-CN: 带范围的全表连接数
en-US: Select Full Range Join
ja-JP: 範囲検索付き全件結合数
- field: select_range
type: 0
i18n:
zh-CN: 索引范围查询数
en-US: Select Range
ja-JP: 範囲検索数
- field: select_range_check
type: 0
i18n:
zh-CN: 索引范围内二次检查数
en-US: Select Range Check
ja-JP: 範囲内二次検索数
- field: select_scan
type: 0
i18n:
zh-CN: 全表扫描查询数
en-US: Select Scan
ja-JP: テーブル全体をスキャンする回数
protocol: jdbc
jdbc:
host: ^_^host^_^
@@ -694,29 +770,34 @@ metrics:
- name: sort
priority: 10
i18n:
zh-CN: 排序 信息
zh-CN: 排序信息
en-US: Sort Info
ja-JP: ソート情報
fields:
- field: sort_rows
type: 0
i18n:
zh-CN: 排序总行数
en-US: Sort Rows
ja-JP: ソートされた行数
- field: sort_range
type: 0
i18n:
zh-CN: 范围条件排序次数
en-US: Sort Range
ja-JP: 範囲付きのソート回数
- field: sort_merge_passes
type: 0
i18n:
zh-CN: 合并排序段次数
en-US: Sort Merge Passes
ja-JP: マージソート回数
- field: sort_scan
type: 0
i18n:
zh-CN: 全表扫描排序次数
en-US: Sort Scan
ja-JP: テーブル全体をスキャンしてソートする回数
protocol: jdbc
jdbc:
host: ^_^host^_^
@@ -733,19 +814,22 @@ metrics:
- name: table_lock
priority: 11
i18n:
zh-CN: 表级锁 信息
zh-CN: 表级锁信息
en-US: Table Lock Info
ja-JP: テーブルロック情報
fields:
- field: table_locks_immediate
type: 0
i18n:
zh-CN: 立即获得表锁次数
en-US: Table Locks Immediate
ja-JP: 即座に取得できた回数
- field: table_locks_waited
type: 0
i18n:
zh-CN: 须等待获取表锁次数
en-US: Table Locks Waited
ja-JP: 取得を待機した回数
protocol: jdbc
jdbc:
host: ^_^host^_^
@@ -762,8 +846,9 @@ metrics:
- name: process_state
priority: 12
i18n:
zh-CN: 进程状态 信息
zh-CN: 进程状态信息
en-US: Process State Info
ja-JP: プロセス状態情報
fields:
- field: state
type: 1
@@ -771,11 +856,13 @@ metrics:
i18n:
zh-CN: 进程状态
en-US: State
ja-JP: プロセス状態
- field: num
type: 0
i18n:
zh-CN: 该状态进程数量
en-US: Num
ja-JP: プロセス総数
protocol: jdbc
jdbc:
host: ^_^host^_^
@@ -792,8 +879,9 @@ metrics:
- name: slow_sql
priority: 13
i18n:
zh-CN: 慢查询 信息
zh-CN: 慢查询信息
en-US: Slow Sql Info
ja-JP: スローSQL情報
fields:
- field: id
type: 1
@@ -801,26 +889,31 @@ metrics:
i18n:
zh-CN: ID
en-US: ID
ja-JP: ID
- field: sql_text
type: 1
i18n:
zh-CN: SQL 文本
en-US: SQL Text
ja-JP: SQL文のテキスト
- field: start_time
type: 1
i18n:
zh-CN: 开始时间
en-US: Start Time
ja-JP: 開始時間
- field: db
type: 1
i18n:
zh-CN: 数据库
en-US: Database
ja-JP: データベース
- field: query_time
type: 3
i18n:
zh-CN: 查询时间
en-US: Query Time
ja-JP: クエリ時間
aliasFields:
- sql_text
- start_time
@@ -21,19 +21,22 @@ app: memcached
name:
zh-CN: Memcached
en-US: Memcached
ja-JP: Memcached
# The description and help of this monitoring type
help:
zh-CN: HertzBeat 支持 Memcached 服务监控,通过(stats、stats settings、stats sizes)命令对(服务器进程ID、服务器已运行秒数、版本、当前线程数、item的大小、item的数量、当前存储占用的字节数)等指标进行监测。
en-US: HertzBeat supports Memcached service monitoring by using commands such as stats, stats settings, and stats sizes to monitor indicators such as server process ID, server running seconds, version, current number of threads, item size, number of items, and current byte storage usage.
zh-TW: HertzBeat支持Memcached服務監控,通過(stats、stats settings、stats sizes)命令對(服務器行程ID、服務器已運行秒數、版本、當前線程數、item的大小、item的數量、當前存儲佔用的位元組數)等名額進行監測。
ja-JP: HertzBeat は Memcached サービスの一般的なメトリクスを監視します。
# Input params define for monitoring(render web ui by the definition)
params:
# field-param field key
- field: host
# name-param field display i18n name
name:
zh-CN: Memcached服务的Host
en-US: Host of Memcached service
zh-CN: 目标Host
en-US: Target Host
ja-JP: 目標ホスト
# type-param field type(most mapping the html input type)
type: host
# required-true or false
@@ -44,6 +47,7 @@ params:
name:
zh-CN: 端口
en-US: Port
ja-JP: ポート
# type-param field type(most mapping the html input type)
type: number
# when type is number, range is required
@@ -59,6 +63,7 @@ metrics:
i18n:
zh-CN: 系统信息
en-US: Server Info
ja-JP: サーバー情報
# metrics scheduling priority(0->127)->(high->low), metrics with the same priority will be scheduled in parallel
# priority 0's metrics is availability metrics, it will be scheduled first, only availability metrics collect success will the scheduling continue
priority: 0
@@ -70,89 +75,106 @@ metrics:
i18n:
zh-CN: 响应时间
en-US: Response Time
ja-JP: 応答時間
- field: pid
type: 1
i18n:
zh-CN: 服务器进程ID
en-US: Pid
en-US: PID
ja-JP: プロセスID
- field: uptime
type: 0
unit: s
i18n:
zh-CN: 已运行秒数
en-US: Uptime
ja-JP: アップタイム
- field: version
type: 1
i18n:
zh-CN: 版本
en-US: Version
ja-JP: バージョン
- field: curr_connections
type: 0
i18n:
zh-CN: 当前连接数量
en-US: Curr Connections
ja-JP: 現在接続数
- field: auth_errors
type: 0
i18n:
zh-CN: 认证失败数目
en-US: Auth Errors
ja-JP: 認証失敗回数
- field: threads
type: 0
i18n:
zh-CN: 当前线程数
en-US: Threads
ja-JP: 現在スレッド数
- field: item_size
type: 0
unit: byte
i18n:
zh-CN: Item 大小
en-US: Item Size
ja-JP: Itemサイズ
- field: item_count
type: 0
i18n:
zh-CN: Item 数量
en-US: Item Count
ja-JP: Item数量
- field: curr_items
type: 0
i18n:
zh-CN: 当前存储的数据总数
en-US: Curr Items
ja-JP: 現在のItem数
- field: total_items
type: 0
i18n:
zh-CN: 启动以来存储的数据总数
en-US: Total Items
ja-JP: Item総数
- field: bytes
type: 0
unit: byte
i18n:
zh-CN: 当前存储占用的字节数
en-US: Bytes
ja-JP: バイト
- field: cmd_get
type: 0
i18n:
zh-CN: Get命令请求次数
en-US: Cmd Get
ja-JP: Getコマンドの実行回数
- field: cmd_set
type: 0
i18n:
zh-CN: Set命令请求次数
en-US: Cmd Set
ja-JP: Setコマンドの実行回数
- field: cmd_flush
type: 0
i18n:
zh-CN: Flush命令请求次数
en-US: Cmd Flush
ja-JP: Flushコマンドの実行回数
- field: get_misses
type: 0
i18n:
zh-CN: Get命令未命中次数
en-US: Get Misses
ja-JP: Getコマンドのミス回数
- field: delete_misses
type: 0
i18n:
zh-CN: Delete命令未命中次数
en-US: Delete Misses
ja-JP: Deleteコマンドのミス回数
# the protocol used for monitoring, eg: sql, ssh, http, telnet, wmi, snmp, sdk
protocol: memcached
# Specific collection configuration when protocol is telnet protocol
@@ -21,11 +21,13 @@ app: modbus
name:
zh-CN: ModBus服务器
en-US: ModBus Server
ja-JP: ModBusサーバー
# The description and help of this monitoring type
help:
zh-CN: HertzBeat对支持Modbus协议的服务进行(保持寄存器和线圈)相关指标进行采集
en-US: HertzBeat collects metrics related to maintaining registers and coils for services that support Modbus protocol
zh-TW: HertzBeat對支持Modbus協定的服務進行(保持寄存器和線圈)相關名額進行採集
ja-JP: HertzBeatはModbusプロトコルをサポートするサービスのメトリクスを監視します。
# Input params define for monitoring(render web ui by the definition)
params:
# field-param field key
@@ -34,6 +36,7 @@ params:
name:
zh-CN: ModBus服务Host
en-US: ModBus Server Host
ja-JP: ModBusサーバーのホスト
# type-param field type(most mapping the html input type)
type: host
# required-true or false
@@ -44,6 +47,7 @@ params:
name:
zh-CN: 端口
en-US: Port
ja-JP: ポート
# type-param field type(most mapping the html input type)
type: number
# when type is number, range is required
@@ -57,6 +61,7 @@ params:
name:
zh-CN: slaveId
en-US: slaveId
ja-JP: slaveId
# type-param field type(most mapping the html input type)
type: text
# required-true or false
@@ -66,6 +71,7 @@ params:
name:
zh-CN: 请求超时时间(ms)
en-US: Request Timeout(ms)
ja-JP: リクエストタイムアウト(ms)
type: number
# when type is number, range is required
range: '[400,200000]'
@@ -77,27 +83,25 @@ params:
name:
zh-CN: 保持寄存器地址
en-US: Holding Registers address
ja-JP: Holdingレジスタのアドレス
# type-param field type(most mapping the html input type)
type: array
# param field input placeholder
placeholder: 'Input RegisterAddress'
# required-true or false
required: true
# hide param-true or false
# hide: true
- field: coilRegisterAddresses
# name-param field display i18n name
name:
zh-CN: 线圈寄存器地址
en-US: Coil Register address
ja-JP: コイルレジスタのアドレス
# type-param field type(most mapping the html input type)
type: array
# param field input placeholder
placeholder: 'Input RegisterAddress'
# required-true or false
required: true
# hide param-true or false
# hide: true
# collect metrics config list
metrics:
@@ -106,6 +110,7 @@ metrics:
i18n:
zh-CN: 保持寄存器 统计信息
en-US: holding-register stats
ja-JP: Holdingレジスタの統計情報
# metrics scheduling priority(0->127)->(high->low), metrics with the same priority will be scheduled in parallel
# priority 0's metrics is availability metrics, it will be scheduled first, only availability metrics collect success will the scheduling continue
priority: 0
@@ -117,38 +122,40 @@ metrics:
i18n:
zh-CN: 响应时间
en-US: Response Time
ja-JP: 応答時間
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
- field: address-0
type: 0
i18n:
zh-CN: address-0
en-US: address-0
ja-JP: address-0
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
- field: address-1-0
type: 0
i18n:
zh-CN: address-1-0
en-US: address-1-0
ja-JP: address-1-0
- field: address-1-1
type: 0
i18n:
zh-CN: address-1-1
en-US: address-1-1
ja-JP: address-1-1
- field: address-1-2
type: 0
i18n:
zh-CN: address-1-2
en-US: address-1-2
# 指标别名列表,按照寄存器地址来进行命名的
# 例如 地址为下标为0 ,值为m holding-register:0
# 地址为下标为1,值为m[2] holding-register:1-0 holding-register:1-1
ja-JP: address-1-2
aliasFields:
- responseTime
- holding-register:0
- holding-register:1-0
- holding-register:1-1
- holding-register:1-2
# mapping and conversion expressions, use thesand aliasField above to calculate metrics value# (可选)指标映射转换计算表达式,与上面的别名一起作用,计算出最终需要的指标值# eg: cores=core1+core2, usage=usage, waitTimeallTime-runningTime
# mapping and conversion expressions, use the same aliasField above to calculate metrics value# eg: cores=core1+core2, usage=usage, waitTimeallTime-runningTime
calculates:
- responseTime=responseTime
- address-0=holding-register:0
@@ -173,6 +180,7 @@ metrics:
i18n:
zh-CN: 线圈 统计信息
en-US: coil stats
ja-JP: コイルの統計情報
# metrics scheduling priority(0->127)->(high->low), metrics with the same priority will be scheduled in parallel
# priority 0's metrics is availability metrics, it will be scheduled first, only availability metrics collect success will the scheduling continue
priority: 1
@@ -184,38 +192,40 @@ metrics:
i18n:
zh-CN: 响应时间
en-US: Response Time
ja-JP: 応答時間
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
- field: address-0
type: 0
i18n:
zh-CN: address-0
en-US: address-0
ja-JP: address-0
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
- field: address-1-0
type: 0
i18n:
zh-CN: address-1-0
en-US: address-1-0
ja-JP: address-1-0
- field: address-1-1
type: 0
i18n:
zh-CN: address-1-1
en-US: address-1-1
ja-JP: address-1-1
- field: address-1-2
type: 0
i18n:
zh-CN: address-1-2
en-US: address-1-2
# 指标别名列表,按照寄存器地址来进行命名的
# 例如 地址为下标为0 ,值为m coil:0
# 地址为下标为1,值为m[2] coil:1-0 coil:1-1
ja-JP: address-1-2
aliasFields:
- responseTime
- coil:0
- coil:1-0
- coil:1-1
- coil:1-2
# mapping and conversion expressions, use thesand aliasField above to calculate metrics value# (可选)指标映射转换计算表达式,与上面的别名一起作用,计算出最终需要的指标值# eg: cores=core1+core2, usage=usage, waitTimeallTime-runningTime
# mapping and conversion expressions, use the same aliasField above to calculate metrics value# eg: cores=core1+core2, usage=usage, waitTimeallTime-runningTime
calculates:
- responseTime=responseTime
- address-0=coil:0
@@ -22,11 +22,13 @@ app: mongodb
name:
zh-CN: MongoDB数据库
en-US: MongoDB
ja-JP: MongoDBデータベース
# The description and help of this monitoring type
help:
zh-CN: Hertzbeat 对 MongoDB 数据库的通用指标进行测量监控。<br>您可以点击 “<i>新建 MongoDB 数据库</i>” 并进行配置,或者选择“<i>更多操作</i>”,导入已有配置。
en-US: Hertzbeat monitors MongoDB general performance metrics. You could click the "<i>New MongoDB</i>" button and proceed with the configuration or import an existing setup through the "<i>More Actions</i>" menu.
zh-TW: Hertzbeat 對 MongoDB 數據庫的通用指標進行測量監控。<br>您可以點擊 “<i>新建 MongoDB 數據庫</i>” 並進行配置,或者選擇“<i>更多操作</i>”,導入已有配置。
ja-JP: Hertzbeat は MongoDBデータベースの一般的なメトリクスを監視します。<br>「<i>新規 MongoDBデータベース</i>」をクリックしてパラメタを設定した後、新規することができます。
helpLink:
zh-CN: https://hertzbeat.apache.org/zh-cn/docs/help/mongodb
en-US: https://hertzbeat.apache.org/docs/help/mongodb
@@ -38,6 +40,7 @@ params:
name:
zh-CN: 目标Host
en-US: Target Host
ja-JP: 目標ホスト
# type-param field type(most mapping the html input type)
type: host
# required-true or false
@@ -48,6 +51,7 @@ params:
name:
zh-CN: 端口
en-US: Port
ja-JP: ポート
# type-param field type(most mapping the html input type)
type: number
# when type is number, range is required
@@ -60,6 +64,7 @@ params:
name:
zh-CN: 用户名
en-US: Username
ja-JP: ユーザー名
type: text
limit: 50
required: false
@@ -67,12 +72,14 @@ params:
name:
zh-CN: 密码
en-US: Password
ja-JP: パスワード
type: password
required: false
- field: database
name:
zh-CN: 数据库
en-US: database
ja-JP: データベース
type: text
# when type is text, use limit to limit string length
limit: 50
@@ -82,6 +89,7 @@ params:
name:
zh-CN: 认证数据库
en-US: authenticationDatabase
ja-JP: 認証データベース
type: text
limit: 50
required: true
@@ -91,6 +99,7 @@ params:
name:
zh-CN: 连接超时时间(ms)
en-US: Connect Timeout(ms)
ja-JP: 接続タイムアウト(ms)
# type-param field type(most mapping the html input type)
type: number
# when type is number, range is required
@@ -115,6 +124,7 @@ metrics:
i18n:
zh-CN: 构建信息
en-US: Build Info
ja-JP: ビルド情報
# monitoring metrics list
fields:
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
@@ -123,36 +133,43 @@ metrics:
i18n:
zh-CN: 版本
en-US: Version
ja-JP: バージョン
- field: gitVersion
type: 1
i18n:
zh-CN: git版本
en-US: Git Version
ja-JP: Gitバージョン
- field: sysInfo
type: 1
i18n:
zh-CN: 系统信息
en-US: Sys Info
ja-JP: システム情報
- field: loaderFlags
type: 1
i18n:
zh-CN: 加载器标志
en-US: Loader Flags
ja-JP: ローダフラグ
- field: compilerFlags
type: 1
i18n:
zh-CN: 编译器标志
en-US: Compiler Flags
ja-JP: コンパイラフラグ
- field: allocator
type: 1
i18n:
zh-CN: 分配器
en-US: Allocator
ja-JP: メモリ割り当て器
- field: javascriptEngine
type: 1
i18n:
zh-CN: JavaScript引擎
en-US: JavaScript Engine
ja-JP: JavaScriptエンジン
# the protocol used for monitoring, eg: sql, ssh, http, telnet, wmi, snmp, sdk
protocol: mongodb
# the config content when protocol is mongodb
@@ -180,27 +197,32 @@ metrics:
i18n:
zh-CN: 服务器文档
en-US: Server Document
ja-JP: サーバードキュメント
fields:
- field: deleted
type: 0
i18n:
zh-CN: 已删除数
en-US: Deleted
ja-JP: 削除回数
- field: inserted
type: 0
i18n:
zh-CN: 已插入数
en-US: Inserted
ja-JP: インサート回数
- field: returned
type: 0
i18n:
zh-CN: 返回结果数
en-US: Returned
ja-JP: 戻る回数
- field: updated
type: 0
i18n:
zh-CN: 已更新数
en-US: Updated
ja-JP: 更新回数
protocol: mongodb
mongodb:
host: ^_^host^_^
@@ -217,17 +239,20 @@ metrics:
i18n:
zh-CN: 服务器操作
en-US: Server Operation
ja-JP: サーバー操作
fields:
- field: scanAndOrder
type: 0
i18n:
zh-CN: 扫描订购
en-US: Scan And Order
ja-JP: スキャンとオーダー
- field: writeConflicts
type: 0
i18n:
zh-CN: 写冲突
en-US: Write And Order
ja-JP: 書き込みとオーダー
protocol: mongodb
mongodb:
host: ^_^host^_^
@@ -242,19 +267,22 @@ metrics:
- name: server_ttl
priority: 3
i18n:
zh-CN: 服务器_ttl
en-US: Max Connections
zh-CN: 服务器_TTL
en-US: server TTL
ja-JP: サーバーTTL
fields:
- field: deletedDocuments
type: 0
i18n:
zh-CN: 已删除文档数
en-US: Deleted Documents
ja-JP: 削除されたドキュメント数
- field: passes
type: 0
i18n:
zh-CN: 通过数
en-US: Passes
ja-JP: パス数
protocol: mongodb
mongodb:
host: ^_^host^_^
@@ -272,49 +300,58 @@ metrics:
i18n:
zh-CN: 系统信息
en-US: System Info
ja-JP: システム情報
fields:
- field: currentTime
type: 1
i18n:
zh-CN: 当前时间
en-US: Current Time
ja-JP: 現在時刻
- field: hostname
type: 1
i18n:
zh-CN: 主机名
en-US: Host Name
ja-JP: ホスト名
- field: cpuAddrSize
type: 0
i18n:
zh-CN: cpu地址大小
en-US: CPU Address Size
ja-JP: CPUアドレスサイズ
- field: memSizeMB
type: 0
unit: MB
i18n:
zh-CN: 内存大小(MB
en-US: Memory Size MB
ja-JP: メモリサイズ(MB)
- field: memLimitMB
type: 0
unit: MB
i18n:
zh-CN: 内存限制(MB
en-US: Memory Limit MB
ja-JP: メモリ制限(MB)
- field: numCores
type: 0
i18n:
zh-CN: CPU 核数
en-US: Num Cores
ja-JP: コア数
- field: cpuArch
type: 1
i18n:
zh-CN: CPU 架构
en-US: CPU Architecture
ja-JP: CPUアーキテクチャ
- field: numaEnabled
type: 1
i18n:
zh-CN: numa是否启用
en-US: Numa Enabled
ja-JP: 有効化
protocol: mongodb
mongodb:
host: ^_^host^_^
@@ -331,22 +368,26 @@ metrics:
i18n:
zh-CN: 操作系统信息
en-US: OS Info
ja-JP: オーエス情報
fields:
- field: type
type: 1
i18n:
zh-CN: 操作系统类型
en-US: Type
ja-JP: タイプ
- field: name
type: 1
i18n:
zh-CN: 名称
en-US: Name
ja-JP: 名前
- field: version
type: 1
i18n:
zh-CN: Version
en-US: 版本号
ja-JP: バージョン
protocol: mongodb
mongodb:
host: ^_^host^_^
@@ -363,47 +404,56 @@ metrics:
i18n:
zh-CN: 额外信息
en-US: Extra Info
ja-JP: 追加情報
fields:
- field: versionString
type: 1
i18n:
zh-CN: 版本字符串
en-US: Version String
ja-JP: バージョン文字列
- field: libcVersion
type: 1
i18n:
zh-CN: 标准库版本
en-US: Libc Version
ja-JP: Libcバージョン
- field: kernelVersion
type: 1
i18n:
zh-CN: 内核版本
en-US: Kernel Version
ja-JP: カーネルバージョン
- field: cpuFrequencyMHz
type: 1
i18n:
zh-CN: CPU 频率(兆赫兹)
en-US: CPU Frequency MHz
ja-JP: CPU周波数(MHz
- field: cpuFeatures
type: 1
i18n:
zh-CN: CPU 特性
en-US: CPU Features
ja-JP: CPU機能
- field: pageSize
type: 0
i18n:
zh-CN: 内存页大小
en-US: Page Size
ja-JP: ページサイズ
- field: numPages
type: 0
i18n:
zh-CN: 内存页数量
en-US: Num Pages
ja-JP: ページ数量
- field: maxOpenFiles
type: 0
i18n:
zh-CN: 最大打开文件数
en-US: Max Open Files
ja-JP: 開けるファイルの最大数
protocol: mongodb
mongodb:
host: ^_^host^_^
@@ -22,11 +22,13 @@ app: mongodb_atlas
name:
zh-CN: MongoDB Atlas 数据库
en-US: MongoDB Atlas Database
ja-JP: MongoDB Atlas データベース
# The description and help of this monitoring type
help:
zh-CN: Hertzbeat 对 MongoDB Atlas 数据库的通用指标进行测量监控。<br>您可以点击 “<i>新建 MongoDB Atlas 数据库</i>” 并进行配置,或者选择“<i>更多操作</i>”,导入已有配置。
en-US: Hertzbeat monitors MongoDB Atlas general performance metrics. You could click the "<i>New MongoDB Atlas</i>" button and proceed with the configuration or import an existing setup through the "<i>More Actions</i>" menu.
zh-TW: Hertzbeat 對 MongoDB Atlas 數據庫的通用指標進行測量監控。<br>您可以點擊 “<i>新建 MongoDB Atlas 數據庫</i>” 並進行配置,或者選擇“<i>更多操作</i>”,導入已有配置。
ja-JP: Hertzbeat は MongoDB Atlas データベースの一般的なメトリクスを監視します。<br>「<i>新規 MongoDB Atlas データベース</i>」をクリックしてパラメタを設定した後、新規することができます。
helpLink:
zh-CN: https://hertzbeat.apache.org/zh-cn/docs/help/mongodb
en-US: https://hertzbeat.apache.org/docs/help/mongodb
@@ -38,6 +40,7 @@ params:
name:
zh-CN: 目标Host
en-US: Target Host
ja-JP: 目標ホスト
# type-param field type(most mapping the html input type)
type: host
# required-true or false
@@ -47,6 +50,7 @@ params:
name:
zh-CN: 用户名
en-US: Username
ja-JP: ユーザー名
type: text
limit: 50
required: false
@@ -54,12 +58,14 @@ params:
name:
zh-CN: 密码
en-US: Password
ja-JP: パスワード
type: password
required: false
- field: database
name:
zh-CN: 数据库
en-US: database
ja-JP: データベース
type: text
# when type is text, use limit to limit string length
limit: 50
@@ -69,6 +75,7 @@ params:
name:
zh-CN: 认证数据库
en-US: authenticationDatabase
ja-JP: 認証データベース
type: text
limit: 50
required: true
@@ -78,6 +85,7 @@ params:
name:
zh-CN: 连接超时时间(ms)
en-US: Connect Timeout(ms)
ja-JP: 接続タイムアウト(ms)
# type-param field type(most mapping the html input type)
type: number
# when type is number, range is required
@@ -93,7 +101,8 @@ params:
# name-param field display i18n name
name:
zh-CN: 集群模式
en-US: Cluster Model
en-US: Cluster Mode
ja-JP: クラスターモード
# type-param field type(most mapping the html input type)
type: text
# can get status information and other config params of MongoDB by running Diagnostic Commands
@@ -112,6 +121,7 @@ metrics:
i18n:
zh-CN: 构建信息
en-US: Build Info
ja-JP: ビルド情報
# monitoring metrics list
fields:
# field-metric name, type-metric type(0-number,1-string), unit-metric unit('%','ms','MB'), label-whether it is a metrics label field
@@ -120,26 +130,31 @@ metrics:
i18n:
zh-CN: 版本
en-US: Version
ja-JP: バージョン
- field: gitVersion
type: 1
i18n:
zh-CN: git版本
en-US: Git Version
ja-JP: Gitバージョン
- field: sysInfo
type: 1
i18n:
zh-CN: 系统信息
en-US: Sys Info
ja-JP: システム情報
- field: allocator
type: 1
i18n:
zh-CN: 分配器
en-US: Allocator
ja-JP: メモリ割り当て器
- field: javascriptEngine
type: 1
i18n:
zh-CN: JavaScript引擎
en-US: JavaScript Engine
ja-JP: JavaScriptエンジン
# the protocol used for monitoring, eg: sql, ssh, http, telnet, wmi, snmp, sdk
protocol: mongodb
# the config content when protocol is mongodb
@@ -169,37 +184,44 @@ metrics:
i18n:
zh-CN: 服务器文档
en-US: Server Document
ja-JP: サーバードキュメント
fields:
- field: delete
type: 0
i18n:
zh-CN: 已删除数
en-US: delete
ja-JP: 削除回数
- field: insert
type: 0
i18n:
zh-CN: 已插入数
en-US: insert
ja-JP: インサート回数
- field: update
type: 0
i18n:
zh-CN: 更新数
en-US: update
ja-JP: 更新回数
- field: query
type: 0
i18n:
zh-CN: 查询数
en-US: query
ja-JP: クエリ回数
- field: getmore
type: 0
i18n:
zh-CN: 光标中剩余文档的请求数
en-US: getmore
ja-JP: getmore
- field: command
type: 0
i18n:
zh-CN: 执行命令操作的总数
en-US: command
ja-JP: 実行されたコマンド数
protocol: mongodb
mongodb:
host: ^_^host^_^
@@ -217,22 +239,26 @@ metrics:
i18n:
zh-CN: 网络操作
en-US: Network Operation
ja-JP: ネットワーク操作
fields:
- field: bytesIn
type: 0
i18n:
zh-CN: 读取字节数
en-US: Bytes In
ja-JP: 読み取りバイト数
- field: bytesOut
type: 0
i18n:
zh-CN: 写入字节数
en-US: Bytes Out
ja-JP: 書き込みバイト数
- field: numRequests
type: 0
i18n:
zh-CN: 请求数
en-US: Request Num
ja-JP: リクエスト数
protocol: mongodb
mongodb:
host: ^_^host^_^
@@ -250,22 +276,26 @@ metrics:
i18n:
zh-CN: 连接信息
en-US: Connection Info
ja-JP: 接続情報
fields:
- field: current
type: 0
i18n:
zh-CN: 当前正在进行连接数
en-US: Current Connections
ja-JP: 現在接続数
- field: available
type: 0
i18n:
zh-CN: 可用连接数
en-US: Available Connections
ja-JP: 利用可能な接続数
- field: totalCreated
type: 0
i18n:
zh-CN: 总创建连接数
en-US: Total Created Connections
ja-JP: 作成された接続総数
protocol: mongodb
mongodb:
host: ^_^host^_^
@@ -283,56 +313,66 @@ metrics:
i18n:
zh-CN: 数据库统计
en-US: Database Statistics
ja-JP: データベース統計
fields:
- field: db
type: 1
i18n:
zh-CN: 数据库名称
en-US: Database Name
ja-JP: データベース名
- field: collections
type: 0
i18n:
zh-CN: 集合数
en-US: Collections
ja-JP: コレクション数
- field: views
type: 0
i18n:
zh-CN: 视图数
en-US: Views
ja-JP: ビュー数
- field: objects
type: 0
i18n:
zh-CN: 文档数
en-US: Objects
ja-JP: ドキュメント数
- field: avgObjSize
type: 0
unit: Bytes
i18n:
zh-CN: 文档平均大小
en-US: Document Avg Size
ja-JP: ドキュメントの平均サイズ
- field: dataSize
type: 0
unit: Bytes
i18n:
zh-CN: 文档大小
en-US: Document Size
ja-JP: ドキュメントサイズ
- field: storageSize
type: 0
unit: Bytes
i18n:
zh-CN: 使用存储空间大小
en-US: Storage Size
ja-JP: ストレージサイズ
- field: indexes
type: 0
i18n:
zh-CN: 索引数
en-US: Indexes
ja-JP: インデックス数
- field: indexSize
type: 0
unit: Bytes
i18n:
zh-CN: 索引大小
en-US: Index Size
ja-JP: インデックスサイズ
protocol: mongodb
mongodb:
host: ^_^host^_^

Some files were not shown because too many files have changed in this diff Show More