Compare commits

...
Author SHA1 Message Date
tomsun28 1f89e1ef5d fix: fix test
Signed-off-by: tomsun28 <tomsun28@outlook.com>
2026-02-15 23:10:29 +08:00
Tomsun28andgithub-actions[bot] abea1d08f7 Update .testcontainers.properties
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: Tomsun28 <tomsun28@outlook.com>
2026-02-15 16:01:19 +08:00
tomsun28 f27e527605 fix: fix the e2e container test
Signed-off-by: tomsun28 <tomsun28@outlook.com>
2026-02-15 15:59:37 +08:00
tomsun28 c4a402f4d6 fix: fix the e2e log test
Signed-off-by: tomsun28 <tomsun28@outlook.com>
2026-02-15 15:05:41 +08:00
Duansg ebfd3c73e3 [fix] Fixed template name mismatch issue (#4021) 2026-02-13 23:06:02 +08:00
Tomsun28 cc60f8453b docs: fix the sidebars i18n and version display, add auto deploy action (#4018)
Signed-off-by: tomsun28 <tomsun28@outlook.com>
2026-02-08 22:14:05 +08:00
0cf286a063 [Docs] Release version 1.8.0 documentation (#4017)
Signed-off-by: Tomsun28 <tomsun28@outlook.com>
Co-authored-by: Tomsun28 <tomsun28@outlook.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-02-07 10:18:41 +08:00
3da977a96f docs: update download page to v1.8.0 (#4015)
Co-authored-by: a-little-fool <2030509072@qq.com>
Co-authored-by: Duansg <siguoduan@gmail.com>
Co-authored-by: Tomsun28 <tomsun28@outlook.com>
2026-02-07 09:01:49 +08:00
Duansg c10761ec9a [improve] improved version validation documentation content (#4013) 2026-02-06 20:50:59 +08:00
Duansg b0c6324713 [fix] Fixed Prometheus task history charts not working in the Favorites tab (#4014) 2026-02-06 16:39:01 +08:00
DuansgandTomsun28 1b51c19c97 [fix]fixed sql formatting error (#4008)
Co-authored-by: Tomsun28 <tomsun28@outlook.com>
2026-01-30 00:00:31 +08:00
LunaRain_079 a83cb0f50f [docs] Fix HTTP Service Discovery API response format (#4005) 2026-01-29 22:10:15 +08:00
Tomsun28 d6022cb587 docs: update doc add contributors and pnpm lock (#4002)
Signed-off-by: tomsun28 <tomsun28@outlook.com>
2026-01-25 20:34:49 +08:00
LogicandTomsun28 67f525a603 [fix] update table name generation to include application context (#4001)
Co-authored-by: Tomsun28 <tomsun28@outlook.com>
2026-01-24 23:26:08 +08:00
LunaRain_079andDuansg 634006aa7e [refactor] Dynamically concatenate app parameter on the front end to accommodate Prometheus apps (#4000)
Co-authored-by: Duansg <siguoduan@gmail.com>
2026-01-24 23:06:25 +08:00
3eb3f926f5 [fix] Update query parameter encoding and error time parser in VictoriaMetricsClusterDataStorage (#3989)
Co-authored-by: Yang Chen <1597081640@qq.com>
Co-authored-by: aias00 <liuhongyu@apache.org>
Co-authored-by: Duansg <siguoduan@gmail.com>
2026-01-24 20:37:08 +08:00
47caa5bdf8 feat: support otlp http binary protobuf format log data and update doc (#3986)
Signed-off-by: Yang Chen <1597081640@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Logic <zqr10159@dromara.org>
2026-01-23 18:34:36 +08:00
Logic 67c3727362 [Refactor] tighten resource constraints for XML/XPath collector (#3999) 2026-01-22 18:51:01 +08:00
b510d6e5ea [feature] add datasource status retrieval and update alert settings UI (#3985)
Signed-off-by: Logic <zqr10159@126.com>
Signed-off-by: Duansg <siguoduan@gmail.com>
Co-authored-by: aias00 <liuhongyu@apache.org>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Duansg <siguoduan@gmail.com>
2026-01-20 22:43:47 +08:00
1ab27bd556 [doc]: update to Docusaurus v3.9.2 (#3984)
Co-authored-by: Tomsun28 <tomsun28@outlook.com>
Co-authored-by: Duansg <siguoduan@gmail.com>
Co-authored-by: aias00 <liuhongyu@apache.org>
2026-01-20 22:04:27 +08:00
Duansg 9600a5e56f [improve] upgrade script merge (#3987) 2026-01-19 23:47:32 +08:00
6c0d5c0db4 [fix]Fix issues such as index failure (#3953)
Co-authored-by: shown <yuluo08290126@gmail.com>
Co-authored-by: Yang Chen <1597081640@qq.com>
Co-authored-by: Logic <zqr10159@dromara.org>
2026-01-18 23:07:42 +08:00
4b010e6523 [monitoring-template] feat(mysql): add database account expiry metrics (#3969)
Co-authored-by: Logic <zqr10159@dromara.org>
Co-authored-by: Duansg <siguoduan@gmail.com>
2026-01-18 22:29:03 +08:00
8962253dd2 [manager] bugfix: fix incorrect status page date calculation (#3962)
Signed-off-by: Tomsun28 <tomsun28@outlook.com>
Co-authored-by: Duansg <siguoduan@gmail.com>
Co-authored-by: Tomsun28 <tomsun28@outlook.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-17 20:08:12 +08:00
Logic afd89e4309 Fix htaccess (#3980)
Signed-off-by: Logic <zqr10159@dromara.org>
2026-01-16 22:16:15 +08:00
Logic 43332f49db [fix]: update Content Security Policy in .htaccess to include new domains (#3979) 2026-01-16 20:53:11 +08:00
Duansg cc90de5541 [fix] fixed cache parsing issues (#3977) 2026-01-16 15:47:09 +08:00
Tomsun28 5e1460bf29 docs: update invitation email template (#3975) 2026-01-16 09:10:51 +08:00
LogicandYang Chen 7dbdca849a [doc]: add monitoring documentation for various service discovery methods (#3965)
Co-authored-by: Yang Chen <1597081640@qq.com>
2026-01-15 21:27:42 +08:00
Yang ChenandDuansg df525d2d32 fix: add GreptimeDB docker-compose assembly configuration (#3960)
Co-authored-by: Duansg <siguoduan@gmail.com>
2026-01-14 20:51:24 +08:00
LunaRain_079 dc96f1766b [Fix]: fix TAG type mismatch by introducing schema version for TDengine (#3963) 2026-01-13 21:29:27 +08:00
Tomsun28andDuansg 70da7d5dce docs: update docs link (#3958)
Co-authored-by: Duansg <siguoduan@gmail.com>
2026-01-11 18:46:13 +08:00
Prakash Kumar 0f1169ce36 [Fix] Fixed alignment of header social links on mobile (#3959) 2026-01-11 09:57:39 +08:00
KOYR 1c38d01abb style: widen table column to prevent label truncation (#3957) 2026-01-10 22:10:53 +08:00
f82b5add11 [feature] Update CSP permissions to include Kapa.ai services (#3948)
Signed-off-by: Duansg <siguoduan@gmail.com>
Co-authored-by: Duansg <siguoduan@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-08 09:49:48 +08:00
KOYR 8e2c1f3b07 style: adjust grid layout width of form labels (#3950) 2026-01-07 15:06:45 +08:00
Logic 8ef92d70d2 [fix] Update overflow properties and enhance swiper container styles (#3947) 2026-01-07 08:10:37 +08:00
Yang Chen b5ce5c9e9d [fix] refactor system built-in labels validation logic (#3946) 2026-01-06 15:47:25 +08:00
P_Peaceful a19a399acd [bugfix&improve] Fix alert label persistence and improve custom label input (#3933) 2026-01-06 12:01:53 +08:00
529d161478 [improve] Optimize Tooltip display logic and support scrollable content (#3941)
Signed-off-by: Tomsun28 <tomsun28@outlook.com>
Co-authored-by: Tomsun28 <tomsun28@outlook.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-04 23:24:46 +08:00
168f78ef23 chore: optimzie docker compose docs hertzbeat version (#3929)
Signed-off-by: yuluo-yx <yuluo08290126@gmail.com>
Co-authored-by: aias00 <liuhongyu@apache.org>
Co-authored-by: Duansg <siguoduan@gmail.com>
2026-01-03 19:05:35 +08:00
KOYRandDuansg a159127572 [improve]: Optimize yAxis scale to better reflect data fluctuations (#3939)
Co-authored-by: Duansg <siguoduan@gmail.com>
2026-01-03 01:57:14 +08:00
88933ce0bb [feature] Update UI text and styles for website (#3934)
Signed-off-by: Logic <zqr10159@126.com>
Signed-off-by: Duansg <siguoduan@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Duansg <siguoduan@gmail.com>
Co-authored-by: aias00 <liuhongyu@apache.org>
2025-12-31 16:00:13 +08:00
Logic 6d703f7eae [feature] add Kapa widget script to docusaurus configuration (#3938) 2025-12-31 10:30:49 +08:00
Duansg f4a0fe3445 [fix] fixed CSP blocking algolia from loading on the page (#3935) 2025-12-30 09:56:58 +08:00
Sahil Shadwal b174c8622c [bugfix]: fix for the collector tab click effect error (#3932) 2025-12-28 18:42:04 +08:00
LathikaBaddam 98e0412ee6 [docs] improve English translation and technical terms for MySQL (#3928) 2025-12-27 20:20:04 +08:00
Logic a138238994 [bugfix] Enhance expiration time parsing to support strict day format (#3924) 2025-12-22 20:44:21 +08:00
Duansgandlynx009 01f5cb1fd6 [feat] Add Grafana to follow the system theme (#3920)
Co-authored-by: lynx009 <2030509072@qq.com>
2025-12-20 19:44:53 +08:00
Tomsun28andYang Chen 472554bc55 [chore] update docker compose hertzbeat version (#3916)
Signed-off-by: tomsun28 <tomsun28@outlook.com>
Co-authored-by: Yang Chen <1597081640@qq.com>
2025-12-19 14:15:28 +08:00
1570 changed files with 108784 additions and 8054 deletions
+47 -1
View File
@@ -2545,7 +2545,8 @@
"avatar_url": "https://avatars.githubusercontent.com/u/176664901?v=4",
"profile": "https://github.com/Saramanda9988",
"contributions": [
"doc"
"doc",
"code"
]
},
{
@@ -2667,6 +2668,51 @@
"contributions": [
"code"
]
},
{
"login": "KOYR",
"name": "KOYR",
"avatar_url": "https://avatars.githubusercontent.com/u/53216619?v=4",
"profile": "https://github.com/KOYR",
"contributions": [
"code"
]
},
{
"login": "Lathika226",
"name": "LathikaBaddam",
"avatar_url": "https://avatars.githubusercontent.com/u/178710568?v=4",
"profile": "https://github.com/Lathika226",
"contributions": [
"doc"
]
},
{
"login": "Sahil-Shadwal",
"name": "Sahil Shadwal",
"avatar_url": "https://avatars.githubusercontent.com/u/119167601?v=4",
"profile": "https://shadwal.space/",
"contributions": [
"code"
]
},
{
"login": "BhanuNidumolu",
"name": "N.Bhanu Prasad",
"avatar_url": "https://avatars.githubusercontent.com/u/180380413?v=4",
"profile": "https://github.com/BhanuNidumolu",
"contributions": [
"code"
]
},
{
"login": "Prakash1185",
"name": "Prakash Kumar",
"avatar_url": "https://avatars.githubusercontent.com/u/183058331?v=4",
"profile": "https://prakashh-portfolio.vercel.app/",
"contributions": [
"code"
]
}
],
"contributorsPerLine": 7,
+82
View File
@@ -0,0 +1,82 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
name: DOC Deploy
on:
# Trigger on push to master branch when home directory files are changed
push:
branches: [ master ]
paths:
- 'home/**'
# Allow manual workflow trigger
workflow_dispatch:
# Set permissions for deployment
permissions:
contents: write
jobs:
deploy:
name: Deploy Documentation to asf-site Branch
runs-on: ubuntu-latest
steps:
# Checkout the repository code
- name: Checkout Repository
uses: actions/checkout@v4
with:
fetch-depth: 0
# Setup Node.js environment
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
# Install dependencies in home directory
- name: Install Dependencies
working-directory: home
run: npm ci
# Build the documentation site
- name: Build Documentation
working-directory: home
run: npm run build
env:
NODE_OPTIONS: --max-old-space-size=4096
# Deploy to asf-site branch using peaceiris/actions-gh-pages
- name: Deploy to asf-site Branch
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./home/build
publish_branch: asf-site
force_orphan: true
user_name: 'github-actions[bot]'
user_email: 'github-actions[bot]@users.noreply.github.com'
commit_message: 'Deploy documentation to asf-site branch'
# Output deployment status
- name: Deployment Summary
run: |
echo "Documentation deployed successfully!"
echo "Deployment branch: asf-site"
echo "Source branch: ${{ github.ref_name }}"
echo "Commit SHA: ${{ github.sha }}"
+22
View File
@@ -0,0 +1,22 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# Testcontainers configuration
# Disable Ryuk container reuse to avoid issues in CI
testcontainers.reuse.enable=false
# Configure Docker host detection
testcontainers.docker.client.strategy=org.testcontainers.dockerclient.UnixSocketClientProviderStrategy
+8 -1
View File
@@ -525,7 +525,7 @@ Thanks to these wonderful people, welcome to join us:
<td align="center" valign="top" width="14.28%"><a href="https://github.com/lx1229"><img src="https://avatars.githubusercontent.com/u/44620005?v=4?s=100" width="100px;" alt="Liuxin"/><br /><sub><b>Liuxin</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=lx1229" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/yy549159265"><img src="https://avatars.githubusercontent.com/u/40821310?v=4?s=100" width="100px;" alt="yy549159265"/><br /><sub><b>yy549159265</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=yy549159265" title="Code">💻</a> <a href="https://github.com/apache/hertzbeat/commits?author=yy549159265" title="Tests">⚠️</a> <a href="#design-yy549159265" title="Design">🎨</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/cto-huhang"><img src="https://avatars.githubusercontent.com/u/53338629?v=4?s=100" width="100px;" alt="cto-huhang"/><br /><sub><b>cto-huhang</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=cto-huhang" title="Documentation">📖</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Saramanda9988"><img src="https://avatars.githubusercontent.com/u/176664901?v=4?s=100" width="100px;" alt="LunaRain_079"/><br /><sub><b>LunaRain_079</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Saramanda9988" title="Documentation">📖</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Saramanda9988"><img src="https://avatars.githubusercontent.com/u/176664901?v=4?s=100" width="100px;" alt="LunaRain_079"/><br /><sub><b>LunaRain_079</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Saramanda9988" title="Documentation">📖</a> <a href="https://github.com/apache/hertzbeat/commits?author=Saramanda9988" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/delei"><img src="https://avatars.githubusercontent.com/u/17263766?v=4?s=100" width="100px;" alt="DeleiGuo"/><br /><sub><b>DeleiGuo</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=delei" title="Documentation">📖</a> <a href="https://github.com/apache/hertzbeat/commits?author=delei" title="Code">💻</a> <a href="https://github.com/apache/hertzbeat/commits?author=delei" title="Tests">⚠️</a> <a href="https://github.com/apache/hertzbeat/issues?q=author%3Adelei" title="Bug reports">🐛</a></td>
</tr>
<tr>
@@ -543,6 +543,13 @@ Thanks to these wonderful people, welcome to join us:
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Jetiaime"><img src="https://avatars.githubusercontent.com/u/93769000?v=4?s=100" width="100px;" alt="TeAmo"/><br /><sub><b>TeAmo</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Jetiaime" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/pentium100"><img src="https://avatars.githubusercontent.com/u/27917?v=4?s=100" width="100px;" alt="pentium100"/><br /><sub><b>pentium100</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=pentium100" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/dedyks"><img src="https://avatars.githubusercontent.com/u/23741665?v=4?s=100" width="100px;" alt="Dedy Kurniawan Santoso"/><br /><sub><b>Dedy Kurniawan Santoso</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=dedyks" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/KOYR"><img src="https://avatars.githubusercontent.com/u/53216619?v=4?s=100" width="100px;" alt="KOYR"/><br /><sub><b>KOYR</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=KOYR" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Lathika226"><img src="https://avatars.githubusercontent.com/u/178710568?v=4?s=100" width="100px;" alt="LathikaBaddam"/><br /><sub><b>LathikaBaddam</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Lathika226" title="Documentation">📖</a></td>
</tr>
<tr>
<td align="center" valign="top" width="14.28%"><a href="https://shadwal.space/"><img src="https://avatars.githubusercontent.com/u/119167601?v=4?s=100" width="100px;" alt="Sahil Shadwal"/><br /><sub><b>Sahil Shadwal</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Sahil-Shadwal" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/BhanuNidumolu"><img src="https://avatars.githubusercontent.com/u/180380413?v=4?s=100" width="100px;" alt="N.Bhanu Prasad"/><br /><sub><b>N.Bhanu Prasad</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=BhanuNidumolu" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://prakashh-portfolio.vercel.app/"><img src="https://avatars.githubusercontent.com/u/183058331?v=4?s=100" width="100px;" alt="Prakash Kumar"/><br /><sub><b>Prakash Kumar</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Prakash1185" title="Code">💻</a></td>
</tr>
</tbody>
</table>
+8 -1
View File
@@ -522,7 +522,7 @@ Thanks these wonderful people, welcome to join us:
<td align="center" valign="top" width="14.28%"><a href="https://github.com/lx1229"><img src="https://avatars.githubusercontent.com/u/44620005?v=4?s=100" width="100px;" alt="Liuxin"/><br /><sub><b>Liuxin</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=lx1229" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/yy549159265"><img src="https://avatars.githubusercontent.com/u/40821310?v=4?s=100" width="100px;" alt="yy549159265"/><br /><sub><b>yy549159265</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=yy549159265" title="Code">💻</a> <a href="https://github.com/apache/hertzbeat/commits?author=yy549159265" title="Tests">⚠️</a> <a href="#design-yy549159265" title="Design">🎨</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/cto-huhang"><img src="https://avatars.githubusercontent.com/u/53338629?v=4?s=100" width="100px;" alt="cto-huhang"/><br /><sub><b>cto-huhang</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=cto-huhang" title="Documentation">📖</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Saramanda9988"><img src="https://avatars.githubusercontent.com/u/176664901?v=4?s=100" width="100px;" alt="LunaRain_079"/><br /><sub><b>LunaRain_079</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Saramanda9988" title="Documentation">📖</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Saramanda9988"><img src="https://avatars.githubusercontent.com/u/176664901?v=4?s=100" width="100px;" alt="LunaRain_079"/><br /><sub><b>LunaRain_079</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Saramanda9988" title="Documentation">📖</a> <a href="https://github.com/apache/hertzbeat/commits?author=Saramanda9988" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/delei"><img src="https://avatars.githubusercontent.com/u/17263766?v=4?s=100" width="100px;" alt="DeleiGuo"/><br /><sub><b>DeleiGuo</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=delei" title="Documentation">📖</a> <a href="https://github.com/apache/hertzbeat/commits?author=delei" title="Code">💻</a> <a href="https://github.com/apache/hertzbeat/commits?author=delei" title="Tests">⚠️</a> <a href="https://github.com/apache/hertzbeat/issues?q=author%3Adelei" title="Bug reports">🐛</a></td>
</tr>
<tr>
@@ -540,6 +540,13 @@ Thanks these wonderful people, welcome to join us:
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Jetiaime"><img src="https://avatars.githubusercontent.com/u/93769000?v=4?s=100" width="100px;" alt="TeAmo"/><br /><sub><b>TeAmo</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Jetiaime" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/pentium100"><img src="https://avatars.githubusercontent.com/u/27917?v=4?s=100" width="100px;" alt="pentium100"/><br /><sub><b>pentium100</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=pentium100" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/dedyks"><img src="https://avatars.githubusercontent.com/u/23741665?v=4?s=100" width="100px;" alt="Dedy Kurniawan Santoso"/><br /><sub><b>Dedy Kurniawan Santoso</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=dedyks" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/KOYR"><img src="https://avatars.githubusercontent.com/u/53216619?v=4?s=100" width="100px;" alt="KOYR"/><br /><sub><b>KOYR</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=KOYR" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Lathika226"><img src="https://avatars.githubusercontent.com/u/178710568?v=4?s=100" width="100px;" alt="LathikaBaddam"/><br /><sub><b>LathikaBaddam</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Lathika226" title="Documentation">📖</a></td>
</tr>
<tr>
<td align="center" valign="top" width="14.28%"><a href="https://shadwal.space/"><img src="https://avatars.githubusercontent.com/u/119167601?v=4?s=100" width="100px;" alt="Sahil Shadwal"/><br /><sub><b>Sahil Shadwal</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Sahil-Shadwal" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/BhanuNidumolu"><img src="https://avatars.githubusercontent.com/u/180380413?v=4?s=100" width="100px;" alt="N.Bhanu Prasad"/><br /><sub><b>N.Bhanu Prasad</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=BhanuNidumolu" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://prakashh-portfolio.vercel.app/"><img src="https://avatars.githubusercontent.com/u/183058331?v=4?s=100" width="100px;" alt="Prakash Kumar"/><br /><sub><b>Prakash Kumar</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Prakash1185" title="Code">💻</a></td>
</tr>
</tbody>
</table>
+8 -1
View File
@@ -525,7 +525,7 @@ Thanks these wonderful people, welcome to join us:
<td align="center" valign="top" width="14.28%"><a href="https://github.com/lx1229"><img src="https://avatars.githubusercontent.com/u/44620005?v=4?s=100" width="100px;" alt="Liuxin"/><br /><sub><b>Liuxin</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=lx1229" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/yy549159265"><img src="https://avatars.githubusercontent.com/u/40821310?v=4?s=100" width="100px;" alt="yy549159265"/><br /><sub><b>yy549159265</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=yy549159265" title="Code">💻</a> <a href="https://github.com/apache/hertzbeat/commits?author=yy549159265" title="Tests">⚠️</a> <a href="#design-yy549159265" title="Design">🎨</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/cto-huhang"><img src="https://avatars.githubusercontent.com/u/53338629?v=4?s=100" width="100px;" alt="cto-huhang"/><br /><sub><b>cto-huhang</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=cto-huhang" title="Documentation">📖</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Saramanda9988"><img src="https://avatars.githubusercontent.com/u/176664901?v=4?s=100" width="100px;" alt="LunaRain_079"/><br /><sub><b>LunaRain_079</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Saramanda9988" title="Documentation">📖</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Saramanda9988"><img src="https://avatars.githubusercontent.com/u/176664901?v=4?s=100" width="100px;" alt="LunaRain_079"/><br /><sub><b>LunaRain_079</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Saramanda9988" title="Documentation">📖</a> <a href="https://github.com/apache/hertzbeat/commits?author=Saramanda9988" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/delei"><img src="https://avatars.githubusercontent.com/u/17263766?v=4?s=100" width="100px;" alt="DeleiGuo"/><br /><sub><b>DeleiGuo</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=delei" title="Documentation">📖</a> <a href="https://github.com/apache/hertzbeat/commits?author=delei" title="Code">💻</a> <a href="https://github.com/apache/hertzbeat/commits?author=delei" title="Tests">⚠️</a> <a href="https://github.com/apache/hertzbeat/issues?q=author%3Adelei" title="Bug reports">🐛</a></td>
</tr>
<tr>
@@ -543,6 +543,13 @@ Thanks these wonderful people, welcome to join us:
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Jetiaime"><img src="https://avatars.githubusercontent.com/u/93769000?v=4?s=100" width="100px;" alt="TeAmo"/><br /><sub><b>TeAmo</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Jetiaime" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/pentium100"><img src="https://avatars.githubusercontent.com/u/27917?v=4?s=100" width="100px;" alt="pentium100"/><br /><sub><b>pentium100</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=pentium100" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/dedyks"><img src="https://avatars.githubusercontent.com/u/23741665?v=4?s=100" width="100px;" alt="Dedy Kurniawan Santoso"/><br /><sub><b>Dedy Kurniawan Santoso</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=dedyks" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/KOYR"><img src="https://avatars.githubusercontent.com/u/53216619?v=4?s=100" width="100px;" alt="KOYR"/><br /><sub><b>KOYR</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=KOYR" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/Lathika226"><img src="https://avatars.githubusercontent.com/u/178710568?v=4?s=100" width="100px;" alt="LathikaBaddam"/><br /><sub><b>LathikaBaddam</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Lathika226" title="Documentation">📖</a></td>
</tr>
<tr>
<td align="center" valign="top" width="14.28%"><a href="https://shadwal.space/"><img src="https://avatars.githubusercontent.com/u/119167601?v=4?s=100" width="100px;" alt="Sahil Shadwal"/><br /><sub><b>Sahil Shadwal</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Sahil-Shadwal" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://github.com/BhanuNidumolu"><img src="https://avatars.githubusercontent.com/u/180380413?v=4?s=100" width="100px;" alt="N.Bhanu Prasad"/><br /><sub><b>N.Bhanu Prasad</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=BhanuNidumolu" title="Code">💻</a></td>
<td align="center" valign="top" width="14.28%"><a href="https://prakashh-portfolio.vercel.app/"><img src="https://avatars.githubusercontent.com/u/183058331?v=4?s=100" width="100px;" alt="Prakash Kumar"/><br /><sub><b>Prakash Kumar</b></sub></a><br /><a href="https://github.com/apache/hertzbeat/commits?author=Prakash1185" title="Code">💻</a></td>
</tr>
</tbody>
</table>
@@ -22,6 +22,7 @@ import io.swagger.v3.oas.annotations.Parameter;
import io.swagger.v3.oas.annotations.tags.Tag;
import jakarta.validation.Valid;
import org.apache.hertzbeat.alert.service.AlertDefineService;
import org.apache.hertzbeat.alert.service.DataSourceService;
import org.apache.hertzbeat.common.entity.alerter.AlertDefine;
import org.apache.hertzbeat.common.entity.dto.Message;
import org.apache.hertzbeat.common.support.exception.AlertExpressionException;
@@ -57,6 +58,9 @@ public class AlertDefineController {
@Autowired
private AlertDefineService alertDefineService;
@Autowired
private DataSourceService dataSourceService;
@PostMapping
@Operation(summary = "New Alarm Definition", description = "Added an alarm definition")
public ResponseEntity<Message<Void>> addNewAlertDefine(@Valid @RequestBody AlertDefine alertDefine) {
@@ -112,4 +116,12 @@ public class AlertDefineController {
}
}
@GetMapping(path = "/datasource/status")
@Operation(summary = "Get available datasource executors status",
description = "Get status of available datasource executors for periodic alerts")
public ResponseEntity<Message<Map<String, Object>>> getDatasourceStatus() {
Map<String, Object> status = dataSourceService.getAvailableExecutors();
return ResponseEntity.ok(Message.successWithData(status));
}
}
@@ -40,4 +40,10 @@ public interface DataSourceService {
* @return result
*/
List<Map<String, Object>> query(String datasource, String expr);
/**
* Get available datasource executors status
* @return map containing available executors by type (promql, sql)
*/
Map<String, Object> getAvailableExecutors();
}
@@ -29,11 +29,14 @@ import org.apache.hertzbeat.alert.dao.AlertDefineDao;
import org.apache.hertzbeat.alert.service.AlertDefineImExportService;
import org.apache.hertzbeat.alert.service.AlertDefineService;
import org.apache.hertzbeat.alert.service.DataSourceService;
import org.apache.hertzbeat.base.dao.LabelDao;
import org.apache.hertzbeat.base.service.LabelService;
import org.apache.hertzbeat.common.cache.CacheFactory;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.constants.ExportFileConstants;
import org.apache.hertzbeat.common.constants.SignConstants;
import org.apache.hertzbeat.common.entity.alerter.AlertDefine;
import org.apache.hertzbeat.common.entity.manager.Label;
import org.apache.hertzbeat.common.util.FileUtil;
import org.apache.hertzbeat.common.util.JexlExpressionRunner;
import org.springframework.beans.factory.annotation.Autowired;
@@ -48,6 +51,7 @@ import org.springframework.transaction.annotation.Transactional;
import org.springframework.util.StringUtils;
import org.springframework.web.multipart.MultipartFile;
import javax.annotation.Resource;
import java.net.URLDecoder;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
@@ -58,6 +62,7 @@ import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import java.util.stream.Collectors;
/**
* Alarm definition management interface implementation
@@ -69,16 +74,31 @@ public class AlertDefineServiceImpl implements AlertDefineService {
@Autowired
private AlertDefineDao alertDefineDao;
@Autowired
private PeriodicAlertRuleScheduler periodicAlertRuleScheduler;
@Resource
private LabelService labelService;
@Resource
private LabelDao labelDao;
private final DataSourceService dataSourceService;
private final Map<String, AlertDefineImExportService> alertDefineImExportServiceMap = new HashMap<>();
private static final String CONTENT_TYPE = MediaType.APPLICATION_OCTET_STREAM_VALUE + SignConstants.SINGLE_MARK + "charset=" + StandardCharsets.UTF_8;
private static final Set<String> SYSTEM_BUILT_IN_LABELS = Set.of(
CommonConstants.LABEL_INSTANCE,
CommonConstants.LABEL_DEFINE_ID,
CommonConstants.LABEL_ALERT_NAME,
CommonConstants.LABEL_INSTANCE_NAME,
CommonConstants.LABEL_ALERT_SEVERITY,
CommonConstants.ALERT_MODE_LABEL
);
public AlertDefineServiceImpl(List<AlertDefineImExportService> alertDefineImExportServiceList, DataSourceService dataSourceService) {
alertDefineImExportServiceList.forEach(it -> alertDefineImExportServiceMap.put(it.type(), it));
this.dataSourceService = dataSourceService;
@@ -93,7 +113,7 @@ public class AlertDefineServiceImpl implements AlertDefineService {
JexlExpressionRunner.compile(alertDefine.getExpr());
} catch (Exception e) {
throw new IllegalArgumentException("alert expr error: " + e.getMessage());
}
}
}
}
// the name of the alarm rule is unique
@@ -107,6 +127,7 @@ public class AlertDefineServiceImpl implements AlertDefineService {
@Override
public void addAlertDefine(AlertDefine alertDefine) throws RuntimeException {
saveNewCustomLabel(alertDefine);
alertDefine = alertDefineDao.saveAndFlush(alertDefine);
periodicAlertRuleScheduler.updateSchedule(alertDefine);
CacheFactory.clearAlertDefineCache();
@@ -114,11 +135,31 @@ public class AlertDefineServiceImpl implements AlertDefineService {
@Override
public void modifyAlertDefine(AlertDefine alertDefine) throws RuntimeException {
saveNewCustomLabel(alertDefine);
alertDefineDao.saveAndFlush(alertDefine);
periodicAlertRuleScheduler.updateSchedule(alertDefine);
CacheFactory.clearAlertDefineCache();
}
private void saveNewCustomLabel(AlertDefine alertDefine) {
Map<String, String> labels = alertDefine.getLabels();
if (labels == null) {
labels = new HashMap<>(8);
alertDefine.setLabels(labels);
}
Map<String, String> customLabels = labels.entrySet().stream()
.filter(entry -> !isSystemBuiltInLabel(entry.getKey()))
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
List<Label> addLabels = labelService.determineNewLabels(customLabels.entrySet());
if (!addLabels.isEmpty()) {
labelDao.saveAll(addLabels);
}
}
private boolean isSystemBuiltInLabel(String labelKey) {
return SYSTEM_BUILT_IN_LABELS.contains(labelKey);
}
@Override
public void deleteAlertDefine(long alertId) throws RuntimeException {
alertDefineDao.deleteById(alertId);
@@ -256,7 +297,7 @@ public class AlertDefineServiceImpl implements AlertDefineService {
if (!StringUtils.hasText(type)) {
throw new IllegalArgumentException("Alert definition type cannot be null or empty");
}
switch (type) {
case CommonConstants.METRIC_ALERT_THRESHOLD_TYPE_REALTIME:
case CommonConstants.METRIC_ALERT_THRESHOLD_TYPE_PERIODIC:
@@ -267,7 +308,7 @@ public class AlertDefineServiceImpl implements AlertDefineService {
default:
throw new IllegalArgumentException("Unsupported alert definition type: " + type);
}
// Query enabled alert definitions by type
return alertDefineDao.findAlertDefinesByTypeAndEnableTrue(type);
}
@@ -72,13 +72,6 @@ public class DataSourceServiceImpl implements DataSourceService {
.recordStats()
.build();
@Getter
private final Cache<String, CommonTokenStream> tokenStreamCache = Caffeine.newBuilder()
.maximumSize(512)
.expireAfterWrite(30, TimeUnit.MINUTES)
.recordStats()
.build();
public DataSourceServiceImpl(@Autowired(required = false) List<QueryExecutor> executors) {
this.executors = executors != null ? executors : Collections.emptyList();
this.sqlSecurityValidator = new SqlSecurityValidator(DEFAULT_ALLOWED_TABLES);
@@ -159,10 +152,12 @@ public class DataSourceServiceImpl implements DataSourceService {
}
private List<Map<String, Object>> evaluate(String expr, QueryExecutor executor) {
CommonTokenStream tokens = tokenStreamCache.get(expr, this::createTokenStream);
CommonTokenStream tokens = createTokenStream(expr);
AlertExpressionParser parser = new AlertExpressionParser(tokens);
ParseTree tree = expressionCache.get(expr, e -> parser.expr());
if (null != tokens && tokens.LA(1) != Token.EOF) {
// Validate EOF only during the first parsing (when tokens.index > 0)
// Skip EOF check when cache hit, as it has already been validated on the first pass.
if (tokens.index() > 0 && tokens.LA(1) != Token.EOF) {
throw new AlertExpressionException(bundle.getString("alerter.calculate.parse.error"));
}
AlertExpressionEvalVisitor visitor = new AlertExpressionEvalVisitor(executor, tokens);
@@ -173,4 +168,34 @@ public class DataSourceServiceImpl implements DataSourceService {
AlertExpressionLexer lexer = new AlertExpressionLexer(CharStreams.fromString(expr));
return new CommonTokenStream(lexer);
}
@Override
public Map<String, Object> getAvailableExecutors() {
boolean hasPromqlExecutor = false;
boolean hasSqlExecutor = false;
java.util.Set<String> availableExecutors = new java.util.HashSet<>();
if (executors != null) {
for (QueryExecutor executor : executors) {
String datasource = executor.getDatasource();
availableExecutors.add(datasource);
// Check if executor supports promql
if (executor.support(WarehouseConstants.PROMQL)) {
hasPromqlExecutor = true;
}
// Check if executor supports sql
if (executor.support(WarehouseConstants.SQL)) {
hasSqlExecutor = true;
}
}
}
Map<String, Object> result = new java.util.HashMap<>(8);
result.put("hasPromqlExecutor", hasPromqlExecutor);
result.put("hasSqlExecutor", hasSqlExecutor);
result.put("availableExecutors", availableExecutors);
return result;
}
}
@@ -21,6 +21,8 @@ import com.google.common.collect.Lists;
import org.apache.hertzbeat.alert.calculate.periodic.PeriodicAlertRuleScheduler;
import org.apache.hertzbeat.alert.dao.AlertDefineDao;
import org.apache.hertzbeat.alert.service.impl.AlertDefineServiceImpl;
import org.apache.hertzbeat.base.dao.LabelDao;
import org.apache.hertzbeat.base.service.LabelService;
import org.apache.hertzbeat.common.cache.CacheFactory;
import org.apache.hertzbeat.common.entity.alerter.AlertDefine;
import org.junit.jupiter.api.BeforeEach;
@@ -69,7 +71,13 @@ class AlertDefineServiceTest {
@Mock
private AlertDefineDao alertDefineDao;
@Mock
private LabelService labelService;
@Mock
private LabelDao labelDao;
@Mock
private PeriodicAlertRuleScheduler periodicAlertRuleScheduler;
@@ -86,6 +94,8 @@ class AlertDefineServiceTest {
void setUp() {
ReflectionTestUtils.setField(this.alertDefineService, "alertDefineDao", alertDefineDao);
ReflectionTestUtils.setField(this.alertDefineService, "periodicAlertRuleScheduler", periodicAlertRuleScheduler);
ReflectionTestUtils.setField(this.alertDefineService, "labelService", labelService);
ReflectionTestUtils.setField(this.alertDefineService, "labelDao", labelDao);
this.alertDefine = AlertDefine.builder()
.id(1L)
@@ -187,40 +197,40 @@ class AlertDefineServiceTest {
AlertDefine.builder().id(1L).type(METRIC_ALERT_THRESHOLD_TYPE_REALTIME).enable(true).build(),
AlertDefine.builder().id(2L).type(METRIC_ALERT_THRESHOLD_TYPE_REALTIME).enable(true).build()
);
try (MockedStatic<CacheFactory> cacheFactoryMock = Mockito.mockStatic(CacheFactory.class)) {
// Mock cache hit
cacheFactoryMock.when(CacheFactory::getMetricsAlertDefineCache).thenReturn(cachedAlertDefines);
List<AlertDefine> result = alertDefineService.getMetricsRealTimeAlertDefines();
assertNotNull(result);
assertEquals(2, result.size());
assertEquals(1L, result.get(0).getId());
assertEquals(2L, result.get(1).getId());
// Verify no database query was called
verify(alertDefineDao, times(0)).findAlertDefinesByTypeAndEnableTrue(any());
cacheFactoryMock.verify(() -> CacheFactory.setMetricsAlertDefineCache(any()), times(0));
}
// Test cache miss scenario
List<AlertDefine> dbAlertDefines = Lists.newArrayList(
AlertDefine.builder().id(3L).type(METRIC_ALERT_THRESHOLD_TYPE_REALTIME).enable(true).build()
);
try (MockedStatic<CacheFactory> cacheFactoryMock = Mockito.mockStatic(CacheFactory.class)) {
// Mock cache miss
cacheFactoryMock.when(CacheFactory::getMetricsAlertDefineCache).thenReturn(null);
when(alertDefineDao.findAlertDefinesByTypeAndEnableTrue(METRIC_ALERT_THRESHOLD_TYPE_REALTIME))
.thenReturn(dbAlertDefines);
List<AlertDefine> result = alertDefineService.getMetricsRealTimeAlertDefines();
assertNotNull(result);
assertEquals(1, result.size());
assertEquals(3L, result.get(0).getId());
// Verify database query and cache setting were called
verify(alertDefineDao, times(1)).findAlertDefinesByTypeAndEnableTrue(METRIC_ALERT_THRESHOLD_TYPE_REALTIME);
cacheFactoryMock.verify(() -> CacheFactory.setMetricsAlertDefineCache(dbAlertDefines), times(1));
@@ -234,40 +244,40 @@ class AlertDefineServiceTest {
AlertDefine.builder().id(4L).type(LOG_ALERT_THRESHOLD_TYPE_REALTIME).enable(true).build(),
AlertDefine.builder().id(5L).type(LOG_ALERT_THRESHOLD_TYPE_REALTIME).enable(true).build()
);
try (MockedStatic<CacheFactory> cacheFactoryMock = Mockito.mockStatic(CacheFactory.class)) {
// Mock cache hit
cacheFactoryMock.when(CacheFactory::getLogAlertDefineCache).thenReturn(cachedAlertDefines);
List<AlertDefine> result = alertDefineService.getLogRealTimeAlertDefines();
assertNotNull(result);
assertEquals(2, result.size());
assertEquals(4L, result.get(0).getId());
assertEquals(5L, result.get(1).getId());
// Verify no database query was called
verify(alertDefineDao, times(0)).findAlertDefinesByTypeAndEnableTrue(any());
cacheFactoryMock.verify(() -> CacheFactory.setLogAlertDefineCache(any()), times(0));
}
// Test cache miss scenario
List<AlertDefine> dbAlertDefines = Lists.newArrayList(
AlertDefine.builder().id(6L).type(LOG_ALERT_THRESHOLD_TYPE_REALTIME).enable(true).build()
);
try (MockedStatic<CacheFactory> cacheFactoryMock = Mockito.mockStatic(CacheFactory.class)) {
// Mock cache miss
cacheFactoryMock.when(CacheFactory::getLogAlertDefineCache).thenReturn(null);
when(alertDefineDao.findAlertDefinesByTypeAndEnableTrue(LOG_ALERT_THRESHOLD_TYPE_REALTIME))
.thenReturn(dbAlertDefines);
List<AlertDefine> result = alertDefineService.getLogRealTimeAlertDefines();
assertNotNull(result);
assertEquals(1, result.size());
assertEquals(6L, result.get(0).getId());
// Verify database query and cache setting were called
verify(alertDefineDao, times(1)).findAlertDefinesByTypeAndEnableTrue(LOG_ALERT_THRESHOLD_TYPE_REALTIME);
cacheFactoryMock.verify(() -> CacheFactory.setLogAlertDefineCache(dbAlertDefines), times(1));
@@ -280,58 +290,58 @@ class AlertDefineServiceTest {
AlertDefine.builder().id(7L).type(METRIC_ALERT_THRESHOLD_TYPE_REALTIME).enable(true).build(),
AlertDefine.builder().id(8L).type(METRIC_ALERT_THRESHOLD_TYPE_REALTIME).enable(true).build()
);
// Test valid metric realtime alert type
when(alertDefineDao.findAlertDefinesByTypeAndEnableTrue(METRIC_ALERT_THRESHOLD_TYPE_REALTIME))
.thenReturn(mockAlertDefines);
List<AlertDefine> result = alertDefineService.getAlertDefinesByType(METRIC_ALERT_THRESHOLD_TYPE_REALTIME);
assertNotNull(result);
assertEquals(2, result.size());
assertEquals(7L, result.get(0).getId());
assertEquals(8L, result.get(1).getId());
verify(alertDefineDao, times(1)).findAlertDefinesByTypeAndEnableTrue(METRIC_ALERT_THRESHOLD_TYPE_REALTIME);
// Test valid metric periodic alert type
reset(alertDefineDao);
when(alertDefineDao.findAlertDefinesByTypeAndEnableTrue(METRIC_ALERT_THRESHOLD_TYPE_PERIODIC))
.thenReturn(Lists.newArrayList());
result = alertDefineService.getAlertDefinesByType(METRIC_ALERT_THRESHOLD_TYPE_PERIODIC);
assertNotNull(result);
assertEquals(0, result.size());
verify(alertDefineDao, times(1)).findAlertDefinesByTypeAndEnableTrue(METRIC_ALERT_THRESHOLD_TYPE_PERIODIC);
// Test valid log realtime alert type
reset(alertDefineDao);
when(alertDefineDao.findAlertDefinesByTypeAndEnableTrue(LOG_ALERT_THRESHOLD_TYPE_REALTIME))
.thenReturn(mockAlertDefines);
result = alertDefineService.getAlertDefinesByType(LOG_ALERT_THRESHOLD_TYPE_REALTIME);
assertNotNull(result);
assertEquals(2, result.size());
verify(alertDefineDao, times(1)).findAlertDefinesByTypeAndEnableTrue(LOG_ALERT_THRESHOLD_TYPE_REALTIME);
// Test valid log periodic alert type
reset(alertDefineDao);
when(alertDefineDao.findAlertDefinesByTypeAndEnableTrue(LOG_ALERT_THRESHOLD_TYPE_PERIODIC))
.thenReturn(Lists.newArrayList());
result = alertDefineService.getAlertDefinesByType(LOG_ALERT_THRESHOLD_TYPE_PERIODIC);
assertNotNull(result);
assertEquals(0, result.size());
verify(alertDefineDao, times(1)).findAlertDefinesByTypeAndEnableTrue(LOG_ALERT_THRESHOLD_TYPE_PERIODIC);
// Test empty string type
assertThrows(IllegalArgumentException.class, () -> alertDefineService.getAlertDefinesByType(""));
// Test null type
assertThrows(IllegalArgumentException.class, () -> alertDefineService.getAlertDefinesByType(null));
// Test invalid type
assertThrows(IllegalArgumentException.class, () -> alertDefineService.getAlertDefinesByType("invalid_type"));
}
@@ -18,7 +18,6 @@
package org.apache.hertzbeat.alert.service;
import com.github.benmanes.caffeine.cache.Cache;
import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.tree.ParseTree;
import org.apache.hertzbeat.alert.service.impl.DataSourceServiceImpl;
import org.apache.hertzbeat.common.support.exception.AlertExpressionException;
@@ -557,13 +556,11 @@ class DataSourceServiceTest {
dataSourceService.setExecutors(List.of(mockExecutor));
String expr = "node_cpu_seconds_total > 50";
Cache<String, ParseTree> expressionCache = dataSourceService.getExpressionCache();
Cache<String, CommonTokenStream> tokenStreamCache = dataSourceService.getTokenStreamCache();
expressionCache.invalidateAll();
tokenStreamCache.invalidateAll();
long beforeHits = tokenStreamCache.stats().hitCount();
long beforeHits = expressionCache.stats().hitCount();
dataSourceService.calculate("promql", expr);
dataSourceService.calculate("promql", expr);
long actualHits = tokenStreamCache.stats().hitCount() - beforeHits;
long actualHits = expressionCache.stats().hitCount() - beforeHits;
assertEquals(1, actualHits, "expression cache should hit but miss");
}
@@ -578,9 +575,7 @@ class DataSourceServiceTest {
dataSourceService.setExecutors(List.of(mockExecutor));
dataSourceService.calculate("promql", "node_cpu_seconds_total > 50");
Cache<String, ParseTree> expressionCache = dataSourceService.getExpressionCache();
Cache<String, CommonTokenStream> tokenStreamCache = dataSourceService.getTokenStreamCache();
expressionCache.invalidateAll();
tokenStreamCache.invalidateAll();
String expr1 = "node_cpu_seconds_total > 30";
String expr2 = "node_cpu_seconds_total > 50";
long beforeHits = expressionCache.stats().hitCount();
@@ -601,15 +596,13 @@ class DataSourceServiceTest {
dataSourceService.setExecutors(List.of(mockExecutor));
dataSourceService.calculate("promql", "node_cpu_seconds_total > 50");
Cache<String, ParseTree> expressionCache = dataSourceService.getExpressionCache();
Cache<String, CommonTokenStream> tokenStreamCache = dataSourceService.getTokenStreamCache();
expressionCache.invalidateAll();
tokenStreamCache.invalidateAll();
String expr1 = "node_cpu_seconds_total > 30";
String expr2 = "node_cpu_seconds_total > 50";
long beforeHits = tokenStreamCache.stats().hitCount();
long beforeHits = expressionCache.stats().hitCount();
dataSourceService.calculate("promql", expr1);
dataSourceService.calculate("promql", expr2);
long actualHits = tokenStreamCache.stats().hitCount() - beforeHits;
long actualHits = expressionCache.stats().hitCount() - beforeHits;
assertEquals(0, actualHits, "expression cache should miss but hit");
}
@@ -15,7 +15,7 @@
* limitations under the License.
*/
package org.apache.hertzbeat.manager.dao;
package org.apache.hertzbeat.base.dao;
import java.util.Optional;
import java.util.Set;
@@ -33,7 +33,7 @@ public interface LabelDao extends JpaRepository<Label, Long>, JpaSpecificationEx
* @param ids id list
*/
void deleteLabelsByIdIn(Set<Long> ids);
/**
* find Label by name and value
* @param name Label name
@@ -41,5 +41,5 @@ public interface LabelDao extends JpaRepository<Label, Long>, JpaSpecificationEx
* @return Label
*/
Optional<Label> findLabelByNameAndTagValue(String name, String value);
}
@@ -15,7 +15,7 @@
* limitations under the License.
*/
package org.apache.hertzbeat.manager.service;
package org.apache.hertzbeat.base.service;
import java.util.HashSet;
import java.util.List;
@@ -39,6 +39,7 @@ import java.util.Objects;
import java.util.Set;
import java.util.stream.Collectors;
import javax.net.ssl.SSLException;
import javax.xml.XMLConstants;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.xpath.XPath;
@@ -106,6 +107,22 @@ import java.util.Collections;
*/
@Slf4j
public class HttpCollectImpl extends AbstractCollect {
/**
* Pre-compiled regex patterns for dangerous Xpath detection.
* Compiled once at class load for performance.
*/
private static final List<Pattern> DANGEROUS_XPATH_PATTERNS;
static {
List<Pattern> patterns = new ArrayList<>();
for (String pattern : CollectorConstants.DANGEROUS_XPATH_PATTERNS) {
// Use CASE_INSENSITIVE to prevent bypass via case variations (e.g., //TEXT() vs //text())
patterns.add(Pattern.compile(pattern, Pattern.CASE_INSENSITIVE | Pattern.DOTALL));
}
DANGEROUS_XPATH_PATTERNS = Collections.unmodifiableList(patterns);
}
private final Set<Integer> defaultSuccessStatusCodes = Set.of(
HttpStatus.SC_OK,
HttpStatus.SC_CREATED,
@@ -357,21 +374,83 @@ public class HttpCollectImpl extends AbstractCollect {
}
}
/**
* Validates the Xpath expression to prevent DoS attacks.
* Checks for dangerous patterns that could traverse the entire XML document.
* Uses pre-compiled patterns for performance and case-insensitive matching for security.
*
* @param xpathExpression the Xpath expression to validate
* @throws IllegalArgumentException if the expression contains dangerous patterns
*/
private void validateXpathExpression(String xpathExpression) throws IllegalArgumentException {
if (!StringUtils.hasText(xpathExpression)) {
return;
}
// Check against dangerous patterns using pre-compiled regex
for (Pattern pattern : DANGEROUS_XPATH_PATTERNS) {
Matcher matcher = pattern.matcher(xpathExpression);
if (matcher.find()) {
throw new IllegalArgumentException(
"Xpath expression contains dangerous pattern that may cause DoS: " + pattern.pattern()
);
}
}
// Check for excessive wildcard usage (more than 3 // or * operators)
long descendantAxisCount = xpathExpression.chars().filter(ch -> ch == '/').count();
long wildcardCount = xpathExpression.chars().filter(ch -> ch == '*').count();
if (descendantAxisCount > 10 || wildcardCount > 5) {
throw new IllegalArgumentException(
"Xpath expression contains too many wildcards or descendant axes, potential DoS risk"
);
}
log.debug("Xpath expression validation passed: {}", xpathExpression);
}
private void parseResponseByXmlPath(String resp, Metrics metrics,
CollectRep.MetricsData.Builder builder, Long responseTime) {
HttpProtocol http = metrics.getHttp();
List<String> aliasFields = metrics.getAliasFields();
String xpathExpression = http.getParseScript();
// Layer 1: Validate Xpath expression is not empty
if (!StringUtils.hasText(xpathExpression)) {
log.warn("Http collect parse type is xmlPath, but the xpath expression is empty.");
builder.setCode(CollectRep.Code.FAIL);
builder.setMsg("XPath expression is empty");
builder.setMsg("Xpath expression is empty");
return;
}
// Layer 2: Check XML response size to prevent memory exhaustion
if (resp != null && resp.length() > CollectorConstants.MAX_XML_RESPONSE_SIZE) {
log.warn("XML response size {} bytes exceeds maximum allowed size {} bytes",
resp.length(), CollectorConstants.MAX_XML_RESPONSE_SIZE);
builder.setCode(CollectRep.Code.FAIL);
builder.setMsg("XML response exceeds maximum allowed size of "
+ (CollectorConstants.MAX_XML_RESPONSE_SIZE / 1024 / 1024) + "MB");
return;
}
// Layer 3: Validate Xpath expression for dangerous patterns
try {
validateXpathExpression(xpathExpression);
} catch (IllegalArgumentException e) {
log.warn("Xpath expression validation failed: {}", e.getMessage());
builder.setCode(CollectRep.Code.FAIL);
builder.setMsg(e.getMessage());
return;
}
int keywordNum = CollectUtil.countMatchKeyword(resp, http.getKeyword());
try {
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
// Layer 4: Enable XML secure processing and XXE protection
dbf.setFeature(XMLConstants.FEATURE_SECURE_PROCESSING, true);
dbf.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true);
dbf.setFeature("http://xml.org/sax/features/external-general-entities", false);
dbf.setFeature("http://xml.org/sax/features/external-parameter-entities", false);
@@ -387,7 +466,7 @@ public class HttpCollectImpl extends AbstractCollect {
NodeList nodeList = (NodeList) xpath.evaluate(xpathExpression, document, XPathConstants.NODESET);
if (nodeList == null || nodeList.getLength() == 0) {
log.debug("XPath expression '{}' returned no nodes.", xpathExpression);
log.debug("Xpath expression '{}' returned no nodes.", xpathExpression);
boolean requestedSummaryFields = aliasFields.stream()
.anyMatch(alias -> NetworkConstants.RESPONSE_TIME.equalsIgnoreCase(alias)
|| CollectorConstants.KEYWORD.equalsIgnoreCase(alias));
@@ -408,7 +487,16 @@ public class HttpCollectImpl extends AbstractCollect {
return;
}
for (int i = 0; i < nodeList.getLength(); i++) {
// Layer 5: Limit the number of results to prevent excessive resource consumption
int resultSize = nodeList.getLength();
int maxResults = CollectorConstants.MAX_XPATH_RESULT_NODES;
if (resultSize > maxResults) {
log.warn("Xpath expression returned {} nodes, exceeding limit of {}. Processing first {} nodes only.",
resultSize, maxResults, maxResults);
resultSize = maxResults;
}
for (int i = 0; i < resultSize; i++) {
Node node = nodeList.item(i);
CollectRep.ValueRow.Builder valueRowBuilder = CollectRep.ValueRow.newBuilder();
@@ -422,7 +510,7 @@ public class HttpCollectImpl extends AbstractCollect {
String value = (String) xpath.evaluate(alias, node, XPathConstants.STRING);
valueRowBuilder.addColumn(StringUtils.hasText(value) ? value : CommonConstants.NULL_VALUE);
} catch (XPathExpressionException e) {
log.warn("Failed to evaluate XPath '{}' for node [{}]: {}", alias, node.getNodeName(), e.getMessage());
log.warn("Failed to evaluate Xpath '{}' for node [{}]: {}", alias, node.getNodeName(), e.getMessage());
valueRowBuilder.addColumn(CommonConstants.NULL_VALUE);
}
}
@@ -431,7 +519,7 @@ public class HttpCollectImpl extends AbstractCollect {
}
} catch (Exception e) {
log.warn("Failed to parse XML response with XPath '{}': {}", xpathExpression, e.getMessage(), e);
log.warn("Failed to parse XML response with Xpath '{}': {}", xpathExpression, e.getMessage(), e);
builder.setCode(CollectRep.Code.FAIL);
builder.setMsg("Failed to parse XML response: " + e.getMessage());
}
@@ -882,4 +970,4 @@ public class HttpCollectImpl extends AbstractCollect {
}
return successCodeSet.contains(statusCode);
}
}
}
@@ -49,4 +49,28 @@ public interface CollectorConstants extends NetworkConstants {
String STATUS_CODE = "statusCode";
/**
* Maximum XML response size in bytes (10MB) to prevent DoS attacks
*/
int MAX_XML_RESPONSE_SIZE = 10 * 1024 * 1024;
/**
* Maximum number of nodes returned by Xpath query to prevent excessive resource consumption
*/
int MAX_XPATH_RESULT_NODES = 1000;
/**
* Dangerous Xpath expression patterns that could cause DoS attacks
* These patterns match expressions that traverse the entire XML document
*/
String[] DANGEROUS_XPATH_PATTERNS = {
"//\\*\\s*\\|\\s*//@\\*\\s*\\|\\s*//text\\(\\)", // //* | //@* | //text()
"//\\*\\s*\\|", // //* | ...
"//@\\*\\s*\\|", // //@* | ...
"//node\\(\\)\\s*\\|", // //node() | ...
"descendant-or-self::node\\(\\)\\s*\\|", // descendant-or-self::node() | ...
"/descendant-or-self::node\\(\\)", // /descendant-or-self::node()
"//\\*[\\s\\S]*//\\*" // //** with multiple wildcards
};
}
@@ -0,0 +1,50 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.common.constants;
import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.ToString;
/**
* Theme Enum
*/
@Getter
@AllArgsConstructor
@ToString
public enum ThemeEnum {
COMPACT("compact"), DEFAULT("default"), LIGHT("light"), DARK("dark");
private final String value;
/**
* Convert theme to standard theme
*
* @param theme the input theme
* @return the converted theme
*/
public static String convert(String theme) {
if ((theme == null || theme.isEmpty())
|| ThemeEnum.COMPACT.getValue().equals(theme)
|| ThemeEnum.DEFAULT.getValue().equals(theme)) {
return ThemeEnum.LIGHT.getValue();
}
return theme;
}
}
@@ -48,8 +48,8 @@ import org.springframework.data.jpa.domain.support.AuditingEntityListener;
*/
@Entity
@Table(name = "hzb_alert_define_monitor_bind", indexes = {
@Index(name = "index_alert_define_monitor", columnList = "alertDefineId"),
@Index(name = "index_alert_define_monitor", columnList = "monitor_id")
@Index(name = "idx_alert_define_id", columnList = "alert_define_id"),
@Index(name = "idx_monitor_id", columnList = "monitor_id")
})
@Data
@Builder
@@ -34,7 +34,7 @@ import lombok.NoArgsConstructor;
@Entity
@Table(name = "hzb_grafana_dashboard")
@Data
@Builder
@Builder(toBuilder = true)
@AllArgsConstructor
@NoArgsConstructor
@Schema(description = "Grafana dashboard entity")
@@ -42,8 +42,8 @@ import org.springframework.data.jpa.domain.support.AuditingEntityListener;
*/
@Entity
@Table(name = "hzb_collector_monitor_bind", indexes = {
@Index(name = "index_collector_monitor", columnList = "collector"),
@Index(name = "index_collector_monitor", columnList = "monitor_id")
@Index(name = "idx_collector_monitor_collector", columnList = "collector"),
@Index(name = "idx_collector_monitor_monitor_id", columnList = "monitor_id")
})
@Data
@Builder
@@ -51,9 +51,9 @@ import org.springframework.data.jpa.domain.support.AuditingEntityListener;
*/
@Entity
@Table(name = "hzb_monitor", indexes = {
@Index(name = "monitor_query_index", columnList = "app"),
@Index(name = "monitor_query_index", columnList = "instance"),
@Index(name = "monitor_query_index", columnList = "name")
@Index(name = "idx_hzb_monitor_app", columnList = "app"),
@Index(name = "idx_hzb_monitor_instance", columnList = "instance"),
@Index(name = "idx_hzb_monitor_name", columnList = "name")
})
@Data
@Builder
@@ -62,51 +62,51 @@ import org.springframework.data.jpa.domain.support.AuditingEntityListener;
@Schema(description = "Monitor Entity")
@EntityListeners(AuditingEntityListener.class)
public class Monitor {
@Id
@Schema(title = "Monitor task ID", example = "87584674384", accessMode = READ_ONLY)
private Long id;
@Schema(title = "Collect task ID", example = "43243543543", accessMode = READ_ONLY)
private Long jobId;
@Schema(title = "task name", example = "Api-TanCloud.cn", accessMode = READ_WRITE)
@Size(max = 100)
private String name;
@Schema(title = "Type of monitoring", example = "TanCloud", accessMode = READ_WRITE)
@Size(max = 100)
private String app;
@Schema(title = "Scrape type: static | http_sd | dns_sd | zookeeper_sd", example = "static", accessMode = READ_WRITE)
@Size(max = 100)
private String scrape;
@Schema(title = "the monitor target: ip/domain+port or ip/domain", example = "192.167.25.11:8081", accessMode = READ_WRITE)
@Size(max = 100)
@HostValid
private String instance;
@Schema(title = "Monitoring of the acquisition interval time in seconds", example = "600", accessMode = READ_WRITE)
@Min(10)
private Integer intervals;
@Schema(title = "Schedule type: interval | cron", example = "interval", accessMode = READ_WRITE)
@Size(max = 20)
private String scheduleType;
@Schema(title = "Cron expression when scheduleType is cron", example = "0/5 * * * * ?", accessMode = READ_WRITE)
@Size(max = 100)
private String cronExpression;
@Schema(title = "Task status 0: Paused, 1: Up, 2: Down", accessMode = READ_WRITE)
@Min(0)
@Max(4)
private byte status;
@Schema(title = "Task type 0: Normal, 1: push auto create, 2: discovery auto create")
private byte type;
@Schema(title = "task label", example = "{env:test}", accessMode = READ_WRITE)
@Convert(converter = JsonMapAttributeConverter.class)
@Column(length = 4096)
@@ -116,23 +116,23 @@ public class Monitor {
@Convert(converter = JsonMapAttributeConverter.class)
@Column(length = 4096)
private Map<String, String> annotations;
@Schema(title = "Monitor note description", example = "Availability monitoring of the SAAS website TanCloud", accessMode = READ_WRITE)
@Size(max = 255)
private String description;
@Schema(title = "The creator of this record", example = "tom", accessMode = READ_ONLY)
@CreatedBy
private String creator;
@Schema(title = "The modifier of this record", example = "tom", accessMode = READ_ONLY)
@LastModifiedBy
private String modifier;
@Schema(title = "Record create time", example = "2024-07-02T20:09:34.903217", accessMode = READ_ONLY)
@CreatedDate
private LocalDateTime gmtCreate;
@Schema(title = "Record modify time", example = "2024-07-02T20:09:34.903217", accessMode = READ_ONLY)
@LastModifiedDate
private LocalDateTime gmtUpdate;
@@ -42,7 +42,7 @@ import org.springframework.data.jpa.domain.support.AuditingEntityListener;
*/
@Entity
@Table(name = "hzb_monitor_bind", indexes = {
@Index(name = "index_monitor_bind", columnList = "bizId"),
@Index(name = "index_monitor_bind", columnList = "biz_id"),
@Index(name = "index_monitor_bin", columnList = "monitor_id")
})
@Data
@@ -59,7 +59,7 @@ public class MonitorBind {
@Schema(title = "key string: ip:port")
private String keyStr;
@Schema(title = "connect bind id", example = "87432674384")
private Long bizId;
@@ -46,8 +46,11 @@ import org.springframework.data.jpa.domain.support.AuditingEntityListener;
* Monitor parameter values
*/
@Entity
@Table(name = "hzb_param", indexes = { @Index(columnList = "monitorId") },
uniqueConstraints = @UniqueConstraint(columnNames = {"monitorId", "field"}))
@Table(
name = "hzb_param",
indexes = {@Index(name = "idx_hzb_param_monitor_id", columnList = "monitor_id")},
uniqueConstraints = {@UniqueConstraint(name = "uk_hzb_param_monitor_field", columnNames = {"monitor_id", "field"})}
)
@Data
@Builder
@AllArgsConstructor
@@ -68,6 +71,7 @@ public class Param {
* Monitor ID
*/
@Schema(title = "Monitor task ID", example = "875846754543", accessMode = READ_WRITE)
@Column(name = "monitor_id")
private Long monitorId;
/**
@@ -75,6 +79,7 @@ public class Param {
*/
@Schema(title = "Parameter identifier field", example = "port", accessMode = READ_WRITE)
@Size(max = 100)
@Column(name = "field")
@NotBlank(message = "field can not null")
private String field;
@@ -41,7 +41,6 @@ import org.springframework.data.jpa.domain.support.AuditingEntityListener;
@Entity
@Table(name = "hzb_status_page_incident_component_bind", indexes = {
@Index(name = "index_incident_component", columnList = "incident_id"),
@Index(name = "index_incident_component", columnList = "component_id")
})
@Data
@Builder
@@ -59,7 +58,7 @@ public class StatusPageIncidentComponentBind {
@Schema(title = "Incident ID", example = "87432674384")
@Column(name = "incident_id")
private Long incidentId;
@Schema(title = "Component ID", example = "87432674336")
@Column(name = "component_id")
private Long componentId;
@@ -71,5 +70,5 @@ public class StatusPageIncidentComponentBind {
@Schema(title = "Record modify time", example = "1612198444000")
@LastModifiedDate
private LocalDateTime gmtUpdate;
}
@@ -35,8 +35,8 @@ import org.springframework.data.jpa.domain.support.AuditingEntityListener;
*/
@Entity
@Table(name = "hzb_push_metrics", indexes = {
@Index(name = "push_query_index", columnList = "monitorId"),
@Index(name = "push_query_index", columnList = "time")
@Index(name = "idx_push_metrics_monitor_id", columnList = "monitor_id"),
@Index(name = "idx_push_metrics_time", columnList = "time")
})
@Data
@Builder
@@ -37,10 +37,10 @@ import lombok.NoArgsConstructor;
*/
@Entity
@Table(name = "hzb_history", indexes = {
@Index(name = "history_query_index", columnList = "instance"),
@Index(name = "history_query_index", columnList = "app"),
@Index(name = "history_query_index", columnList = "metrics"),
@Index(name = "history_query_index", columnList = "metric")
@Index(name = "idx_hzb_history_instance", columnList = "instance"),
@Index(name = "idx_hzb_history_app", columnList = "app"),
@Index(name = "idx_hzb_history_metrics", columnList = "metrics"),
@Index(name = "idx_hzb_history_metric", columnList = "metric")
})
@Data
@Builder
@@ -65,7 +65,7 @@ public class History {
@Schema(title = "Monitoring Metric usage speed count")
private String metric;
@Column(length = 5000)
private String metricLabels;
@@ -159,4 +159,8 @@ public class StrBuffer {
}
return Double.parseDouble(s);
}
public static String escapeForFormat(String value) {
return value.replace("%", "%%");
}
}
@@ -31,6 +31,7 @@ import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.condition.EnabledIf;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.junit.jupiter.MockitoExtension;
import org.testcontainers.containers.GenericContainer;
@@ -48,8 +49,17 @@ import java.util.Map;
*/
@Slf4j
@ExtendWith(MockitoExtension.class)
@EnabledIf("isDockerAvailable")
public class ZookeeperMonitorE2eTest extends AbstractCollectE2eTest {
static boolean isDockerAvailable() {
try {
return org.testcontainers.DockerClientFactory.instance().isDockerAvailable();
} catch (Exception e) {
return false;
}
}
private static final String ZOOKEEPER_IMAGE_NAME = "zookeeper:3.8.4";
private static final String ZOOKEEPER_NAME = "zookeeper";
private static final Integer ZOOKEEPER_PORT = 2181;
@@ -0,0 +1,38 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.collector.collect;
import org.testcontainers.DockerClientFactory;
/**
* Utility class for E2E tests
*/
public class TestUtil {
/**
* Check if Docker is available for testcontainers
* @return true if Docker is available, false otherwise
*/
public static boolean isDockerAvailable() {
try {
return DockerClientFactory.instance().isDockerAvailable();
} catch (Exception e) {
return false;
}
}
}
@@ -28,6 +28,7 @@ import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.condition.EnabledIf;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.KafkaContainer;
import org.testcontainers.containers.Network;
@@ -50,8 +51,17 @@ import java.util.stream.Stream;
* KafkaCollectE2E
*/
@Slf4j
@EnabledIf("isDockerAvailable")
public class KafkaCollectE2eTest {
static boolean isDockerAvailable() {
try {
return org.testcontainers.DockerClientFactory.instance().isDockerAvailable();
} catch (Exception e) {
return false;
}
}
private static final String ZOOKEEPER_IMAGE_NAME = "zookeeper:3.8.4";
private static final String ZOOKEEPER_NAME = "zookeeper";
private static final Integer ZOOKEEPER_PORT = 2181;
@@ -65,34 +75,53 @@ public class KafkaCollectE2eTest {
@AfterAll
public static void tearDown() {
kafkaContainer.stop();
zookeeperContainer.stop();
if (kafkaContainer != null) {
kafkaContainer.stop();
}
if (zookeeperContainer != null) {
zookeeperContainer.stop();
}
}
@BeforeEach
public void setUp() {
kafkaCollect = new KafkaCollectImpl();
metrics = new Metrics();
Network.NetworkImpl network = Network.builder().build();
zookeeperContainer = new GenericContainer<>(DockerImageName.parse(ZOOKEEPER_IMAGE_NAME))
.withExposedPorts(ZOOKEEPER_PORT)
.withNetwork(network)
.withNetworkAliases(ZOOKEEPER_NAME)
.waitingFor(Wait.forListeningPort())
.withStartupTimeout(Duration.ofSeconds(120));
zookeeperContainer.setPortBindings(Collections.singletonList(ZOOKEEPER_PORT + ":" + ZOOKEEPER_PORT));
try {
Network.NetworkImpl network = Network.builder().build();
zookeeperContainer = new GenericContainer<>(DockerImageName.parse(ZOOKEEPER_IMAGE_NAME))
.withExposedPorts(ZOOKEEPER_PORT)
.withNetwork(network)
.withNetworkAliases(ZOOKEEPER_NAME)
.waitingFor(Wait.forListeningPort())
.withStartupTimeout(Duration.ofSeconds(120));
zookeeperContainer.setPortBindings(Collections.singletonList(ZOOKEEPER_PORT + ":" + ZOOKEEPER_PORT));
Startables.deepStart(Stream.of(zookeeperContainer)).join();
Startables.deepStart(Stream.of(zookeeperContainer)).join();
kafkaContainer = new KafkaContainer(DockerImageName.parse(KAFKA_IMAGE_NAME))
.withExternalZookeeper(ZOOKEEPER_NAME + ":2181")
.withNetwork(network)
.withNetworkAliases(KAFKA_NAME)
.withLogConsumer(
new Slf4jLogConsumer(
DockerLoggerFactory.getLogger(KAFKA_IMAGE_NAME)))
.withStartupTimeout(Duration.ofSeconds(120));
Startables.deepStart(Stream.of(kafkaContainer)).join();
kafkaContainer = new KafkaContainer(DockerImageName.parse(KAFKA_IMAGE_NAME))
.withExternalZookeeper(ZOOKEEPER_NAME + ":2181")
.withNetwork(network)
.withNetworkAliases(KAFKA_NAME)
.withLogConsumer(
new Slf4jLogConsumer(
DockerLoggerFactory.getLogger(KAFKA_IMAGE_NAME)))
.withStartupTimeout(Duration.ofSeconds(120));
Startables.deepStart(Stream.of(kafkaContainer)).join();
} catch (Exception e) {
log.error("Failed to start Kafka containers", e);
// Clean up any partially started containers
if (kafkaContainer != null) {
kafkaContainer.stop();
kafkaContainer = null;
}
if (zookeeperContainer != null) {
zookeeperContainer.stop();
zookeeperContainer = null;
}
throw new RuntimeException("Failed to start Kafka test environment", e);
}
}
@Test
@@ -26,12 +26,14 @@ import org.apache.hertzbeat.common.entity.alerter.SingleAlert;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.api.condition.EnabledIf;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.mock.mockito.SpyBean;
import org.springframework.boot.test.web.server.LocalServerPort;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.testcontainers.DockerClientFactory;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.Network;
import org.testcontainers.containers.wait.strategy.Wait;
@@ -61,8 +63,17 @@ import static org.mockito.Mockito.doAnswer;
@SpringBootTest(classes = org.apache.hertzbeat.startup.HertzBeatApplication.class, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@Slf4j
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
@EnabledIf("isDockerAvailable")
public class LogPeriodicAlertE2eTest {
static boolean isDockerAvailable() {
try {
return org.testcontainers.DockerClientFactory.instance().isDockerAvailable();
} catch (Exception e) {
return false;
}
}
private static final String VECTOR_IMAGE = "timberio/vector:latest-alpine";
private static final int VECTOR_PORT = 8686;
private static final String VECTOR_CONFIG_PATH = "/etc/vector/vector.yml";
@@ -85,18 +96,24 @@ public class LogPeriodicAlertE2eTest {
private AlarmCommonReduce alarmCommonReduce;
static GenericContainer<?> vector;
static GenericContainer<?> greptimedb;
static {
greptimedb = new GenericContainer<>(DockerImageName.parse(GREPTIME_IMAGE))
.withExposedPorts(GREPTIME_HTTP_PORT, GREPTIME_GRPC_PORT)
.withCommand("standalone", "start",
"--http-addr", "0.0.0.0:" + GREPTIME_HTTP_PORT,
"--rpc-bind-addr", "0.0.0.0:" + GREPTIME_GRPC_PORT)
.waitingFor(Wait.forListeningPorts(GREPTIME_HTTP_PORT, GREPTIME_GRPC_PORT))
.withStartupTimeout(CONTAINER_STARTUP_TIMEOUT);
greptimedb.start();
try {
if (DockerClientFactory.instance().isDockerAvailable()) {
greptimedb = new GenericContainer<>(DockerImageName.parse(GREPTIME_IMAGE))
.withExposedPorts(GREPTIME_HTTP_PORT, GREPTIME_GRPC_PORT)
.withCommand("standalone", "start",
"--http-addr", "0.0.0.0:" + GREPTIME_HTTP_PORT,
"--rpc-bind-addr", "0.0.0.0:" + GREPTIME_GRPC_PORT)
.waitingFor(Wait.forListeningPorts(GREPTIME_HTTP_PORT, GREPTIME_GRPC_PORT))
.withStartupTimeout(CONTAINER_STARTUP_TIMEOUT);
greptimedb.start();
}
} catch (Exception e) {
log.warn("Docker not available, skipping GreptimeDB container startup: {}", e.getMessage());
}
}
@DynamicPropertySource
@@ -26,6 +26,7 @@ import org.apache.hertzbeat.common.entity.alerter.SingleAlert;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.api.condition.EnabledIf;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.mock.mockito.SpyBean;
import org.springframework.boot.test.web.server.LocalServerPort;
@@ -58,8 +59,17 @@ import static org.mockito.Mockito.doAnswer;
@SpringBootTest(classes = org.apache.hertzbeat.startup.HertzBeatApplication.class, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@Slf4j
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
@EnabledIf("isDockerAvailable")
public class LogRealTimeAlertE2eTest {
static boolean isDockerAvailable() {
try {
return org.testcontainers.DockerClientFactory.instance().isDockerAvailable();
} catch (Exception e) {
return false;
}
}
private static final String VECTOR_IMAGE = "timberio/vector:latest-alpine";
private static final int VECTOR_PORT = 8686;
private static final String VECTOR_CONFIG_PATH = "/etc/vector/vector.yml";
@@ -81,7 +91,7 @@ public class LogRealTimeAlertE2eTest {
void setUpAll() {
// Setup test alert definitions
setupTestAlertDefines();
// Expose host ports for testcontainers
Testcontainers.exposeHostPorts(port);
vector = new GenericContainer<>(DockerImageName.parse(VECTOR_IMAGE))
@@ -23,9 +23,11 @@ import org.apache.hertzbeat.common.queue.CommonDataQueue;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.api.condition.EnabledIf;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.server.LocalServerPort;
import org.testcontainers.DockerClientFactory;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.Network;
import org.testcontainers.containers.wait.strategy.Wait;
@@ -47,8 +49,17 @@ import static org.junit.jupiter.api.Assertions.assertNotNull;
@SpringBootTest(classes = org.apache.hertzbeat.startup.HertzBeatApplication.class, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@Slf4j
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
@EnabledIf("isDockerAvailable")
public class LogIngestionE2eTest {
static boolean isDockerAvailable() {
try {
return DockerClientFactory.instance().isDockerAvailable();
} catch (Exception e) {
return false;
}
}
private static final String VECTOR_IMAGE = "timberio/vector:latest-alpine";
private static final int VECTOR_PORT = 8686;
private static final String VECTOR_CONFIG_PATH = "/etc/vector/vector.yml";
@@ -24,11 +24,13 @@ import org.apache.hertzbeat.warehouse.store.history.tsdb.greptime.GreptimeDbData
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.api.condition.EnabledIf;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.server.LocalServerPort;
import org.springframework.test.context.DynamicPropertyRegistry;
import org.springframework.test.context.DynamicPropertySource;
import org.testcontainers.DockerClientFactory;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.Network;
import org.testcontainers.containers.wait.strategy.Wait;
@@ -50,8 +52,17 @@ import static org.junit.jupiter.api.Assertions.assertNotNull;
@SpringBootTest(classes = org.apache.hertzbeat.startup.HertzBeatApplication.class, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@Slf4j
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
@EnabledIf("isDockerAvailable")
public class GreptimeLogStorageE2eTest {
static boolean isDockerAvailable() {
try {
return DockerClientFactory.instance().isDockerAvailable();
} catch (Exception e) {
return false;
}
}
private static final String VECTOR_IMAGE = "timberio/vector:latest-alpine";
private static final int VECTOR_PORT = 8686;
private static final String VECTOR_CONFIG_PATH = "/etc/vector/vector.yml";
@@ -74,14 +85,20 @@ public class GreptimeLogStorageE2eTest {
static GenericContainer<?> greptimedb;
static {
greptimedb = new GenericContainer<>(DockerImageName.parse(GREPTIME_IMAGE))
.withExposedPorts(GREPTIME_HTTP_PORT, GREPTIME_GRPC_PORT)
.withCommand("standalone", "start",
"--http-addr", "0.0.0.0:" + GREPTIME_HTTP_PORT,
"--rpc-bind-addr", "0.0.0.0:" + GREPTIME_GRPC_PORT)
.waitingFor(Wait.forListeningPorts(GREPTIME_HTTP_PORT, GREPTIME_GRPC_PORT))
.withStartupTimeout(CONTAINER_STARTUP_TIMEOUT);
greptimedb.start();
try {
if (DockerClientFactory.instance().isDockerAvailable()) {
greptimedb = new GenericContainer<>(DockerImageName.parse(GREPTIME_IMAGE))
.withExposedPorts(GREPTIME_HTTP_PORT, GREPTIME_GRPC_PORT)
.withCommand("standalone", "start",
"--http-addr", "0.0.0.0:" + GREPTIME_HTTP_PORT,
"--rpc-bind-addr", "0.0.0.0:" + GREPTIME_GRPC_PORT)
.waitingFor(Wait.forListeningPorts(GREPTIME_HTTP_PORT, GREPTIME_GRPC_PORT))
.withStartupTimeout(CONTAINER_STARTUP_TIMEOUT);
greptimedb.start();
}
} catch (Exception e) {
log.warn("Docker not available, skipping GreptimeDB container startup: {}", e.getMessage());
}
}
@DynamicPropertySource
@@ -161,6 +178,6 @@ public class GreptimeLogStorageE2eTest {
long startTime = endTime - Duration.ofMinutes(5).toMillis(); // Look back 5 minutes
return greptimeDbDataStorage.queryLogsByMultipleConditions(
startTime, endTime, null, null, null, null);
startTime, endTime, null, null, null, null, null);
}
}
+1 -1
View File
@@ -87,5 +87,5 @@
</dependency>
</dependencies>
</dependencyManagement>
</project>
+2 -2
View File
@@ -32,10 +32,10 @@
</properties>
<dependencies>
<!-- hertzbeat common -->
<!-- hertzbeat base -->
<dependency>
<groupId>org.apache.hertzbeat</groupId>
<artifactId>hertzbeat-common</artifactId>
<artifactId>hertzbeat-base</artifactId>
</dependency>
<!-- hertzbeat warehouse -->
<dependency>
@@ -33,6 +33,8 @@ public interface GrafanaConstants {
String INSTANCE = "&var-instance=";
String THEME = "&theme=";
String CREATE_DASHBOARD_API = "/api/dashboards/db";
String DELETE_DASHBOARD_API = "/api/dashboards/uid/%s";
@@ -92,4 +94,4 @@ public interface GrafanaConstants {
static String generateUseDatasource(String datasourceName) {
return "&var-ds=" + datasourceName;
}
}
}
@@ -17,17 +17,15 @@
package org.apache.hertzbeat.grafana.service;
import static org.apache.hertzbeat.grafana.common.GrafanaConstants.CREATE_DASHBOARD_API;
import static org.apache.hertzbeat.grafana.common.GrafanaConstants.DELETE_DASHBOARD_API;
import static org.apache.hertzbeat.grafana.common.GrafanaConstants.INSTANCE;
import static org.apache.hertzbeat.grafana.common.GrafanaConstants.KIOSK;
import static org.apache.hertzbeat.grafana.common.GrafanaConstants.REFRESH;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.hertzbeat.base.dao.GeneralConfigDao;
import org.apache.hertzbeat.common.constants.GeneralConfigTypeEnum;
import org.apache.hertzbeat.common.constants.ThemeEnum;
import org.apache.hertzbeat.common.entity.grafana.GrafanaDashboard;
import org.apache.hertzbeat.common.entity.manager.GeneralConfig;
import org.apache.hertzbeat.common.util.JsonUtil;
import org.apache.hertzbeat.grafana.common.GrafanaConstants;
import org.apache.hertzbeat.grafana.config.GrafanaProperties;
@@ -42,7 +40,17 @@ import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.web.client.HttpClientErrorException;
import org.springframework.web.client.RestTemplate;
import lombok.extern.slf4j.Slf4j;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import static org.apache.hertzbeat.grafana.common.GrafanaConstants.CREATE_DASHBOARD_API;
import static org.apache.hertzbeat.grafana.common.GrafanaConstants.DELETE_DASHBOARD_API;
import static org.apache.hertzbeat.grafana.common.GrafanaConstants.INSTANCE;
import static org.apache.hertzbeat.grafana.common.GrafanaConstants.KIOSK;
import static org.apache.hertzbeat.grafana.common.GrafanaConstants.REFRESH;
/**
* Service for managing Grafana dashboards.
@@ -66,6 +74,11 @@ public class DashboardService {
@Autowired
private DatasourceService datasourceService;
@Autowired
private GeneralConfigDao generalConfigDao;
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
/**
* Creates or updates a dashboard in Grafana.
* The "id" field will be removed from the dashboard JSON before sending
@@ -225,7 +238,16 @@ public class DashboardService {
* @return GrafanaDashboard object
*/
public GrafanaDashboard getDashboardByMonitorId(Long monitorId) {
return dashboardDao.findByMonitorId(monitorId);
GrafanaDashboard dashboard = dashboardDao.findByMonitorId(monitorId);
if (null != dashboard
&& StringUtils.isNotBlank(dashboard.getUrl())
&& !dashboard.getUrl().contains(GrafanaConstants.THEME)) {
String theme = loadThemeConfig();
return dashboard.toBuilder()
.url(dashboard.getUrl() + GrafanaConstants.THEME + theme)
.build();
}
return dashboard;
}
/**
@@ -248,4 +270,21 @@ public class DashboardService {
log.warn("No Grafana dashboard record found for monitorId {} to disable.", monitorId);
}
}
}
private String loadThemeConfig() {
try {
GeneralConfig config = generalConfigDao.findByType(GeneralConfigTypeEnum.system.name());
if (config != null && config.getContent() != null) {
JsonNode root = OBJECT_MAPPER.readTree(config.getContent());
JsonNode node = root.get("theme");
if (node != null && !node.isNull()) {
return ThemeEnum.convert(node.asText());
}
return ThemeEnum.LIGHT.getValue();
}
} catch (Exception e) {
log.error("Failed to load database theme configuration", e);
}
return ThemeEnum.LIGHT.getValue();
}
}
@@ -17,6 +17,7 @@
package org.apache.hertzbeat.grafana.service;
import org.apache.hertzbeat.base.dao.GeneralConfigDao;
import org.apache.hertzbeat.common.entity.grafana.GrafanaDashboard;
import org.apache.hertzbeat.grafana.config.GrafanaProperties;
import org.apache.hertzbeat.grafana.dao.DashboardDao;
@@ -34,10 +35,12 @@ import org.springframework.web.client.RestTemplate;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import static org.mockito.Mockito.lenient;
/**
* Test case for {@link DashboardService}
@@ -60,6 +63,9 @@ public class DashboardServiceTest {
@Mock
private DatasourceService datasourceService;
@Mock
private GeneralConfigDao generalConfigDao;
@InjectMocks
private DashboardService dashboardService;
@@ -73,25 +79,27 @@ public class DashboardServiceTest {
+ " \"version\": 3"
+ "}";
@BeforeEach
void setUp() {
when(datasourceService.getCurrentDatasourceName()).thenReturn("hertzbeat-vm-localhost-8428");
when(grafanaProperties.enabled()).thenReturn(Boolean.TRUE);
when(grafanaProperties.getPrefix()).thenReturn("");
when(grafanaProperties.getUrl()).thenReturn("http://127.0.0.1:3000");
when(grafanaProperties.exposeUrl()).thenReturn("http://127.0.0.1:3000");
when(serviceAccountService.getToken()).thenReturn("test-token");
lenient().when(datasourceService.getCurrentDatasourceName()).thenReturn("hertzbeat-vm-localhost-8428");
lenient().when(grafanaProperties.enabled()).thenReturn(Boolean.TRUE);
lenient().when(grafanaProperties.getPrefix()).thenReturn("");
lenient().when(grafanaProperties.getUrl()).thenReturn("http://127.0.0.1:3000");
lenient().when(grafanaProperties.exposeUrl()).thenReturn("http://127.0.0.1:3000");
lenient().when(serviceAccountService.getToken()).thenReturn("test-token");
}
@Test
void testCreateOrUpdateDashboard() {
ResponseEntity<String> responseEntity = new ResponseEntity<>(GRAFANA_API_RESULT, HttpStatus.OK);
when(restTemplate.postForEntity(
eq("http://127.0.0.1:3000/api/dashboards/db"), any(HttpEntity.class), eq(String.class)
)).thenReturn(responseEntity);
ArgumentCaptor<GrafanaDashboard> dashboardCaptor = ArgumentCaptor.forClass(GrafanaDashboard.class);
dashboardService.createOrUpdateDashboard("{\"id\":11}", 1L);
verify(dashboardDao).save(dashboardCaptor.capture());
@@ -109,7 +117,7 @@ public class DashboardServiceTest {
assertTrue(savedDashboard.getUrl().contains("refresh=15s"), "URL should contain refresh parameter");
assertTrue(savedDashboard.getUrl().contains("var-instance=1"), "URL should contain instance parameter");
}
@Test
void testCreateOrUpdateDashboardWithTrailingSlash() {
when(grafanaProperties.exposeUrl()).thenReturn("http://127.0.0.1:3000/");
@@ -138,4 +146,31 @@ public class DashboardServiceTest {
assertTrue(savedDashboard.getUrl().contains("var-instance=1"), "URL should contain instance parameter");
}
}
@Test
void testGetDashboardByMonitorId() {
Long monitorId = 1L;
String originalUrl = "http://localhost:3000/d/123/my-dashboard?kiosk";
GrafanaDashboard originalDashboard = GrafanaDashboard.builder()
.monitorId(monitorId)
.url(originalUrl)
.build();
when(dashboardDao.findByMonitorId(eq(monitorId))).thenReturn(originalDashboard);
String themeConfigJson = "{\"theme\":\"light\"}";
org.apache.hertzbeat.common.entity.manager.GeneralConfig runConfig = new org.apache.hertzbeat.common.entity.manager.GeneralConfig();
runConfig.setContent(themeConfigJson);
when(generalConfigDao.findByType(eq(org.apache.hertzbeat.common.constants.GeneralConfigTypeEnum.system.name())))
.thenReturn(runConfig);
GrafanaDashboard resultDashboard = dashboardService.getDashboardByMonitorId(monitorId);
assertNotNull(resultDashboard);
assertTrue(resultDashboard.getUrl().contains("&theme=light"));
assertTrue(originalDashboard.getUrl().endsWith("?kiosk"));
assertFalse(originalDashboard.getUrl().contains("&theme="));
}
}
@@ -17,6 +17,7 @@
package org.apache.hertzbeat.log.controller;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.extern.slf4j.Slf4j;
import java.util.List;
@@ -25,8 +26,8 @@ import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.entity.dto.Message;
import org.apache.hertzbeat.log.service.LogProtocolAdapter;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.util.StringUtils;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
@@ -34,15 +35,16 @@ import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* Log Ingestion Controller
* Generic Log Ingestion Controller
* Provides a fallback endpoint for log protocols that don't have dedicated controllers.
* For OTLP protocol, use OtlpLogController instead.
*/
@Tag(name = "Log Ingestion Controller")
@RestController
@RequestMapping(path = "/api/logs", produces = "application/json")
@RequestMapping(path = "/api/logs", produces = MediaType.APPLICATION_JSON_VALUE)
@Slf4j
public class LogIngestionController {
private static final String DEFAULT_PROTOCOL = "otlp";
private final List<LogProtocolAdapter> protocolAdapters;
public LogIngestionController(List<LogProtocolAdapter> protocolAdapters) {
@@ -51,26 +53,23 @@ public class LogIngestionController {
/**
* Receive log payload pushed from external system specifying the log protocol.
* Examples:
* - POST /api/logs/ingest/otlp (content body is OTLP JSON)
*
* @param protocol log protocol identifier
* @param content raw request body
* @param protocol log protocol identifier (e.g., "vector", "loki")
* @param content raw request body
*/
@PostMapping("/ingest/{protocol}")
public ResponseEntity<Message<Void>> ingestExternLog(@PathVariable("protocol") String protocol,
@RequestBody String content) {
log.debug("Receive extern log from protocol: {}, content length: {}", protocol, content == null ? 0 : content.length());
if (!StringUtils.hasText(protocol)) {
protocol = DEFAULT_PROTOCOL; // Default to OTLP if no protocol specified
}
@Operation(summary = "Ingest logs by protocol name")
@PostMapping(value = "/ingest/{protocol}", consumes = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<Message<Void>> ingestLog(@PathVariable("protocol") String protocol,
@RequestBody String content) {
log.debug("Receive log from protocol: {}, content length: {}", protocol, content == null ? 0 : content.length());
for (LogProtocolAdapter adapter : protocolAdapters) {
if (adapter.supportProtocol().equalsIgnoreCase(protocol)) {
try {
adapter.ingest(content);
return ResponseEntity.ok(Message.success("Add extern log success"));
} catch (Exception e) {
log.error("Add extern log failed: {}", e.getMessage(), e);
log.error("Add log failed: {}", e.getMessage(), e);
return ResponseEntity.status(HttpStatus.BAD_REQUEST)
.body(Message.fail(CommonConstants.FAIL_CODE, "Add extern log failed: " + e.getMessage()));
}
@@ -80,30 +79,4 @@ public class LogIngestionController {
return ResponseEntity.status(HttpStatus.BAD_REQUEST)
.body(Message.fail(CommonConstants.FAIL_CODE, "Not support the " + protocol + " protocol log"));
}
/**
* Receive default log payload (when protocol is not specified).
* It will look for a service whose supportProtocol() returns "otlp".
*/
@PostMapping("/ingest")
public ResponseEntity<Message<Void>> ingestDefaultExternLog(@RequestBody String content) {
log.info("Receive default extern log content, length: {}", content == null ? 0 : content.length());
LogProtocolAdapter adapter = protocolAdapters.stream()
.filter(item -> DEFAULT_PROTOCOL.equalsIgnoreCase(item.supportProtocol()))
.findFirst()
.orElse(null);
if (adapter != null) {
try {
adapter.ingest(content);
return ResponseEntity.ok(Message.success("Add extern log success"));
} catch (Exception e) {
log.error("Add extern log failed: {}", e.getMessage(), e);
return ResponseEntity.status(HttpStatus.BAD_REQUEST)
.body(Message.fail(CommonConstants.FAIL_CODE, "Add extern log failed: " + e.getMessage()));
}
}
log.error("Not support default extern log protocol");
return ResponseEntity.status(HttpStatus.BAD_REQUEST)
.body(Message.fail(CommonConstants.FAIL_CODE, "Not support the default protocol log"));
}
}
@@ -61,50 +61,54 @@ public class LogQueryController {
}
@GetMapping("/list")
@Operation(summary = "Query logs by time range with optional filters",
description = "Query logs by [start,end] in ms and optional filters with pagination. Returns paginated log entries sorted by timestamp in descending order.")
@Operation(summary = "Query logs by time range with optional filters",
description = "Query logs by [start,end] in ms and optional filters with pagination. Returns paginated log entries sorted by timestamp in descending order.")
public ResponseEntity<Message<Page<LogEntry>>> list(
@Parameter(description = "Start timestamp in milliseconds (Unix timestamp)", example = "1640995200000")
@Parameter(description = "Start timestamp in milliseconds (Unix timestamp)", example = "1640995200000")
@RequestParam(value = "start", required = false) Long start,
@Parameter(description = "End timestamp in milliseconds (Unix timestamp)", example = "1641081600000")
@Parameter(description = "End timestamp in milliseconds (Unix timestamp)", example = "1641081600000")
@RequestParam(value = "end", required = false) Long end,
@Parameter(description = "Trace ID for distributed tracing", example = "1234567890abcdef")
@Parameter(description = "Trace ID for distributed tracing", example = "1234567890abcdef")
@RequestParam(value = "traceId", required = false) String traceId,
@Parameter(description = "Span ID for distributed tracing", example = "abcdef1234567890")
@Parameter(description = "Span ID for distributed tracing", example = "abcdef1234567890")
@RequestParam(value = "spanId", required = false) String spanId,
@Parameter(description = "Log severity number (1-24 according to OpenTelemetry standard)", example = "9")
@Parameter(description = "Log severity number (1-24 according to OpenTelemetry standard)", example = "9")
@RequestParam(value = "severityNumber", required = false) Integer severityNumber,
@Parameter(description = "Log severity text (TRACE, DEBUG, INFO, WARN, ERROR, FATAL)", example = "INFO")
@Parameter(description = "Log severity text (TRACE, DEBUG, INFO, WARN, ERROR, FATAL)", example = "INFO")
@RequestParam(value = "severityText", required = false) String severityText,
@Parameter(description = "Page index starting from 0", example = "0")
@Parameter(description = "Log content search keyword", example = "error")
@RequestParam(value = "search", required = false) String search,
@Parameter(description = "Page index starting from 0", example = "0")
@RequestParam(value = "pageIndex", required = false, defaultValue = "0") Integer pageIndex,
@Parameter(description = "Number of items per page", example = "20")
@Parameter(description = "Number of items per page", example = "20")
@RequestParam(value = "pageSize", required = false, defaultValue = "20") Integer pageSize) {
Page<LogEntry> result = getPagedLogs(start, end, traceId, spanId, severityNumber, severityText, pageIndex, pageSize);
Page<LogEntry> result = getPagedLogs(start, end, traceId, spanId, severityNumber, severityText, search, pageIndex, pageSize);
return ResponseEntity.ok(Message.success(result));
}
@GetMapping("/stats/overview")
@Operation(summary = "Log overview statistics",
description = "Overall counts and basic statistics with filters. Provides counts by severity levels according to OpenTelemetry standard.")
@Operation(summary = "Log overview statistics",
description = "Overall counts and basic statistics with filters. Provides counts by severity levels according to OpenTelemetry standard.")
public ResponseEntity<Message<Map<String, Object>>> overviewStats(
@Parameter(description = "Start timestamp in milliseconds (Unix timestamp)", example = "1640995200000")
@Parameter(description = "Start timestamp in milliseconds (Unix timestamp)", example = "1640995200000")
@RequestParam(value = "start", required = false) Long start,
@Parameter(description = "End timestamp in milliseconds (Unix timestamp)", example = "1641081600000")
@Parameter(description = "End timestamp in milliseconds (Unix timestamp)", example = "1641081600000")
@RequestParam(value = "end", required = false) Long end,
@Parameter(description = "Trace ID for distributed tracing", example = "1234567890abcdef")
@Parameter(description = "Trace ID for distributed tracing", example = "1234567890abcdef")
@RequestParam(value = "traceId", required = false) String traceId,
@Parameter(description = "Span ID for distributed tracing", example = "abcdef1234567890")
@Parameter(description = "Span ID for distributed tracing", example = "abcdef1234567890")
@RequestParam(value = "spanId", required = false) String spanId,
@Parameter(description = "Log severity number (1-24 according to OpenTelemetry standard)", example = "9")
@Parameter(description = "Log severity number (1-24 according to OpenTelemetry standard)", example = "9")
@RequestParam(value = "severityNumber", required = false) Integer severityNumber,
@Parameter(description = "Log severity text (TRACE, DEBUG, INFO, WARN, ERROR, FATAL)", example = "INFO")
@RequestParam(value = "severityText", required = false) String severityText) {
List<LogEntry> logs = getFilteredLogs(start, end, traceId, spanId, severityNumber, severityText);
@Parameter(description = "Log severity text (TRACE, DEBUG, INFO, WARN, ERROR, FATAL)", example = "INFO")
@RequestParam(value = "severityText", required = false) String severityText,
@Parameter(description = "Log content search keyword", example = "error")
@RequestParam(value = "search", required = false) String search) {
List<LogEntry> logs = getFilteredLogs(start, end, traceId, spanId, severityNumber, severityText, search);
Map<String, Object> overview = new HashMap<>();
overview.put("totalCount", logs.size());
// Count by severity levels according to OpenTelemetry standard
// TRACE: 1-4, DEBUG: 5-8, INFO: 9-12, WARN: 13-16, ERROR: 17-20, FATAL: 21-24
long fatalCount = logs.stream().filter(log -> log.getSeverityNumber() != null && log.getSeverityNumber() >= 21 && log.getSeverityNumber() <= 24).count();
@@ -113,114 +117,117 @@ public class LogQueryController {
long infoCount = logs.stream().filter(log -> log.getSeverityNumber() != null && log.getSeverityNumber() >= 9 && log.getSeverityNumber() <= 12).count();
long debugCount = logs.stream().filter(log -> log.getSeverityNumber() != null && log.getSeverityNumber() >= 5 && log.getSeverityNumber() <= 8).count();
long traceCount = logs.stream().filter(log -> log.getSeverityNumber() != null && log.getSeverityNumber() >= 1 && log.getSeverityNumber() <= 4).count();
overview.put("fatalCount", fatalCount);
overview.put("errorCount", errorCount);
overview.put("warnCount", warnCount);
overview.put("infoCount", infoCount);
overview.put("debugCount", debugCount);
overview.put("traceCount", traceCount);
return ResponseEntity.ok(Message.success(overview));
}
@GetMapping("/stats/trace-coverage")
@Operation(summary = "Trace coverage statistics",
description = "Statistics about trace information availability. Shows how many logs have trace IDs, span IDs, or both for distributed tracing analysis.")
@Operation(summary = "Trace coverage statistics",
description = "Statistics about trace information availability. Shows how many logs have trace IDs, span IDs, or both for distributed tracing analysis.")
public ResponseEntity<Message<Map<String, Object>>> traceCoverageStats(
@Parameter(description = "Start timestamp in milliseconds (Unix timestamp)", example = "1640995200000")
@Parameter(description = "Start timestamp in milliseconds (Unix timestamp)", example = "1640995200000")
@RequestParam(value = "start", required = false) Long start,
@Parameter(description = "End timestamp in milliseconds (Unix timestamp)", example = "1641081600000")
@Parameter(description = "End timestamp in milliseconds (Unix timestamp)", example = "1641081600000")
@RequestParam(value = "end", required = false) Long end,
@Parameter(description = "Trace ID for distributed tracing", example = "1234567890abcdef")
@Parameter(description = "Trace ID for distributed tracing", example = "1234567890abcdef")
@RequestParam(value = "traceId", required = false) String traceId,
@Parameter(description = "Span ID for distributed tracing", example = "abcdef1234567890")
@Parameter(description = "Span ID for distributed tracing", example = "abcdef1234567890")
@RequestParam(value = "spanId", required = false) String spanId,
@Parameter(description = "Log severity number (1-24 according to OpenTelemetry standard)", example = "9")
@Parameter(description = "Log severity number (1-24 according to OpenTelemetry standard)", example = "9")
@RequestParam(value = "severityNumber", required = false) Integer severityNumber,
@Parameter(description = "Log severity text (TRACE, DEBUG, INFO, WARN, ERROR, FATAL)", example = "INFO")
@RequestParam(value = "severityText", required = false) String severityText) {
List<LogEntry> logs = getFilteredLogs(start, end, traceId, spanId, severityNumber, severityText);
@Parameter(description = "Log severity text (TRACE, DEBUG, INFO, WARN, ERROR, FATAL)", example = "INFO")
@RequestParam(value = "severityText", required = false) String severityText,
@Parameter(description = "Log content search keyword", example = "error")
@RequestParam(value = "search", required = false) String search) {
List<LogEntry> logs = getFilteredLogs(start, end, traceId, spanId, severityNumber, severityText, search);
Map<String, Object> result = new HashMap<>();
// Trace coverage statistics
long withTraceId = logs.stream().filter(log -> log.getTraceId() != null && !log.getTraceId().isEmpty()).count();
long withSpanId = logs.stream().filter(log -> log.getSpanId() != null && !log.getSpanId().isEmpty()).count();
long withBothTraceAndSpan = logs.stream().filter(log ->
log.getTraceId() != null && !log.getTraceId().isEmpty()
&& log.getSpanId() != null && !log.getSpanId().isEmpty()).count();
long withBothTraceAndSpan = logs.stream().filter(log ->
log.getTraceId() != null && !log.getTraceId().isEmpty()
&& log.getSpanId() != null && !log.getSpanId().isEmpty()).count();
long withoutTrace = logs.size() - withTraceId;
Map<String, Long> traceCoverage = new HashMap<>();
traceCoverage.put("withTrace", withTraceId);
traceCoverage.put("withoutTrace", withoutTrace);
traceCoverage.put("withSpan", withSpanId);
traceCoverage.put("withBothTraceAndSpan", withBothTraceAndSpan);
result.put("traceCoverage", traceCoverage);
return ResponseEntity.ok(Message.success(result));
}
@GetMapping("/stats/trend")
@Operation(summary = "Log trend over time",
description = "Count logs by hour intervals with filters. Groups logs by hour and provides time-series data for trend analysis.")
@Operation(summary = "Log trend over time",
description = "Count logs by hour intervals with filters. Groups logs by hour and provides time-series data for trend analysis.")
public ResponseEntity<Message<Map<String, Object>>> trendStats(
@Parameter(description = "Start timestamp in milliseconds (Unix timestamp)", example = "1640995200000")
@Parameter(description = "Start timestamp in milliseconds (Unix timestamp)", example = "1640995200000")
@RequestParam(value = "start", required = false) Long start,
@Parameter(description = "End timestamp in milliseconds (Unix timestamp)", example = "1641081600000")
@Parameter(description = "End timestamp in milliseconds (Unix timestamp)", example = "1641081600000")
@RequestParam(value = "end", required = false) Long end,
@Parameter(description = "Trace ID for distributed tracing", example = "1234567890abcdef")
@Parameter(description = "Trace ID for distributed tracing", example = "1234567890abcdef")
@RequestParam(value = "traceId", required = false) String traceId,
@Parameter(description = "Span ID for distributed tracing", example = "abcdef1234567890")
@Parameter(description = "Span ID for distributed tracing", example = "abcdef1234567890")
@RequestParam(value = "spanId", required = false) String spanId,
@Parameter(description = "Log severity number (1-24 according to OpenTelemetry standard)", example = "9")
@Parameter(description = "Log severity number (1-24 according to OpenTelemetry standard)", example = "9")
@RequestParam(value = "severityNumber", required = false) Integer severityNumber,
@Parameter(description = "Log severity text (TRACE, DEBUG, INFO, WARN, ERROR, FATAL)", example = "INFO")
@RequestParam(value = "severityText", required = false) String severityText) {
List<LogEntry> logs = getFilteredLogs(start, end, traceId, spanId, severityNumber, severityText);
@Parameter(description = "Log severity text (TRACE, DEBUG, INFO, WARN, ERROR, FATAL)", example = "INFO")
@RequestParam(value = "severityText", required = false) String severityText,
@Parameter(description = "Log content search keyword", example = "error")
@RequestParam(value = "search", required = false) String search) {
List<LogEntry> logs = getFilteredLogs(start, end, traceId, spanId, severityNumber, severityText, search);
// Group by hour
Map<String, Long> hourlyStats = logs.stream()
.filter(log -> log.getTimeUnixNano() != null)
.collect(Collectors.groupingBy(
log -> {
long timestampMs = log.getTimeUnixNano() / 1_000_000L;
LocalDateTime dateTime = LocalDateTime.ofInstant(
Instant.ofEpochMilli(timestampMs),
ZoneId.systemDefault());
return dateTime.format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:00"));
},
Collectors.counting()));
.filter(log -> log.getTimeUnixNano() != null)
.collect(Collectors.groupingBy(
log -> {
long timestampMs = log.getTimeUnixNano() / 1_000_000L;
LocalDateTime dateTime = LocalDateTime.ofInstant(
Instant.ofEpochMilli(timestampMs),
ZoneId.systemDefault());
return dateTime.format(DateTimeFormatter.ofPattern("yyyy-MM-dd HH:00"));
},
Collectors.counting()));
Map<String, Object> result = new HashMap<>();
result.put("hourlyStats", hourlyStats);
return ResponseEntity.ok(Message.success(result));
}
private List<LogEntry> getFilteredLogs(Long start, Long end, String traceId, String spanId,
Integer severityNumber, String severityText) {
private List<LogEntry> getFilteredLogs(Long start, Long end, String traceId, String spanId,
Integer severityNumber, String severityText, String search) {
// Use the new multi-condition query method
return historyDataReader.queryLogsByMultipleConditions(start, end, traceId, spanId, severityNumber, severityText);
return historyDataReader.queryLogsByMultipleConditions(start, end, traceId, spanId, severityNumber, severityText, search);
}
private Page<LogEntry> getPagedLogs(Long start, Long end, String traceId, String spanId,
Integer severityNumber, String severityText, Integer pageIndex, Integer pageSize) {
private Page<LogEntry> getPagedLogs(Long start, Long end, String traceId, String spanId,
Integer severityNumber, String severityText, String search,
Integer pageIndex, Integer pageSize) {
// Calculate pagination parameters
int offset = pageIndex * pageSize;
// Get total count and paginated data
long totalElements = historyDataReader.countLogsByMultipleConditions(start, end, traceId, spanId, severityNumber, severityText);
long totalElements = historyDataReader.countLogsByMultipleConditions(start, end, traceId, spanId, severityNumber, severityText, search);
List<LogEntry> pagedLogs = historyDataReader.queryLogsByMultipleConditionsWithPagination(
start, end, traceId, spanId, severityNumber, severityText, offset, pageSize);
start, end, traceId, spanId, severityNumber, severityText, search, offset, pageSize);
// Create PageRequest (sorted by timestamp descending)
Sort sort = Sort.by(Sort.Direction.DESC, "timeUnixNano");
PageRequest pageRequest = PageRequest.of(pageIndex, pageSize, sort);
// Return Spring Data Page object
return new PageImpl<>(pagedLogs, pageRequest, totalElements);
}
}
}
@@ -0,0 +1,160 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.log.controller;
import com.fasterxml.jackson.core.io.JsonStringEncoder;
import com.google.protobuf.InvalidProtocolBufferException;
import com.google.protobuf.util.JsonFormat;
import com.google.rpc.Status;
import io.opentelemetry.proto.collector.logs.v1.ExportLogsServiceResponse;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.log.service.impl.OtlpLogProtocolAdapter;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
/**
* OTLP Log Ingestion Controller
* Implements OTLP/HTTP specification for log ingestion.
* Supports both binary-encoded Protobuf (application/x-protobuf) and JSON-encoded Protobuf (application/json).
*
* @see <a href="https://opentelemetry.io/docs/specs/otlp/#otlphttp">OTLP/HTTP Specification</a>
*/
@Tag(name = "OTLP Log Controller")
@RestController
@RequestMapping(path = "/api/logs/otlp")
@Slf4j
public class OtlpLogController {
private static final String CONTENT_TYPE_PROTOBUF = "application/x-protobuf";
private static final ExportLogsServiceResponse EMPTY_RESPONSE = ExportLogsServiceResponse.newBuilder().build();
private final OtlpLogProtocolAdapter otlpLogProtocolAdapter;
public OtlpLogController(OtlpLogProtocolAdapter otlpLogProtocolAdapter) {
this.otlpLogProtocolAdapter = otlpLogProtocolAdapter;
}
/**
* OTLP/HTTP standard endpoint for logs with JSON-encoded Protobuf payload.
* Content-Type: application/json
*
* Response follows OTLP specification:
* - Success: HTTP 200 with ExportLogsServiceResponse (JSON encoded)
* - Failure: HTTP 400 with google.rpc.Status (JSON encoded)
*
* @param content JSON-encoded ExportLogsServiceRequest
* @return ExportLogsServiceResponse on success, Status on failure
*/
@Operation(summary = "Ingest OTLP logs (JSON format)")
@PostMapping(value = "/v1/logs", consumes = MediaType.APPLICATION_JSON_VALUE, produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<String> ingestJsonLogs(@RequestBody String content) {
log.debug("Receive OTLP JSON logs, content length: {}", content == null ? 0 : content.length());
try {
otlpLogProtocolAdapter.ingest(content);
return ResponseEntity.ok(toJsonResponse(EMPTY_RESPONSE));
} catch (IllegalArgumentException e) {
return ResponseEntity.status(HttpStatus.BAD_REQUEST)
.body(toJsonErrorResponse(e.getMessage()));
} catch (Exception e) {
// Server-side errors - unexpected failure
log.error("Unexpected error ingesting OTLP JSON logs: {}", e.getMessage(), e);
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(toJsonErrorResponse(e.getMessage()));
}
}
/**
* OTLP/HTTP standard endpoint for logs with binary-encoded Protobuf payload.
* Content-Type: application/x-protobuf
*
* Response follows OTLP specification:
* - Success: HTTP 200 with ExportLogsServiceResponse (binary encoded)
* - Failure: HTTP 400 with google.rpc.Status (binary encoded)
*
* @param content binary-encoded ExportLogsServiceRequest
* @return ExportLogsServiceResponse on success, Status on failure
*/
@Operation(summary = "Ingest OTLP logs (binary Protobuf format)")
@PostMapping(value = "/v1/logs", consumes = CONTENT_TYPE_PROTOBUF, produces = CONTENT_TYPE_PROTOBUF)
public ResponseEntity<byte[]> ingestBinaryLogs(@RequestBody byte[] content) {
log.debug("Receive OTLP binary logs, content length: {}", content == null ? 0 : content.length);
try {
otlpLogProtocolAdapter.ingestBinary(content);
return ResponseEntity.ok(EMPTY_RESPONSE.toByteArray());
} catch (IllegalArgumentException e) {
// Client-side validation errors - malformed request
return ResponseEntity.status(HttpStatus.BAD_REQUEST)
.body(createBinaryErrorResponse(e.getMessage()));
} catch (Exception e) {
// Server-side errors - unexpected failure
log.error("Unexpected error ingesting OTLP binary logs: {}", e.getMessage(), e);
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(createBinaryErrorResponse(e.getMessage()));
}
}
private String toJsonResponse(ExportLogsServiceResponse response) {
try {
return JsonFormat.printer().print(response);
} catch (InvalidProtocolBufferException e) {
log.error("Failed to convert ExportLogsServiceResponse to JSON: {}", e.getMessage(), e);
return "{}";
}
}
private String toJsonErrorResponse(String message) {
Status status = Status.newBuilder()
.setMessage(message != null ? message : "Unknown error")
.build();
try {
return JsonFormat.printer().print(status);
} catch (InvalidProtocolBufferException e) {
return "{\"message\":\"" + escapeJson(message) + "\"}";
}
}
/**
* Escapes a string value for safe inclusion in JSON.
*
* @param message the string to escape
* @return the escaped string, or empty string if message is null
*/
private String escapeJson(String message) {
if (message == null) {
return "";
}
char[] escaped = JsonStringEncoder.getInstance().quoteAsString(message);
return new String(escaped);
}
private byte[] createBinaryErrorResponse(String message) {
return Status.newBuilder()
.setMessage(message != null ? message : "Unknown error")
.build()
.toByteArray();
}
}
@@ -49,7 +49,13 @@ public class LogSseFilterCriteria {
*/
@Schema(description = "The severity text (also known as log level).", example = "INFO", accessMode = READ_WRITE)
private String severityText;
/**
* Log content text filtering (case-insensitive contains match).
*/
@Schema(description = "Log content text filtering", example = "error occurred", accessMode = READ_WRITE)
private String logContent;
/**
* A unique identifier for a trace.
* All spans from the same trace share the same trace_id.
@@ -77,18 +83,30 @@ public class LogSseFilterCriteria {
if (StringUtils.hasText(severityText) && !severityText.equalsIgnoreCase(log.getSeverityText())) {
return false;
}
// Check severity number match (if both are present)
if (severityNumber != null && log.getSeverityNumber() != null
&& !severityNumber.equals(log.getSeverityNumber())) {
return false;
}
// Check log content match
if (StringUtils.hasText(logContent)) {
Object body = log.getBody();
if (body == null) {
return false;
}
String bodyStr = body.toString();
if (!StringUtils.hasText(bodyStr) || !bodyStr.toLowerCase().contains(logContent.toLowerCase())) {
return false;
}
}
// Check trace ID match
if (StringUtils.hasText(traceId) && !traceId.equalsIgnoreCase(log.getTraceId())) {
return false;
}
// Check span ID match
if (StringUtils.hasText(spanId) && !spanId.equalsIgnoreCase(log.getSpanId())) {
return false;
@@ -28,7 +28,7 @@ package org.apache.hertzbeat.log.service;
public interface LogProtocolAdapter {
/**
* Ingest raw log payload pushed from external system.
* Ingest log payload pushed from external system.
*
* @param content raw request body string
*/
@@ -39,7 +39,10 @@ import java.util.List;
import java.util.Map;
/**
* Adapter for OpenTelemetry OTLP/HTTP JSON log ingestion.
* Adapter for OpenTelemetry OTLP/HTTP log ingestion.
* Supports both JSON-encoded and binary-encoded Protobuf formats.
*
* @see <a href="https://opentelemetry.io/docs/specs/otlp/#otlphttp">OTLP/HTTP Specification</a>
*/
@Slf4j
@Service
@@ -58,26 +61,47 @@ public class OtlpLogProtocolAdapter implements LogProtocolAdapter {
@Override
public void ingest(String content) {
if (content == null || content.isEmpty()) {
log.warn("Received empty OTLP log payload - skip processing.");
log.warn("Received empty OTLP JSON log payload - skip processing.");
return;
}
ExportLogsServiceRequest.Builder builder = ExportLogsServiceRequest.newBuilder();
try {
JsonFormat.parser().ignoringUnknownFields().merge(content, builder);
ExportLogsServiceRequest request = builder.build();
// Extract LogEntry instances from the request
List<LogEntry> logEntries = extractLogEntries(request);
log.debug("Successfully extracted {} log entries from OTLP payload {}", logEntries.size(), content);
commonDataQueue.sendLogEntryToStorageBatch(logEntries);
commonDataQueue.sendLogEntryToAlertBatch(logEntries);
logEntries.forEach(logSseManager::broadcast);
processLogsRequest(request, "JSON");
} catch (InvalidProtocolBufferException e) {
log.error("Failed to parse OTLP log payload: {}", e.getMessage());
throw new IllegalArgumentException("Invalid OTLP log content", e);
log.error("Failed to parse OTLP JSON log payload: {}", e.getMessage());
throw new IllegalArgumentException("Invalid OTLP JSON log content", e);
}
}
/**
* Ingest binary-encoded Protobuf log payload (OTLP-specific).
*
* @param content binary-encoded ExportLogsServiceRequest
*/
public void ingestBinary(byte[] content) {
if (content == null || content.length == 0) {
log.warn("Received empty OTLP binary log payload - skip processing.");
return;
}
try {
ExportLogsServiceRequest request = ExportLogsServiceRequest.parseFrom(content);
processLogsRequest(request, "binary");
} catch (InvalidProtocolBufferException e) {
log.error("Failed to parse OTLP binary log payload: {}", e.getMessage());
throw new IllegalArgumentException("Invalid OTLP binary log content", e);
}
}
private void processLogsRequest(ExportLogsServiceRequest request, String format) {
List<LogEntry> logEntries = extractLogEntries(request);
log.debug("Successfully extracted {} log entries from OTLP {} payload", logEntries.size(), format);
commonDataQueue.sendLogEntryToStorageBatch(logEntries);
commonDataQueue.sendLogEntryToAlertBatch(logEntries);
logEntries.forEach(logSseManager::broadcast);
}
/**
* Extract LogEntry instances from ExportLogsServiceRequest.
*
@@ -18,24 +18,22 @@
package org.apache.hertzbeat.log.controller;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.doNothing;
import static org.mockito.Mockito.doThrow;
import static org.mockito.Mockito.when;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
import static org.springframework.test.web.servlet.result.MockMvcResultHandlers.print;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.entity.log.LogEntry;
import org.apache.hertzbeat.common.util.JsonUtil;
import org.apache.hertzbeat.log.service.LogProtocolAdapter;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
@@ -51,34 +49,29 @@ class LogIngestionControllerTest {
private MockMvc mockMvc;
@Mock
private LogProtocolAdapter otlpAdapter;
private LogProtocolAdapter vectorAdapter;
private LogIngestionController logIngestionController;
@BeforeEach
void setUp() {
List<LogProtocolAdapter> adapters = Arrays.asList(otlpAdapter);
List<LogProtocolAdapter> adapters = Arrays.asList(vectorAdapter);
this.logIngestionController = new LogIngestionController(adapters);
this.mockMvc = MockMvcBuilders.standaloneSetup(logIngestionController).build();
}
@Test
void testIngestExternLogWithOtlpProtocol() throws Exception {
LogEntry logEntry = LogEntry.builder()
.timeUnixNano(1734005477630L)
.severityNumber(1)
.severityText("INFO")
.body("Test log message")
.attributes(new HashMap<>())
.build();
void testIngestLogWithKnownProtocol() throws Exception {
String logContent = "{\"message\":\"Test log message\"}";
when(otlpAdapter.supportProtocol()).thenReturn("otlp");
when(vectorAdapter.supportProtocol()).thenReturn("vector");
doNothing().when(vectorAdapter).ingest(anyString());
mockMvc.perform(
MockMvcRequestBuilders
.post("/api/logs/ingest/otlp")
.post("/api/logs/ingest/vector")
.contentType(MediaType.APPLICATION_JSON)
.content(JsonUtil.toJson(logEntry))
.content(logContent)
)
.andDo(print())
.andExpect(status().isOk())
@@ -88,16 +81,16 @@ class LogIngestionControllerTest {
}
@Test
void testIngestExternLogWithUnsupportedProtocol() throws Exception {
String unsupportedLogContent = "{\"message\":\"Unsupported protocol log\"}";
void testIngestLogWithUnsupportedProtocol() throws Exception {
String logContent = "{\"message\":\"Unsupported protocol log\"}";
when(otlpAdapter.supportProtocol()).thenReturn("otlp");
when(vectorAdapter.supportProtocol()).thenReturn("vector");
mockMvc.perform(
MockMvcRequestBuilders
.post("/api/logs/ingest/unsupported")
.contentType(MediaType.APPLICATION_JSON)
.content(unsupportedLogContent)
.content(logContent)
)
.andExpect(status().isBadRequest())
.andExpect(jsonPath("$.code").value((int) CommonConstants.FAIL_CODE))
@@ -105,38 +98,15 @@ class LogIngestionControllerTest {
}
@Test
void testIngestDefaultExternLog() throws Exception {
LogEntry logEntry = LogEntry.builder()
.timeUnixNano(1734005477630L)
.severityNumber(2)
.severityText("WARN")
.body("Default protocol log message")
.attributes(new HashMap<>())
.build();
void testIngestLogWithAdapterException() throws Exception {
String logContent = "{\"message\":\"Log message that will cause exception\"}";
when(otlpAdapter.supportProtocol()).thenReturn("otlp");
when(vectorAdapter.supportProtocol()).thenReturn("vector");
doThrow(new IllegalArgumentException("Invalid log format")).when(vectorAdapter).ingest(anyString());
mockMvc.perform(
MockMvcRequestBuilders
.post("/api/logs/ingest")
.contentType(MediaType.APPLICATION_JSON)
.content(JsonUtil.toJson(logEntry))
)
.andExpect(status().isOk())
.andExpect(jsonPath("$.code").value((int) CommonConstants.SUCCESS_CODE))
.andExpect(jsonPath("$.msg").value("Add extern log success"));
}
@Test
void testIngestDefaultExternLogWithAdapterException() throws Exception {
String logContent = "{\"message\":\"Default log message that will cause exception\"}";
when(otlpAdapter.supportProtocol()).thenReturn("otlp");
Mockito.doThrow(new IllegalArgumentException("Invalid log format")).when(otlpAdapter).ingest(anyString());
mockMvc.perform(
MockMvcRequestBuilders
.post("/api/logs/ingest")
.post("/api/logs/ingest/vector")
.contentType(MediaType.APPLICATION_JSON)
.content(logContent)
)
@@ -17,11 +17,10 @@
package org.apache.hertzbeat.log.controller;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyInt;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.ArgumentMatchers.isNull;
import static org.mockito.Mockito.when;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
@@ -87,10 +86,10 @@ class LogQueryControllerTest {
List<LogEntry> mockLogs = Arrays.asList(logEntry1, logEntry2);
when(historyDataReader.countLogsByMultipleConditions(anyLong(), anyLong(), anyString(),
anyString(), anyInt(), anyString())).thenReturn(2L);
when(historyDataReader.queryLogsByMultipleConditionsWithPagination(anyLong(), anyLong(),
anyString(), anyString(), anyInt(), anyString(), anyInt(), anyInt()))
when(historyDataReader.countLogsByMultipleConditions(anyLong(), anyLong(), any(),
any(), any(), any(), any())).thenReturn(2L);
when(historyDataReader.queryLogsByMultipleConditionsWithPagination(anyLong(), anyLong(),
any(), any(), any(), any(), any(), anyInt(), anyInt()))
.thenReturn(mockLogs);
mockMvc.perform(
@@ -127,10 +126,10 @@ class LogQueryControllerTest {
.build()
);
when(historyDataReader.countLogsByMultipleConditions(isNull(), isNull(), isNull(),
isNull(), isNull(), isNull())).thenReturn(1L);
when(historyDataReader.queryLogsByMultipleConditionsWithPagination(isNull(), isNull(),
isNull(), isNull(), isNull(), isNull(), eq(0), eq(20)))
when(historyDataReader.countLogsByMultipleConditions(any(), any(), any(),
any(), any(), any(), any())).thenReturn(1L);
when(historyDataReader.queryLogsByMultipleConditionsWithPagination(any(), any(),
any(), any(), any(), any(), any(), eq(0), eq(20)))
.thenReturn(mockLogs);
mockMvc.perform(
@@ -163,8 +162,8 @@ class LogQueryControllerTest {
LogEntry.builder().severityNumber(21).build()
);
when(historyDataReader.queryLogsByMultipleConditions(isNull(), isNull(), isNull(),
isNull(), isNull(), isNull())).thenReturn(mockLogs);
when(historyDataReader.queryLogsByMultipleConditions(any(), any(), any(),
any(), any(), any(), any())).thenReturn(mockLogs);
mockMvc.perform(
MockMvcRequestBuilders
@@ -188,8 +187,8 @@ class LogQueryControllerTest {
LogEntry.builder().severityNumber(17).build()
);
when(historyDataReader.queryLogsByMultipleConditions(eq(1734005477000L), eq(1734005478000L),
isNull(), isNull(), isNull(), isNull())).thenReturn(mockLogs);
when(historyDataReader.queryLogsByMultipleConditions(eq(1734005477000L), eq(1734005478000L),
any(), any(), any(), any(), any())).thenReturn(mockLogs);
mockMvc.perform(
MockMvcRequestBuilders
@@ -217,8 +216,8 @@ class LogQueryControllerTest {
LogEntry.builder().build() // null values
);
when(historyDataReader.queryLogsByMultipleConditions(isNull(), isNull(), isNull(),
isNull(), isNull(), isNull())).thenReturn(mockLogs);
when(historyDataReader.queryLogsByMultipleConditions(any(), any(), any(),
any(), any(), any(), any())).thenReturn(mockLogs);
mockMvc.perform(
MockMvcRequestBuilders
@@ -244,8 +243,8 @@ class LogQueryControllerTest {
LogEntry.builder().timeUnixNano(1734009077630000000L).build()
);
when(historyDataReader.queryLogsByMultipleConditions(isNull(), isNull(), isNull(),
isNull(), isNull(), isNull())).thenReturn(mockLogs);
when(historyDataReader.queryLogsByMultipleConditions(any(), any(), any(),
any(), any(), any(), any())).thenReturn(mockLogs);
mockMvc.perform(
MockMvcRequestBuilders
@@ -263,8 +262,8 @@ class LogQueryControllerTest {
LogEntry.builder().timeUnixNano(null).build() // This should be filtered out
);
when(historyDataReader.queryLogsByMultipleConditions(isNull(), isNull(), isNull(),
isNull(), isNull(), isNull())).thenReturn(mockLogs);
when(historyDataReader.queryLogsByMultipleConditions(any(), any(), any(),
any(), any(), any(), any())).thenReturn(mockLogs);
mockMvc.perform(
MockMvcRequestBuilders
@@ -0,0 +1,131 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.log.controller;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.doNothing;
import static org.mockito.Mockito.doThrow;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content;
import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status;
import static org.springframework.test.web.servlet.result.MockMvcResultHandlers.print;
import org.apache.hertzbeat.log.service.impl.OtlpLogProtocolAdapter;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.http.MediaType;
import org.springframework.test.web.servlet.MockMvc;
import org.springframework.test.web.servlet.request.MockMvcRequestBuilders;
import org.springframework.test.web.servlet.setup.MockMvcBuilders;
/**
* Unit test for {@link OtlpLogController}
*/
@ExtendWith(MockitoExtension.class)
class OtlpLogControllerTest {
private static final String CONTENT_TYPE_PROTOBUF = "application/x-protobuf";
private MockMvc mockMvc;
@Mock
private OtlpLogProtocolAdapter otlpLogProtocolAdapter;
private OtlpLogController otlpLogController;
@BeforeEach
void setUp() {
this.otlpLogController = new OtlpLogController(otlpLogProtocolAdapter);
this.mockMvc = MockMvcBuilders.standaloneSetup(otlpLogController).build();
}
@Test
void testIngestJsonLogsSuccess() throws Exception {
String jsonContent = "{\"resourceLogs\":[]}";
doNothing().when(otlpLogProtocolAdapter).ingest(anyString());
mockMvc.perform(
MockMvcRequestBuilders
.post("/api/logs/otlp/v1/logs")
.contentType(MediaType.APPLICATION_JSON)
.content(jsonContent)
)
.andDo(print())
.andExpect(status().isOk())
.andExpect(content().contentType(MediaType.APPLICATION_JSON))
.andReturn();
}
@Test
void testIngestJsonLogsFailure() throws Exception {
String jsonContent = "{\"invalid\":\"content\"}";
doThrow(new IllegalArgumentException("Invalid OTLP JSON log content"))
.when(otlpLogProtocolAdapter).ingest(anyString());
mockMvc.perform(
MockMvcRequestBuilders
.post("/api/logs/otlp/v1/logs")
.contentType(MediaType.APPLICATION_JSON)
.content(jsonContent)
)
.andExpect(status().isBadRequest())
.andExpect(content().contentType(MediaType.APPLICATION_JSON))
.andReturn();
}
@Test
void testIngestBinaryLogsSuccess() throws Exception {
byte[] binaryContent = new byte[]{0x0a, 0x0b, 0x0c};
doNothing().when(otlpLogProtocolAdapter).ingestBinary(any(byte[].class));
mockMvc.perform(
MockMvcRequestBuilders
.post("/api/logs/otlp/v1/logs")
.contentType(CONTENT_TYPE_PROTOBUF)
.content(binaryContent)
)
.andDo(print())
.andExpect(status().isOk())
.andExpect(content().contentType(CONTENT_TYPE_PROTOBUF))
.andReturn();
}
@Test
void testIngestBinaryLogsFailure() throws Exception {
byte[] binaryContent = new byte[]{0x0a, 0x0b, 0x0c};
doThrow(new IllegalArgumentException("Invalid OTLP binary log content"))
.when(otlpLogProtocolAdapter).ingestBinary(any(byte[].class));
mockMvc.perform(
MockMvcRequestBuilders
.post("/api/logs/otlp/v1/logs")
.contentType(CONTENT_TYPE_PROTOBUF)
.content(binaryContent)
)
.andExpect(status().isBadRequest())
.andExpect(content().contentType(CONTENT_TYPE_PROTOBUF))
.andReturn();
}
}
@@ -105,16 +105,44 @@ class LogSseFilterCriteriaTest {
// Test Span ID filter - match
filterCriteria.setSpanId("1234567890abcdef");
assertTrue(filterCriteria.matches(testLogEntry));
// Test Span ID filter - no match
filterCriteria.setSpanId("abcdef1234567890");
assertFalse(filterCriteria.matches(testLogEntry));
// Test Span ID filter - case insensitive
filterCriteria.setSpanId("1234567890ABCDEF");
assertTrue(filterCriteria.matches(testLogEntry));
}
@Test
void testMatchesWithLogContentFilter() {
// Test log content filter - match
filterCriteria.setLogContent("Test log");
assertTrue(filterCriteria.matches(testLogEntry));
// Test log content filter - no match
filterCriteria.setLogContent("Error message");
assertFalse(filterCriteria.matches(testLogEntry));
// Test log content filter - case insensitive
filterCriteria.setLogContent("test log");
assertTrue(filterCriteria.matches(testLogEntry));
// Test log content filter - partial match
filterCriteria.setLogContent("message");
assertTrue(filterCriteria.matches(testLogEntry));
// Test log content filter with null body
LogEntry nullBodyLog = LogEntry.builder()
.severityNumber(9)
.severityText("INFO")
.body(null)
.build();
filterCriteria.setLogContent("test");
assertFalse(filterCriteria.matches(nullBodyLog));
}
@Test
void testMatchesWithMultipleFilters() {
// Test multiple filter combinations - all match
@@ -163,14 +191,15 @@ class LogSseFilterCriteriaTest {
void testConstructorWithAllParameters() {
// Test constructor with all parameters
LogSseFilterCriteria criteria = new LogSseFilterCriteria(
9, "INFO", "1234567890abcdef1234567890abcdef", "1234567890abcdef"
9, "INFO", null, "1234567890abcdef1234567890abcdef", "1234567890abcdef"
);
assertEquals(9, criteria.getSeverityNumber());
assertEquals("INFO", criteria.getSeverityText());
assertEquals(null, criteria.getLogContent());
assertEquals("1234567890abcdef1234567890abcdef", criteria.getTraceId());
assertEquals("1234567890abcdef", criteria.getSpanId());
// Test matching
assertTrue(criteria.matches(testLogEntry));
}
@@ -179,17 +208,19 @@ class LogSseFilterCriteriaTest {
void testNoArgsConstructorAndSetters() {
// Test no-args constructor and setter methods
LogSseFilterCriteria criteria = new LogSseFilterCriteria();
criteria.setSeverityNumber(9);
criteria.setSeverityText("INFO");
criteria.setLogContent("Test log");
criteria.setTraceId("1234567890abcdef1234567890abcdef");
criteria.setSpanId("1234567890abcdef");
assertEquals(9, criteria.getSeverityNumber());
assertEquals("INFO", criteria.getSeverityText());
assertEquals("Test log", criteria.getLogContent());
assertEquals("1234567890abcdef1234567890abcdef", criteria.getTraceId());
assertEquals("1234567890abcdef", criteria.getSpanId());
// Test matching
assertTrue(criteria.matches(testLogEntry));
}
@@ -80,6 +80,47 @@ class OtlpLogProtocolAdapterTest {
verifyNoInteractions(commonDataQueue, logSseManager);
}
@Test
void testIngestBinaryWithNullContent() {
adapter.ingestBinary(null);
verifyNoInteractions(commonDataQueue, logSseManager);
}
@Test
void testIngestBinaryWithEmptyContent() {
adapter.ingestBinary(new byte[0]);
verifyNoInteractions(commonDataQueue, logSseManager);
}
@Test
void testIngestBinaryWithValidOtlpLogData() throws Exception {
byte[] binaryPayload = createValidOtlpLogBinaryPayload();
adapter.ingestBinary(binaryPayload);
ArgumentCaptor<List<LogEntry>> listCaptor = ArgumentCaptor.forClass(List.class);
verify(commonDataQueue, times(1)).sendLogEntryToStorageBatch(listCaptor.capture());
verify(commonDataQueue, times(1)).sendLogEntryToAlertBatch(anyList());
verify(logSseManager, times(1)).broadcast(any(LogEntry.class));
List<LogEntry> capturedList = listCaptor.getValue();
assertNotNull(capturedList);
assertEquals(1, capturedList.size());
LogEntry capturedEntry = capturedList.get(0);
assertEquals("binary-test-service", capturedEntry.getResource().get("service_name"));
assertEquals("binary log message", capturedEntry.getBody());
assertEquals("INFO", capturedEntry.getSeverityText());
}
@Test
void testIngestBinaryWithInvalidContent() {
byte[] invalidBinary = "not a valid protobuf".getBytes();
assertThrows(IllegalArgumentException.class, () -> adapter.ingestBinary(invalidBinary));
verifyNoInteractions(commonDataQueue, logSseManager);
}
@Test
void testIngestWithValidOtlpLogData() throws Exception {
String otlpPayload = createValidOtlpLogPayload();
@@ -315,4 +356,32 @@ class OtlpLogProtocolAdapterTest {
ExportLogsServiceRequest request = ExportLogsServiceRequest.newBuilder().build();
return JsonFormat.printer().print(request);
}
private byte[] createValidOtlpLogBinaryPayload() {
ExportLogsServiceRequest request = ExportLogsServiceRequest.newBuilder()
.addResourceLogs(ResourceLogs.newBuilder()
.setResource(Resource.newBuilder()
.addAttributes(KeyValue.newBuilder()
.setKey("service.name")
.setValue(AnyValue.newBuilder().setStringValue("binary-test-service").build())
.build())
.build())
.addScopeLogs(ScopeLogs.newBuilder()
.setScope(InstrumentationScope.newBuilder()
.setName("binary-test-scope")
.setVersion("1.0.0")
.build())
.addLogRecords(LogRecord.newBuilder()
.setTimeUnixNano(System.currentTimeMillis() * 1_000_000)
.setObservedTimeUnixNano(System.currentTimeMillis() * 1_000_000)
.setSeverityNumberValue(9)
.setSeverityText("INFO")
.setBody(AnyValue.newBuilder().setStringValue("binary log message").build())
.build())
.build())
.build())
.build();
return request.toByteArray();
}
}
@@ -26,7 +26,7 @@ import java.util.HashSet;
import java.util.List;
import org.apache.hertzbeat.common.entity.dto.Message;
import org.apache.hertzbeat.common.entity.manager.Label;
import org.apache.hertzbeat.manager.service.LabelService;
import org.apache.hertzbeat.base.service.LabelService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.domain.Page;
import org.springframework.http.ResponseEntity;
@@ -46,7 +46,7 @@ import org.springframework.web.bind.annotation.RestController;
@RestController
@RequestMapping(path = "/api/label", produces = {APPLICATION_JSON_VALUE})
public class LabelController {
@Autowired
private LabelService labelService;
@@ -45,8 +45,11 @@ import org.springframework.data.jpa.domain.support.AuditingEntityListener;
* PluginParam
*/
@Entity
@Table(name = "hzb_plugin_param", indexes = { @Index(columnList = "pluginMetadataId") },
uniqueConstraints = @UniqueConstraint(columnNames = {"pluginMetadataId", "field"}))
@Table(
name = "hzb_plugin_param",
indexes = {@Index(name = "idx_hzb_plugin_param_plugin_metadata_id", columnList = "plugin_metadata_id")},
uniqueConstraints = @UniqueConstraint(name = "uk_hzb_plugin_param_metadata_field", columnNames = {"plugin_metadata_id", "field"})
)
@Data
@Builder
@AllArgsConstructor
@@ -65,8 +68,9 @@ public class PluginParam {
/**
* Monitor ID
*/
@Schema(title = "Plugin task ID", example = "875846754543", accessMode = READ_WRITE)
@NotNull
@Column(name = "plugin_metadata_id")
@Schema(title = "Plugin task ID", example = "875846754543", accessMode = READ_WRITE)
private Long pluginMetadataId;
/**
@@ -75,6 +79,7 @@ public class PluginParam {
@Schema(title = "Parameter identifier field", example = "port", accessMode = READ_WRITE)
@Size(max = 100)
@NotNull
@Column(name = "field")
private String field;
/**
@@ -31,7 +31,7 @@ import org.apache.hertzbeat.manager.config.ManagerSseManager;
import org.apache.hertzbeat.manager.pojo.dto.MonitorDto;
import org.apache.hertzbeat.manager.service.ImExportService;
import org.apache.hertzbeat.manager.service.MonitorService;
import org.apache.hertzbeat.manager.service.LabelService;
import org.apache.hertzbeat.base.service.LabelService;
import org.springframework.beans.BeanUtils;
import org.springframework.context.annotation.Lazy;
import org.springframework.util.CollectionUtils;
@@ -31,8 +31,8 @@ import lombok.extern.slf4j.Slf4j;
import org.apache.commons.collections.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.hertzbeat.common.entity.manager.Label;
import org.apache.hertzbeat.manager.dao.LabelDao;
import org.apache.hertzbeat.manager.service.LabelService;
import org.apache.hertzbeat.base.dao.LabelDao;
import org.apache.hertzbeat.base.service.LabelService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.PageRequest;
@@ -50,7 +50,7 @@ import org.apache.hertzbeat.grafana.service.DashboardService;
import org.apache.hertzbeat.manager.component.validator.ParamValidatorManager;
import org.apache.hertzbeat.manager.dao.CollectorDao;
import org.apache.hertzbeat.manager.dao.CollectorMonitorBindDao;
import org.apache.hertzbeat.manager.dao.LabelDao;
import org.apache.hertzbeat.base.dao.LabelDao;
import org.apache.hertzbeat.manager.dao.MonitorBindDao;
import org.apache.hertzbeat.manager.dao.MonitorDao;
import org.apache.hertzbeat.manager.dao.ParamDao;
@@ -59,7 +59,7 @@ import org.apache.hertzbeat.manager.pojo.dto.MetricsInfo;
import org.apache.hertzbeat.manager.pojo.dto.MonitorDto;
import org.apache.hertzbeat.manager.scheduler.CollectJobScheduling;
import org.apache.hertzbeat.manager.service.AppService;
import org.apache.hertzbeat.manager.service.LabelService;
import org.apache.hertzbeat.base.service.LabelService;
import org.apache.hertzbeat.manager.service.MetricsFavoriteService;
import org.apache.hertzbeat.manager.service.MonitorService;
import org.apache.hertzbeat.manager.service.helper.MonitorImExportHelper;
@@ -18,9 +18,8 @@
package org.apache.hertzbeat.manager.service.impl;
import java.time.Instant;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.ZoneOffset;
import java.time.ZonedDateTime;
import java.util.ArrayList;
import java.util.LinkedList;
import java.util.List;
@@ -131,45 +130,70 @@ public class StatusPageServiceImpl implements StatusPageService {
componentStatus.setInfo(component);
List<StatusPageHistory> histories = new LinkedList<>();
// query today status
LocalDateTime nowTime = LocalDateTime.now();
LocalDateTime todayStartTime = nowTime.withHour(0).withMinute(0).withSecond(0).withNano(0);
ZoneOffset zoneOffset = ZoneId.systemDefault().getRules().getOffset(Instant.now());
long nowTimestamp = nowTime.toInstant(zoneOffset).toEpochMilli();
long todayStartTimestamp = todayStartTime.toInstant(zoneOffset).toEpochMilli();
ZoneId zoneId = ZoneId.systemDefault();
Instant now = Instant.now();
long nowTimestamp = now.toEpochMilli();
long todayStartTimestamp = now
.atZone(zoneId)
.toLocalDate()
.atStartOfDay(zoneId)
.toInstant()
.toEpochMilli();
List<StatusPageHistory> todayStatusPageHistoryList = statusPageHistoryDao
.findStatusPageHistoriesByComponentIdAndTimestampBetween(component.getId(), todayStartTimestamp, nowTimestamp);
StatusPageHistory todayStatus = combineOneDayStatusPageHistory(todayStatusPageHistoryList, component, nowTimestamp);
histories.add(todayStatus);
// query 30d component status history
LocalDateTime preTime = todayStartTime.minusDays(HISTORY_SPAN_DAYS);
long preTimestamp = preTime.toInstant(zoneOffset).toEpochMilli();
long preTimestamp = now
.atZone(zoneId)
.toLocalDate()
.minusDays(HISTORY_SPAN_DAYS)
.atStartOfDay(zoneId)
.toInstant()
.toEpochMilli();
List<StatusPageHistory> history = statusPageHistoryDao
.findStatusPageHistoriesByComponentIdAndTimestampBetween(component.getId(), preTimestamp, todayStartTimestamp);
LinkedList<StatusPageHistory> historyList = new LinkedList<>(history);
historyList.sort((o1, o2) -> (int) (o1.getTimestamp() - o2.getTimestamp()));
LocalDateTime endTime = todayStartTime.minusSeconds(1);
LocalDateTime startTime = endTime.withHour(0).withMinute(0).withSecond(0).withNano(0);
for (int index = 0; index < HISTORY_SPAN_DAYS; index++) {
long startTimestamp = startTime.toInstant(zoneOffset).toEpochMilli();
long endTimestamp = endTime.toInstant(zoneOffset).toEpochMilli();
List<StatusPageHistory> thisDayHistory = historyList.stream().filter(item ->
item.getTimestamp() >= startTimestamp && item.getTimestamp() <= endTimestamp)
.collect(Collectors.toList());
historyList.sort((o1, o2) -> Long.compare(o1.getTimestamp(), o2.getTimestamp()));
ZonedDateTime end = Instant.ofEpochMilli(todayStartTimestamp)
.atZone(zoneId)
.minusSeconds(1); // yesterday 23:59:59 local time
for (int i = 0; i < HISTORY_SPAN_DAYS; i++) {
long endTimestamp = end.toInstant().toEpochMilli();
long startTimestamp = end.toLocalDate()
.atStartOfDay(zoneId)
.toInstant()
.toEpochMilli();
List<StatusPageHistory> thisDayHistory = historyList.stream()
.filter(h -> h.getTimestamp() >= startTimestamp && h.getTimestamp() <= endTimestamp)
.collect(Collectors.toList());
if (thisDayHistory.isEmpty()) {
StatusPageHistory statusPageHistory = StatusPageHistory.builder().timestamp(endTimestamp)
.componentId(component.getId()).state(CommonConstants.STATUS_PAGE_COMPONENT_STATE_UNKNOWN).build();
histories.add(statusPageHistory);
histories.add(StatusPageHistory.builder()
.timestamp(endTimestamp)
.componentId(component.getId())
.state(CommonConstants.STATUS_PAGE_COMPONENT_STATE_UNKNOWN)
.build());
} else if (thisDayHistory.size() == 1) {
histories.add(thisDayHistory.get(0));
} else {
StatusPageHistory statusPageHistory = combineOneDayStatusPageHistory(thisDayHistory, component, endTimestamp);
histories.add(statusPageHistory);
StatusPageHistory merged =
combineOneDayStatusPageHistory(thisDayHistory, component, endTimestamp);
histories.add(merged);
statusPageHistoryDao.deleteAll(thisDayHistory);
statusPageHistoryDao.save(statusPageHistory);
statusPageHistoryDao.save(merged);
}
startTime = startTime.minusDays(1);
endTime = endTime.minusDays(1);
end = end.minusDays(1);
}
componentStatus.setHistory(histories);
componentStatusList.add(componentStatus);
}
@@ -215,50 +239,86 @@ public class StatusPageServiceImpl implements StatusPageService {
@Override
public ComponentStatus queryComponentStatus(long id) {
StatusPageComponent component = statusPageComponentDao.findById(id).orElseThrow(() -> new IllegalArgumentException("component not found"));
StatusPageComponent component = statusPageComponentDao.findById(id)
.orElseThrow(() -> new IllegalArgumentException("component not found"));
ComponentStatus componentStatus = new ComponentStatus();
componentStatus.setInfo(component);
List<StatusPageHistory> histories = new LinkedList<>();
// query today status
LocalDateTime nowTime = LocalDateTime.now();
LocalDateTime todayStartTime = nowTime.withHour(0).withMinute(0).withSecond(0).withNano(0);
ZoneOffset zoneOffset = ZoneId.systemDefault().getRules().getOffset(Instant.now());
long nowTimestamp = nowTime.toInstant(zoneOffset).toEpochMilli();
long todayStartTimestamp = todayStartTime.toInstant(zoneOffset).toEpochMilli();
List<StatusPageHistory> todayStatusPageHistoryList = statusPageHistoryDao
.findStatusPageHistoriesByComponentIdAndTimestampBetween(component.getId(), todayStartTimestamp, nowTimestamp);
StatusPageHistory todayStatus = combineOneDayStatusPageHistory(todayStatusPageHistoryList, component, nowTimestamp);
ZoneId zoneId = ZoneId.systemDefault();
Instant now = Instant.now();
long nowTimestamp = now.toEpochMilli();
long todayStartTimestamp = now
.atZone(zoneId)
.toLocalDate()
.atStartOfDay(zoneId)
.toInstant()
.toEpochMilli();
// Today
List<StatusPageHistory> todayStatusPageHistoryList =
statusPageHistoryDao.findStatusPageHistoriesByComponentIdAndTimestampBetween(
component.getId(), todayStartTimestamp, nowTimestamp);
StatusPageHistory todayStatus =
combineOneDayStatusPageHistory(todayStatusPageHistoryList, component, nowTimestamp);
histories.add(todayStatus);
// query 30d component status history
LocalDateTime preTime = todayStartTime.minusDays(HISTORY_SPAN_DAYS);
long preTimestamp = preTime.toInstant(zoneOffset).toEpochMilli();
List<StatusPageHistory> history = statusPageHistoryDao
.findStatusPageHistoriesByComponentIdAndTimestampBetween(component.getId(), preTimestamp, todayStartTimestamp);
// Previous HISTORY_SPAN_DAYS days (excluding today)
long preTimestamp = now
.atZone(zoneId)
.toLocalDate()
.minusDays(HISTORY_SPAN_DAYS)
.atStartOfDay(zoneId)
.toInstant()
.toEpochMilli();
List<StatusPageHistory> history =
statusPageHistoryDao.findStatusPageHistoriesByComponentIdAndTimestampBetween(
component.getId(), preTimestamp, todayStartTimestamp);
LinkedList<StatusPageHistory> historyList = new LinkedList<>(history);
historyList.sort((o1, o2) -> (int) (o1.getTimestamp() - o2.getTimestamp()));
LocalDateTime endTime = todayStartTime.minusSeconds(1);
LocalDateTime startTime = endTime.withHour(0).withMinute(0).withSecond(0).withNano(0);
for (int index = 0; index < HISTORY_SPAN_DAYS; index++) {
long startTimestamp = startTime.toInstant(zoneOffset).toEpochMilli();
long endTimestamp = endTime.toInstant(zoneOffset).toEpochMilli();
List<StatusPageHistory> thisDayHistory = historyList.stream().filter(item ->
item.getTimestamp() >= startTimestamp && item.getTimestamp() <= endTimestamp)
.collect(Collectors.toList());
historyList.sort((o1, o2) -> Long.compare(o1.getTimestamp(), o2.getTimestamp()));
ZonedDateTime end = Instant.ofEpochMilli(todayStartTimestamp)
.atZone(zoneId)
.minusSeconds(1); // yesterday 23:59:59 local time
for (int i = 0; i < HISTORY_SPAN_DAYS; i++) {
long endTimestamp = end.toInstant().toEpochMilli();
long startTimestamp = end.toLocalDate()
.atStartOfDay(zoneId)
.toInstant()
.toEpochMilli();
List<StatusPageHistory> thisDayHistory = historyList.stream()
.filter(h -> h.getTimestamp() >= startTimestamp && h.getTimestamp() <= endTimestamp)
.collect(Collectors.toList());
if (thisDayHistory.isEmpty()) {
StatusPageHistory statusPageHistory = StatusPageHistory.builder().timestamp(endTimestamp)
.componentId(component.getId()).state(CommonConstants.STATUS_PAGE_COMPONENT_STATE_UNKNOWN).build();
histories.add(statusPageHistory);
histories.add(StatusPageHistory.builder()
.timestamp(endTimestamp)
.componentId(component.getId())
.state(CommonConstants.STATUS_PAGE_COMPONENT_STATE_UNKNOWN)
.build());
} else if (thisDayHistory.size() == 1) {
histories.add(thisDayHistory.get(0));
} else {
StatusPageHistory statusPageHistory = combineOneDayStatusPageHistory(thisDayHistory, component, endTimestamp);
histories.add(statusPageHistory);
StatusPageHistory merged =
combineOneDayStatusPageHistory(thisDayHistory, component, endTimestamp);
histories.add(merged);
statusPageHistoryDao.deleteAll(thisDayHistory);
statusPageHistoryDao.save(statusPageHistory);
statusPageHistoryDao.save(merged);
}
startTime = startTime.minusDays(1);
endTime = endTime.minusDays(1);
end = end.minusDays(1);
}
componentStatus.setHistory(histories);
return componentStatus;
}
@@ -25,7 +25,7 @@ name:
# The description and help of this monitoring type
help:
zh-CN: HertzBeat 使用 <a class='help_module_content' href='https://hertzbeat.apache.org/docs/advanced/extend-jdbc'> JDBC 协议</a> 通过配置 SQL 对 Mysql 数据库的通用性能指标(系统信息、性能状态、Innodb、缓存、事物、用户线程、慢SQL等)进行采集监控,支持版本为 Mysql5+。<br>您可以点击“<i>新建 Mysql 数据库</i>”并进行配置,或者选择“<i>更多操作</i>”,导入已有配置。
en-US: HertzBeat uses <a class='help_module_content' href='https://hertzbeat.apache.org/docs/advanced/extend-jdbc'> JDBC Protocol</a> to configure SQL for collecting general metrics of Mysql database (system information, performance status, Innodb, cache, things, user threads, slow SQL, etc.). Supported version is Mysql5+. <br>You can click "<i>New Mysql Database</i>" and configure it, or select "<i>More Operations</i>" to import the existing configuration.
en-US: HertzBeat uses <a class='help_module_content' href='https://hertzbeat.apache.org/docs/advanced/extend-jdbc'>JDBC Protocol</a> to collect general performance metrics of MySQL databases (system information, performance status, InnoDB, cache, transactions, user threads, slow SQL, etc.) by configuring SQL. Supported versions include MySQL 5.0 and above. <br>You can click "<i>New MySQL Database</i>" to configure it, or select "<i>More Operations</i>" to import an existing configuration.
zh-TW: HertzBeat 使用 <a class='help_module_content' href='https://hertzbeat.apache.org/docs/advanced/extend-jdbc'> JDBC 協議</a> 通過配置 SQL 對 Mysql 數據庫的通用性能指標(系統信息、性能狀態、Innodb、緩存、事物、用戶線程、慢SQL等)進行采集監控,支持版本爲 Mysql5+。<br>您可以點擊“<i>新建 Mysql 數據庫</i>”並進行配置,或者選擇“<i>更多操作</i>”,導入已有配置。
ja-JP: HertzBeat は <a class='help_module_content' href='https://hertzbeat.apache.org/docs/advanced/extend-jdbc'> JDBC プロトコル</a> を介して Mysql データベースの一般的なパフォーマンスのメトリクス(システム情報、パフォーマンスステータス、Innodb、キャッシュ、オブジェクト、ユーザースレッド、遅いSQLなど)収集監視を行う,サポートバージョンは Mysql5+。<br>クリックすることができます“<i>新規 Mysql データベース</i>”および構成,または“<i>その他の操作</i>”,既存の構成のインポートすることができます。
helpLink:
@@ -1178,3 +1178,69 @@ metrics:
privateKey: ^_^sshPrivateKey^_^
privateKeyPassphrase: ^_^sshPrivateKeyPassphrase^_^
shareConnection: ^_^sshShareConnection^_^
- name: account_expiry
priority: 100
i18n:
zh-CN: 数据库账号有效期
en-US: Account Expiry
ja-JP: アカウント有効期限
fields:
- field: user
type: 1
label: true
i18n:
zh-CN: 用户
en-US: User
ja-JP: ユーザー
- field: host
type: 1
label: true
i18n:
zh-CN: 主机
en-US: Host
ja-JP: ホスト
- field: days_left
type: 0
i18n:
zh-CN: 剩余天数
en-US: Days Left
ja-JP: 残り日数
- field: password_expired
type: 1
i18n:
zh-CN: 是否已过期
en-US: Password Expired
ja-JP: 期限切れ
protocol: jdbc
jdbc:
host: ^_^host^_^
port: ^_^port^_^
platform: mysql
username: ^_^username^_^
password: ^_^password^_^
database: mysql
timeout: ^_^timeout^_^
queryType: multiRow
sql: |
SELECT
user,
host,
IF(password_lifetime IS NULL,
NULL,
password_lifetime - DATEDIFF(NOW(), password_last_changed)
) AS days_left,
password_expired
FROM mysql.user;
url: ^_^url^_^
sshTunnel:
enable: ^_^enableSshTunnel^_^
host: ^_^sshHost^_^
port: ^_^sshPort^_^
timeout: ^_^sshTimeout^_^
username: ^_^sshUsername^_^
password: ^_^sshPassword^_^
privateKey: ^_^sshPrivateKey^_^
privateKeyPassphrase: ^_^sshPrivateKeyPassphrase^_^
shareConnection: ^_^sshShareConnection^_^
@@ -16,7 +16,7 @@
# The monitoring type categoryservice-application service monitoring db-database monitoring mid-middleware custom-custom monitoring os-operating system monitoring
category: server
# The monitoring type eg: linux windows tomcat mysql aws...
app: synology
app: synology_nas
# The monitoring i18n name
name:
zh-CN: Synology NAS
@@ -17,6 +17,7 @@
package org.apache.hertzbeat.manager.service;
import org.apache.hertzbeat.base.service.LabelService;
import org.junit.jupiter.api.Assertions;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertNotNull;
@@ -34,7 +35,7 @@ import java.util.Map;
import java.util.Optional;
import org.apache.hertzbeat.common.entity.manager.Label;
import org.apache.hertzbeat.manager.dao.LabelDao;
import org.apache.hertzbeat.base.dao.LabelDao;
import org.apache.hertzbeat.manager.service.impl.LabelServiceImpl;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
@@ -35,6 +35,7 @@ import jakarta.persistence.criteria.CriteriaBuilder;
import jakarta.persistence.criteria.CriteriaQuery;
import jakarta.persistence.criteria.Root;
import org.apache.hertzbeat.alert.dao.AlertDefineBindDao;
import org.apache.hertzbeat.base.service.LabelService;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.entity.job.Job;
import org.apache.hertzbeat.common.entity.job.Metrics;
@@ -0,0 +1,142 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.manager.service.impl;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.Mockito.when;
import java.lang.reflect.Field;
import java.time.Duration;
import java.time.Instant;
import java.time.LocalDate;
import java.time.ZoneId;
import java.time.ZonedDateTime;
import java.util.List;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.entity.manager.StatusPageComponent;
import org.apache.hertzbeat.common.entity.manager.StatusPageHistory;
import org.apache.hertzbeat.manager.component.status.CalculateStatus;
import org.apache.hertzbeat.manager.dao.StatusPageComponentDao;
import org.apache.hertzbeat.manager.dao.StatusPageHistoryDao;
import org.apache.hertzbeat.manager.dao.StatusPageIncidentComponentBindDao;
import org.apache.hertzbeat.manager.dao.StatusPageIncidentDao;
import org.apache.hertzbeat.manager.dao.StatusPageOrgDao;
import org.apache.hertzbeat.manager.pojo.dto.ComponentStatus;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;
class StatusPageServiceImplTimeTest {
@Mock private StatusPageHistoryDao historyDao;
@Mock private StatusPageComponentDao componentDao;
@Mock private StatusPageOrgDao orgDao;
@Mock private StatusPageIncidentDao incidentDao;
@Mock private StatusPageIncidentComponentBindDao bindDao;
@Mock private CalculateStatus calculateStatus;
private StatusPageServiceImpl service;
private StatusPageComponent component;
@BeforeEach
void setup() throws Exception {
MockitoAnnotations.openMocks(this);
// Create service using its real constructor
service = new StatusPageServiceImpl(bindDao);
// Inject all @Autowired fields manually
inject("statusPageHistoryDao", historyDao);
inject("statusPageComponentDao", componentDao);
inject("statusPageOrgDao", orgDao);
inject("statusPageIncidentDao", incidentDao);
inject("calculateStatus", calculateStatus);
component = new StatusPageComponent();
component.setId(1L);
component.setState((byte) CommonConstants.STATUS_PAGE_COMPONENT_STATE_NORMAL);
when(componentDao.findAll()).thenReturn(List.of(component));
when(calculateStatus.getCalculateStatusIntervals()).thenReturn(300);
when(bindDao.countByComponentId(anyLong())).thenReturn(0L);
}
private void inject(String field, Object value) throws Exception {
Field f = StatusPageServiceImpl.class.getDeclaredField(field);
f.setAccessible(true);
f.set(service, value);
}
@Test
void testMidnightBoundary() {
Instant midnight = LocalDate.of(2026, 1, 14)
.atStartOfDay(ZoneId.of("UTC"))
.toInstant();
StatusPageHistory before = history(midnight.minusSeconds(1),
CommonConstants.STATUS_PAGE_COMPONENT_STATE_ABNORMAL);
StatusPageHistory after = history(midnight.plusSeconds(1),
CommonConstants.STATUS_PAGE_COMPONENT_STATE_NORMAL);
when(historyDao.findStatusPageHistoriesByComponentIdAndTimestampBetween(anyLong(), anyLong(), anyLong()))
.thenReturn(List.of(before, after));
List<ComponentStatus> result = service.queryComponentsStatus();
assertEquals(30, result.get(0).getHistory().size());
}
@Test
void testDstDay() {
ZoneId zone = ZoneId.of("America/New_York");
Instant dstDay = ZonedDateTime.of(2026, 3, 8, 12, 0, 0, 0, zone).toInstant();
StatusPageHistory history = history(dstDay,
CommonConstants.STATUS_PAGE_COMPONENT_STATE_ABNORMAL);
when(historyDao.findStatusPageHistoriesByComponentIdAndTimestampBetween(anyLong(), anyLong(), anyLong()))
.thenReturn(List.of(history));
List<ComponentStatus> result = service.queryComponentsStatus();
assertEquals(30, result.get(0).getHistory().size());
}
@Test
void testHistoryWindowSize() {
Instant tenDaysAgo = Instant.now().minus(Duration.ofDays(10));
StatusPageHistory history = history(tenDaysAgo,
CommonConstants.STATUS_PAGE_COMPONENT_STATE_NORMAL);
when(historyDao.findStatusPageHistoriesByComponentIdAndTimestampBetween(anyLong(), anyLong(), anyLong()))
.thenReturn(List.of(history));
List<ComponentStatus> result = service.queryComponentsStatus();
assertEquals(30, result.get(0).getHistory().size());
}
private StatusPageHistory history(Instant time, int state) {
return StatusPageHistory.builder()
.timestamp(time.toEpochMilli())
.state((byte) state)
.componentId(1L)
.build();
}
}
@@ -72,7 +72,7 @@ void migrateHistoryTable(java.sql.Connection conn) throws java.sql.SQLException
try (java.sql.ResultSet rs = conn.getMetaData().getColumns(null, null, "HZB_HISTORY", "METRIC_LABELS")) {
if (rs.next()) metricLabelsExists = true;
}
if (monitorIdExists) {
try (java.sql.Statement stmt = conn.createStatement()) {
if (!metricLabelsExists) {
@@ -92,3 +92,518 @@ void migrateHistoryTable(java.sql.Connection conn) throws java.sql.SQLException
$$;
CALL MIGRATE_HISTORY_TABLE();
DROP ALIAS MIGRATE_HISTORY_TABLE;
-- ========================================
-- hzb_alert_define_monitor_bind table
-- ========================================
CREATE ALIAS UPDATE_ALERT_DEFINE_MONITOR_BIND_INDEXES AS $$
void updateAlertDefineMonitorBindIndexes(java.sql.Connection conn) throws java.sql.SQLException {
boolean tableExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getTables(null, null, "HZB_ALERT_DEFINE_MONITOR_BIND", null)) {
if (rs.next()) tableExists = true;
}
if (tableExists) {
try (java.sql.Statement stmt = conn.createStatement()) {
// Drop old index if exists
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_ALERT_DEFINE_MONITOR_BIND", false, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if ("INDEX_ALERT_DEFINE_MONITOR".equalsIgnoreCase(indexName)) {
stmt.execute("DROP INDEX INDEX_ALERT_DEFINE_MONITOR");
break;
}
}
} catch (Exception e) {
// Index may not exist, continue
}
// Create new indexes if not exist
boolean alertDefineIdIndexExists = false;
boolean monitorIdIndexExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_ALERT_DEFINE_MONITOR_BIND", false, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if ("IDX_ALERT_DEFINE_ID".equalsIgnoreCase(indexName)) {
alertDefineIdIndexExists = true;
}
if ("IDX_MONITOR_ID".equalsIgnoreCase(indexName)) {
monitorIdIndexExists = true;
}
}
}
if (!alertDefineIdIndexExists) {
stmt.execute("CREATE INDEX IDX_ALERT_DEFINE_ID ON HZB_ALERT_DEFINE_MONITOR_BIND(ALERT_DEFINE_ID)");
}
if (!monitorIdIndexExists) {
stmt.execute("CREATE INDEX IDX_MONITOR_ID ON HZB_ALERT_DEFINE_MONITOR_BIND(MONITOR_ID)");
}
}
}
}
$$;
CALL UPDATE_ALERT_DEFINE_MONITOR_BIND_INDEXES();
DROP ALIAS UPDATE_ALERT_DEFINE_MONITOR_BIND_INDEXES;
-- ========================================
-- hzb_collector_monitor_bind table
-- ========================================
CREATE ALIAS UPDATE_COLLECTOR_MONITOR_BIND_INDEXES AS $$
void updateCollectorMonitorBindIndexes(java.sql.Connection conn) throws java.sql.SQLException {
boolean tableExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getTables(null, null, "HZB_COLLECTOR_MONITOR_BIND", null)) {
if (rs.next()) tableExists = true;
}
if (tableExists) {
try (java.sql.Statement stmt = conn.createStatement()) {
// Drop old index if exists
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_COLLECTOR_MONITOR_BIND", false, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if ("INDEX_COLLECTOR_MONITOR".equalsIgnoreCase(indexName)) {
stmt.execute("DROP INDEX INDEX_COLLECTOR_MONITOR");
break;
}
}
} catch (Exception e) {
// Index may not exist, continue
}
// Create new indexes if not exist
boolean collectorIndexExists = false;
boolean monitorIdIndexExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_COLLECTOR_MONITOR_BIND", false, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if ("IDX_COLLECTOR_MONITOR_COLLECTOR".equalsIgnoreCase(indexName)) {
collectorIndexExists = true;
}
if ("IDX_COLLECTOR_MONITOR_MONITOR_ID".equalsIgnoreCase(indexName)) {
monitorIdIndexExists = true;
}
}
}
if (!collectorIndexExists) {
stmt.execute("CREATE INDEX IDX_COLLECTOR_MONITOR_COLLECTOR ON HZB_COLLECTOR_MONITOR_BIND(COLLECTOR)");
}
if (!monitorIdIndexExists) {
stmt.execute("CREATE INDEX IDX_COLLECTOR_MONITOR_MONITOR_ID ON HZB_COLLECTOR_MONITOR_BIND(MONITOR_ID)");
}
}
}
}
$$;
CALL UPDATE_COLLECTOR_MONITOR_BIND_INDEXES();
DROP ALIAS UPDATE_COLLECTOR_MONITOR_BIND_INDEXES;
-- ========================================
-- hzb_monitor table
-- ========================================
CREATE ALIAS UPDATE_MONITOR_INDEXES AS $$
void updateMonitorIndexes(java.sql.Connection conn) throws java.sql.SQLException {
boolean tableExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getTables(null, null, "HZB_MONITOR", null)) {
if (rs.next()) tableExists = true;
}
if (tableExists) {
try (java.sql.Statement stmt = conn.createStatement()) {
// Drop old index if exists
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_MONITOR", false, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if ("MONITOR_QUERY_INDEX".equalsIgnoreCase(indexName)) {
stmt.execute("DROP INDEX MONITOR_QUERY_INDEX");
break;
}
}
} catch (Exception e) {
// Index may not exist, continue
}
// Create new indexes if not exist
boolean appIndexExists = false;
boolean instanceIndexExists = false;
boolean nameIndexExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_MONITOR", false, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if ("IDX_HZB_MONITOR_APP".equalsIgnoreCase(indexName)) {
appIndexExists = true;
}
if ("IDX_HZB_MONITOR_INSTANCE".equalsIgnoreCase(indexName)) {
instanceIndexExists = true;
}
if ("IDX_HZB_MONITOR_NAME".equalsIgnoreCase(indexName)) {
nameIndexExists = true;
}
}
}
if (!appIndexExists) {
stmt.execute("CREATE INDEX IDX_HZB_MONITOR_APP ON HZB_MONITOR(APP)");
}
if (!instanceIndexExists) {
stmt.execute("CREATE INDEX IDX_HZB_MONITOR_INSTANCE ON HZB_MONITOR(INSTANCE)");
}
if (!nameIndexExists) {
stmt.execute("CREATE INDEX IDX_HZB_MONITOR_NAME ON HZB_MONITOR(NAME)");
}
}
}
}
$$;
CALL UPDATE_MONITOR_INDEXES();
DROP ALIAS UPDATE_MONITOR_INDEXES;
-- ========================================
-- hzb_monitor_bind table
-- ========================================
CREATE ALIAS UPDATE_MONITOR_BIND_INDEXES AS $$
void updateMonitorBindIndexes(java.sql.Connection conn) throws java.sql.SQLException {
boolean tableExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getTables(null, null, "HZB_MONITOR_BIND", null)) {
if (rs.next()) tableExists = true;
}
if (tableExists) {
try (java.sql.Statement stmt = conn.createStatement()) {
// Create new index if not exist
boolean bindIndexExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_MONITOR_BIND", false, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if ("INDEX_MONITOR_BIND".equalsIgnoreCase(indexName)) {
bindIndexExists = true;
}
}
}
if (!bindIndexExists) {
stmt.execute("CREATE INDEX INDEX_MONITOR_BIND ON HZB_MONITOR_BIND(BIZ_ID)");
}
}
}
}
$$;
CALL UPDATE_MONITOR_BIND_INDEXES();
DROP ALIAS UPDATE_MONITOR_BIND_INDEXES;
-- ========================================
-- hzb_status_page_incident_component_bind table
-- Special handling: component_id might have auto-created index from FK
-- ========================================
CREATE ALIAS UPDATE_STATUS_PAGE_INCIDENT_COMPONENT_BIND_INDEXES AS $$
void updateStatusPageIncidentComponentBindIndexes(java.sql.Connection conn) throws java.sql.SQLException {
boolean tableExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getTables(null, null, "HZB_STATUS_PAGE_INCIDENT_COMPONENT_BIND", null)) {
if (rs.next()) tableExists = true;
}
if (tableExists) {
try (java.sql.Statement stmt = conn.createStatement()) {
// Check if component_id column already has any index
boolean hasComponentIdIndex = false;
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_STATUS_PAGE_INCIDENT_COMPONENT_BIND", false, false)) {
while (rs.next()) {
String columnName = rs.getString("COLUMN_NAME");
if (columnName != null && "COMPONENT_ID".equalsIgnoreCase(columnName)) {
hasComponentIdIndex = true;
break;
}
}
}
// Create index on component_id only if no index exists on this column
if (!hasComponentIdIndex) {
stmt.execute("CREATE INDEX IDX_INCIDENT_COMPONENT_COMPONENT_ID ON HZB_STATUS_PAGE_INCIDENT_COMPONENT_BIND(COMPONENT_ID)");
}
}
}
}
$$;
CALL UPDATE_STATUS_PAGE_INCIDENT_COMPONENT_BIND_INDEXES();
DROP ALIAS UPDATE_STATUS_PAGE_INCIDENT_COMPONENT_BIND_INDEXES;
-- ========================================
-- hzb_push_metrics table
-- ========================================
CREATE ALIAS UPDATE_PUSH_METRICS_INDEXES AS $$
void updatePushMetricsIndexes(java.sql.Connection conn) throws java.sql.SQLException {
boolean tableExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getTables(null, null, "HZB_PUSH_METRICS", null)) {
if (rs.next()) tableExists = true;
}
if (tableExists) {
try (java.sql.Statement stmt = conn.createStatement()) {
// Drop old index if exists
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_PUSH_METRICS", false, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if ("PUSH_QUERY_INDEX".equalsIgnoreCase(indexName)) {
stmt.execute("DROP INDEX PUSH_QUERY_INDEX");
break;
}
}
} catch (Exception e) {
// Index may not exist, continue
}
// Create new indexes if not exist
boolean monitorIdIndexExists = false;
boolean timeIndexExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_PUSH_METRICS", false, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if ("IDX_PUSH_METRICS_MONITOR_ID".equalsIgnoreCase(indexName)) {
monitorIdIndexExists = true;
}
if ("IDX_PUSH_METRICS_TIME".equalsIgnoreCase(indexName)) {
timeIndexExists = true;
}
}
}
if (!monitorIdIndexExists) {
stmt.execute("CREATE INDEX IDX_PUSH_METRICS_MONITOR_ID ON HZB_PUSH_METRICS(MONITOR_ID)");
}
if (!timeIndexExists) {
stmt.execute("CREATE INDEX IDX_PUSH_METRICS_TIME ON HZB_PUSH_METRICS(TIME)");
}
}
}
}
$$;
CALL UPDATE_PUSH_METRICS_INDEXES();
DROP ALIAS UPDATE_PUSH_METRICS_INDEXES;
-- ========================================
-- hzb_history table
-- ========================================
CREATE ALIAS UPDATE_HISTORY_INDEXES AS $$
void updateHistoryIndexes(java.sql.Connection conn) throws java.sql.SQLException {
boolean tableExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getTables(null, null, "HZB_HISTORY", null)) {
if (rs.next()) tableExists = true;
}
if (tableExists) {
try (java.sql.Statement stmt = conn.createStatement()) {
// Drop old index if exists
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_HISTORY", false, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if ("HISTORY_QUERY_INDEX".equalsIgnoreCase(indexName)) {
stmt.execute("DROP INDEX HISTORY_QUERY_INDEX");
break;
}
}
} catch (Exception e) {
// Index may not exist, continue
}
// Create new indexes if not exist
boolean instanceIndexExists = false;
boolean appIndexExists = false;
boolean metricsIndexExists = false;
boolean metricIndexExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_HISTORY", false, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if ("IDX_HZB_HISTORY_INSTANCE".equalsIgnoreCase(indexName)) {
instanceIndexExists = true;
}
if ("IDX_HZB_HISTORY_APP".equalsIgnoreCase(indexName)) {
appIndexExists = true;
}
if ("IDX_HZB_HISTORY_METRICS".equalsIgnoreCase(indexName)) {
metricsIndexExists = true;
}
if ("IDX_HZB_HISTORY_METRIC".equalsIgnoreCase(indexName)) {
metricIndexExists = true;
}
}
}
if (!instanceIndexExists) {
stmt.execute("CREATE INDEX IDX_HZB_HISTORY_INSTANCE ON HZB_HISTORY(INSTANCE)");
}
if (!appIndexExists) {
stmt.execute("CREATE INDEX IDX_HZB_HISTORY_APP ON HZB_HISTORY(APP)");
}
if (!metricsIndexExists) {
stmt.execute("CREATE INDEX IDX_HZB_HISTORY_METRICS ON HZB_HISTORY(METRICS)");
}
if (!metricIndexExists) {
stmt.execute("CREATE INDEX IDX_HZB_HISTORY_METRIC ON HZB_HISTORY(METRIC)");
}
}
}
}
$$;
CALL UPDATE_HISTORY_INDEXES();
DROP ALIAS UPDATE_HISTORY_INDEXES;
-- ========================================
-- hzb_param table
-- ========================================
CREATE ALIAS UPDATE_PARAM_TABLE_INDEXES AS $$
void updateParamTableIndexes(java.sql.Connection conn) throws java.sql.SQLException {
boolean tableExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getTables(null, null, "HZB_PARAM", null)) {
if (rs.next()) tableExists = true;
}
if (tableExists) {
try (java.sql.Statement stmt = conn.createStatement()) {
// Drop old index if exists
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_PARAM", false, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if ("IDX_HZB_PARAM_MONITOR_ID".equalsIgnoreCase(indexName)) {
stmt.execute("DROP INDEX IDX_HZB_PARAM_MONITOR_ID");
break;
}
}
} catch (Exception e) {
// Index may not exist, continue
}
// Create new index if not exist
boolean indexExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_PARAM", false, false)) {
while (rs.next()) {
if ("IDX_HZB_PARAM_MONITOR_ID".equalsIgnoreCase(rs.getString("INDEX_NAME"))) {
indexExists = true;
break;
}
}
}
if (!indexExists) {
stmt.execute("CREATE INDEX IDX_HZB_PARAM_MONITOR_ID ON HZB_PARAM(MONITOR_ID)");
}
// Drop old unique constraint if exists
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_PARAM", true, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if (indexName != null && indexName.equalsIgnoreCase("UK_HZB_PARAM_MONITOR_FIELD")) {
stmt.execute("DROP INDEX UK_HZB_PARAM_MONITOR_FIELD");
break;
}
}
} catch (Exception e) {
// Constraint may not exist, continue
}
// Create new unique constraint if not exist
boolean constraintExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_PARAM", true, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if (indexName != null && indexName.equalsIgnoreCase("UK_HZB_PARAM_MONITOR_FIELD")) {
constraintExists = true;
break;
}
}
}
if (!constraintExists) {
stmt.execute("CREATE UNIQUE INDEX UK_HZB_PARAM_MONITOR_FIELD ON HZB_PARAM(MONITOR_ID, FIELD)");
}
}
}
}
$$;
CALL UPDATE_PARAM_TABLE_INDEXES();
DROP ALIAS UPDATE_PARAM_TABLE_INDEXES;
-- ========================================
-- hzb_plugin_param table
-- ========================================
CREATE ALIAS UPDATE_PLUGIN_PARAM_TABLE_INDEXES AS $$
void updatePluginParamTableIndexes(java.sql.Connection conn) throws java.sql.SQLException {
boolean tableExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getTables(null, null, "HZB_PLUGIN_PARAM", null)) {
if (rs.next()) tableExists = true;
}
if (tableExists) {
try (java.sql.Statement stmt = conn.createStatement()) {
// Drop old index if exists
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_PLUGIN_PARAM", false, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if ("IDX_HZB_PLUGIN_PARAM_PLUGIN_METADATA_ID".equalsIgnoreCase(indexName)) {
stmt.execute("DROP INDEX IDX_HZB_PLUGIN_PARAM_PLUGIN_METADATA_ID");
break;
}
}
} catch (Exception e) {
// Index may not exist, continue
}
// Create new index if not exist
boolean indexExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_PLUGIN_PARAM", false, false)) {
while (rs.next()) {
if ("IDX_HZB_PLUGIN_PARAM_PLUGIN_METADATA_ID".equalsIgnoreCase(rs.getString("INDEX_NAME"))) {
indexExists = true;
break;
}
}
}
if (!indexExists) {
stmt.execute("CREATE INDEX IDX_HZB_PLUGIN_PARAM_PLUGIN_METADATA_ID ON HZB_PLUGIN_PARAM(PLUGIN_METADATA_ID)");
}
// Drop old unique constraint if exists
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_PLUGIN_PARAM", true, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if (indexName != null && indexName.equalsIgnoreCase("UK_HZB_PLUGIN_PARAM_METADATA_FIELD")) {
stmt.execute("DROP INDEX UK_HZB_PLUGIN_PARAM_METADATA_FIELD");
break;
}
}
} catch (Exception e) {
// Constraint may not exist, continue
}
// Create new unique constraint if not exist
boolean constraintExists = false;
try (java.sql.ResultSet rs = conn.getMetaData().getIndexInfo(null, null, "HZB_PLUGIN_PARAM", true, false)) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
if (indexName != null && indexName.equalsIgnoreCase("UK_HZB_PLUGIN_PARAM_METADATA_FIELD")) {
constraintExists = true;
break;
}
}
}
if (!constraintExists) {
stmt.execute("CREATE UNIQUE INDEX UK_HZB_PLUGIN_PARAM_METADATA_FIELD ON HZB_PLUGIN_PARAM(PLUGIN_METADATA_ID, FIELD)");
}
}
}
}
$$;
CALL UPDATE_PLUGIN_PARAM_TABLE_INDEXES();
DROP ALIAS UPDATE_PLUGIN_PARAM_TABLE_INDEXES;
@@ -91,7 +91,7 @@ BEGIN
SELECT COUNT(*) INTO monitor_id_exists FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'hzb_history' AND COLUMN_NAME = 'monitor_id';
SELECT COUNT(*) INTO metric_labels_exists FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'hzb_history' AND COLUMN_NAME = 'metric_labels';
IF monitor_id_exists > 0 THEN
IF metric_labels_exists = 0 THEN
SET @sql_rename = 'ALTER TABLE hzb_history CHANGE instance metric_labels VARCHAR(5000)';
@@ -111,9 +111,9 @@ BEGIN
EXECUTE stmt_resize;
DEALLOCATE PREPARE stmt_resize;
END IF;
UPDATE hzb_history h JOIN hzb_monitor m ON h.monitor_id = m.id SET h.instance = m.instance;
SET @sql_drop = 'ALTER TABLE hzb_history DROP COLUMN monitor_id';
PREPARE stmt_drop FROM @sql_drop;
EXECUTE stmt_drop;
@@ -124,4 +124,433 @@ DELIMITER ;
CALL MigrateHistoryTable();
DROP PROCEDURE IF EXISTS MigrateHistoryTable;
-- ========================================
-- hzb_alert_define_monitor_bind table
-- ========================================
DELIMITER //
CREATE PROCEDURE UpdateAlertDefineMonitorBindIndexes()
BEGIN
DECLARE table_exists INT;
SELECT COUNT(*) INTO table_exists
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'hzb_alert_define_monitor_bind';
IF table_exists = 1 THEN
-- Drop old index if exists
IF EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_alert_define_monitor_bind'
AND INDEX_NAME = 'index_alert_define_monitor') THEN
SET @drop_sql = 'DROP INDEX index_alert_define_monitor ON hzb_alert_define_monitor_bind';
PREPARE stmt FROM @drop_sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END IF;
-- Create new indexes if not exist
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_alert_define_monitor_bind'
AND INDEX_NAME = 'idx_alert_define_id') THEN
CREATE INDEX idx_alert_define_id ON hzb_alert_define_monitor_bind(alert_define_id);
END IF;
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_alert_define_monitor_bind'
AND INDEX_NAME = 'idx_monitor_id') THEN
CREATE INDEX idx_monitor_id ON hzb_alert_define_monitor_bind(monitor_id);
END IF;
END IF;
END //
DELIMITER ;
CALL UpdateAlertDefineMonitorBindIndexes();
DROP PROCEDURE IF EXISTS UpdateAlertDefineMonitorBindIndexes;
-- ========================================
-- hzb_collector_monitor_bind table
-- ========================================
DELIMITER //
CREATE PROCEDURE UpdateCollectorMonitorBindIndexes()
BEGIN
DECLARE table_exists INT;
SELECT COUNT(*) INTO table_exists
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'hzb_collector_monitor_bind';
IF table_exists = 1 THEN
-- Drop old index if exists
IF EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_collector_monitor_bind'
AND INDEX_NAME = 'index_collector_monitor') THEN
SET @drop_sql = 'DROP INDEX index_collector_monitor ON hzb_collector_monitor_bind';
PREPARE stmt FROM @drop_sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END IF;
-- Create new indexes if not exist
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_collector_monitor_bind'
AND INDEX_NAME = 'idx_collector_monitor_collector') THEN
CREATE INDEX idx_collector_monitor_collector ON hzb_collector_monitor_bind(collector);
END IF;
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_collector_monitor_bind'
AND INDEX_NAME = 'idx_collector_monitor_monitor_id') THEN
CREATE INDEX idx_collector_monitor_monitor_id ON hzb_collector_monitor_bind(monitor_id);
END IF;
END IF;
END //
DELIMITER ;
CALL UpdateCollectorMonitorBindIndexes();
DROP PROCEDURE IF EXISTS UpdateCollectorMonitorBindIndexes;
-- ========================================
-- hzb_monitor table
-- ========================================
DELIMITER //
CREATE PROCEDURE UpdateMonitorIndexes()
BEGIN
DECLARE table_exists INT;
SELECT COUNT(*) INTO table_exists
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'hzb_monitor';
IF table_exists = 1 THEN
-- Drop old index if exists
IF EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_monitor'
AND INDEX_NAME = 'monitor_query_index') THEN
SET @drop_sql = 'DROP INDEX monitor_query_index ON hzb_monitor';
PREPARE stmt FROM @drop_sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END IF;
-- Create new indexes if not exist
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_monitor'
AND INDEX_NAME = 'idx_hzb_monitor_app') THEN
CREATE INDEX idx_hzb_monitor_app ON hzb_monitor(app);
END IF;
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_monitor'
AND INDEX_NAME = 'idx_hzb_monitor_instance') THEN
CREATE INDEX idx_hzb_monitor_instance ON hzb_monitor(instance);
END IF;
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_monitor'
AND INDEX_NAME = 'idx_hzb_monitor_name') THEN
CREATE INDEX idx_hzb_monitor_name ON hzb_monitor(name);
END IF;
END IF;
END //
DELIMITER ;
CALL UpdateMonitorIndexes();
DROP PROCEDURE IF EXISTS UpdateMonitorIndexes;
-- ========================================
-- hzb_monitor_bind table
-- ========================================
DELIMITER //
CREATE PROCEDURE UpdateMonitorBindIndexes()
BEGIN
DECLARE table_exists INT;
SELECT COUNT(*) INTO table_exists
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'hzb_monitor_bind';
IF table_exists = 1 THEN
-- Create new index if not exist
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_monitor_bind'
AND INDEX_NAME = 'index_monitor_bind') THEN
CREATE INDEX index_monitor_bind ON hzb_monitor_bind(biz_id);
END IF;
END IF;
END //
DELIMITER ;
CALL UpdateMonitorBindIndexes();
DROP PROCEDURE IF EXISTS UpdateMonitorBindIndexes;
-- ========================================
-- hzb_status_page_incident_component_bind table
-- Special handling: component_id might have auto-created index from FK
-- ========================================
DELIMITER //
CREATE PROCEDURE UpdateStatusPageIncidentComponentBindIndexes()
BEGIN
DECLARE table_exists INT;
DECLARE component_id_has_index INT;
SELECT COUNT(*) INTO table_exists
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'hzb_status_page_incident_component_bind';
IF table_exists = 1 THEN
-- Check if component_id column already has any index (including auto-created by FK)
SELECT COUNT(*) INTO component_id_has_index
FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_status_page_incident_component_bind'
AND COLUMN_NAME = 'component_id'
AND INDEX_NAME != 'PRIMARY';
-- Create index on component_id only if no index exists on this column
IF component_id_has_index = 0 THEN
CREATE INDEX idx_incident_component_component_id ON hzb_status_page_incident_component_bind(component_id);
END IF;
END IF;
END //
DELIMITER ;
CALL UpdateStatusPageIncidentComponentBindIndexes();
DROP PROCEDURE IF EXISTS UpdateStatusPageIncidentComponentBindIndexes;
-- ========================================
-- hzb_push_metrics table
-- ========================================
DELIMITER //
CREATE PROCEDURE UpdatePushMetricsIndexes()
BEGIN
DECLARE table_exists INT;
SELECT COUNT(*) INTO table_exists
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'hzb_push_metrics';
IF table_exists = 1 THEN
-- Drop old index if exists
IF EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_push_metrics'
AND INDEX_NAME = 'push_query_index') THEN
SET @drop_sql = 'DROP INDEX push_query_index ON hzb_push_metrics';
PREPARE stmt FROM @drop_sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END IF;
-- Create new indexes if not exist
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_push_metrics'
AND INDEX_NAME = 'idx_push_metrics_monitor_id') THEN
CREATE INDEX idx_push_metrics_monitor_id ON hzb_push_metrics(monitor_id);
END IF;
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_push_metrics'
AND INDEX_NAME = 'idx_push_metrics_time') THEN
CREATE INDEX idx_push_metrics_time ON hzb_push_metrics(time);
END IF;
END IF;
END //
DELIMITER ;
CALL UpdatePushMetricsIndexes();
DROP PROCEDURE IF EXISTS UpdatePushMetricsIndexes;
-- ========================================
-- hzb_history table
-- ========================================
DELIMITER //
CREATE PROCEDURE UpdateHistoryIndexes()
BEGIN
DECLARE table_exists INT;
SELECT COUNT(*) INTO table_exists
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'hzb_history';
IF table_exists = 1 THEN
-- Drop old index if exists
IF EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_history'
AND INDEX_NAME = 'history_query_index') THEN
SET @drop_sql = 'DROP INDEX history_query_index ON hzb_history';
PREPARE stmt FROM @drop_sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END IF;
-- Create new indexes if not exist
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_history'
AND INDEX_NAME = 'idx_hzb_history_instance') THEN
CREATE INDEX idx_hzb_history_instance ON hzb_history(instance);
END IF;
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_history'
AND INDEX_NAME = 'idx_hzb_history_app') THEN
CREATE INDEX idx_hzb_history_app ON hzb_history(app);
END IF;
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_history'
AND INDEX_NAME = 'idx_hzb_history_metrics') THEN
CREATE INDEX idx_hzb_history_metrics ON hzb_history(metrics);
END IF;
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_history'
AND INDEX_NAME = 'idx_hzb_history_metric') THEN
CREATE INDEX idx_hzb_history_metric ON hzb_history(metric);
END IF;
END IF;
END //
DELIMITER ;
CALL UpdateHistoryIndexes();
DROP PROCEDURE IF EXISTS UpdateHistoryIndexes;
-- ========================================
-- hzb_param table
-- ========================================
DELIMITER //
CREATE PROCEDURE UpdateParamTableIndexes()
BEGIN
DECLARE table_exists INT;
SELECT COUNT(*) INTO table_exists
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'hzb_param';
IF table_exists = 1 THEN
-- Drop old index if exists
IF EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_param'
AND INDEX_NAME = 'idx_hzb_param_monitor_id') THEN
SET @drop_sql = 'DROP INDEX idx_hzb_param_monitor_id ON hzb_param';
PREPARE stmt FROM @drop_sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END IF;
-- Create new index if not exist
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_param'
AND INDEX_NAME = 'idx_hzb_param_monitor_id') THEN
CREATE INDEX idx_hzb_param_monitor_id ON hzb_param(monitor_id);
END IF;
-- Drop old unique constraint if exists
IF EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_param'
AND CONSTRAINT_TYPE = 'UNIQUE'
AND CONSTRAINT_NAME = 'uk_hzb_param_monitor_field') THEN
SET @drop_sql = 'ALTER TABLE hzb_param DROP INDEX uk_hzb_param_monitor_field';
PREPARE stmt FROM @drop_sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END IF;
-- Create new unique constraint if not exist
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_param'
AND CONSTRAINT_TYPE = 'UNIQUE'
AND CONSTRAINT_NAME = 'uk_hzb_param_monitor_field') THEN
ALTER TABLE hzb_param
ADD CONSTRAINT uk_hzb_param_monitor_field
UNIQUE (monitor_id, field);
END IF;
END IF;
END //
DELIMITER ;
CALL UpdateParamTableIndexes();
DROP PROCEDURE IF EXISTS UpdateParamTableIndexes;
-- ========================================
-- hzb_plugin_param table
-- ========================================
DELIMITER //
CREATE PROCEDURE UpdatePluginParamTableIndexes()
BEGIN
DECLARE table_exists INT;
SELECT COUNT(*) INTO table_exists
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'hzb_plugin_param';
IF table_exists = 1 THEN
-- Drop old index if exists
IF EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_plugin_param'
AND INDEX_NAME = 'idx_hzb_plugin_param_plugin_metadata_id') THEN
SET @drop_sql = 'DROP INDEX idx_hzb_plugin_param_plugin_metadata_id ON hzb_plugin_param';
PREPARE stmt FROM @drop_sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END IF;
-- Create new index if not exist
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.STATISTICS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_plugin_param'
AND INDEX_NAME = 'idx_hzb_plugin_param_plugin_metadata_id') THEN
CREATE INDEX idx_hzb_plugin_param_plugin_metadata_id ON hzb_plugin_param(plugin_metadata_id);
END IF;
-- Drop old unique constraint if exists
IF EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_plugin_param'
AND CONSTRAINT_TYPE = 'UNIQUE'
AND CONSTRAINT_NAME = 'uk_hzb_plugin_param_metadata_field') THEN
SET @drop_sql = 'ALTER TABLE hzb_plugin_param DROP INDEX uk_hzb_plugin_param_metadata_field';
PREPARE stmt FROM @drop_sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END IF;
-- Create new unique constraint if not exist
IF NOT EXISTS (SELECT 1 FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'hzb_plugin_param'
AND CONSTRAINT_TYPE = 'UNIQUE'
AND CONSTRAINT_NAME = 'uk_hzb_plugin_param_metadata_field') THEN
ALTER TABLE hzb_plugin_param
ADD CONSTRAINT uk_hzb_plugin_param_metadata_field
UNIQUE (plugin_metadata_id, field);
END IF;
END IF;
END //
DELIMITER ;
CALL UpdatePluginParamTableIndexes();
DROP PROCEDURE IF EXISTS UpdatePluginParamTableIndexes;
COMMIT;
@@ -62,14 +62,240 @@ BEGIN
UPDATE hzb_history SET instance = NULL;
ALTER TABLE hzb_history ALTER COLUMN instance TYPE VARCHAR(255);
END IF;
UPDATE hzb_history h
SET instance = m.instance
FROM hzb_monitor m
WHERE h.monitor_id = m.id;
ALTER TABLE hzb_history DROP COLUMN monitor_id;
END IF;
END $$;
commit;
-- ========================================
-- hzb_alert_define_monitor_bind table
-- ========================================
DO $$
BEGIN
IF EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'hzb_alert_define_monitor_bind') THEN
-- Drop old index if exists
IF EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_alert_define_monitor_bind' AND indexname = 'index_alert_define_monitor') THEN
DROP INDEX IF EXISTS index_alert_define_monitor;
END IF;
-- Create new indexes if not exist
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_alert_define_monitor_bind' AND indexname = 'idx_alert_define_id') THEN
CREATE INDEX idx_alert_define_id ON hzb_alert_define_monitor_bind(alert_define_id);
END IF;
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_alert_define_monitor_bind' AND indexname = 'idx_monitor_id') THEN
CREATE INDEX idx_monitor_id ON hzb_alert_define_monitor_bind(monitor_id);
END IF;
END IF;
END $$;
-- ========================================
-- hzb_collector_monitor_bind table
-- ========================================
DO $$
BEGIN
IF EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'hzb_collector_monitor_bind') THEN
-- Drop old index if exists
IF EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_collector_monitor_bind' AND indexname = 'index_collector_monitor') THEN
DROP INDEX IF EXISTS index_collector_monitor;
END IF;
-- Create new indexes if not exist
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_collector_monitor_bind' AND indexname = 'idx_collector_monitor_collector') THEN
CREATE INDEX idx_collector_monitor_collector ON hzb_collector_monitor_bind(collector);
END IF;
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_collector_monitor_bind' AND indexname = 'idx_collector_monitor_monitor_id') THEN
CREATE INDEX idx_collector_monitor_monitor_id ON hzb_collector_monitor_bind(monitor_id);
END IF;
END IF;
END $$;
-- ========================================
-- hzb_monitor table
-- ========================================
DO $$
BEGIN
IF EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'hzb_monitor') THEN
-- Drop old index if exists
IF EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_monitor' AND indexname = 'monitor_query_index') THEN
DROP INDEX IF EXISTS monitor_query_index;
END IF;
-- Create new indexes if not exist
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_monitor' AND indexname = 'idx_hzb_monitor_app') THEN
CREATE INDEX idx_hzb_monitor_app ON hzb_monitor(app);
END IF;
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_monitor' AND indexname = 'idx_hzb_monitor_instance') THEN
CREATE INDEX idx_hzb_monitor_instance ON hzb_monitor(instance);
END IF;
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_monitor' AND indexname = 'idx_hzb_monitor_name') THEN
CREATE INDEX idx_hzb_monitor_name ON hzb_monitor(name);
END IF;
END IF;
END $$;
-- ========================================
-- hzb_monitor_bind table
-- ========================================
DO $$
BEGIN
IF EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'hzb_monitor_bind') THEN
-- Create new index if not exist
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_monitor_bind' AND indexname = 'index_monitor_bind') THEN
CREATE INDEX index_monitor_bind ON hzb_monitor_bind(biz_id);
END IF;
END IF;
END $$;
-- ========================================
-- hzb_status_page_incident_component_bind table
-- Special handling: component_id might have auto-created index from FK
-- ========================================
DO $$
BEGIN
IF EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'hzb_status_page_incident_component_bind') THEN
-- Check if component_id column already has any index, create if not
IF NOT EXISTS (
SELECT 1 FROM pg_indexes
WHERE tablename = 'hzb_status_page_incident_component_bind'
AND indexdef LIKE '%component_id%'
AND indexname NOT LIKE '%_pkey'
) THEN
CREATE INDEX idx_incident_component_component_id ON hzb_status_page_incident_component_bind(component_id);
END IF;
END IF;
END $$;
-- ========================================
-- hzb_push_metrics table
-- ========================================
DO $$
BEGIN
IF EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'hzb_push_metrics') THEN
-- Drop old index if exists
IF EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_push_metrics' AND indexname = 'push_query_index') THEN
DROP INDEX IF EXISTS push_query_index;
END IF;
-- Create new indexes if not exist
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_push_metrics' AND indexname = 'idx_push_metrics_monitor_id') THEN
CREATE INDEX idx_push_metrics_monitor_id ON hzb_push_metrics(monitor_id);
END IF;
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_push_metrics' AND indexname = 'idx_push_metrics_time') THEN
CREATE INDEX idx_push_metrics_time ON hzb_push_metrics(time);
END IF;
END IF;
END $$;
-- ========================================
-- hzb_history table
-- ========================================
DO $$
BEGIN
IF EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'hzb_history') THEN
-- Drop old index if exists
IF EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_history' AND indexname = 'history_query_index') THEN
DROP INDEX IF EXISTS history_query_index;
END IF;
-- Create new indexes if not exist
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_history' AND indexname = 'idx_hzb_history_instance') THEN
CREATE INDEX idx_hzb_history_instance ON hzb_history(instance);
END IF;
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_history' AND indexname = 'idx_hzb_history_app') THEN
CREATE INDEX idx_hzb_history_app ON hzb_history(app);
END IF;
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_history' AND indexname = 'idx_hzb_history_metrics') THEN
CREATE INDEX idx_hzb_history_metrics ON hzb_history(metrics);
END IF;
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_history' AND indexname = 'idx_hzb_history_metric') THEN
CREATE INDEX idx_hzb_history_metric ON hzb_history(metric);
END IF;
END IF;
END $$;
-- ========================================
-- hzb_param table
-- ========================================
DO $$
BEGIN
IF EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'hzb_param') THEN
-- Drop old index if exists
IF EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_param' AND indexname = 'idx_hzb_param_monitor_id') THEN
DROP INDEX IF EXISTS idx_hzb_param_monitor_id;
END IF;
-- Create new index if not exist
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_param' AND indexname = 'idx_hzb_param_monitor_id') THEN
CREATE INDEX idx_hzb_param_monitor_id ON hzb_param(monitor_id);
END IF;
-- Drop old unique constraint if exists
IF EXISTS (SELECT 1 FROM information_schema.table_constraints
WHERE table_name = 'hzb_param'
AND constraint_type = 'UNIQUE'
AND constraint_name = 'uk_hzb_param_monitor_field') THEN
ALTER TABLE hzb_param DROP CONSTRAINT IF EXISTS uk_hzb_param_monitor_field;
END IF;
-- Create new unique constraint if not exist
IF NOT EXISTS (SELECT 1 FROM information_schema.table_constraints
WHERE table_name = 'hzb_param'
AND constraint_type = 'UNIQUE'
AND constraint_name = 'uk_hzb_param_monitor_field') THEN
ALTER TABLE hzb_param
ADD CONSTRAINT uk_hzb_param_monitor_field
UNIQUE (monitor_id, field);
END IF;
END IF;
END $$;
-- ========================================
-- hzb_plugin_param table
-- ========================================
DO $$
BEGIN
IF EXISTS (SELECT 1 FROM information_schema.tables WHERE table_name = 'hzb_plugin_param') THEN
-- Drop old index if exists
IF EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_plugin_param' AND indexname = 'idx_hzb_plugin_param_plugin_metadata_id') THEN
DROP INDEX IF EXISTS idx_hzb_plugin_param_plugin_metadata_id;
END IF;
-- Create new index if not exist
IF NOT EXISTS (SELECT 1 FROM pg_indexes WHERE tablename = 'hzb_plugin_param' AND indexname = 'idx_hzb_plugin_param_plugin_metadata_id') THEN
CREATE INDEX idx_hzb_plugin_param_plugin_metadata_id ON hzb_plugin_param(plugin_metadata_id);
END IF;
-- Drop old unique constraint if exists
IF EXISTS (SELECT 1 FROM information_schema.table_constraints
WHERE table_name = 'hzb_plugin_param'
AND constraint_type = 'UNIQUE'
AND constraint_name = 'uk_hzb_plugin_param_metadata_field') THEN
ALTER TABLE hzb_plugin_param DROP CONSTRAINT IF EXISTS uk_hzb_plugin_param_metadata_field;
END IF;
-- Create new unique constraint if not exist
IF NOT EXISTS (SELECT 1 FROM information_schema.table_constraints
WHERE table_name = 'hzb_plugin_param'
AND constraint_type = 'UNIQUE'
AND constraint_name = 'uk_hzb_plugin_param_metadata_field') THEN
ALTER TABLE hzb_plugin_param
ADD CONSTRAINT uk_hzb_plugin_param_metadata_field
UNIQUE (plugin_metadata_id, field);
END IF;
END IF;
END $$;
COMMIT;
@@ -0,0 +1,185 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.hertzbeat.startup;
import jakarta.persistence.Index;
import jakarta.persistence.Table;
import jakarta.persistence.UniqueConstraint;
import lombok.extern.slf4j.Slf4j;
import org.apache.hertzbeat.common.entity.alerter.AlertDefineMonitorBind;
import org.apache.hertzbeat.common.entity.manager.CollectorMonitorBind;
import org.apache.hertzbeat.common.entity.manager.Monitor;
import org.apache.hertzbeat.common.entity.manager.MonitorBind;
import org.apache.hertzbeat.common.entity.manager.Param;
import org.apache.hertzbeat.common.entity.manager.StatusPageIncidentComponentBind;
import org.apache.hertzbeat.common.entity.push.PushMetrics;
import org.apache.hertzbeat.common.entity.warehouse.History;
import org.apache.hertzbeat.manager.pojo.dto.PluginParam;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.web.WebProperties;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.context.TestConfiguration;
import org.springframework.context.annotation.Bean;
import org.springframework.test.context.ActiveProfiles;
import org.springframework.test.context.TestPropertySource;
import javax.sql.DataSource;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import static org.junit.jupiter.api.Assertions.assertTrue;
/**
* Integration test for validating that entity indexes are actually created in the database.
* This test uses H2 in-memory database to verify DDL generation and index creation.
*/
@Slf4j
@SpringBootTest(classes = HertzBeatApplication.class, webEnvironment = SpringBootTest.WebEnvironment.NONE)
@ActiveProfiles("test")
@TestPropertySource(properties = {
"spring.jpa.hibernate.ddl-auto=create-drop",
"spring.datasource.url=jdbc:h2:mem:testdb;MODE=MySQL;DB_CLOSE_DELAY=-1",
"spring.datasource.driver-class-name=org.h2.Driver",
"spring.jpa.show-sql=false"
})
@DisplayName("Entity Index Creation Integration Test")
class EntityIndexIntegrationTest {
@Autowired
private DataSource dataSource;
public List<Class<?>> entities = List.of(
AlertDefineMonitorBind.class,
CollectorMonitorBind.class,
Monitor.class,
MonitorBind.class,
Param.class,
StatusPageIncidentComponentBind.class,
PushMetrics.class,
History.class,
PluginParam.class
);
@Test
@DisplayName("All entity indexes should be created in database")
public void testAllEntityIndexes() throws SQLException {
for (Class<?> entity : entities) {
List<ExpectedIndex> expectedIndexes = parseExpectedIndexes(entity);
if (expectedIndexes.isEmpty()) {
continue;
}
Map<String, IndexMeta> actual = getIndexes(expectedIndexes.get(0).tableName());
for (ExpectedIndex expected : expectedIndexes) {
assertIndexExists(expected, actual);
}
}
}
private List<ExpectedIndex> parseExpectedIndexes(Class<?> entityClass) {
Table table = entityClass.getAnnotation(Table.class);
if (table == null) {
return List.of();
}
String tableName = table.name().toUpperCase();
List<ExpectedIndex> result = new ArrayList<>();
for (Index idx : table.indexes()) {
Set<String> columns = Arrays.stream(idx.columnList().split(","))
.map(String::trim)
.map(String::toUpperCase)
.collect(Collectors.toSet());
result.add(new ExpectedIndex(tableName, columns, false));
}
for (UniqueConstraint uc : table.uniqueConstraints()) {
Set<String> columns = Arrays.stream(uc.columnNames())
.map(String::toUpperCase).collect(Collectors.toSet());
result.add(new ExpectedIndex(tableName, columns, true));
}
return result;
}
private void assertIndexExists(ExpectedIndex expected, Map<String, IndexMeta> actualIndexes) {
boolean found = actualIndexes.values().stream().filter(idx -> !expected.unique || idx.unique).anyMatch(idx -> idx.columns.containsAll(expected.columns));
assertTrue(found, "Expected index not found. Table=" + expected.tableName + ", columns=" + expected.columns + ", unique=" + expected.unique + ", actual=" + actualIndexes);
}
private Map<String, IndexMeta> getIndexes(String tableName) throws SQLException {
Map<String, IndexMeta> indexes = new HashMap<>();
String sql = """
SELECT
INDEX_NAME,
COLUMN_NAME,
IS_UNIQUE
FROM INFORMATION_SCHEMA.INDEX_COLUMNS
WHERE TABLE_NAME = ?
""";
try (Connection conn = dataSource.getConnection(); var ps = conn.prepareStatement(sql)) {
ps.setString(1, tableName.toUpperCase());
try (ResultSet rs = ps.executeQuery()) {
while (rs.next()) {
String indexName = rs.getString("INDEX_NAME");
String column = rs.getString("COLUMN_NAME");
boolean unique = rs.getBoolean("IS_UNIQUE");
if ("PRIMARY".equalsIgnoreCase(indexName)) {
continue;
}
indexes.computeIfAbsent(indexName, k -> new IndexMeta(unique)).columns.add(column.toUpperCase());
}
}
}
return indexes;
}
static class IndexMeta {
final boolean unique;
final Set<String> columns = new HashSet<>();
IndexMeta(boolean unique) {
this.unique = unique;
}
}
record ExpectedIndex(String tableName, Set<String> columns, boolean unique) {
}
@TestConfiguration
static class TestConfig {
@Bean
public WebProperties webProperties() {
return new WebProperties();
}
}
}
@@ -27,7 +27,7 @@ import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;
import org.apache.hertzbeat.common.entity.manager.Label;
import org.apache.hertzbeat.manager.dao.LabelDao;
import org.apache.hertzbeat.base.dao.LabelDao;
import org.apache.hertzbeat.startup.AbstractSpringIntegrationTest;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
@@ -65,45 +65,67 @@ public interface HistoryDataReader {
* @param spanId span ID filter
* @param severityNumber severity number filter
* @param severityText severity text filter
* @param searchContent search content in log body
* @return filtered log entries
*/
default List<LogEntry> queryLogsByMultipleConditions(Long startTime, Long endTime, String traceId,
String spanId, Integer severityNumber,
String severityText) {
String severityText, String searchContent) {
throw new UnsupportedOperationException("query logs by multiple conditions is not supported");
}
/**
* Query logs with multiple filter conditions and pagination
* Query logs with multiple filter conditions and pagination (Legacy)
*/
default List<LogEntry> queryLogsByMultipleConditionsWithPagination(Long startTime, Long endTime, String traceId,
String spanId, Integer severityNumber,
String severityText, Integer offset, Integer limit) {
return queryLogsByMultipleConditionsWithPagination(startTime, endTime, traceId, spanId, severityNumber, severityText, null, offset, limit);
}
/**
* Query logs with multiple filter conditions and pagination including search content
* @param startTime start time in milliseconds
* @param endTime end time in milliseconds
* @param traceId trace ID filter
* @param spanId span ID filter
* @param severityNumber severity number filter
* @param severityText severity text filter
* @param searchContent search content in log body
* @param offset pagination offset
* @param limit pagination limit
* @return filtered log entries with pagination
*/
default List<LogEntry> queryLogsByMultipleConditionsWithPagination(Long startTime, Long endTime, String traceId,
String spanId, Integer severityNumber,
String severityText, Integer offset, Integer limit) {
String severityText, String searchContent,
Integer offset, Integer limit) {
throw new UnsupportedOperationException("query logs by multiple conditions with pagination is not supported");
}
/**
* Count logs with multiple filter conditions
* Count logs with multiple filter conditions (Legacy)
*/
default long countLogsByMultipleConditions(Long startTime, Long endTime, String traceId,
String spanId, Integer severityNumber,
String severityText) {
return countLogsByMultipleConditions(startTime, endTime, traceId, spanId, severityNumber, severityText, null);
}
/**
* Count logs with multiple filter conditions including search content
* @param startTime start time in milliseconds
* @param endTime end time in milliseconds
* @param traceId trace ID filter
* @param spanId span ID filter
* @param severityNumber severity number filter
* @param severityText severity text filter
* @param searchContent search content in log body
* @return count of matching log entries
*/
default long countLogsByMultipleConditions(Long startTime, Long endTime, String traceId,
String spanId, Integer severityNumber,
String severityText) {
String severityText, String searchContent) {
throw new UnsupportedOperationException("count logs by multiple conditions is not supported");
}
}
}
@@ -49,6 +49,8 @@ import java.util.Map;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
/**
* data storage by duckdb
@@ -64,6 +66,9 @@ public class DuckdbDatabaseDataStorage extends AbstractHistoryDataStorage {
// Ideal number of data points for charting (avoids frontend lag)
private static final int TARGET_CHART_POINTS = 800;
// Regex to strictly match days format, e.g., "90d", "7D". Group 1 captures the digits.
private static final Pattern DAY_PATTERN = Pattern.compile("^(\\d+)[dD]$");
private final String expireTimeStr;
private final String dbPath;
private HikariDataSource dataSource;
@@ -136,11 +141,21 @@ public class DuckdbDatabaseDataStorage extends AbstractHistoryDataStorage {
log.info("[duckdb] start data cleaner and checkpoint...");
long expireTime;
try {
if (NumberUtils.isParsable(expireTimeStr)) {
expireTime = NumberUtils.toLong(expireTimeStr);
// Ensure no whitespace issues
String cleanExpireStr = expireTimeStr == null ? "" : expireTimeStr.trim();
Matcher dayMatcher = DAY_PATTERN.matcher(cleanExpireStr);
if (NumberUtils.isParsable(cleanExpireStr)) {
expireTime = NumberUtils.toLong(cleanExpireStr);
expireTime = (ZonedDateTime.now().toEpochSecond() - expireTime) * 1000L;
} else if (dayMatcher.matches()) {
// Strictly matched "90d" or "90D" format
long days = Long.parseLong(dayMatcher.group(1));
ZonedDateTime dateTime = ZonedDateTime.now().minus(Duration.ofDays(days));
expireTime = dateTime.toEpochSecond() * 1000L;
} else {
TemporalAmount temporalAmount = TimePeriodUtil.parseTokenTime(expireTimeStr);
// Fallback to existing utility for other units (h, m, s, etc.)
TemporalAmount temporalAmount = TimePeriodUtil.parseTokenTime(cleanExpireStr);
ZonedDateTime dateTime = ZonedDateTime.now().minus(temporalAmount);
expireTime = dateTime.toEpochSecond() * 1000L;
}
@@ -43,6 +43,7 @@ import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
@@ -143,7 +144,8 @@ public class GreptimeDbDataStorage extends AbstractHistoryDataStorage {
return;
}
String instance = metricsData.getInstance();
String tableName = getTableName(metricsData.getMetrics());
String app = metricsData.getApp();
String tableName = getTableName(app, metricsData.getMetrics());
TableSchema.Builder tableSchemaBuilder = TableSchema.newBuilder(tableName);
tableSchemaBuilder.addTag("instance", DataType.String)
@@ -220,8 +222,8 @@ public class GreptimeDbDataStorage extends AbstractHistoryDataStorage {
return getHistoryData(start, end, step, instance, app, metrics, metric);
}
private String getTableName(String metrics) {
return metrics;
private String getTableName(String app, String metrics) {
return app + "_" + metrics;
}
@Override
@@ -246,7 +248,7 @@ public class GreptimeDbDataStorage extends AbstractHistoryDataStorage {
long effectiveStart = values.get(0).getTime() / 1000;
long effectiveEnd = values.get(values.size() - 1).getTime() / 1000 + Duration.ofHours(4).getSeconds();
String name = getTableName(metrics);
String name = getTableName(app, metrics);
String timeSeriesSelector = name + "{" + LABEL_KEY_INSTANCE + "=\"" + instance + "\"";
if (!CommonConstants.PROMETHEUS.equals(app)) {
timeSeriesSelector = timeSeriesSelector + "," + LABEL_KEY_FIELD + "=\"" + metric + "\"";
@@ -330,7 +332,7 @@ public class GreptimeDbDataStorage extends AbstractHistoryDataStorage {
* @return history metric data
*/
private Map<String, List<Value>> getHistoryData(long start, long end, String step, String instance, String app, String metrics, String metric) {
String name = getTableName(metrics);
String name = getTableName(app, metrics);
String timeSeriesSelector = LABEL_KEY_NAME + "=\"" + name + "\""
+ "," + LABEL_KEY_INSTANCE + "=\"" + instance + "\"";
if (!CommonConstants.PROMETHEUS.equals(app)) {
@@ -541,10 +543,10 @@ public class GreptimeDbDataStorage extends AbstractHistoryDataStorage {
@Override
public List<LogEntry> queryLogsByMultipleConditions(Long startTime, Long endTime, String traceId,
String spanId, Integer severityNumber,
String severityText) {
String severityText, String searchContent) {
try {
StringBuilder sql = new StringBuilder("SELECT * FROM ").append(LOG_TABLE_NAME);
buildWhereConditions(sql, startTime, endTime, traceId, spanId, severityNumber, severityText);
buildWhereConditions(sql, startTime, endTime, traceId, spanId, severityNumber, severityText, searchContent);
sql.append(" ORDER BY time_unix_nano DESC");
List<Map<String, Object>> rows = greptimeSqlQueryExecutor.execute(sql.toString());
@@ -558,10 +560,11 @@ public class GreptimeDbDataStorage extends AbstractHistoryDataStorage {
@Override
public List<LogEntry> queryLogsByMultipleConditionsWithPagination(Long startTime, Long endTime, String traceId,
String spanId, Integer severityNumber,
String severityText, Integer offset, Integer limit) {
String severityText, String searchContent,
Integer offset, Integer limit) {
try {
StringBuilder sql = new StringBuilder("SELECT * FROM ").append(LOG_TABLE_NAME);
buildWhereConditions(sql, startTime, endTime, traceId, spanId, severityNumber, severityText);
buildWhereConditions(sql, startTime, endTime, traceId, spanId, severityNumber, severityText, searchContent);
sql.append(" ORDER BY time_unix_nano DESC");
// Add pagination
@@ -583,10 +586,10 @@ public class GreptimeDbDataStorage extends AbstractHistoryDataStorage {
@Override
public long countLogsByMultipleConditions(Long startTime, Long endTime, String traceId,
String spanId, Integer severityNumber,
String severityText) {
String severityText, String searchContent) {
try {
StringBuilder sql = new StringBuilder("SELECT COUNT(*) as count FROM ").append(LOG_TABLE_NAME);
buildWhereConditions(sql, startTime, endTime, traceId, spanId, severityNumber, severityText);
buildWhereConditions(sql, startTime, endTime, traceId, spanId, severityNumber, severityText, searchContent);
List<Map<String, Object>> rows = greptimeSqlQueryExecutor.execute(sql.toString());
if (rows != null && !rows.isEmpty()) {
@@ -623,7 +626,7 @@ public class GreptimeDbDataStorage extends AbstractHistoryDataStorage {
* @param severityNumber severity number
*/
private void buildWhereConditions(StringBuilder sql, Long startTime, Long endTime, String traceId,
String spanId, Integer severityNumber, String severityText) {
String spanId, Integer severityNumber, String severityText, String searchContent) {
List<String> conditions = new ArrayList<>();
// Time range condition
@@ -651,6 +654,11 @@ public class GreptimeDbDataStorage extends AbstractHistoryDataStorage {
conditions.add("severity_text = '" + safeString(severityText) + "'");
}
// Search content condition - search in body field
if (StringUtils.hasText(searchContent)) {
conditions.add("body LIKE '%" + safeString(searchContent) + "%'");
}
// Add WHERE clause if there are conditions
if (!conditions.isEmpty()) {
sql.append(" WHERE ").append(String.join(" AND ", conditions));
@@ -758,7 +766,7 @@ public class GreptimeDbDataStorage extends AbstractHistoryDataStorage {
try {
StringBuilder sql = new StringBuilder("DELETE FROM ").append(LOG_TABLE_NAME).append(" WHERE time_unix_nano IN (");
sql.append(timeUnixNanos.stream()
.filter(time -> time != null)
.filter(Objects::nonNull)
.map(String::valueOf)
.collect(Collectors.joining(", ")));
sql.append(")");
@@ -47,6 +47,7 @@ import org.apache.hertzbeat.common.entity.arrow.RowWrapper;
import org.apache.hertzbeat.common.entity.dto.Value;
import org.apache.hertzbeat.common.entity.message.CollectRep;
import org.apache.hertzbeat.common.util.JsonUtil;
import org.apache.hertzbeat.common.util.StrBuffer;
import org.apache.hertzbeat.warehouse.store.history.tsdb.AbstractHistoryDataStorage;
import org.jetbrains.annotations.NotNull;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
@@ -68,8 +69,14 @@ public class TdEngineDataStorage extends AbstractHistoryDataStorage {
private static final String INSTANCE_NULL = "''";
private static final String CONSTANTS_CREATE_DATABASE = "CREATE DATABASE IF NOT EXISTS %s";
private static final String INSERT_TABLE_DATA_SQL = "INSERT INTO `%s` USING `%s` TAGS (%s) VALUES %s";
private static final String CREATE_SUPER_TABLE_SQL = "CREATE STABLE IF NOT EXISTS `%s` %s TAGS (monitor BIGINT)";
private static final String CREATE_SUPER_TABLE_SQL = "CREATE STABLE IF NOT EXISTS `%s` %s TAGS (instance NCHAR(128))";
private static final String NO_SUPER_TABLE_ERROR = "Table does not exist";
/**
* schema version suffix - increment when TAG definition changes.
* v1: instance BIGINT (original monitorId like 123456789)
* v2: instance NCHAR(128) (supports string instance values like 127.0.0.1:8080)
*/
private static final String SUPER_TABLE_VERSION = "_v2";
private static final String QUERY_HISTORY_WITH_INSTANCE_SQL =
"SELECT ts, metric_labels, `%s` FROM `%s` WHERE metric_labels = '%s' AND ts >= now - %s order by ts desc";
@@ -185,7 +192,7 @@ public class TdEngineDataStorage extends AbstractHistoryDataStorage {
String instance = metricsData.getInstance();
String app = metricsData.getApp();
String metrics = metricsData.getMetrics();
String superTable = getTable(app, metrics, "_super");
String superTable = getSuperTable(app, metrics);
String table = getTable(app, metrics, instance);
StringBuilder sqlBuffer = new StringBuilder();
int i = 0;
@@ -215,7 +222,12 @@ public class TdEngineDataStorage extends AbstractHistoryDataStorage {
} else {
try {
double number = Double.parseDouble(value);
sqlRowBuffer.append(number);
// TDengine doesn't support NaN or Infinity literals, convert to NULL
if (Double.isNaN(number) || Double.isInfinite(number)) {
sqlRowBuffer.append("NULL");
} else {
sqlRowBuffer.append(number);
}
} catch (Exception e) {
if (log.isWarnEnabled()) {
@@ -230,7 +242,7 @@ public class TdEngineDataStorage extends AbstractHistoryDataStorage {
if (CommonConstants.NULL_VALUE.equals(value)) {
sqlRowBuffer.append("NULL");
} else {
sqlRowBuffer.append("'").append(formatStringValue(value)).append("'");
sqlRowBuffer.append("'").append(StrBuffer.escapeForFormat(formatStringValue(value))).append("'");
}
}
@@ -246,7 +258,7 @@ public class TdEngineDataStorage extends AbstractHistoryDataStorage {
sqlBuffer.append(" ").append(String.format(sqlRowBuffer.toString(), formatStringValue(JsonUtil.toJson(labels))));
}
String insertDataSql = String.format(INSERT_TABLE_DATA_SQL, table, superTable, instance, sqlBuffer);
String insertDataSql = String.format(INSERT_TABLE_DATA_SQL, table, superTable, "'" + formatStringValue(instance) + "'", sqlBuffer);
if (log.isDebugEnabled()) {
log.debug(insertDataSql);
@@ -325,7 +337,12 @@ public class TdEngineDataStorage extends AbstractHistoryDataStorage {
.replace("[", "_")
.replace("]", "_");
}
return app + "_" + metrics + "_" + instance;
return app + "_" + metrics + "_" + instance + SUPER_TABLE_VERSION;
}
@NotNull
private static String getSuperTable(String app, String metrics) {
return app + "_" + metrics + "_super" + SUPER_TABLE_VERSION;
}
private String formatStringValue(String value) {
@@ -6,7 +6,7 @@
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
@@ -22,10 +22,7 @@ import com.fasterxml.jackson.databind.JsonNode;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.net.URI;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.time.Duration;
import java.time.Instant;
import java.time.ZonedDateTime;
import java.time.temporal.TemporalAmount;
import java.util.ArrayList;
@@ -150,17 +147,17 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
String selectNodeStatusUrl = vmClusterProps.select().url() + VM_SELECT_BASE_PATH.formatted(vmClusterProps.accountID(), STATUS_PATH);
HttpHeaders headers = new HttpHeaders();
if (StringUtils.hasText(vmInsertProps.username())
&& StringUtils.hasText(vmInsertProps.password())) {
&& StringUtils.hasText(vmInsertProps.password())) {
String authStr = vmInsertProps.username() + ":" + vmInsertProps.password();
String encodedAuth = Base64Util.encode(authStr);
headers.add(HttpHeaders.AUTHORIZATION, NetworkConstants.BASIC + SignConstants.BLANK + encodedAuth);
}
HttpEntity<Void> requestEntity = new HttpEntity<>(headers);
ResponseEntity<String> responseEntity = restTemplate.exchange(
selectNodeStatusUrl,
HttpMethod.GET,
requestEntity,
String.class
selectNodeStatusUrl,
HttpMethod.GET,
requestEntity,
String.class
);
String result = responseEntity.getBody();
@@ -186,7 +183,7 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
}
if (metricsData.getValues().isEmpty()) {
log.info("[warehouse victoria-metrics] flush metrics data {} {} {} is null, ignore.",
metricsData.getId(), metricsData.getApp(), metricsData.getMetrics());
metricsData.getId(), metricsData.getApp(), metricsData.getMetrics());
return;
}
Map<String, String> defaultLabels = Maps.newHashMapWithExpectedSize(8);
@@ -196,7 +193,7 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
isPrometheusAuto = true;
defaultLabels.remove(MONITOR_METRICS_KEY);
defaultLabels.put(LABEL_KEY_JOB, metricsData.getApp()
.substring(CommonConstants.PROMETHEUS_APP_PREFIX.length()));
.substring(CommonConstants.PROMETHEUS_APP_PREFIX.length()));
} else {
isPrometheusAuto = false;
defaultLabels.put(LABEL_KEY_JOB, metricsData.getApp());
@@ -239,7 +236,7 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
try {
labels.putAll(defaultLabels);
String labelName = isPrometheusAuto ? metricsData.getMetrics()
: metricsData.getMetrics() + SPILT + entry.getKey();
: metricsData.getMetrics() + SPILT + entry.getKey();
labels.put(LABEL_KEY_NAME, labelName);
if (!isPrometheusAuto) {
labels.put(MONITOR_METRIC_KEY, entry.getKey());
@@ -251,10 +248,10 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
labels.putAll(customizedLabels);
}
VictoriaMetricsDataStorage.VictoriaMetricsContent content = VictoriaMetricsDataStorage.VictoriaMetricsContent.builder()
.metric(new HashMap<>(labels))
.values(new Double[]{entry.getValue()})
.timestamps(timestamp)
.build();
.metric(new HashMap<>(labels))
.values(new Double[]{entry.getValue()})
.timestamps(timestamp)
.build();
contentList.add(content);
} catch (Exception e) {
log.error("combine metrics data error: {}.", e.getMessage(), e);
@@ -290,13 +287,13 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
public Map<String, List<Value>> getHistoryMetricData(String instance, String app, String metrics, String metric,
String history) {
String labelName = metrics + SPILT + metric;
if (CommonConstants.PROMETHEUS.equals(app)) {
if (app.startsWith(CommonConstants.PROMETHEUS_APP_PREFIX)) {
labelName = metrics;
}
String timeSeriesSelector = Stream.of(
LABEL_KEY_NAME + "=\"" + labelName + "\"",
LABEL_KEY_INSTANCE + "=\"" + instance + "\"",
CommonConstants.PROMETHEUS.equals(app) ? null : MONITOR_METRIC_KEY + "=\"" + metric + "\""
app.startsWith(CommonConstants.PROMETHEUS_APP_PREFIX) ? null : MONITOR_METRIC_KEY + "=\"" + metric + "\""
).filter(Objects::nonNull).collect(Collectors.joining(","));
Map<String, List<Value>> instanceValuesMap = new HashMap<>(8);
try {
@@ -307,26 +304,25 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
String authStr = vmSelectProps.username() + ":" + vmSelectProps.password();
String encodedAuth = Base64Util.encode(authStr);
headers.add(HttpHeaders.AUTHORIZATION, NetworkConstants.BASIC
+ SignConstants.BLANK + encodedAuth);
+ SignConstants.BLANK + encodedAuth);
}
HttpEntity<Void> httpEntity = new HttpEntity<>(headers);
Instant end = Instant.now();
Duration duration = Duration.ofHours(Long.parseLong(history.replace("h", "")));
Instant start = end.minus(duration);
String exportUrl = vmClusterProps.select().url() + VM_SELECT_BASE_PATH.formatted(vmClusterProps.accountID(), EXPORT_PATH);
String exportUrl = vmClusterProps.select().url() + VM_SELECT_BASE_PATH.formatted(vmClusterProps.accountID(), EXPORT_PATH);
URI uri = UriComponentsBuilder.fromUriString(exportUrl)
.queryParam("match", URLEncoder.encode("{" + timeSeriesSelector + "}", StandardCharsets.UTF_8))
.queryParam("start", String.valueOf(start.getEpochSecond()))
.queryParam("end", String.valueOf(end.getEpochSecond()))
.build(true).toUri();
.queryParam("match[]", "{" + timeSeriesSelector + "}")
.queryParam("start", "now-" + history)
.queryParam("end", "now")
.build()
.encode()
.toUri();
ResponseEntity<String> responseEntity = restTemplate.exchange(uri, HttpMethod.GET, httpEntity,
String.class);
String.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
log.debug("query metrics data from victoria-metrics success. {}", uri);
if (StringUtils.hasText(responseEntity.getBody())) {
String[] contentJsonArr = responseEntity.getBody().split("\n");
List<VictoriaMetricsContent> contents = Arrays.stream(contentJsonArr)
.map(item -> JsonUtil.fromJson(item, VictoriaMetricsContent.class)).toList();
.map(item -> JsonUtil.fromJson(item, VictoriaMetricsContent.class)).toList();
for (VictoriaMetricsContent content : contents) {
Map<String, String> labels = content.getMetric();
labels.remove(LABEL_KEY_NAME);
@@ -338,7 +334,7 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
String labelStr = JsonUtil.toJson(labels);
if (content.getValues() != null && content.getTimestamps() != null) {
List<Value> valueList = instanceValuesMap.computeIfAbsent(labelStr,
k -> new LinkedList<>());
k -> new LinkedList<>());
if (content.getValues().length != content.getTimestamps().length) {
log.error("content.getValues().length != content.getTimestamps().length");
continue;
@@ -347,7 +343,7 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
Long[] timestamps = content.getTimestamps();
for (int index = 0; index < content.getValues().length; index++) {
String strValue = BigDecimal.valueOf(values[index]).setScale(4, RoundingMode.HALF_UP)
.stripTrailingZeros().toPlainString();
.stripTrailingZeros().toPlainString();
// read timestamp here is ms unit
valueList.add(new Value(strValue, timestamps[index]));
}
@@ -368,7 +364,7 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
String metric, String history) {
if (!serverAvailable) {
log.error("""
\t---------------VictoriaMetrics Init Failed---------------
\t--------------Please Config VictoriaMetrics--------------
\t----------Can Not Use Metric History Now----------
@@ -392,13 +388,13 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
startTime = dateTime.toEpochSecond();
}
String labelName = metrics + SPILT + metric;
if (CommonConstants.PROMETHEUS.equals(app)) {
if (app.startsWith(CommonConstants.PROMETHEUS_APP_PREFIX)) {
labelName = metrics;
}
String timeSeriesSelector = Stream.of(
LABEL_KEY_NAME + "=\"" + labelName + "\"",
LABEL_KEY_INSTANCE + "=\"" + instance + "\"",
CommonConstants.PROMETHEUS.equals(app) ? null : MONITOR_METRIC_KEY + "=\"" + metric + "\""
app.startsWith(CommonConstants.PROMETHEUS_APP_PREFIX) ? null : MONITOR_METRIC_KEY + "=\"" + metric + "\""
).filter(Objects::nonNull).collect(Collectors.joining(","));
Map<String, List<Value>> instanceValuesMap = new HashMap<>(8);
try {
@@ -409,25 +405,26 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
String authStr = vmSelectProps.username() + ":" + vmSelectProps.password();
String encodedAuth = Base64Util.encode(authStr);
headers.add(HttpHeaders.AUTHORIZATION, NetworkConstants.BASIC
+ SignConstants.BLANK + encodedAuth);
+ SignConstants.BLANK + encodedAuth);
}
HttpEntity<Void> httpEntity = new HttpEntity<>(headers);
String rangeUrl = VM_SELECT_BASE_PATH.formatted(vmClusterProps.accountID(), QUERY_RANGE_PATH);
String rangeUrl = vmClusterProps.select().url() + VM_SELECT_BASE_PATH.formatted(vmClusterProps.accountID(), QUERY_RANGE_PATH);
URI uri = UriComponentsBuilder.fromUriString(rangeUrl)
.queryParam("query", URLEncoder.encode("{" + timeSeriesSelector + "}", StandardCharsets.UTF_8))
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build(true)
.toUri();
.queryParam("query", "{" + timeSeriesSelector + "}")
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build()
.encode()
.toUri();
ResponseEntity<PromQlQueryContent> responseEntity = restTemplate.exchange(uri, HttpMethod.GET,
httpEntity, PromQlQueryContent.class);
httpEntity, PromQlQueryContent.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
log.debug("query metrics data from victoria-metrics success. {}", uri);
if (responseEntity.getBody() != null && responseEntity.getBody().getData() != null
&& responseEntity.getBody().getData().getResult() != null) {
&& responseEntity.getBody().getData().getResult() != null) {
List<PromQlQueryContent.ContentData.Content> contents = responseEntity.getBody().getData()
.getResult();
.getResult();
for (PromQlQueryContent.ContentData.Content content : contents) {
Map<String, String> labels = content.getMetric();
labels.remove(LABEL_KEY_NAME);
@@ -439,11 +436,11 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
String labelStr = JsonUtil.toJson(labels);
if (content.getValues() != null && !content.getValues().isEmpty()) {
List<Value> valueList = instanceValuesMap.computeIfAbsent(labelStr,
k -> new LinkedList<>());
k -> new LinkedList<>());
for (Object[] valueArr : content.getValues()) {
long timestamp = Long.parseLong(String.valueOf(valueArr[0]));
String value = new BigDecimal(String.valueOf(valueArr[1])).setScale(4,
RoundingMode.HALF_UP).stripTrailingZeros().toPlainString();
RoundingMode.HALF_UP).stripTrailingZeros().toPlainString();
// read timestamp here is s unit
valueList.add(new Value(value, timestamp * 1000));
}
@@ -455,18 +452,19 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
}
// max
uri = UriComponentsBuilder.fromUriString(rangeUrl)
.queryParam("query", URLEncoder.encode("max_over_time({" + timeSeriesSelector + "})", StandardCharsets.UTF_8))
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build(true)
.toUri();
.queryParam("query", "max_over_time({" + timeSeriesSelector + "})")
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build()
.encode()
.toUri();
responseEntity = restTemplate.exchange(uri, HttpMethod.GET, httpEntity, PromQlQueryContent.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
if (responseEntity.getBody() != null && responseEntity.getBody().getData() != null
&& responseEntity.getBody().getData().getResult() != null) {
&& responseEntity.getBody().getData().getResult() != null) {
List<PromQlQueryContent.ContentData.Content> contents = responseEntity.getBody().getData()
.getResult();
.getResult();
for (PromQlQueryContent.ContentData.Content content : contents) {
Map<String, String> labels = content.getMetric();
labels.remove(LABEL_KEY_NAME);
@@ -478,13 +476,13 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
String labelStr = JsonUtil.toJson(labels);
if (content.getValues() != null && !content.getValues().isEmpty()) {
List<Value> valueList = instanceValuesMap.computeIfAbsent(labelStr,
k -> new LinkedList<>());
k -> new LinkedList<>());
if (valueList.size() == content.getValues().size()) {
for (int timestampIndex = 0; timestampIndex < valueList.size(); timestampIndex++) {
Value value = valueList.get(timestampIndex);
Object[] valueArr = content.getValues().get(timestampIndex);
String maxValue = new BigDecimal(String.valueOf(valueArr[1])).setScale(4,
RoundingMode.HALF_UP).stripTrailingZeros().toPlainString();
RoundingMode.HALF_UP).stripTrailingZeros().toPlainString();
value.setMax(maxValue);
}
}
@@ -494,18 +492,19 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
}
// min
uri = UriComponentsBuilder.fromUriString(rangeUrl)
.queryParam("query", URLEncoder.encode("min_over_time({" + timeSeriesSelector + "})", StandardCharsets.UTF_8))
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build(true)
.toUri();
.queryParam("query", "min_over_time({" + timeSeriesSelector + "})")
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build()
.encode()
.toUri();
responseEntity = restTemplate.exchange(uri, HttpMethod.GET, httpEntity, PromQlQueryContent.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
if (responseEntity.getBody() != null && responseEntity.getBody().getData() != null
&& responseEntity.getBody().getData().getResult() != null) {
&& responseEntity.getBody().getData().getResult() != null) {
List<PromQlQueryContent.ContentData.Content> contents = responseEntity.getBody().getData()
.getResult();
.getResult();
for (PromQlQueryContent.ContentData.Content content : contents) {
Map<String, String> labels = content.getMetric();
labels.remove(LABEL_KEY_NAME);
@@ -517,13 +516,13 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
String labelStr = JsonUtil.toJson(labels);
if (content.getValues() != null && !content.getValues().isEmpty()) {
List<Value> valueList = instanceValuesMap.computeIfAbsent(labelStr,
k -> new LinkedList<>());
k -> new LinkedList<>());
if (valueList.size() == content.getValues().size()) {
for (int timestampIndex = 0; timestampIndex < valueList.size(); timestampIndex++) {
Value value = valueList.get(timestampIndex);
Object[] valueArr = content.getValues().get(timestampIndex);
String minValue = new BigDecimal(String.valueOf(valueArr[1])).setScale(4,
RoundingMode.HALF_UP).stripTrailingZeros().toPlainString();
RoundingMode.HALF_UP).stripTrailingZeros().toPlainString();
value.setMin(minValue);
}
}
@@ -533,18 +532,19 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
}
// avg
uri = UriComponentsBuilder.fromUriString(rangeUrl)
.queryParam("query", URLEncoder.encode("avg_over_time({" + timeSeriesSelector + "})", StandardCharsets.UTF_8))
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build(true)
.toUri();
.queryParam("query", "avg_over_time({" + timeSeriesSelector + "})")
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build()
.encode()
.toUri();
responseEntity = restTemplate.exchange(uri, HttpMethod.GET, httpEntity, PromQlQueryContent.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
if (responseEntity.getBody() != null && responseEntity.getBody().getData() != null
&& responseEntity.getBody().getData().getResult() != null) {
&& responseEntity.getBody().getData().getResult() != null) {
List<PromQlQueryContent.ContentData.Content> contents = responseEntity.getBody().getData()
.getResult();
.getResult();
for (PromQlQueryContent.ContentData.Content content : contents) {
Map<String, String> labels = content.getMetric();
labels.remove(LABEL_KEY_NAME);
@@ -556,13 +556,13 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
String labelStr = JsonUtil.toJson(labels);
if (content.getValues() != null && !content.getValues().isEmpty()) {
List<Value> valueList = instanceValuesMap.computeIfAbsent(labelStr,
k -> new LinkedList<>());
k -> new LinkedList<>());
if (valueList.size() == content.getValues().size()) {
for (int timestampIndex = 0; timestampIndex < valueList.size(); timestampIndex++) {
Value value = valueList.get(timestampIndex);
Object[] valueArr = content.getValues().get(timestampIndex);
String avgValue = new BigDecimal(String.valueOf(valueArr[1])).setScale(4,
RoundingMode.HALF_UP).stripTrailingZeros().toPlainString();
RoundingMode.HALF_UP).stripTrailingZeros().toPlainString();
value.setMean(avgValue);
}
}
@@ -584,7 +584,7 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
if (StringUtils.hasText(vmInsertProps.username())
&& StringUtils.hasText(vmInsertProps.password())) {
&& StringUtils.hasText(vmInsertProps.password())) {
String authStr = vmInsertProps.username() + ":" + vmInsertProps.password();
String encodedAuth = Base64Util.encode(authStr);
headers.add(HttpHeaders.AUTHORIZATION, NetworkConstants.BASIC + SignConstants.BLANK + encodedAuth);
@@ -596,7 +596,7 @@ public class VictoriaMetricsClusterDataStorage extends AbstractHistoryDataStorag
HttpEntity<String> httpEntity = new HttpEntity<>(stringBuilder.toString(), headers);
String importUrl = vmClusterProps.insert().url() + VM_INSERT_BASE_PATH.formatted(vmClusterProps.accountID(), IMPORT_PATH);
ResponseEntity<String> responseEntity = restTemplate.postForEntity(importUrl,
httpEntity, String.class);
httpEntity, String.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
log.debug("insert metrics data to victoria-metrics success.");
} else {
@@ -6,7 +6,7 @@
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
@@ -21,7 +21,6 @@ import java.io.ByteArrayOutputStream;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.net.URI;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.time.Duration;
import java.time.ZonedDateTime;
@@ -86,7 +85,7 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
private static final String IMPORT_PATH = "/api/v1/import";
/**
* <a href="https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-export-data-in-json-line-format">
* https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-export-data-in-json-line-format
* https://docs.victoriametrics.com/Single-server-VictoriaMetrics.html#how-to-export-data-in-json-line-format
* </a>
*/
private static final String EXPORT_PATH = "/api/v1/export";
@@ -105,7 +104,7 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
private final VictoriaMetricsProperties victoriaMetricsProp;
private final RestTemplate restTemplate;
private final BlockingQueue<VictoriaMetricsDataStorage.VictoriaMetricsContent> metricsBufferQueue;
private HashedWheelTimer metricsFlushTimer = null;
private final VictoriaMetricsProperties.InsertConfig insertConfig;
private final AtomicBoolean draining = new AtomicBoolean(false);
@@ -119,7 +118,7 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
victoriaMetricsProp = victoriaMetricsProperties;
serverAvailable = checkVictoriaMetricsDatasourceAvailable();
insertConfig = victoriaMetricsProperties.insert() == null ? new VictoriaMetricsProperties.InsertConfig(100, 3,
new VictoriaMetricsProperties.Compression(false)) : victoriaMetricsProperties.insert();
new VictoriaMetricsProperties.Compression(false)) : victoriaMetricsProperties.insert();
metricsBufferQueue = new LinkedBlockingQueue<>(insertConfig.bufferSize());
initializeFlushTimer();
}
@@ -139,14 +138,14 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
try {
HttpHeaders headers = new HttpHeaders();
if (StringUtils.hasText(victoriaMetricsProp.username())
&& StringUtils.hasText(victoriaMetricsProp.password())) {
&& StringUtils.hasText(victoriaMetricsProp.password())) {
String authStr = victoriaMetricsProp.username() + SignConstants.DOUBLE_MARK + victoriaMetricsProp.password();
String encodedAuth = Base64Util.encode(authStr);
headers.add(HttpHeaders.AUTHORIZATION, NetworkConstants.BASIC + " " + encodedAuth);
}
HttpEntity<Void> httpEntity = new HttpEntity<>(headers);
ResponseEntity<String> responseEntity = restTemplate.exchange(victoriaMetricsProp.url() + STATUS_PATH,
HttpMethod.GET, httpEntity, String.class);
HttpMethod.GET, httpEntity, String.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
log.info("check victoria metrics server status success.");
return true;
@@ -168,7 +167,7 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
}
if (metricsData.getValues().isEmpty()) {
log.info("[warehouse victoria-metrics] flush metrics data {} {} {} is null, ignore.",
metricsData.getId(), metricsData.getApp(), metricsData.getMetrics());
metricsData.getId(), metricsData.getApp(), metricsData.getMetrics());
return;
}
Map<String, String> defaultLabels = Maps.newHashMapWithExpectedSize(8);
@@ -178,7 +177,7 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
isPrometheusAuto = true;
defaultLabels.remove(MONITOR_METRICS_KEY);
defaultLabels.put(LABEL_KEY_JOB, metricsData.getApp()
.substring(CommonConstants.PROMETHEUS_APP_PREFIX.length()));
.substring(CommonConstants.PROMETHEUS_APP_PREFIX.length()));
} else {
defaultLabels.put(LABEL_KEY_JOB, metricsData.getApp());
}
@@ -220,7 +219,7 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
try {
labels.putAll(defaultLabels);
String labelName = isPrometheusAuto ? metricsData.getMetrics()
: metricsData.getMetrics() + SPILT + entry.getKey();
: metricsData.getMetrics() + SPILT + entry.getKey();
labels.put(LABEL_KEY_NAME, labelName);
if (!isPrometheusAuto) {
labels.put(MONITOR_METRIC_KEY, entry.getKey());
@@ -232,10 +231,10 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
labels.putAll(customizedLabels);
}
VictoriaMetricsContent content = VictoriaMetricsContent.builder()
.metric(new HashMap<>(labels))
.values(new Double[]{entry.getValue()})
.timestamps(timestamp)
.build();
.metric(new HashMap<>(labels))
.values(new Double[]{entry.getValue()})
.timestamps(timestamp)
.build();
contentList.add(content);
} catch (Exception e) {
log.error("combine metrics data error: {}.", e.getMessage(), e);
@@ -266,37 +265,39 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
@Override
public Map<String, List<Value>> getHistoryMetricData(String instance, String app, String metrics, String metric, String history) {
String labelName = metrics + SPILT + metric;
if (CommonConstants.PROMETHEUS.equals(app)) {
if (app.startsWith(CommonConstants.PROMETHEUS_APP_PREFIX)) {
labelName = metrics;
}
String timeSeriesSelector = LABEL_KEY_NAME + "=\"" + labelName + "\""
+ "," + LABEL_KEY_INSTANCE + "=\"" + instance + "\""
+ (CommonConstants.PROMETHEUS.equals(app) ? "" : "," + MONITOR_METRIC_KEY + "=\"" + metric + "\"");
+ (app.startsWith(CommonConstants.PROMETHEUS_APP_PREFIX) ? "" : "," + MONITOR_METRIC_KEY + "=\"" + metric + "\"");
Map<String, List<Value>> instanceValuesMap = new HashMap<>(8);
try {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.setAccept(List.of(MediaType.APPLICATION_JSON));
if (StringUtils.hasText(victoriaMetricsProp.username())
&& StringUtils.hasText(victoriaMetricsProp.password())) {
&& StringUtils.hasText(victoriaMetricsProp.password())) {
String authStr = victoriaMetricsProp.username() + ":" + victoriaMetricsProp.password();
String encodedAuth = Base64Util.encode(authStr);
headers.add(HttpHeaders.AUTHORIZATION, NetworkConstants.BASIC + SignConstants.BLANK + encodedAuth);
}
HttpEntity<Void> httpEntity = new HttpEntity<>(headers);
URI uri = UriComponentsBuilder.fromUriString(victoriaMetricsProp.url() + EXPORT_PATH)
.queryParam(URLEncoder.encode("match[]", StandardCharsets.UTF_8), URLEncoder.encode("{" + timeSeriesSelector + "}", StandardCharsets.UTF_8))
.queryParam("start", URLEncoder.encode("now-" + history, StandardCharsets.UTF_8))
.queryParam("end", "now")
.build(true).toUri();
.queryParam("match[]", "{" + timeSeriesSelector + "}")
.queryParam("start", "now-" + history)
.queryParam("end", "now")
.build()
.encode()
.toUri();
ResponseEntity<String> responseEntity = restTemplate.exchange(uri,
HttpMethod.GET, httpEntity, String.class);
HttpMethod.GET, httpEntity, String.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
log.debug("query metrics data from victoria-metrics success. {}", uri);
if (StringUtils.hasText(responseEntity.getBody())) {
String[] contentJsonArr = responseEntity.getBody().split("\n");
List<VictoriaMetricsContent> contents = Arrays.stream(contentJsonArr).map(
item -> JsonUtil.fromJson(item, VictoriaMetricsContent.class)
item -> JsonUtil.fromJson(item, VictoriaMetricsContent.class)
).toList();
for (VictoriaMetricsContent content : contents) {
Map<String, String> labels = content.getMetric();
@@ -337,7 +338,7 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
String metric, String history) {
if (!serverAvailable) {
log.error("""
\t---------------VictoriaMetrics Init Failed---------------
\t--------------Please Config VictoriaMetrics--------------
\t----------Can Not Use Metric History Now----------
@@ -361,37 +362,39 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
startTime = dateTime.toEpochSecond();
}
String labelName = metrics + SPILT + metric;
if (CommonConstants.PROMETHEUS.equals(app)) {
if (app.startsWith(CommonConstants.PROMETHEUS_APP_PREFIX)) {
labelName = metrics;
}
String timeSeriesSelector = LABEL_KEY_NAME + "=\"" + labelName + "\""
+ "," + LABEL_KEY_INSTANCE + "=\"" + instance + "\""
+ (CommonConstants.PROMETHEUS.equals(app) ? "" : "," + MONITOR_METRIC_KEY + "=\"" + metric + "\"");
+ (app.startsWith(CommonConstants.PROMETHEUS_APP_PREFIX) ? "" : "," + MONITOR_METRIC_KEY + "=\"" + metric + "\"");
Map<String, List<Value>> instanceValuesMap = new HashMap<>(8);
try {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.setAccept(List.of(MediaType.APPLICATION_JSON));
if (StringUtils.hasText(victoriaMetricsProp.username())
&& StringUtils.hasText(victoriaMetricsProp.password())) {
&& StringUtils.hasText(victoriaMetricsProp.password())) {
String authStr = victoriaMetricsProp.username() + ":" + victoriaMetricsProp.password();
String encodedAuth = Base64Util.encode(authStr);
headers.add(HttpHeaders.AUTHORIZATION, NetworkConstants.BASIC
+ SignConstants.BLANK + encodedAuth);
+ SignConstants.BLANK + encodedAuth);
}
HttpEntity<Void> httpEntity = new HttpEntity<>(headers);
URI uri = UriComponentsBuilder.fromUriString(victoriaMetricsProp.url() + QUERY_RANGE_PATH)
.queryParam(URLEncoder.encode("query", StandardCharsets.UTF_8), URLEncoder.encode("{" + timeSeriesSelector + "}", StandardCharsets.UTF_8))
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build(true).toUri();
.queryParam("query", "{" + timeSeriesSelector + "}")
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build()
.encode()
.toUri();
ResponseEntity<PromQlQueryContent> responseEntity = restTemplate.exchange(uri,
HttpMethod.GET, httpEntity, PromQlQueryContent.class);
HttpMethod.GET, httpEntity, PromQlQueryContent.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
log.debug("query metrics data from victoria-metrics success. {}", uri);
if (responseEntity.getBody() != null && responseEntity.getBody().getData() != null
&& responseEntity.getBody().getData().getResult() != null) {
&& responseEntity.getBody().getData().getResult() != null) {
List<PromQlQueryContent.ContentData.Content> contents = responseEntity.getBody().getData().getResult();
for (PromQlQueryContent.ContentData.Content content : contents) {
Map<String, String> labels = content.getMetric();
@@ -418,16 +421,18 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
}
// max
uri = UriComponentsBuilder.fromUriString(victoriaMetricsProp.url() + QUERY_RANGE_PATH)
.queryParam(URLEncoder.encode("query", StandardCharsets.UTF_8), URLEncoder.encode("max_over_time({" + timeSeriesSelector + "})", StandardCharsets.UTF_8))
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build(true).toUri();
.queryParam("query", "max_over_time({" + timeSeriesSelector + "})")
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build()
.encode()
.toUri();
responseEntity = restTemplate.exchange(uri,
HttpMethod.GET, httpEntity, PromQlQueryContent.class);
HttpMethod.GET, httpEntity, PromQlQueryContent.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
if (responseEntity.getBody() != null && responseEntity.getBody().getData() != null
&& responseEntity.getBody().getData().getResult() != null) {
&& responseEntity.getBody().getData().getResult() != null) {
List<PromQlQueryContent.ContentData.Content> contents = responseEntity.getBody().getData().getResult();
for (PromQlQueryContent.ContentData.Content content : contents) {
Map<String, String> labels = content.getMetric();
@@ -454,16 +459,18 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
}
// min
uri = UriComponentsBuilder.fromUriString(victoriaMetricsProp.url() + QUERY_RANGE_PATH)
.queryParam(URLEncoder.encode("query", StandardCharsets.UTF_8), URLEncoder.encode("min_over_time({" + timeSeriesSelector + "})", StandardCharsets.UTF_8))
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build(true).toUri();
.queryParam("query", "min_over_time({" + timeSeriesSelector + "})")
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build()
.encode()
.toUri();
responseEntity = restTemplate.exchange(uri,
HttpMethod.GET, httpEntity, PromQlQueryContent.class);
HttpMethod.GET, httpEntity, PromQlQueryContent.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
if (responseEntity.getBody() != null && responseEntity.getBody().getData() != null
&& responseEntity.getBody().getData().getResult() != null) {
&& responseEntity.getBody().getData().getResult() != null) {
List<PromQlQueryContent.ContentData.Content> contents = responseEntity.getBody().getData().getResult();
for (PromQlQueryContent.ContentData.Content content : contents) {
Map<String, String> labels = content.getMetric();
@@ -490,16 +497,18 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
}
// avg
uri = UriComponentsBuilder.fromUriString(victoriaMetricsProp.url() + QUERY_RANGE_PATH)
.queryParam(URLEncoder.encode("query", StandardCharsets.UTF_8), URLEncoder.encode("avg_over_time({" + timeSeriesSelector + "})", StandardCharsets.UTF_8))
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build(true).toUri();
.queryParam("query", "avg_over_time({" + timeSeriesSelector + "})")
.queryParam("step", "4h")
.queryParam("start", startTime)
.queryParam("end", endTime)
.build()
.encode()
.toUri();
responseEntity = restTemplate.exchange(uri,
HttpMethod.GET, httpEntity, PromQlQueryContent.class);
HttpMethod.GET, httpEntity, PromQlQueryContent.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
if (responseEntity.getBody() != null && responseEntity.getBody().getData() != null
&& responseEntity.getBody().getData().getResult() != null) {
&& responseEntity.getBody().getData().getResult() != null) {
List<PromQlQueryContent.ContentData.Content> contents = responseEntity.getBody().getData().getResult();
for (PromQlQueryContent.ContentData.Content content : contents) {
Map<String, String> labels = content.getMetric();
@@ -598,7 +607,7 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
}
// Refresh in advance to avoid waiting
if (metricsBufferQueue.size() >= insertConfig.bufferSize() * 0.8
&& draining.compareAndSet(false, true)) {
&& draining.compareAndSet(false, true)) {
triggerImmediateFlush();
}
}
@@ -664,7 +673,7 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
if (StringUtils.hasText(victoriaMetricsProp.username())
&& StringUtils.hasText(victoriaMetricsProp.password())) {
&& StringUtils.hasText(victoriaMetricsProp.password())) {
String authStr = victoriaMetricsProp.username() + ":" + victoriaMetricsProp.password();
String encodedAuth = Base64Util.encode(authStr);
headers.add(HttpHeaders.AUTHORIZATION, NetworkConstants.BASIC + SignConstants.BLANK + encodedAuth);
@@ -693,7 +702,7 @@ public class VictoriaMetricsDataStorage extends AbstractHistoryDataStorage {
}
ResponseEntity<String> responseEntity = restTemplate.postForEntity(victoriaMetricsProp.url() + IMPORT_PATH,
httpEntity, String.class);
httpEntity, String.class);
if (responseEntity.getStatusCode().is2xxSuccessful()) {
log.debug("insert metrics data to victoria-metrics success.");
} else {
@@ -17,17 +17,67 @@
package org.apache.hertzbeat.warehouse.store;
import com.zaxxer.hikari.HikariDataSource;
import org.apache.arrow.vector.types.pojo.ArrowType;
import org.apache.arrow.vector.types.pojo.Field;
import org.apache.arrow.vector.types.pojo.FieldType;
import org.apache.hertzbeat.common.constants.CommonConstants;
import org.apache.hertzbeat.common.constants.MetricDataConstants;
import org.apache.hertzbeat.common.entity.arrow.ArrowCell;
import org.apache.hertzbeat.common.entity.arrow.RowWrapper;
import org.apache.hertzbeat.common.entity.message.CollectRep;
import org.apache.hertzbeat.warehouse.store.history.tsdb.tdengine.TdEngineDataStorage;
import org.apache.hertzbeat.warehouse.store.history.tsdb.tdengine.TdEngineProperties;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.ArgumentCaptor;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import org.mockito.junit.jupiter.MockitoSettings;
import org.mockito.quality.Strictness;
import java.sql.Connection;
import java.sql.Statement;
import java.util.List;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.Mockito.atLeastOnce;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
/**
* Test case for {@link TdEngineDataStorage}
*/
@ExtendWith(MockitoExtension.class)
@MockitoSettings(strictness = Strictness.LENIENT)
class TdEngineDataStorageTest {
@Mock
private TdEngineProperties tdEngineProperties;
@Mock
private HikariDataSource mockHikariDataSource;
@Mock
private Connection mockConnection;
@Mock
private Statement mockStatement;
private TdEngineDataStorage tdEngineDataStorage;
@BeforeEach
void setUp() {
void setUp() throws Exception {
when(tdEngineProperties.enabled()).thenReturn(true);
when(tdEngineProperties.url()).thenReturn("jdbc:TAOS-RS://localhost:6041/demo");
when(tdEngineProperties.username()).thenReturn("root");
when(tdEngineProperties.password()).thenReturn("root");
when(tdEngineProperties.tableStrColumnDefineMaxLength()).thenReturn(200);
when(mockHikariDataSource.getConnection()).thenReturn(mockConnection);
when(mockConnection.createStatement()).thenReturn(mockStatement);
}
@Test
@@ -35,7 +85,34 @@ class TdEngineDataStorageTest {
}
@Test
void saveData() {
void testSaveData() throws Exception {
tdEngineDataStorage = new TdEngineDataStorage(tdEngineProperties);
setPrivateField(tdEngineDataStorage, "hikariDataSource", mockHikariDataSource);
setParentPrivateField(tdEngineDataStorage, "serverAvailable", true);
CollectRep.MetricsData metricsData = generateMockedMetricsData();
tdEngineDataStorage.saveData(metricsData);
ArgumentCaptor<String> sqlCaptor = ArgumentCaptor.forClass(String.class);
verify(mockStatement, atLeastOnce()).execute(sqlCaptor.capture());
String executedSql = sqlCaptor.getValue();
// Verify SQL structure
assertNotNull(executedSql);
assertTrue(executedSql.startsWith("INSERT INTO"), "SQL should start with INSERT INTO");
assertTrue(executedSql.contains("USING"), "SQL should contain USING clause");
assertTrue(executedSql.contains("TAGS"), "SQL should contain TAGS clause");
assertTrue(executedSql.contains("VALUES"), "SQL should contain VALUES clause");
// Verify table name format: app_metrics_instance_v2
assertTrue(executedSql.contains("app_cpu_test-%server-01_v2"), "Should contain correct table name");
// Verify super table name format: app_metrics_super_v2
assertTrue(executedSql.contains("app_cpu_super_v2"), "Should contain correct super table name");
// Verify tags format: test-%server-01
assertTrue(executedSql.contains("TAGS ('test-%server-01')"), "Should contain correct super table name");
// Verify VALUES clause structure (timestamp + data values)
assertTrue(executedSql.matches(".*VALUES\\s+\\(\\d+.*68\\.7\\)"), "Should contain timestamp and value 68.7");
}
@Test
@@ -49,4 +126,81 @@ class TdEngineDataStorageTest {
@Test
void getHistoryIntervalMetricData() {
}
/**
* Helper method to set private field using reflection
*/
private void setPrivateField(Object target, String fieldName, Object value) throws Exception {
java.lang.reflect.Field field = target.getClass().getDeclaredField(fieldName);
field.setAccessible(true);
field.set(target, value);
}
/**
* Helper method to set private field from parent class using reflection
*/
private void setParentPrivateField(Object target, String fieldName, Object value) throws Exception {
java.lang.reflect.Field field = target.getClass().getSuperclass().getDeclaredField(fieldName);
field.setAccessible(true);
field.set(target, value);
}
public static CollectRep.MetricsData generateMockedMetricsData() {
CollectRep.MetricsData mockMetricsData = Mockito.mock(CollectRep.MetricsData.class);
when(mockMetricsData.getId()).thenReturn(0L);
when(mockMetricsData.getMetrics()).thenReturn("cpu");
when(mockMetricsData.getTime()).thenReturn(System.currentTimeMillis());
when(mockMetricsData.getCode()).thenReturn(CollectRep.Code.SUCCESS);
when(mockMetricsData.getApp()).thenReturn("app");
when(mockMetricsData.getInstance()).thenReturn("test-%server-01");
CollectRep.ValueRow mockValueRow = Mockito.mock(CollectRep.ValueRow.class);
List<String> columnValues = List.of("test-%server-01", "68.7");
when(mockValueRow.getColumnsList()).thenReturn(columnValues);
when(mockValueRow.getColumns(0)).thenReturn("test-%server-01");
when(mockValueRow.getColumns(1)).thenReturn("68.7");
List<CollectRep.ValueRow> mockValueRowsList = List.of(mockValueRow);
when(mockMetricsData.getValues()).thenReturn(mockValueRowsList);
CollectRep.Field instanceField = Mockito.mock(CollectRep.Field.class);
when(instanceField.getName()).thenReturn("instance");
CollectRep.Field usageField = Mockito.mock(CollectRep.Field.class);
when(usageField.getName()).thenReturn("usage");
CollectRep.Field systemField = Mockito.mock(CollectRep.Field.class);
when(systemField.getName()).thenReturn("system");
List<CollectRep.Field> mockFields = List.of(instanceField, usageField, systemField);
when(mockMetricsData.getFields()).thenReturn(mockFields);
ArrowType instanceArrowType = new ArrowType.Utf8();
FieldType instanceFieldType = new FieldType(true, instanceArrowType, null, null);
Field instanceArrowField = new Field("instance", instanceFieldType, null);
ArrowCell instanceCell = Mockito.mock(ArrowCell.class);
when(instanceCell.getField()).thenReturn(instanceArrowField);
when(instanceCell.getValue()).thenReturn("test-%server-01");
when(instanceCell.getMetadataAsBoolean(MetricDataConstants.LABEL)).thenReturn(true);
when(instanceCell.getMetadataAsByte(MetricDataConstants.TYPE)).thenReturn(CommonConstants.TYPE_STRING);
ArrowType usageArrowType = new ArrowType.Utf8();
FieldType usageFieldType = new FieldType(true, usageArrowType, null, null);
Field usageArrowField = new Field("usage", usageFieldType, null);
ArrowCell usageCell = Mockito.mock(ArrowCell.class);
when(usageCell.getField()).thenReturn(usageArrowField);
when(usageCell.getValue()).thenReturn("68.7");
when(usageCell.getMetadataAsBoolean(MetricDataConstants.LABEL)).thenReturn(false);
when(usageCell.getMetadataAsByte(MetricDataConstants.TYPE)).thenReturn(CommonConstants.TYPE_NUMBER);
List<ArrowCell> mockCells = List.of(instanceCell, usageCell);
// Create Arrow Field list for RowWrapper
List<org.apache.arrow.vector.types.pojo.Field> arrowFields = List.of(instanceArrowField, usageArrowField);
RowWrapper mockRowWrapper = Mockito.mock(RowWrapper.class);
when(mockRowWrapper.hasNextRow()).thenReturn(true).thenReturn(false);
when(mockRowWrapper.nextRow()).thenReturn(mockRowWrapper);
when(mockRowWrapper.cellStream()).thenAnswer(invocation -> mockCells.stream());
when(mockRowWrapper.getFieldList()).thenReturn(arrowFields);
when(mockMetricsData.readRow()).thenReturn(mockRowWrapper);
return mockMetricsData;
}
}
@@ -82,7 +82,7 @@ class GreptimeDbDataStorageTest {
@Mock
private GreptimeDB greptimeDb;
private GreptimeDbDataStorage greptimeDbDataStorage;
@BeforeEach
@@ -120,14 +120,14 @@ class GreptimeDbDataStorageTest {
void testSaveData() {
try (MockedStatic<GreptimeDB> mockedStatic = mockStatic(GreptimeDB.class)) {
mockedStatic.when(() -> GreptimeDB.create(any())).thenReturn(greptimeDb);
// Mock the write result
@SuppressWarnings("unchecked")
Result<WriteOk, Err> mockResult = mock(Result.class);
when(mockResult.isOk()).thenReturn(true);
CompletableFuture<Result<WriteOk, Err>> mockFuture = CompletableFuture.completedFuture(mockResult);
when(greptimeDb.write(any(Table.class))).thenReturn(mockFuture);
greptimeDbDataStorage = new GreptimeDbDataStorage(greptimeProperties, restTemplate, greptimeSqlQueryExecutor);
// Test with valid metrics data
@@ -153,7 +153,7 @@ class GreptimeDbDataStorageTest {
@Test
void testGetHistoryMetricData() {
greptimeDbDataStorage = new GreptimeDbDataStorage(greptimeProperties, restTemplate, greptimeSqlQueryExecutor);
PromQlQueryContent content = createMockPromQlQueryContent();
ResponseEntity<PromQlQueryContent> responseEntity = new ResponseEntity<>(content, HttpStatus.OK);
@@ -172,14 +172,14 @@ class GreptimeDbDataStorageTest {
void testSaveLogData() {
try (MockedStatic<GreptimeDB> mockedStatic = mockStatic(GreptimeDB.class)) {
mockedStatic.when(() -> GreptimeDB.create(any())).thenReturn(greptimeDb);
// Mock the write result
@SuppressWarnings("unchecked")
Result<WriteOk, Err> mockResult = mock(Result.class);
when(mockResult.isOk()).thenReturn(true);
CompletableFuture<Result<WriteOk, Err>> mockFuture = CompletableFuture.completedFuture(mockResult);
when(greptimeDb.write(any(Table.class))).thenReturn(mockFuture);
greptimeDbDataStorage = new GreptimeDbDataStorage(greptimeProperties, restTemplate, greptimeSqlQueryExecutor);
LogEntry logEntry = createMockLogEntry();
@@ -196,10 +196,10 @@ class GreptimeDbDataStorageTest {
greptimeDbDataStorage = new GreptimeDbDataStorage(greptimeProperties, restTemplate, greptimeSqlQueryExecutor);
List<Map<String, Object>> mockLogRows = createMockLogRows();
when(greptimeSqlQueryExecutor.execute(anyString())).thenReturn(mockLogRows);
// Test basic query
List<LogEntry> result = greptimeDbDataStorage.queryLogsByMultipleConditions(
System.currentTimeMillis() - 3600000, System.currentTimeMillis(), "trace123", "span456", 1, "INFO"
System.currentTimeMillis() - 3600000, System.currentTimeMillis(), "trace123", "span456", 1, "INFO", "content"
);
assertNotNull(result);
assertEquals(1, result.size());
@@ -209,34 +209,34 @@ class GreptimeDbDataStorageTest {
List<Map<String, Object>> mockCountResult = List.of(Map.of("count", 5L));
when(greptimeSqlQueryExecutor.execute(anyString())).thenReturn(mockCountResult);
long count = greptimeDbDataStorage.countLogsByMultipleConditions(
System.currentTimeMillis() - 3600000, System.currentTimeMillis(), null, null, null, null
System.currentTimeMillis() - 3600000, System.currentTimeMillis(), null, null, null, null, null
);
assertEquals(5L, count);
// Test count query with executor error
when(greptimeSqlQueryExecutor.execute(anyString())).thenThrow(new RuntimeException("Database error"));
long errorCount = greptimeDbDataStorage.countLogsByMultipleConditions(System.currentTimeMillis() - 3600000, System.currentTimeMillis(), null, null, null, null);
long errorCount = greptimeDbDataStorage.countLogsByMultipleConditions(System.currentTimeMillis() - 3600000, System.currentTimeMillis(), null, null, null, null, null);
assertEquals(0L, errorCount);
}
}
@Test
void testQueryLogsWithPagination() {
try (MockedStatic<GreptimeDB> mockedStatic = mockStatic(GreptimeDB.class)) {
mockedStatic.when(() -> GreptimeDB.create(any())).thenReturn(greptimeDb);
greptimeDbDataStorage = new GreptimeDbDataStorage(greptimeProperties, restTemplate, greptimeSqlQueryExecutor);
when(greptimeSqlQueryExecutor.execute(anyString())).thenReturn(createMockLogRows());
ArgumentCaptor<String> sqlCaptor = ArgumentCaptor.forClass(String.class);
greptimeDbDataStorage.queryLogsByMultipleConditionsWithPagination(
System.currentTimeMillis() - 3600000, System.currentTimeMillis(),
null, null, null, null, 1, 10
null, null, null, null, null, 1, 10
);
verify(greptimeSqlQueryExecutor).execute(sqlCaptor.capture());
String capturedSql = sqlCaptor.getValue();
assertTrue(capturedSql.toLowerCase().contains("limit 10"));
assertTrue(capturedSql.toLowerCase().contains("offset 1"));
}
@@ -299,7 +299,7 @@ class GreptimeDbDataStorageTest {
lenient().when(mockMetricsData.getCode()).thenReturn(CollectRep.Code.SUCCESS);
lenient().when(mockMetricsData.getMetrics()).thenReturn("cpu");
lenient().when(mockMetricsData.getId()).thenReturn(1L);
if (!hasValues) {
lenient().when(mockMetricsData.getValues()).thenReturn(Collections.emptyList());
return mockMetricsData;
@@ -311,36 +311,36 @@ class GreptimeDbDataStorageTest {
lenient().when(mockField1.getName()).thenReturn("usage");
lenient().when(mockField1.getLabel()).thenReturn(false);
lenient().when(mockField1.getType()).thenReturn((int) CommonConstants.TYPE_NUMBER);
CollectRep.Field mockField2 = mock(CollectRep.Field.class);
lenient().when(mockField2.getName()).thenReturn("instance");
lenient().when(mockField2.getLabel()).thenReturn(true);
lenient().when(mockField2.getType()).thenReturn((int) CommonConstants.TYPE_STRING);
lenient().when(mockMetricsData.getFields()).thenReturn(List.of(mockField1, mockField2));
// Create ValueRow mock
CollectRep.ValueRow mockValueRow = mock(CollectRep.ValueRow.class);
lenient().when(mockValueRow.getColumnsList()).thenReturn(List.of("server1", "85.5"));
lenient().when(mockMetricsData.getValues()).thenReturn(List.of(mockValueRow));
// Mock RowWrapper for readRow()
RowWrapper mockRowWrapper = mock(RowWrapper.class);
lenient().when(mockRowWrapper.hasNextRow()).thenReturn(true, false);
lenient().when(mockRowWrapper.nextRow()).thenReturn(mockRowWrapper);
// Mock cell stream
ArrowCell mockCell1 = mock(ArrowCell.class);
lenient().when(mockCell1.getValue()).thenReturn("85.5");
lenient().when(mockCell1.getMetadataAsBoolean(any())).thenReturn(false);
lenient().when(mockCell1.getMetadataAsByte(any())).thenReturn(CommonConstants.TYPE_NUMBER);
ArrowCell mockCell2 = mock(ArrowCell.class);
lenient().when(mockCell2.getValue()).thenReturn("server1");
lenient().when(mockCell2.getMetadataAsBoolean(any())).thenReturn(true);
lenient().when(mockCell2.getMetadataAsByte(any())).thenReturn(CommonConstants.TYPE_STRING);
lenient().when(mockRowWrapper.cellStream()).thenReturn(java.util.stream.Stream.of(mockCell1, mockCell2));
lenient().when(mockMetricsData.readRow()).thenReturn(mockRowWrapper);
@@ -384,7 +384,7 @@ class GreptimeDbDataStorageTest {
row.put("body", "\"Test log message\"");
row.put("trace_id", "trace123");
row.put("span_id", "span456");
return List.of(row);
}
}
}
+1
View File
@@ -0,0 +1 @@
enable-pre-post-scripts=true
+20 -20
View File
@@ -1,12 +1,12 @@
---
title: Cloud Monitoring System HertzBeat v1.0 Officially Released
author: tom
author_title: tom
author_url: https://github.com/tomsun28
title: Cloud Monitoring System HertzBeat v1.0 Officially Released
author: tom
author_title: tom
author_url: https://github.com/tomsun28
tags: [opensource]
---
HertzBeat, incubated by Dromara and open-sourced by TanCloud, is an open-source monitoring and alerting project that supports a variety of monitoring types including websites, APIs, PING, ports, databases, full-site, operating systems, middleware, etc.
HertzBeat, incubated by Dromara and open-sourced by TanCloud, is an open-source monitoring and alerting project that supports a variety of monitoring types including websites, APIs, PING, ports, databases, full-site, operating systems, middleware, etc.
It supports threshold alarms and notification alerts (email, webhook, DingTalk, WeCom, Feishu robots) and has an easy-to-use, friendly visual operation interface.
Official Website: hertzbeat.com | tancloud.cn
@@ -17,20 +17,20 @@ We would like to thank the HertzBeat Contributors for their contributions since
<table>
<tr>
<td align="center"><a href="https://github.com/tomsun28"><img src="https://avatars.githubusercontent.com/u/24788200?v=4?s=100" width="100px;" alt=""/><br /><sub><b>tomsun28</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=tomsun28" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/commits?author=tomsun28" title="Documentation">📖</a> <a href="#design-tomsun28" title="Design">🎨</a></td>
<td align="center"><a href="https://github.com/wang1027-wqh"><img src="https://avatars.githubusercontent.com/u/71161318?v=4?s=100" width="100px;" alt=""/><br /><sub><b>会编程的王学长</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=wang1027-wqh" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/commits?author=wang1027-wqh" title="Documentation">📖</a> <a href="#design-wang1027-wqh" title="Design">🎨</a></td>
<td align="center"><a href="https://www.maxkey.top/"><img src="https://avatars.githubusercontent.com/u/1563377?v=4?s=100" width="100px;" alt=""/><br /><sub><b>MaxKey</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=shimingxy" title="Code">💻</a> <a href="#design-shimingxy" title="Design">🎨</a> <a href="#ideas-shimingxy" title="Ideas, Planning, & Feedback">🤔</a></td>
<td align="center"><a href="https://blog.gcdd.top/"><img src="https://avatars.githubusercontent.com/u/26523525?v=4?s=100" width="100px;" alt=""/><br /><sub><b>观沧海</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=gcdd1993" title="Code">💻</a> <a href="#design-gcdd1993" title="Design">🎨</a> <a href="https://github.com/tomsun28/hertzbeat/issues?q=author%3Agcdd1993" title="Bug reports">🐛</a></td>
<td align="center"><a href="https://github.com/a25017012"><img src="https://avatars.githubusercontent.com/u/32265356?v=4?s=100" width="100px;" alt=""/><br /><sub><b>yuye</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=a25017012" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/commits?author=a25017012" title="Documentation">📖</a></td>
<td align="center"><a href="https://github.com/jx10086"><img src="https://avatars.githubusercontent.com/u/5323228?v=4?s=100" width="100px;" alt=""/><br /><sub><b>jx10086</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=jx10086" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/issues?q=author%3Ajx10086" title="Bug reports">🐛</a></td>
<td align="center"><a href="https://github.com/winnerTimer"><img src="https://avatars.githubusercontent.com/u/76024658?v=4?s=100" width="100px;" alt=""/><br /><sub><b>winnerTimer</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=winnerTimer" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/issues?q=author%3AwinnerTimer" title="Bug reports">🐛</a></td>
<td align="center"><a href="https://github.com/tomsun28"><img src="https://avatars.githubusercontent.com/u/24788200?v=4?s=100" width="100px;" alt="" /><br /><sub><b>tomsun28</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=tomsun28" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/commits?author=tomsun28" title="Documentation">📖</a> <a href="#design-tomsun28" title="Design">🎨</a></td>
<td align="center"><a href="https://github.com/wang1027-wqh"><img src="https://avatars.githubusercontent.com/u/71161318?v=4?s=100" width="100px;" alt="" /><br /><sub><b>会编程的王学长</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=wang1027-wqh" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/commits?author=wang1027-wqh" title="Documentation">📖</a> <a href="#design-wang1027-wqh" title="Design">🎨</a></td>
<td align="center"><a href="https://www.maxkey.top/"><img src="https://avatars.githubusercontent.com/u/1563377?v=4?s=100" width="100px;" alt="" /><br /><sub><b>MaxKey</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=shimingxy" title="Code">💻</a> <a href="#design-shimingxy" title="Design">🎨</a> <a href="#ideas-shimingxy" title="Ideas, Planning, & Feedback">🤔</a></td>
<td align="center"><a href="https://blog.gcdd.top/"><img src="https://avatars.githubusercontent.com/u/26523525?v=4?s=100" width="100px;" alt="" /><br /><sub><b>观沧海</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=gcdd1993" title="Code">💻</a> <a href="#design-gcdd1993" title="Design">🎨</a> <a href="https://github.com/tomsun28/hertzbeat/issues?q=author%3Agcdd1993" title="Bug reports">🐛</a></td>
<td align="center"><a href="https://github.com/a25017012"><img src="https://avatars.githubusercontent.com/u/32265356?v=4?s=100" width="100px;" alt="" /><br /><sub><b>yuye</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=a25017012" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/commits?author=a25017012" title="Documentation">📖</a></td>
<td align="center"><a href="https://github.com/jx10086"><img src="https://avatars.githubusercontent.com/u/5323228?v=4?s=100" width="100px;" alt="" /><br /><sub><b>jx10086</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=jx10086" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/issues?q=author%3Ajx10086" title="Bug reports">🐛</a></td>
<td align="center"><a href="https://github.com/winnerTimer"><img src="https://avatars.githubusercontent.com/u/76024658?v=4?s=100" width="100px;" alt="" /><br /><sub><b>winnerTimer</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=winnerTimer" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/issues?q=author%3AwinnerTimer" title="Bug reports">🐛</a></td>
</tr>
<tr>
<td align="center"><a href="https://github.com/goo-kits"><img src="https://avatars.githubusercontent.com/u/13163673?v=4?s=100" width="100px;" alt=""/><br /><sub><b>goo-kits</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=goo-kits" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/issues?q=author%3Agoo-kits" title="Bug reports">🐛</a></td>
<td align="center"><a href="https://github.com/brave4Time"><img src="https://avatars.githubusercontent.com/u/105094014?v=4?s=100" width="100px;" alt=""/><br /><sub><b>brave4Time</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=brave4Time" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/issues?q=author%3Abrave4Time" title="Bug reports">🐛</a></td>
<td align="center"><a href="https://github.com/walkerlee-lab"><img src="https://avatars.githubusercontent.com/u/8426753?v=4?s=100" width="100px;" alt=""/><br /><sub><b>WalkerLee</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=walkerlee-lab" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/issues?q=author%3Awalkerlee-lab" title="Bug reports">🐛</a></td>
<td align="center"><a href="https://github.com/fullofjoy"><img src="https://avatars.githubusercontent.com/u/30247571?v=4?s=100" width="100px;" alt=""/><br /><sub><b>jianghang</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=fullofjoy" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/issues?q=author%3Afullofjoy" title="Bug reports">🐛</a></td>
<td align="center"><a href="https://github.com/ChineseTony"><img src="https://avatars.githubusercontent.com/u/24618786?v=4?s=100" width="100px;" alt=""/><br /><sub><b>ChineseTony</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=ChineseTony" title="Code">💻</a></td>
<td align="center"><a href="https://github.com/goo-kits"><img src="https://avatars.githubusercontent.com/u/13163673?v=4?s=100" width="100px;" alt="" /><br /><sub><b>goo-kits</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=goo-kits" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/issues?q=author%3Agoo-kits" title="Bug reports">🐛</a></td>
<td align="center"><a href="https://github.com/brave4Time"><img src="https://avatars.githubusercontent.com/u/105094014?v=4?s=100" width="100px;" alt="" /><br /><sub><b>brave4Time</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=brave4Time" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/issues?q=author%3Abrave4Time" title="Bug reports">🐛</a></td>
<td align="center"><a href="https://github.com/walkerlee-lab"><img src="https://avatars.githubusercontent.com/u/8426753?v=4?s=100" width="100px;" alt="" /><br /><sub><b>WalkerLee</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=walkerlee-lab" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/issues?q=author%3Awalkerlee-lab" title="Bug reports">🐛</a></td>
<td align="center"><a href="https://github.com/fullofjoy"><img src="https://avatars.githubusercontent.com/u/30247571?v=4?s=100" width="100px;" alt="" /><br /><sub><b>jianghang</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=fullofjoy" title="Code">💻</a> <a href="https://github.com/tomsun28/hertzbeat/issues?q=author%3Afullofjoy" title="Bug reports">🐛</a></td>
<td align="center"><a href="https://github.com/ChineseTony"><img src="https://avatars.githubusercontent.com/u/24618786?v=4?s=100" width="100px;" alt="" /><br /><sub><b>ChineseTony</b></sub></a><br /><a href="https://github.com/tomsun28/hertzbeat/commits?author=ChineseTony" title="Code">💻</a></td>
</tr>
</table>
@@ -52,7 +52,7 @@ Bug fix.
5. [[collector]bugfix: fix warehouse data queue consume error #153](https://github.com/apache/hertzbeat/pull/153). issue by @daqianxiaoyao
6. [[web-app]bugfix:fix input blocking when input error in dark theme #157](https://github.com/apache/hertzbeat/pull/157). issue by @ConradWen
Online <https://console.tancloud.cn>.
Online: [tancloud](https://console.tancloud.cn)
-----------------------
@@ -69,5 +69,5 @@ Online <https://console.tancloud.cn>.
**Repository url**
[Github](https://github.com/apache/hertzbeat) <https://github.com/apache/hertzbeat>
[Gitee](https://gitee.com/hertzbeat/hertzbeat) <https://gitee.com/hertzbeat/hertzbeat>
[Github](https://github.com/apache/hertzbeat) [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)
[Gitee](https://gitee.com/hertzbeat/hertzbeat) [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)
+3 -3
View File
@@ -32,7 +32,7 @@ Bugfixes:
3. [[monitor] bugfix: Fix for Elasticsearch monitoring failure under basic authentication #174](https://github.com/apache/hertzbeat/pull/174) Contributed by @weifuqing
4. [Fix for monitoring failure due to ambiguous Oracle monitoring parameter "database name" #182](https://github.com/apache/hertzbeat/pull/182) @zklmcookle
Online at <https://console.tancloud.cn>.
Online at [https://console.tancloud.cn](https://console.tancloud.cn).
---
@@ -68,5 +68,5 @@ Have Fun!
**Repository Addresses**
[Github](https://github.com/apache/hertzbeat) <https://github.com/apache/hertzbeat>
[Gitee](https://gitee.com/hertzbeat/hertzbeat) <https://gitee.com/hertzbeat/hertzbeat>
[Github](https://github.com/apache/hertzbeat) [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)
[Gitee](https://gitee.com/hertzbeat/hertzbeat) [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)
+2 -2
View File
@@ -67,5 +67,5 @@ Have Fun!
**Repository Addresses**
[Github](https://github.com/apache/hertzbeat) <https://github.com/apache/hertzbeat>
[Gitee](https://gitee.com/hertzbeat/hertzbeat) <https://gitee.com/hertzbeat/hertzbeat>
[Github](https://github.com/apache/hertzbeat) [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)
[Gitee](https://gitee.com/hertzbeat/hertzbeat) [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)
+3 -3
View File
@@ -39,7 +39,7 @@ Bugfix.
8. [[manager] 修改监控页面取消监控功能再启动监控导致多生成jobId,原有监控项目并没有真实取消 #215](https://github.com/apache/hertzbeat/pull/215) contribute by @yangshihui
9. [[warehouse] bugfix exception when tdengine create table SQL contain special char #220](https://github.com/apache/hertzbeat/pull/220)
Online <https://console.tancloud.cn>.
Online [https://console.tancloud.cn](https://console.tancloud.cn).
Have Fun!
@@ -57,5 +57,5 @@ Have Fun!
**If you like HertzBeat, star us on GitHub**
[Github](https://github.com/apache/hertzbeat) <https://github.com/apache/hertzbeat>
[Gitee](https://gitee.com/hertzbeat/hertzbeat) <https://gitee.com/hertzbeat/hertzbeat>
[Github](https://github.com/apache/hertzbeat) [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)
[Gitee](https://gitee.com/hertzbeat/hertzbeat) [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)
+3 -3
View File
@@ -32,9 +32,9 @@ Bugfix.
1. [[docs] fix extend-http-jsonpath.md parseScript error #262](https://github.com/apache/hertzbeat/pull/262) contribute by @woshiniusange .
2. [[monitor] update help docs, refactor redis metrics name #264](https://github.com/apache/hertzbeat/pull/264)
3. [[manager] bugfix alert tags is null when tags map key normal value null. #270](https://github.com/apache/hertzbeat/pull/270) issue by <https://gitee.com/hello_brother_niu>
4. [[alert] bugfix: the alert global preset config do not take effect #275](https://github.com/apache/hertzbeat/pull/275) issue by <https://gitee.com/hello_brother_niu>
3. [[manager] bugfix alert tags is null when tags map key normal value null. #270](https://github.com/apache/hertzbeat/pull/270) issue by [https://gitee.com/hello_brother_niu](https://gitee.com/hello_brother_niu)
4. [[alert] bugfix: the alert global preset config do not take effect #275](https://github.com/apache/hertzbeat/pull/275) issue by [https://gitee.com/hello_brother_niu](https://gitee.com/hello_brother_niu)
Online <https://console.tancloud.cn>.
Online [https://console.tancloud.cn](https://console.tancloud.cn).
## Have Fun
+7 -7
View File
@@ -16,10 +16,10 @@ Today's article describes how to use hertzbeat monitoring system to detect the v
HertzBeat is a real-time monitoring tool with powerful customizable monitoring capabilities without the need for an agent. Website monitoring, PING connectivity, port availability, database, OS, middleware, API monitoring, threshold alerts, alert notifications (email weChat pinning flybook).
**Official website: <https://hertzbeat.apache.org> | <https://tancloud.cn>**
**Official website: [https://hertzbeat.apache.org](https://hertzbeat.apache.org) | [https://tancloud.cn](https://tancloud.cn)**
github: <https://github.com/apache/hertzbeat>
gitee: <https://gitee.com/hertzbeat/hertzbeat>
github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)
gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)
#### Install HertzBeat
@@ -86,8 +86,8 @@ gitee: <https://gitee.com/hertzbeat/hertzbeat>
You can refer to the help file for the token configuration of Nail WeChat Flying Book, etc.
<https://hertzbeat.apache.org/docs/help/alert_dingtalk>
<https://tancloud.cn/docs/help/alert_dingtalk>
[https://hertzbeat.apache.org/docs/help/alert_dingtalk](https://hertzbeat.apache.org/docs/help/alert_dingtalk)
[https://tancloud.cn/docs/help/alert_dingtalk](https://tancloud.cn/docs/help/alert_dingtalk)
> Alert Notification -> Add new alert notification policy -> Enable notification for the recipients you just configured
@@ -101,6 +101,6 @@ gitee: <https://gitee.com/hertzbeat/hertzbeat>
The practice of monitoring SSL certificates here, of course, for hertzbeat this function is just the tip of the iceberg, if you think hertzbeat this open source project is good if you welcome to give us in the GitHub Gitee star oh, thank you very much. Thank you for your support. The author!
**github: <https://github.com/apache/hertzbeat>**
**github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)**
**gitee: <https://gitee.com/hertzbeat/hertzbeat>**
**gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)**
+2 -2
View File
@@ -44,13 +44,13 @@ Bugfixes.
2. [[web-app] fix redirect when monitors app is null #286](https://github.com/apache/hertzbeat/pull/286)
3. [[alerter] bugfix aviator expression match npe #297](https://github.com/apache/hertzbeat/pull/297)
4. [[doc] fix project name error #294](https://github.com/apache/hertzbeat/pull/294) contributed by @CharlieXCL
5. [[common]feature:use "apache.http.conn.util" replace "sun.net.util" for upgrading java version #299](<https://github.com/dromara/> hertzbeat/pull/299) contributed by @Privauto
5. [[common]feature:use "apache.http.conn.util" replace "sun.net.util" for upgrading java version #299]([https://github.com/dromara/](https://github.com/dromara/) hertzbeat/pull/299) contributed by @Privauto
6. [Update docker-deploy.md #304](https://github.com/apache/hertzbeat/pull/304) contributed by @emrys-he
7. [fix(sec): upgrade snakeyaml to 1.31 #313](https://github.com/apache/hertzbeat/pull/313) contributed by @SxLiuYu
8. [[script] add startup log and optimize port service judgment #321](https://github.com/apache/hertzbeat/pull/321)
9. [[web-app] fix echarts y-axis value tip overflow #325](https://github.com/apache/hertzbeat/pull/325)
10. [[webapp] fix interceptor http resp common error-msg when error #329](https://github.com/apache/hertzbeat/pull/329)
Online <https://console.tancloud.cn>.
Online [https://console.tancloud.cn](https://console.tancloud.cn).
Have Fun!
+1 -1
View File
@@ -50,6 +50,6 @@ Bugfix.
6. [[manager] bugfix the gmtUpdate not change when update monitor param #459](https://github.com/apache/hertzbeat/pull/459)
7. [[home] fix typo in springboot2.md #464](https://github.com/apache/hertzbeat/pull/464) @eltociear
Online <https://console.tancloud.cn>.
Online [https://console.tancloud.cn](https://console.tancloud.cn).
Have Fun!
+1 -1
View File
@@ -56,7 +56,7 @@ github[wang1027-wqh](https://github.com/wang1027-wqh)
现从事:某互联网公司Java开发工程师
email<1758619238@qq.com>
email[1758619238@qq.com](mailto:1758619238@qq.com)
Hertzbeat Committer
+1 -1
View File
@@ -48,6 +48,6 @@ Bugfix.
7. [监控k8s问题issue描述与解决方案 #511](https://github.com/apache/hertzbeat/pull/511) @MrAndyMing
8. [[manager] springboot2 monitor support base path config #515](https://github.com/apache/hertzbeat/pull/515)
Online <https://console.tancloud.cn>.
Online [https://console.tancloud.cn](https://console.tancloud.cn).
Have Fun!
+5 -5
View File
@@ -40,7 +40,7 @@ tags: [opensource, practice]
- PROMETHEUS
```
2. Restart IoTDB, open a browser or use curl to access <http://ip:9091/metrics>, and you can see the metric data.
2. Restart IoTDB, open a browser or use curl to access [http://ip:9091/metrics](http://ip:9091/metrics), and you can see the metric data.
#### 2. Add IoTDB monitoring on the HertzBeat monitoring page
@@ -53,7 +53,7 @@ tags: [opensource, practice]
2. Configure the parameters required for monitoring IoTDB
Fill in the IoTDB **service IP** and **monitoring port** (default 9091) on the monitoring page, and finally click OK to add.
For other parameters such as **collection interval**, **timeout period**, etc., please refer to [Help Documentation](https://hertzbeat.apache.org/docs/help/iotdb/) <https://hertzbeat.apache.org/docs/help> /iotdb/
For other parameters such as **collection interval**, **timeout period**, etc., please refer to [Help Documentation](https://hertzbeat.apache.org/docs/help/iotdb/) [https://hertzbeat.apache.org/docs/help](https://hertzbeat.apache.org/docs/help) /iotdb/
![HertzBeat](/img/blog/monitor-iotdb-2.png)
@@ -95,7 +95,7 @@ tags: [opensource, practice]
Message notification methods support **email, DingTalk, WeChat Work, Feishu, WebHook, SMS**, etc. Here we take the commonly used DingTalk as an example.
- Refer to this [Help Documentation](https://hertzbeat.apache.org/docs/help/alert_dingtalk) <https://hertzbeat.apache.org/docs/help/alert_dingtalk> to configure the robot on DingTalk and set the security custom keyword `HertzBeat`, get the corresponding `access_token` value.
- Refer to this [Help Documentation](https://hertzbeat.apache.org/docs/help/alert_dingtalk) [https://hertzbeat.apache.org/docs/help/alert_dingtalk](https://hertzbeat.apache.org/docs/help/alert_dingtalk) to configure the robot on DingTalk and set the security custom keyword `HertzBeat`, get the corresponding `access_token` value.
- Configure the receiver parameters in HertzBeat as follows.
【Alarm Notification】->【New Recipient】->【Select DingTalk Robot Notification Method】->【Set DingTalk Robot ACCESS_TOKEN】->【OK】
@@ -122,8 +122,8 @@ Content details: The status of IOTDB node 127.0.0.1 is monitored as OFFLINE, ple
This practical article took us to experience that if HertzBeat is used to monitor the IoTDB database metric data, we can find that HertzBeat, which integrates monitoring-alarm-notification, is more convenient in operation and use, and IoTDB can be included in the monitoring with a simple click on the page, it is no longer necessary to deploy multiple components and write multiple YML configuration files with thresholds.
IoTDB Github: <https://github.com/apache/iotdb>
HertzBeat Github: <https://github.com/apache/hertzbeat>
IoTDB Github: [https://github.com/apache/iotdb](https://github.com/apache/iotdb)
HertzBeat Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)
**Welcome to learn about using Star Support!**
+9 -9
View File
@@ -42,11 +42,11 @@ tags: [opensource, practice]
1. Add the `metrics plugin` dependency to the `pom.xml` file of the gateway.
```xml
<dependency>
<groupId>org.apache.shenyu</groupId>
<artifactId>shenyu-spring-boot-starter-plugin-metrics</artifactId>
<version>${project.version}</version>
</dependency>
`<dependency>`
`<groupId>`org.apache.shenyu`</groupId>`
`<artifactId>`shenyu-spring-boot-starter-plugin-metrics`</artifactId>`
`<version>`${project.version}`</version>`
`</dependency>`
```
2. `metric` plugin Capture is turned off by default, edit the following in the gateway's configuration `yaml` file:
@@ -76,7 +76,7 @@ tags: [opensource, practice]
2. Configure the parameters required for monitoring ShenYu
On the monitor page, fill in ShenYu **service IP**, **monitor port** (default 8090), and click OK to add.
For other parameters such as **collection interval**, **timeout**, etc., you can refer to the [help file](https://hertzbeat.apache.org/docs/help/shenyu/) <https://hertzbeat.apache.org/docs/help/shenyu/>
For other parameters such as **collection interval**, **timeout**, etc., you can refer to the [help file](https://hertzbeat.apache.org/docs/help/shenyu/) [https://hertzbeat.apache.org/docs/help/shenyu/](https://hertzbeat.apache.org/docs/help/shenyu/)
![HertzBeat](/img/blog/monitor-shenyu-1.png)
@@ -125,7 +125,7 @@ Of course, just looking at it is not perfect, monitoring is often accompanied by
Message notification methods support **Email, Nail, WeChat, Flybook, WebHook, SMS**, etc. Here we take the commonly used Nail as an example.
- Refer to this [help document](https://hertzbeat.apache.org/docs/help/alert_dingtalk) <https://hertzbeat.apache.org/docs/help/alert_dingtalk> Configure the bot on the pinning side, set the security customization keyword `HertzBeat`, get the corresponding `access_token` value.
- Refer to this [help document](https://hertzbeat.apache.org/docs/help/alert_dingtalk) [https://hertzbeat.apache.org/docs/help/alert_dingtalk](https://hertzbeat.apache.org/docs/help/alert_dingtalk) Configure the bot on the pinning side, set the security customization keyword `HertzBeat`, get the corresponding `access_token` value.
- Configure the recipient parameters in HertzBeat as follows.
[Alert Notification] -> [Add Recipient] -> [Select Nailed Bot Notification Method] -> [Set Nailed Bot ACCESS_TOKEN] -> [OK]
@@ -156,8 +156,8 @@ Details : Please note that the number of file descriptors opened by ⚠️ ShenY
This hands-on article takes us through how to use HertzBeat to monitor Apache ShenYu metrics data, and we can find that HertzBeat, which combines ``Monitoring-Alert-Notification``, is much more convenient to operate and use, and you can include ShenYu in the monitoring by simply clicking on a page. There is no need to deploy multiple components and write multiple YML configuration files.
:::
Apache ShenYu Github: <https://github.com/apache/shenyu>
HertzBeat Github: <https://github.com/apache/hertzbeat>
Apache ShenYu Github: [https://github.com/apache/shenyu](https://github.com/apache/shenyu)
HertzBeat Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)
**Welcome to learn about using Star Support Oh!**
+4 -4
View File
@@ -88,7 +88,7 @@ tags: [opensource, practice]
2. Configure the parameters required for monitoring DynamicTp.
On the monitor page, fill in DynamicTp **service IP**, **monitoring port** (default 8080), and finally click OK to add it.
For other parameters such as **collection interval**, **timeout**, etc., you can refer to [help](https://hertzbeat.apache.org/docs/help/dynamic_tp/) <https://hertzbeat.apache.org/docs/help/dynamic_tp/>
For other parameters such as **collection interval**, **timeout**, etc., you can refer to [help](https://hertzbeat.apache.org/docs/help/dynamic_tp/) [https://hertzbeat.apache.org/docs/help/dynamic_tp/](https://hertzbeat.apache.org/docs/help/dynamic_tp/)
![HertzBeat](/img/blog/monitor-dynamic-tp-2.png)
@@ -137,7 +137,7 @@ Of course, just watching is not perfect, monitoring is often accompanied by alar
Message notification methods support **Email, Dingtalk, WeChat, Flybook, WebHook, SMS**, etc. We take the commonly used Dingtalk as an example.
- Refer to this [help document](https://hertzbeat.apache.org/docs/help/alert_dingtalk) <https://hertzbeat.apache.org/docs/help/alert_dingtalk> Configure the bot on Dingtalk side, set the security customization keyword `HertzBeat`, get the corresponding `access_token` value.
- Refer to this [help document](https://hertzbeat.apache.org/docs/help/alert_dingtalk) [https://hertzbeat.apache.org/docs/help/alert_dingtalk](https://hertzbeat.apache.org/docs/help/alert_dingtalk) Configure the bot on Dingtalk side, set the security customization keyword `HertzBeat`, get the corresponding `access_token` value.
- Configure the recipient parameters in HertzBeat as follows.
[Alert Notification] -> [Add Recipient] -> [Choose Dingtalk bot notification method] -> [Set Dingtalk bot ACCESS_TOKEN] -> [OK]
@@ -168,8 +168,8 @@ Details : DynamicTp has run timeout thread, count is 2
This practical article takes us to experience how to use HertzBeat to monitor DynamicTp thread pool metrics data, and we can find that HertzBeat with ``monitoring-alerting-notification`` is much more convenient to operate and use, and you only need to point and click on a page to include DynamicTp thread pool into the monitoring and alert notification, and you don't need to deploy multiple components to write YML configuration files anymore. There is no need to deploy multiple components and write YML configuration files.
:::
DynamicTp Github: <https://github.com/dromara/dynamic-tp>
HertzBeat Github: <https://github.com/apache/hertzbeat>
DynamicTp Github: [https://github.com/dromara/dynamic-tp](https://github.com/dromara/dynamic-tp)
HertzBeat Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)
**Welcome to learn how to use Star Support!**
+2 -2
View File
@@ -112,8 +112,8 @@ First of all, I am also a newcomer to Novice Village, but I can share some of my
>
> The powerful customization of `HertzBeat`, multi-type support, easy expansion, and low coupling, hope to help developers and small and medium-sized teams quickly build their own monitoring tools.
**Github: <https://github.com/apache/hertzbeat>**
**Gitee: <https://gitee.com/hertzbeat/hertzbeat>**
**Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)**
**Gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)**
## ⛄ Supported
+4 -4
View File
@@ -39,7 +39,7 @@ Keywords: [Open source monitoring tool, open source database monitoring, Mysql d
2. Configure the parameters required for the new monitoring Mysql database
On the monitoring page, fill in Mysql **service IP**, **monitoring port** (default 3306), **account password, etc.**, and finally click OK to add.
For other parameters such as **collection interval**, **timeout period**, etc., please refer to [Help Documentation](https://hertzbeat.apache.org/docs/help/mysql/) <https://hertzbeat.apache.org/docs/help> /mysql/
For other parameters such as **collection interval**, **timeout period**, etc., please refer to [Help Documentation](https://hertzbeat.apache.org/docs/help/mysql/) [https://hertzbeat.apache.org/docs/help](https://hertzbeat.apache.org/docs/help) /mysql/
![HertzBeat](/img/blog/monitor-mysql-2.png)
@@ -87,7 +87,7 @@ Of course, just looking at it is definitely not perfect. Monitoring is often acc
Message notification methods support **email, DingTalk, WeChat Work, Feishu, WebHook, SMS**, etc. Here we take the commonly used DingTalk as an example.
- Refer to this [Help Documentation](https://hertzbeat.apache.org/docs/help/alert_dingtalk) <https://hertzbeat.apache.org/docs/help/alert_dingtalk> to configure the robot on DingTalk and set the security custom keyword `HertzBeat`, get the corresponding `access_token` value.
- Refer to this [Help Documentation](https://hertzbeat.apache.org/docs/help/alert_dingtalk) [https://hertzbeat.apache.org/docs/help/alert_dingtalk](https://hertzbeat.apache.org/docs/help/alert_dingtalk) to configure the robot on DingTalk and set the security custom keyword `HertzBeat`, get the corresponding `access_token` value.
- Configure the receiver parameters in HertzBeat as follows.
【Alarm Notification】->【New Recipient】->【Select DingTalk Robot Notification Method】->【Set DingTalk Robot ACCESS_TOKEN】->【OK】
@@ -118,8 +118,8 @@ Content details: mysql db query_cache_hit_rate is too low, now is 20.
This practical article took us to experience how to use the open source real-time monitoring tool HertzBeat to monitor Mysql database metric data. We can find that HertzBeat, which integrates `monitoring-alarm-notification`, is more convenient in operation and use, just click on the page The Mysql database can be included in the monitoring and alarm notification, and the tedious operations of deploying multiple components and writing configuration files are no longer needed.
:::
Mysql Github: <https://github.com/mysql/mysql-server>
HertzBeat Github: <https://github.com/apache/hertzbeat>
Mysql Github: [https://github.com/mysql/mysql-server](https://github.com/mysql/mysql-server)
HertzBeat Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)
**Welcome to learn, use and star!**
+5 -5
View File
@@ -17,7 +17,7 @@ keywords: [Open source monitoring tool, operating system monitoring, Linux monit
- It configurable protocol specifications such as Http, Jmx, Ssh, Snmp, Jdbc, Prometheus, etc. You only need to configure YML to use these protocols to customize and collect any metrics you want to collect. Do you believe that you can immediately adapt to a new monitoring type such as K8s or Docker just by configuring YML?
- HertzBeat's powerful customization, multi-type support, easy expansion, and low coupling, hope to help developers and small and medium teams quickly build their own monitoring tools.
Github: <https://github.com/apache/hertzbeat>
Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)
### Get Linux Monitoring Done in HertzBeat in 5 Minutes
@@ -36,7 +36,7 @@ Github: <https://github.com/apache/hertzbeat>
2. Configure the parameters required for new monitoring Linux
Fill in the Linux **peer IP**, **SSH port** (default 22), **account password, etc.** on the monitoring page, and finally click OK to add.
For other parameters such as **collection interval**, **timeout period**, etc., please refer to the help document <https://hertzbeat.apache.org/docs/help/mysql/>
For other parameters such as **collection interval**, **timeout period**, etc., please refer to the help document [https://hertzbeat.apache.org/docs/help/mysql/](https://hertzbeat.apache.org/docs/help/mysql/)
![HertzBeat](/img/blog/monitor-linux-2.png)
@@ -88,7 +88,7 @@ Of course, just looking at it is definitely not perfect. Monitoring is often acc
Message notification methods support **email, DingTalk, WeChat Work, Feishu, WebHook, SMS**, etc. Here we take the commonly used DingTalk as an example.
- Refer to this [Help Documentation](https://hertzbeat.apache.org/docs/help/alert_dingtalk) <https://hertzbeat.apache.org/docs/help/alert_dingtalk> to configure the robot on DingTalk and set the security custom keyword `HertzBeat`, get the corresponding `access_token` value.
- Refer to this [Help Documentation](https://hertzbeat.apache.org/docs/help/alert_dingtalk) [https://hertzbeat.apache.org/docs/help/alert_dingtalk](https://hertzbeat.apache.org/docs/help/alert_dingtalk) to configure the robot on DingTalk and set the security custom keyword `HertzBeat`, get the corresponding `access_token` value.
- Configure the receiver parameters in HertzBeat as follows.
【Alarm Notification】->【New Recipient】->【Select DingTalk Robot Notification Method】->【Set DingTalk Robot ACCESS_TOKEN】->【OK】
@@ -132,8 +132,8 @@ This practical article took us to experience how to use the open source real-tim
>
> The powerful customization of `HertzBeat`, multi-type support, easy expansion, and low coupling, hope to help developers and small and medium-sized teams quickly build their own monitoring tools.
**Github: <https://github.com/apache/hertzbeat>**
**Gitee: <https://gitee.com/hertzbeat/hertzbeat>**
**Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)**
**Gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)**
## ⛄ Supported
+4 -4
View File
@@ -21,9 +21,9 @@ Website: hertzbeat.com | tancloud.cn
>
> Do you believe that you can immediately adapt a new monitoring type such as K8s or Docker just by defining YML?
**Github: <https://github.com/apache/hertzbeat>**
**Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)**
**Gitee: <https://gitee.com/hertzbeat/hertzbeat>**
**Gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)**
### v1.3.0 release is here
@@ -86,5 +86,5 @@ COMMIT;
---
**Github: <https://github.com/apache/hertzbeat>**
**Gitee: <https://gitee.com/hertzbeat/hertzbeat>**
**Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)**
**Gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)**
+5 -5
View File
@@ -21,7 +21,7 @@ keywords: [opensource monitoring, SpringBoot monitoring, alert]
>
> `HertzBeat`'s powerful custom-define, multi-type support, easy expansion, low coupling, hope to help developers and micro teams to quickly build their own monitoring system.
Github: <https://github.com/apache/hertzbeat>
Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)
### Monitoring SpringBoot2 Metrics with HertzBeat in 5 minutes
@@ -93,7 +93,7 @@ Github: <https://github.com/apache/hertzbeat>
2. Configure the parameters required for new monitoring SpringBoot2
Fill in the SpringBoot2 application **peer IP**, **service port** (default 8080), **account password, etc.** on the monitoring page, and finally click OK to add.
For other parameters such as **collection interval**, **timeout period**, etc., please refer to the help document <https://hertzbeat.apache.org/docs/help/>
For other parameters such as **collection interval**, **timeout period**, etc., please refer to the help document [https://hertzbeat.apache.org/docs/help/](https://hertzbeat.apache.org/docs/help/)
![HertzBeat](/img/blog/monitor-springboot2-2.png)
@@ -141,7 +141,7 @@ Of course, it is impossible to manually check the metrics in real time. Monitori
Message notification methods support **email, DingTalk, WeChat Work, Feishu, WebHook, SMS**, etc. Here we take the commonly used DingTalk as an example.
- Refer to this [Help Documentation](https://hertzbeat.apache.org/docs/help/alert_dingtalk) <https://hertzbeat.apache.org/docs/help/alert_dingtalk> to configure the robot on DingTalk and set the security custom keyword `HertzBeat`, get the corresponding `access_token` value.
- Refer to this [Help Documentation](https://hertzbeat.apache.org/docs/help/alert_dingtalk) [https://hertzbeat.apache.org/docs/help/alert_dingtalk](https://hertzbeat.apache.org/docs/help/alert_dingtalk) to configure the robot on DingTalk and set the security custom keyword `HertzBeat`, get the corresponding `access_token` value.
- Configure the receiver parameters in HertzBeat as follows.
【Alarm Notification】->【New Recipient】->【Select DingTalk Robot Notification Method】->【Set DingTalk Robot ACCESS_TOKEN】->【OK】
@@ -208,5 +208,5 @@ This practical article took us to experience how to use the open source real-tim
----
**Github: <https://github.com/apache/hertzbeat>**
**Gitee: <https://gitee.com/hertzbeat/hertzbeat>**
**Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)**
**Gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)**
+4 -4
View File
@@ -24,9 +24,9 @@ Website: hertzbeat.com | tancloud.cn
> `HertzBeat`'s powerful custom-define, multi-type support, easy expansion, low coupling, hope to help developers and micro teams to quickly build their own monitoring system.
> We also provide **[Monitoring SaaS Cloud](https://console.tancloud.cn)**, users no longer need to deploy a cumbersome monitoring system in order to monitor resources. **[Get started for free](https://console.tancloud.cn)**.
**Github: <https://github.com/apache/hertzbeat>**
**Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)**
**Gitee: <https://gitee.com/hertzbeat/hertzbeat>**
**Gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)**
### v1.3.1 release is here
@@ -66,5 +66,5 @@ COMMIT;
---
**Github: <https://github.com/apache/hertzbeat>**
**Gitee: <https://gitee.com/hertzbeat/hertzbeat>**
**Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)**
**Gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)**
+10 -10
View File
@@ -1,8 +1,8 @@
---
title: GreptimeDB & HertzBeat, using the open source temporal database GreptimeDB to store metrics for the open source real-time monitoring HertzBeat
author: tom
author_title: tom
author_url: https://github.com/tomsun28
title: GreptimeDB & HertzBeat, using the open source temporal database GreptimeDB to store metrics for the open source real-time monitoring HertzBeat
author: tom
author_title: tom
author_url: https://github.com/tomsun28
tags: [opensource, practice]
keywords: [open source monitoring system, open source temporal database, HertzBeat, GreptimeDB]
---
@@ -18,7 +18,7 @@ keywords: [open source monitoring system, open source temporal database, HertzBe
- Efficient storage and computation, with object storage and high data compression rate to achieve ultra-low storage costs. Built-in data analytics solution to avoid replicating data to external data warehouses.
- Distributed, Highly Reliable & Highly Available, easily scale each module independently with decoupled cloud-native architecture. Ensure data reliability and availability with configurable replicas and automated failover mechanisms.
Cloud: **[GreptimePlay](https://greptime.com/playground)**
Cloud: **[GreptimePlay](https://greptime.com/product/cloud)**
### What is HertzBeat?
@@ -75,9 +75,9 @@ See the [official documentation](https://hertzbeat.apache.org/zh-cn/docs/start/d
- `-v /opt/application.yml:/opt/hertzbeat/config/application.yml` : Mount customized local configuration files to the container, i.e. use local configuration files to overwrite the container configuration files.
Note that the ⚠️ local mount configuration file `application.yml` needs to exist in advance, and the full contents of the file can be found in the project repository [/script/application.yml](<https://github.com/apache/hertzbeat/raw/master/script/> application.yml)
Note that the ⚠️ local mount configuration file `application.yml` needs to exist in advance, and the full contents of the file can be found in the project repository [/script/application.yml]([https://github.com/apache/hertzbeat/raw/master/script/](https://github.com/apache/hertzbeat/raw/master/script/) application.yml)
2. Go to <http://ip:1157/> with the default account and password admin/hertzbeat to see if HertzBeat starts successfully.
2. Go to [http://ip:1157/](http://ip:1157/) with the default account and password admin/hertzbeat to see if HertzBeat starts successfully.
#### Configure to use GreptimeDB to store HertzBeat monitoring metrics metrics data
@@ -105,7 +105,7 @@ See the [official documentation](https://hertzbeat.apache.org/zh-cn/docs/start/d
#### Observe the authentication effect
1. visit HertzBeat in your browser <http://ip:1157/> default account password admin/hertzbeat
1. visit HertzBeat in your browser [http://ip:1157/](http://ip:1157/) default account password admin/hertzbeat
2. Use HertzBeat to add application monitors, such as website monitors, Linux monitors, Mysql monitors, and so on.
3. After monitoring and collecting several cycles, check whether GreptimeDB database stores the metrics data and whether HertzBeat metrics data graph data is displayed normally.
@@ -122,8 +122,8 @@ Here's the picture: !
This article took us to experience how to use the open source time-series database GreptimeDB to store the metrics data of the open source real-time monitoring HertzBeat, in general, the two open source products is very simple to get started, the key is that if it is too much trouble do not want to deploy both of them still have cloud services 😂 let you toss.
As one of the developers of the feature [HertzBeat supports GreptimeDB](https://github.com/apache/hertzbeat/pull/834), in the actual adaptation process, GreptimeDB's silky-smooth native SDK and relational database-like SQL, let us from other GreptimeDB native SDK and relational database-like SQL make it very easy to switch from other time-series databases like `TDengine, IotDB, InfluxDB` to GreptimeDB, and the experience is very smooth.
GreptimeDB Github: <https://github.com/GreptimeTeam/greptimedb>
HertzBeat Github: <https://github.com/apache/hertzbeat>
GreptimeDB Github: [https://github.com/GreptimeTeam/greptimedb](https://github.com/GreptimeTeam/greptimedb)
HertzBeat Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)
**Finally, you are welcome to be more understanding, more use, more comments, more ISSUE, more PR, more Star support these two did not come out for a long time hope to get care of open source cattle are not afraid of difficulties a small star oh! Do open source, we are sincere, love 💗**
+4 -4
View File
@@ -24,9 +24,9 @@ Website: hertzbeat.com | tancloud.cn
> `HertzBeat`'s powerful custom-define, multi-type support, easy expansion, low coupling, hope to help developers and micro teams to quickly build their own monitoring system.
> We also provide **[Monitoring SaaS Cloud](https://console.tancloud.cn)**, users no longer need to deploy a cumbersome monitoring system in order to monitor resources. **[Get started for free](https://console.tancloud.cn)**.
**Github: <https://github.com/apache/hertzbeat>**
**Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)**
**Gitee: <https://gitee.com/hertzbeat/hertzbeat>**
**Gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)**
### v1.3.2
@@ -92,5 +92,5 @@ spring:
---
**Github: <https://github.com/apache/hertzbeat>**
**Gitee: <https://gitee.com/hertzbeat/hertzbeat>**
**Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)**
**Gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)**
+51 -51
View File
@@ -25,9 +25,9 @@ keywords: [open source monitoring system, alerting system, Linux monitoring]
![HertzBeat](/img/docs/hertzbeat-arch.png)
**Github: <https://github.com/apache/hertzbeat>**
**Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)**
**Gitee: <https://gitee.com/hertzbeat/hertzbeat>**
**Gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)**
### The cluster version is coming
@@ -100,53 +100,53 @@ Detailed config refer to [Install HertzBeat via Docker](https://hertzbeat.apache
> Welcome to explore more new version updates, thanks to the hard work of the community partners, love 💗!
* [doc] add v1.3.2 publish doc by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1075>
* remove elasticsearch unused param index by @Ceilzcx in <https://github.com/apache/hertzbeat/pull/1080>
* feature support monitoring apache airflow by @luoxuanzao in <https://github.com/apache/hertzbeat/pull/1081>
* add luoxuanzao as a contributor for code by @allcontributors in <https://github.com/apache/hertzbeat/pull/1083>
* [collector] bugfix sshd cannot use private key to connect by @gcdd1993 in <https://github.com/apache/hertzbeat/pull/1084>
* bugfix update dashboard alerts cards height not consist by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1087>
* Feature#serverchan by @zqr10159 in <https://github.com/apache/hertzbeat/pull/1092>
* bugfix dm database monitoring connect error by @lisongning in <https://github.com/apache/hertzbeat/pull/1094>
* add lisongning as a contributor for code by @allcontributors in <https://github.com/apache/hertzbeat/pull/1096>
* update alert rule operator display "<=" to ">=" by @Ceilzcx in <https://github.com/apache/hertzbeat/pull/1097>
* [doc] add custom monitoring relate document by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1098>
* add YutingNie as a contributor for code by @allcontributors in <https://github.com/apache/hertzbeat/pull/1103>
* Remove unreachable status by @YutingNie in <https://github.com/apache/hertzbeat/pull/1102>
* 139 auto update alert status by @l646505418 in <https://github.com/apache/hertzbeat/pull/1104>
* feat: aviator fn for str contains, exists & matches by @mikezzb in <https://github.com/apache/hertzbeat/pull/1106>
* add mikezzb as a contributor for code by @allcontributors in <https://github.com/apache/hertzbeat/pull/1107>
* bugfix common alarm do not need monitorId tag existed by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1108>
* bugfix extern alert do not have labels mapping inner monitor by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1111>
* feature: support apache spark metrics monitoring by @a-little-fool in <https://github.com/apache/hertzbeat/pull/1114>
* add a-little-fool as a contributor for code by @allcontributors in <https://github.com/apache/hertzbeat/pull/1116>
* [Feature]Add third report of TenCloud by @zqr10159 in <https://github.com/apache/hertzbeat/pull/1113>
* [Feature]Add third report of TenCloud (#1113) by @zqr10159 in <https://github.com/apache/hertzbeat/pull/1119>
* [manager] fix: can query by tags when tagValue is null by @l646505418 in <https://github.com/apache/hertzbeat/pull/1118>
* bugfix the notification template environment variable display error by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1120>
* add littlezhongzer as a contributor for code by @allcontributors in <https://github.com/apache/hertzbeat/pull/1127>
* feature:monitor brearer token api, ignore letter case to comparison by @littlezhongzer in <https://github.com/apache/hertzbeat/pull/1122>
* docs: enhance README by @mikezzb in <https://github.com/apache/hertzbeat/pull/1128>
* Update app-oracle.yml by @ChenXiangxxxxx in <https://github.com/apache/hertzbeat/pull/1129>
* add ChenXiangxxxxx as a contributor for code by @allcontributors in <https://github.com/apache/hertzbeat/pull/1130>
* fix alarm silence strategy setting failed by @Ceilzcx in <https://github.com/apache/hertzbeat/pull/1131>
* support run sql script file in jdbc protocol config by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1117>
* bugfix return old cache json file when upgrade version by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1137>
* support ssh protocol config choose if reuse connection by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1136>
* feat(web): alert threshold UI support matches & contains by @mikezzb in <https://github.com/apache/hertzbeat/pull/1138>
* support hertzbeat metrics collector cluster by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1101>
* add collector card in dashboard by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1147>
* bugfix: linux collect warning: bad syntax, perhaps a bogus '-' by @Mr-zhou315 in <https://github.com/apache/hertzbeat/pull/1151>
* add Mr-zhou315 as a contributor for code by @allcontributors in <https://github.com/apache/hertzbeat/pull/1157>
* support config timezone locale language region on web ui by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1154>
* bugfix monitoring template app name already exists by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1152>
* bugfix can not startup when error monitoring template yml file by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1153>
* tags also deleted when the monitor is deleted by @Ceilzcx in <https://github.com/apache/hertzbeat/pull/1159>
* monitoring param host with http head will not be error reported by @littlezhongzer in <https://github.com/apache/hertzbeat/pull/1155>
* [script] feature update build.sh and Dockerfile: detect app version a… by @XimfengYao in <https://github.com/apache/hertzbeat/pull/1162>
* add XimfengYao as a contributor for code by @allcontributors in <https://github.com/apache/hertzbeat/pull/1163>
* [doc] add collector clusters document by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1161>
* [hertzbeat] release hertzbeat version v1.4.0 by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1168>
* [doc] add v1.3.2 publish doc by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1075](https://github.com/apache/hertzbeat/pull/1075)
* remove elasticsearch unused param index by @Ceilzcx in [https://github.com/apache/hertzbeat/pull/1080](https://github.com/apache/hertzbeat/pull/1080)
* feature support monitoring apache airflow by @luoxuanzao in [https://github.com/apache/hertzbeat/pull/1081](https://github.com/apache/hertzbeat/pull/1081)
* add luoxuanzao as a contributor for code by @allcontributors in [https://github.com/apache/hertzbeat/pull/1083](https://github.com/apache/hertzbeat/pull/1083)
* [collector] bugfix sshd cannot use private key to connect by @gcdd1993 in [https://github.com/apache/hertzbeat/pull/1084](https://github.com/apache/hertzbeat/pull/1084)
* bugfix update dashboard alerts cards height not consist by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1087](https://github.com/apache/hertzbeat/pull/1087)
* Feature#serverchan by @zqr10159 in [https://github.com/apache/hertzbeat/pull/1092](https://github.com/apache/hertzbeat/pull/1092)
* bugfix dm database monitoring connect error by @lisongning in [https://github.com/apache/hertzbeat/pull/1094](https://github.com/apache/hertzbeat/pull/1094)
* add lisongning as a contributor for code by @allcontributors in [https://github.com/apache/hertzbeat/pull/1096](https://github.com/apache/hertzbeat/pull/1096)
* update alert rule operator display "\<=" to "\>=" by @Ceilzcx in [https://github.com/apache/hertzbeat/pull/1097](https://github.com/apache/hertzbeat/pull/1097)
* [doc] add custom monitoring relate document by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1098](https://github.com/apache/hertzbeat/pull/1098)
* add YutingNie as a contributor for code by @allcontributors in [https://github.com/apache/hertzbeat/pull/1103](https://github.com/apache/hertzbeat/pull/1103)
* Remove unreachable status by @YutingNie in [https://github.com/apache/hertzbeat/pull/1102](https://github.com/apache/hertzbeat/pull/1102)
* 139 auto update alert status by @l646505418 in [https://github.com/apache/hertzbeat/pull/1104](https://github.com/apache/hertzbeat/pull/1104)
* feat: aviator fn for str contains, exists & matches by @mikezzb in [https://github.com/apache/hertzbeat/pull/1106](https://github.com/apache/hertzbeat/pull/1106)
* add mikezzb as a contributor for code by @allcontributors in [https://github.com/apache/hertzbeat/pull/1107](https://github.com/apache/hertzbeat/pull/1107)
* bugfix common alarm do not need monitorId tag existed by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1108](https://github.com/apache/hertzbeat/pull/1108)
* bugfix extern alert do not have labels mapping inner monitor by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1111](https://github.com/apache/hertzbeat/pull/1111)
* feature: support apache spark metrics monitoring by @a-little-fool in [https://github.com/apache/hertzbeat/pull/1114](https://github.com/apache/hertzbeat/pull/1114)
* add a-little-fool as a contributor for code by @allcontributors in [https://github.com/apache/hertzbeat/pull/1116](https://github.com/apache/hertzbeat/pull/1116)
* [Feature]Add third report of TenCloud by @zqr10159 in [https://github.com/apache/hertzbeat/pull/1113](https://github.com/apache/hertzbeat/pull/1113)
* [Feature]Add third report of TenCloud (#1113) by @zqr10159 in [https://github.com/apache/hertzbeat/pull/1119](https://github.com/apache/hertzbeat/pull/1119)
* [manager] fix: can query by tags when tagValue is null by @l646505418 in [https://github.com/apache/hertzbeat/pull/1118](https://github.com/apache/hertzbeat/pull/1118)
* bugfix the notification template environment variable display error by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1120](https://github.com/apache/hertzbeat/pull/1120)
* add littlezhongzer as a contributor for code by @allcontributors in [https://github.com/apache/hertzbeat/pull/1127](https://github.com/apache/hertzbeat/pull/1127)
* feature:monitor brearer token api, ignore letter case to comparison by @littlezhongzer in [https://github.com/apache/hertzbeat/pull/1122](https://github.com/apache/hertzbeat/pull/1122)
* docs: enhance README by @mikezzb in [https://github.com/apache/hertzbeat/pull/1128](https://github.com/apache/hertzbeat/pull/1128)
* Update app-oracle.yml by @ChenXiangxxxxx in [https://github.com/apache/hertzbeat/pull/1129](https://github.com/apache/hertzbeat/pull/1129)
* add ChenXiangxxxxx as a contributor for code by @allcontributors in [https://github.com/apache/hertzbeat/pull/1130](https://github.com/apache/hertzbeat/pull/1130)
* fix alarm silence strategy setting failed by @Ceilzcx in [https://github.com/apache/hertzbeat/pull/1131](https://github.com/apache/hertzbeat/pull/1131)
* support run sql script file in jdbc protocol config by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1117](https://github.com/apache/hertzbeat/pull/1117)
* bugfix return old cache json file when upgrade version by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1137](https://github.com/apache/hertzbeat/pull/1137)
* support ssh protocol config choose if reuse connection by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1136](https://github.com/apache/hertzbeat/pull/1136)
* feat(web): alert threshold UI support matches & contains by @mikezzb in [https://github.com/apache/hertzbeat/pull/1138](https://github.com/apache/hertzbeat/pull/1138)
* support hertzbeat metrics collector cluster by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1101](https://github.com/apache/hertzbeat/pull/1101)
* add collector card in dashboard by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1147](https://github.com/apache/hertzbeat/pull/1147)
* bugfix: linux collect warning: bad syntax, perhaps a bogus '-' by @Mr-zhou315 in [https://github.com/apache/hertzbeat/pull/1151](https://github.com/apache/hertzbeat/pull/1151)
* add Mr-zhou315 as a contributor for code by @allcontributors in [https://github.com/apache/hertzbeat/pull/1157](https://github.com/apache/hertzbeat/pull/1157)
* support config timezone locale language region on web ui by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1154](https://github.com/apache/hertzbeat/pull/1154)
* bugfix monitoring template app name already exists by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1152](https://github.com/apache/hertzbeat/pull/1152)
* bugfix can not startup when error monitoring template yml file by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1153](https://github.com/apache/hertzbeat/pull/1153)
* tags also deleted when the monitor is deleted by @Ceilzcx in [https://github.com/apache/hertzbeat/pull/1159](https://github.com/apache/hertzbeat/pull/1159)
* monitoring param host with http head will not be error reported by @littlezhongzer in [https://github.com/apache/hertzbeat/pull/1155](https://github.com/apache/hertzbeat/pull/1155)
* [script] feature update build.sh and Dockerfile: detect app version a… by @XimfengYao in [https://github.com/apache/hertzbeat/pull/1162](https://github.com/apache/hertzbeat/pull/1162)
* add XimfengYao as a contributor for code by @allcontributors in [https://github.com/apache/hertzbeat/pull/1163](https://github.com/apache/hertzbeat/pull/1163)
* [doc] add collector clusters document by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1161](https://github.com/apache/hertzbeat/pull/1161)
* [hertzbeat] release hertzbeat version v1.4.0 by @tomsun28 in [https://github.com/apache/hertzbeat/pull/1168](https://github.com/apache/hertzbeat/pull/1168)
---
@@ -163,5 +163,5 @@ Detailed config refer to [Install HertzBeat via Docker](https://hertzbeat.apache
---
**Github: <https://github.com/apache/hertzbeat>**
**Gitee: <https://gitee.com/hertzbeat/hertzbeat>**
**Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)**
**Gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)**
+1 -1
View File
@@ -84,6 +84,6 @@ HertzBeat HertzBeat is an open source real-time monitoring and alerting system w
> `HertzBeat`'s powerful customization, multi-type support, high performance, easy to extend, low-coupling, and hopefully can help developers and teams to quickly build their own monitoring system.
Github: <https://github.com/apache/hertzbeat>
Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)
More users are welcome to participate in `HertzBeat` open source collaboration, no matter a typo or punctuation we are very welcome.
+56 -56
View File
@@ -25,9 +25,9 @@ keywords: [open source monitoring system, alerting system, Linux monitoring]
![HertzBeat](/img/docs/hertzbeat-arch.png)
**Github: <https://github.com/apache/hertzbeat>**
**Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)**
**Gitee: <https://gitee.com/hertzbeat/hertzbeat>**
**Gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)**
### HertzBeat's 1.4.1 version is coming
@@ -83,58 +83,58 @@ Detailed config refer to [Install HertzBeat via Docker](https://hertzbeat.apache
> Welcome to explore more new version updates, thanks to the hard work of the community partners, love 💗!
* bugfix npe when get undefined name monitor template yml by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1173>
* [bug fixed]When importing and exporting monitoring, support export collectors, configure collectors when importing by @zqr10159 in <https://github.com/apache/hertzbeat/pull/1178>
* support alert threshold rule config system value row count by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1180>
* Update README.md by @zqr10159 in <https://github.com/apache/hertzbeat/pull/1182>
* support config alert threshold tags bind by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1181>
* the back-end of help component has been built by @YutingNie in <https://github.com/apache/hertzbeat/pull/1160>
* support enable alert threshold auto resolved notice by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1185>
* Delete tag of the dashboard's homepage on the top four pages by @Ceilzcx in <https://github.com/apache/hertzbeat/pull/1189>
* replace obsolete `registry.npm.taobao.org` to`registry.npmmirror.com` by @zqr10159 in <https://github.com/apache/hertzbeat/pull/1192>
* refactor MonitorServiceImpl by @Carpe-Wang in <https://github.com/apache/hertzbeat/pull/1190>
* config default system timezone and fix monitor status auto recover by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1187>
* update-doc-doris by @zqr10159 in <https://github.com/apache/hertzbeat/pull/1193>
* [manager] support tidb database monitoring by @luxx-lq in <https://github.com/apache/hertzbeat/pull/733>
* refactor fix potential npe by @Carpe-Wang in <https://github.com/apache/hertzbeat/pull/1197>
* [ospp] support ui help massage component by @YutingNie in <https://github.com/apache/hertzbeat/pull/1199>
* support monitor metrics name i18n by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1198>
* support google analytics by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1202>
* refactor code and fix some npe by @Carpe-Wang in <https://github.com/apache/hertzbeat/pull/1201>
* bugfix fix found 2 dataQueue bean when not config common.queue param by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1205>
* Help component update by @YutingNie in <https://github.com/apache/hertzbeat/pull/1207>
* bugfix enterprise wechat push display content is too cumbersome by @l646505418 in <https://github.com/apache/hertzbeat/pull/1149>
* bugfix WeChatAppAlertNotifyHandlerImpl by @LINGLUOJUN in <https://github.com/apache/hertzbeat/pull/1208>
* add LINGLUOJUN as a contributor for code by @allcontributors in <https://github.com/apache/hertzbeat/pull/1209>
* fix jmx jndi inject vulnerability by @luelueking in <https://github.com/apache/hertzbeat/pull/1215>
* add luelueking as a contributor for code by @allcontributors in <https://github.com/apache/hertzbeat/pull/1217>
* bugfix monitoring param number limit range by @qyaaaa in <https://github.com/apache/hertzbeat/pull/1216>
* add qyaaaa as a contributor for code by @allcontributors in <https://github.com/apache/hertzbeat/pull/1218>
* add app-ping i18n by @qyaaaa in <https://github.com/apache/hertzbeat/pull/1220>
* some codes opt by @LINGLUOJUN in <https://github.com/apache/hertzbeat/pull/1214>
* support deploy hertzbeat by kubernetes helm charts by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1221>
* bugfix threshold setting template variables has repeated parameters by @qyaaaa in <https://github.com/apache/hertzbeat/pull/1223>
* support display metrics i18n label when threshold setting by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1225>
* bugfix user role display not correctly on webui by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1227>
* add hertzbeat about msg card by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1229>
* add app-api i18n by @novohit in <https://github.com/apache/hertzbeat/pull/1236>
* add novohit as a contributor for code by @allcontributors in <https://github.com/apache/hertzbeat/pull/1238>
* [feature]Add `getAlertDefinesByName`. by @zqr10159 in <https://github.com/apache/hertzbeat/pull/1237>
* thread pool executor support shutdown gracefully by @LINGLUOJUN in <https://github.com/apache/hertzbeat/pull/1240>
* fix: expression injection RCE by @mikezzb in <https://github.com/apache/hertzbeat/pull/1241>
* [bugfix]Replace schema "{key1:value1}" to "{\"key1\":\"value1\"}" by @zqr10159 in <https://github.com/apache/hertzbeat/pull/1245>
* [Refactor] Use static methods instead of constructors for Message.java by @gcdd1993 in <https://github.com/apache/hertzbeat/pull/1247>
* bugfix snake yaml decode rce by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1239>
* bugfix jackson deserialize localDatetime error by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1249>
* netty as an independent module, add new feature about collector list by @Ceilzcx in <https://github.com/apache/hertzbeat/pull/1244>
* support show deploy collector script in web by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1251>
* bugfix mongodb collect extra metrics npe by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1257>
* bugfix fix collector run cyclic when connect auth failed by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1256>
* update webapp login ui by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1260>
* bugfix collector can not auto reconnect when channel idle by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1259>
* update alarm notice wework app send content ui by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1258>
* [hertzbeat] release hertzbeat version v1.4.1 by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1261>
* auto split webhook token when user input hook url by @tomsun28 in <https://github.com/apache/hertzbeat/pull/1262>
* bugfix npe when get undefined name monitor template yml by @tomsun28 in [#1173](https://github.com/apache/hertzbeat/pull/1173)
* [bug fixed]When importing and exporting monitoring, support export collectors, configure collectors when importing by @zqr10159 in [#1178](https://github.com/apache/hertzbeat/pull/1178)
* support alert threshold rule config system value row count by @tomsun28 in [#1180](https://github.com/apache/hertzbeat/pull/1180)
* Update README.md by @zqr10159 in [#1182](https://github.com/apache/hertzbeat/pull/1182)
* support config alert threshold tags bind by @tomsun28 in [#1181](https://github.com/apache/hertzbeat/pull/1181)
* the back-end of help component has been built by @YutingNie in [#1160](https://github.com/apache/hertzbeat/pull/1160)
* support enable alert threshold auto resolved notice by @tomsun28 in [#1185](https://github.com/apache/hertzbeat/pull/1185)
* Delete tag of the dashboard's homepage on the top four pages by @Ceilzcx in [#1189](https://github.com/apache/hertzbeat/pull/1189)
* replace obsolete `registry.npm.taobao.org` to`registry.npmmirror.com` by @zqr10159 in [#1192](https://github.com/apache/hertzbeat/pull/1192)
* refactor MonitorServiceImpl by @Carpe-Wang in [#1190](https://github.com/apache/hertzbeat/pull/1190)
* config default system timezone and fix monitor status auto recover by @tomsun28 in [#1187](https://github.com/apache/hertzbeat/pull/1187)
* update-doc-doris by @zqr10159 in [#1193](https://github.com/apache/hertzbeat/pull/1193)
* [manager] support tidb database monitoring by @luxx-lq in [#733](https://github.com/apache/hertzbeat/pull/733)
* refactor fix potential npe by @Carpe-Wang in [#1197](https://github.com/apache/hertzbeat/pull/1197)
* [ospp] support ui help massage component by @YutingNie in [#1199](https://github.com/apache/hertzbeat/pull/1199)
* support monitor metrics name i18n by @tomsun28 in [#1198](https://github.com/apache/hertzbeat/pull/1198)
* support google analytics by @tomsun28 in [#1202](https://github.com/apache/hertzbeat/pull/1202)
* refactor code and fix some npe by @Carpe-Wang in [#1201](https://github.com/apache/hertzbeat/pull/1201)
* bugfix fix found 2 dataQueue bean when not config common.queue param by @tomsun28 in [#1205](https://github.com/apache/hertzbeat/pull/1205)
* Help component update by @YutingNie in [#1207](https://github.com/apache/hertzbeat/pull/1207)
* bugfix enterprise wechat push display content is too cumbersome by @l646505418 in [#1149](https://github.com/apache/hertzbeat/pull/1149)
* bugfix WeChatAppAlertNotifyHandlerImpl by @LINGLUOJUN in [#1208](https://github.com/apache/hertzbeat/pull/1208)
* add LINGLUOJUN as a contributor for code by @allcontributors in [#1209](https://github.com/apache/hertzbeat/pull/1209)
* fix jmx jndi inject vulnerability by @luelueking in [#1215](https://github.com/apache/hertzbeat/pull/1215)
* add luelueking as a contributor for code by @allcontributors in [#1217](https://github.com/apache/hertzbeat/pull/1217)
* bugfix monitoring param number limit range by @qyaaaa in [#1216](https://github.com/apache/hertzbeat/pull/1216)
* add qyaaaa as a contributor for code by @allcontributors in [#1218](https://github.com/apache/hertzbeat/pull/1218)
* add app-ping i18n by @qyaaaa in [#1220](https://github.com/apache/hertzbeat/pull/1220)
* some codes opt by @LINGLUOJUN in [#1214](https://github.com/apache/hertzbeat/pull/1214)
* support deploy hertzbeat by kubernetes helm charts by @tomsun28 in [#1221](https://github.com/apache/hertzbeat/pull/1221)
* bugfix threshold setting template variables has repeated parameters by @qyaaaa in [#1223](https://github.com/apache/hertzbeat/pull/1223)
* support display metrics i18n label when threshold setting by @tomsun28 in [#1225](https://github.com/apache/hertzbeat/pull/1225)
* bugfix user role display not correctly on webui by @tomsun28 in [#1227](https://github.com/apache/hertzbeat/pull/1227)
* add hertzbeat about msg card by @tomsun28 in [#1229](https://github.com/apache/hertzbeat/pull/1229)
* add app-api i18n by @novohit in [#1236](https://github.com/apache/hertzbeat/pull/1236)
* add novohit as a contributor for code by @allcontributors in [#1238](https://github.com/apache/hertzbeat/pull/1238)
* [feature]Add `getAlertDefinesByName`. by @zqr10159 in [#1237](https://github.com/apache/hertzbeat/pull/1237)
* thread pool executor support shutdown gracefully by @LINGLUOJUN in [#1240](https://github.com/apache/hertzbeat/pull/1240)
* fix: expression injection RCE by @mikezzb in [#1241](https://github.com/apache/hertzbeat/pull/1241)
* [bugfix]Replace schema `"{key1:value1}"` to `"{\"key1\":\"value1\"}"` by @zqr10159 in [#1245](https://github.com/apache/hertzbeat/pull/1245)
* [Refactor] Use static methods instead of constructors for Message.java by @gcdd1993 in [#1247](https://github.com/apache/hertzbeat/pull/1247)
* bugfix snake yaml decode rce by @tomsun28 in [#1239](https://github.com/apache/hertzbeat/pull/1239)
* bugfix jackson deserialize localDatetime error by @tomsun28 in [#1249](https://github.com/apache/hertzbeat/pull/1249)
* netty as an independent module, add new feature about collector list by @Ceilzcx in [#1244](https://github.com/apache/hertzbeat/pull/1244)
* support show deploy collector script in web by @tomsun28 in [#1251](https://github.com/apache/hertzbeat/pull/1251)
* bugfix mongodb collect extra metrics npe by @tomsun28 in [#1257](https://github.com/apache/hertzbeat/pull/1257)
* bugfix fix collector run cyclic when connect auth failed by @tomsun28 in [#1256](https://github.com/apache/hertzbeat/pull/1256)
* update webapp login ui by @tomsun28 in [#1260](https://github.com/apache/hertzbeat/pull/1260)
* bugfix collector can not auto reconnect when channel idle by @tomsun28 in [#1259](https://github.com/apache/hertzbeat/pull/1259)
* update alarm notice wework app send content ui by @tomsun28 in [#1258](https://github.com/apache/hertzbeat/pull/1258)
* [hertzbeat] release hertzbeat version v1.4.1 by @tomsun28 in [#1261](https://github.com/apache/hertzbeat/pull/1261)
* auto split webhook token when user input hook url by @tomsun28 in [#1262](https://github.com/apache/hertzbeat/pull/1262)
---
@@ -151,5 +151,5 @@ Detailed config refer to [Install HertzBeat via Docker](https://hertzbeat.apache
---
**Github: <https://github.com/apache/hertzbeat>**
**Gitee: <https://gitee.com/hertzbeat/hertzbeat>**
**Github: [https://github.com/apache/hertzbeat](https://github.com/apache/hertzbeat)**
**Gitee: [https://gitee.com/hertzbeat/hertzbeat](https://gitee.com/hertzbeat/hertzbeat)**

Some files were not shown because too many files have changed in this diff Show More